Commit Graph

147 Commits

Author SHA1 Message Date
xPushkin f24d00fcc6 A lot of typo fixes 2017-10-08 13:30:35 +01:00
Aliaksandr Valialkin 5116aa64e0 Mention that the body returned from Request.Body and Response.Body is valid until the Request/Response modification 2017-09-12 13:15:57 +03:00
Aliaksandr Valialkin 8948e047bf Issue #176: reset Content-Length when compression is enabled for streamed response body 2017-06-27 20:50:04 +03:00
Aliaksandr Valialkin b154429ca9 Compress responses only if their content-type starts with text/ or application/ 2017-05-17 17:10:29 +03:00
Aliaksandr Valialkin 103adc311e Do not compress responses with sizes smaller than 200 bytes
Compressing small bodies has little sense, since the compressed result size
may exceed the original body size.

This should save CPU time when the server responds with small responses.
2017-05-17 16:17:53 +03:00
Aliaksandr Valialkin 30e92af08f Limit heap memory usage when compressing high number of concurrent responses
Previously each concurrent compression could allocate huge compression state
with the size up to 1Mb each. So 10K concurrent connections could result in
10Gb of compression state in the heap.

This CL limits the number of compression states among concurrent requests
when {Append,Write}{Gzip,Deflate}* functions are called to O(GOMAXPROCS).
These functions are used by CompressHandler* for non-streaming responses,
i.e. it should cover the majority of use cases.

Memory usage for 10K concurrent connections that compress responses drops
from 10Gb to 200Mb after this CL.
2017-05-17 14:45:31 +03:00
Aliaksandr Valialkin b0de56d13b Properly set "https" scheme in RequestCtx.URI() for TLS connections 2017-01-15 00:26:21 +02:00
Aliaksandr Valialkin 4ee6517626 Issue #193: properly handle nil body when calling WriteGzip* and WriteDeflate* on Response struct 2016-10-31 13:41:50 +02:00
Aliaksandr Valialkin 3e1cb31dcd Added SwapBody to Request and Response for zero-copy body pass between Request and Response objects 2016-09-12 13:42:59 +03:00
Aliaksandr Valialkin 834fb48f10 Immediately write compressed response data to the connection after the user flushes the data from StreamWriter 2016-08-18 13:03:57 +03:00
Aliaksandr Valialkin 8e21bc7f60 do not compress response body again if Content-Encoding is set 2016-08-17 14:50:58 +03:00
Aliaksandr Valialkin d055141f64 Propagate 'https' scheme to request URI for TLS connections 2016-08-17 14:01:35 +03:00
Aliaksandr Valialkin 45915c69f6 Optimization: recycle request and response body buffers only if Server.ReduceMemoryUsage is set 2016-07-18 17:46:52 +03:00
Aliaksandr Valialkin 778f83ed89 Discourage using ReleaseBody, since now bytebufferpool should properly handle the case mentioned in #102 2016-06-27 18:26:56 +03:00
Aliaksandr Valialkin 00868f2226 Use separate pools for request and response bodies.
This should reduce potential memory waste related to byte buffers' pool usage.
2016-06-24 15:15:59 +03:00
Aliaksandr Valialkin 8ca66d3de8 Use github.com/valyala/bytebufferpool, which should protect against memory fragmentation in the pool of byte buffers like mentioned in the PR #102 2016-06-22 20:32:00 +03:00
Aliaksandr Valialkin 83a24c301b Issue #114: added SetHost and Host helper methods to Request 2016-06-15 20:32:54 +03:00
Aliaksandr Valialkin 8db7d05839 Close base response body streams when wrapping them into gzip or deflate stream writers 2016-06-10 20:19:15 +03:00
Aliaksandr Valialkin 90baa054bc Export IsBodyStream on RequestCtx, Request and Response
These methods may help writing proper request handler wrappers
for request handlers, which may set body either via SetBodyStream*
or via usual body methods.
2016-06-03 17:07:13 +03:00
Aliaksandr Valialkin 1fc1f4cbf5 Reduce memory usage under high load by pooling request/response bodies.
- Use request/response body pools.
- Defer request/response body allocation until it is really required.
- Return request/response bodies to the pool as soon as they become unused.

This minimizes the total amount of memory occupied by active request/response
bodies under high load.
2016-06-03 16:54:22 +03:00
Aliaksandr Valialkin bdd73c3261 small cleanup after 117f109c29 2016-05-27 16:11:29 +03:00
Jason Fesler 117f109c29 Add func (resp *Response) ReleaseBody(size int) (#102) 2016-05-24 15:49:08 +03:00
Aliaksandr Valialkin fa50251b46 Issue #100: accept POST query args with arbitrary charset set via Content-Type 2016-05-20 18:14:06 +03:00
Aliaksandr Valialkin c9c8d72607 Issue #78: immediately flush chunked data to client 2016-04-15 23:14:38 +03:00
Aliaksandr Valialkin 8280b7a162 Moved empty noCopy struct to the top of container structs. See @stemar94 's comment at https://github.com/golang/go/issues/12884 for details 2016-03-06 00:17:08 +02:00
Aliaksandr Valialkin 9f43aa1601 Do not expose noCopy.Lock 2016-03-04 21:20:52 +02:00
Aliaksandr Valialkin 9fa69c74af Embed noCopy struct into structs, which mustn't be copied
This should help `go vet` detecting invalid structs' copyings.
See https://github.com/golang/go/issues/8005#issuecomment-190753527 for details.
2016-03-04 16:57:24 +02:00
Aliaksandr Valialkin 8581513fa2 Issue #61: server: properly read multipart/form-data 2016-03-01 18:07:52 +02:00
Aliaksandr Valialkin 3546c31c2b Issue #60: skip body copying in DoTimeout. This should improve DoTimeout performance when dealing with big request and/or response bodies 2016-03-01 01:13:50 +02:00
Aliaksandr Valialkin 70316d0535 Issue #60: give up to find an optimal strategy for body buffers' re-use 2016-02-29 18:35:15 +02:00
Aliaksandr Valialkin 8985a565b2 Issue #60: throughput tuning: re-use body buffers if their size is smaller than 8Kb 2016-02-29 18:11:17 +02:00
Aliaksandr Valialkin fcfda9f312 Issue #60: increased client and server throughput when working with big bodies 2016-02-29 17:31:47 +02:00
Aliaksandr Valialkin c1437a71e6 Shuffled Request and Response members in order to reduce object sizes 2016-02-25 13:57:21 +02:00
Aliaksandr Valialkin f3d9e7ae9b golint fix 2016-02-25 13:30:25 +02:00
Sebastian Schepens 889c65bae6 Added BodyInflate and WriteInflate to match BodyGunzip and WriteGunzip 2016-02-22 14:36:58 -03:00
Aliaksandr Valialkin b98beaa40f Added RequestURI helper to Request 2016-02-21 13:22:13 +02:00
Aliaksandr Valialkin be49d3027a Allow updating request's RequestURI and Host header via Request.URI() 2016-02-19 18:53:13 +02:00
Aliaksandr Valialkin 14b2ff3d2b Issue #14: Added BodyGunzip helper method to Request to be consistent with Response 2016-02-18 17:01:05 +02:00
Aliaksandr Valialkin d7f433999f Document that it is safe re-using []byte buffers after SetBody and AppendBody function calls 2016-02-18 16:49:23 +02:00
Aliaksandr Valialkin 7cde0cf1fa Reduce memory waste and fragmentation by limiting the maximum request and response body size which may be re-used 2016-02-18 12:23:11 +02:00
Aliaksandr Valialkin fd2887a5fc Issue #53: Clarify that the following instances mustn't be used from concurrently running goroutines: Args, Cookie, URI, RequestCtx, Request, Response, RequestHeader and ResponseHeader 2016-02-17 11:45:21 +02:00
Aliaksandr Valialkin 83e1796359 Do not copy body stream in Request.CopyTo to be consistent with Response.CopyTo 2016-02-14 14:38:00 +02:00
Aliaksandr Valialkin a8c4ad4d10 document that WriteGzip* and WriteDeflate* methods on Response compress Response.Body() before writing it to output stream 2016-02-14 14:33:19 +02:00
Aliaksandr Valialkin c53eccc930 always close body stream after writing request/response 2016-02-14 14:18:49 +02:00
Aliaksandr Valialkin e7e436064f Keep response and request body after calling Body() if SetBodyStream is called 2016-02-13 11:42:22 +02:00
Aliaksandr Valialkin 87628147e5 Use zero-alloc copy instead of io.Copy 2016-02-13 11:01:18 +02:00
Aliaksandr Valialkin 77a12cbf68 Added SetBodyStream* to Request. This allows streaming multi-GB data in request bodies 2016-02-12 20:52:29 +02:00
Aliaksandr Valialkin 1ee286887c Substituted bytes.Buffer with ByteBuffer in marshalMultipartForm 2016-02-12 14:00:26 +02:00
Aliaksandr Valialkin dd96137a25 Substitute bytes.Buffer with ByteBuffer when compressing response body and converting request and response to string. This shaves off one memory allocation 2016-02-11 20:03:15 +02:00
Aliaksandr Valialkin 6cd79f25f2 Added BodyWriteTo to Request and Response 2016-02-09 20:04:46 +02:00