* client: simplify (*HostClient).do()
Remove an allocation in favour of deferring a call to release the
response.
* client: remove panic in dialAddr
Return an error instead of panicking if the user supplied a nonsensical
DialFunc.
* compression: remove panic on invalid compression level
If a compression level exceeding gzip's boundaries is provided, fasthttp
will panic. Instead it would be better to handle this error for them by
limiting it to the minimum or maximum value, depending on the direction
the user has exceeded the limits.
Clamp the value of gzip to always be between gzip.BestSpeed and
gzip.BestCompression.
* peripconn: remove panic on negative connection count
When a negative count is reached when unregistering a connection, a
panic is caused even though data-integrity is not at risk.
Replace the panic() with a simple clamp on the value to ensure the
value does not exceed it's expected lower bounds.
References: #1504
* compress: remove error on failed nonblocking writes
Since there is no way of handling or even logging non-critical errors in
stateless non-blocking writecalls, just drop them and hope the user
notices and tries again.
* workerPool: remove panic on redundant Start and Stop calls
Instead of panicking for invalid behaviour, it's preferable to just turn
the function into a noop.
* http: remove panic on invalid form boundary
* http: remove panic on negative reads
Since bufio already panics on negative reads, it is not necessary to do
so as well. If the length is zero and for some reason no error is
returned, readBodyIdentity and appendBodyFixedSize now errors in these
cases.
Link: https://github.com/golang/go/blob/851f6fd61425c810959c7ab51e6dc86f8a63c970/src/bufio/bufio.go#L246
* fs: remove panic on negative reader count
When a negative count is reached when unregistering a reader, a panic is
thrown even though data-integrity is not at risk.
Replace the panic() with a simple clamp on the value to ensure the
value does not exceed it's expected lower bounds.
* server: remove panic in favour of a segfault
Panicking with "BUG: " obscures the error. As the segfault causes a
panic anyway, just let the chaos unfold.
* server: remove panic in favour of returning an error
Writing on a timed-out response is not endangering data integrity and
just fails.
* chore: add comments to all panics
* chore: fix minor typo
* Request.SetTimeout
This functionally works the same as e.g. Client.DoTimeout(), but we can
also use it for Client.DoRedirect(). There is no way as far as I can
tell to set a timeout on a DoRedirect call, so this makes it possible.
* tests
* docs
* Update Go Version to Go1.19.x And add cache
* Fix CI Line endings
* Update test CI Go Version to Go1.19.x And add cache
* Update Gosec Security Scanner CI to securego/gosec@v2.12.0
* Format comment
Go 1.19 adds support for links, lists, and clearer headings in doc comments. As part of this change, gofmt now reformats doc comments to make their rendered meaning clearer. See “Go Doc Comments” for syntax details and descriptions of common mistakes now highlighted by gofmt. As another part of this change, the new package go/doc/comment provides parsing and reformatting of doc comments as well as support for rendering them to HTML, Markdown, and text.
ref: https://tip.golang.org/doc/go1.19
ref: https://tip.golang.org/doc/comment
* Fix doc structure
* header.go ContentEncoding() getter and setters
For Response the CE header is stored into a separate field because compressed responses are often used.
But for the Request let's just peek and store it from headers map
* http.go: New BodyUncompressed() method for request and responses
The new method returns a body and uncompress if it's gzipped
* Response.ContentEncoding(): store as field
The CE is not so often used for plain APIs responses and even not so often used for static files and on the fly compression.
But still it should be checked each time.
Also having a dedicated field getter and setter simplifies code
* header.go Use shorter Response.setNonSpecial() and Request.setNonSpecial() methods instead of SetCanonical()
The change should improve performance because the setSpecialHeader() call is omitted.
As a downside on adding a new basic header field all putHeader() must be replaced with a direct getter and setter.
* Read response when client closes connection #1232
* Fix edge case were client responds with invalid header
* Follow linter suggestions for tests
* Changes after review
* Reafactor error check after review
* Handle connection reset on windows
* Remove format string from test where not needed
* Run connection reset tests not on Windows
* feature: add request and response body size limit, it prevents the large body from slowly stretching the memory of the entire service
* fix: http.go fmt
* refact: optimize code naming
* Update http.go
Co-authored-by: Erik Dubbelboer <erik@dubbelboer.com>
* Update http.go
Co-authored-by: Erik Dubbelboer <erik@dubbelboer.com>
* Update http.go
Co-authored-by: Erik Dubbelboer <erik@dubbelboer.com>
* Update http.go
Co-authored-by: Erik Dubbelboer <erik@dubbelboer.com>
* Update http.go
Co-authored-by: Erik Dubbelboer <erik@dubbelboer.com>
Co-authored-by: Erik Dubbelboer <erik@dubbelboer.com>
* Allow to set Host header for Client
* Allow to change Host header without tests violation
* Rename AllowToChangeHostHeader and add tests.
* Allow to use empty uri.Host() when req.Header.Host() does not empty
Currently, the only way to set URI for a request is to call SetRequestURI(string).
Then when a request performed the string will be parsed into a fasthttp.URI struct.
If there are many requests with the same URI then we'll waste CPU for a parsing of the same URI string.
With the new SetURI(*URI) method we can once parse a URI string into a fasthttp.URI struct and then reuse it for many requests.
Unfortunately the URI will be copied because may be modified inside the request.
But anyway this will be more lightweight than parsing.
* Adding ConvertHTTPRequest and renaming ConvertRequest to ConvertFastRequest
* Removing forServer boolean from ConvertHTTPRequest
* Preparing for PR
* Reverting adaptor changes
* Fixing godoc, adding req.ReadBody function as well
* Update comment to be more clear
As per @erikdubbelboer suggestion
Co-authored-by: Erik Dubbelboer <erik@dubbelboer.com>
Co-authored-by: Erik Dubbelboer <erik@dubbelboer.com>
* http: refactor out crlf reading function
Make it a bit simpler and make it reusable.
* streaming: fix chunked stream test
This test is supposed to check for stream unchunking, but it's not really
effective with that because chunks created by createChunkedBody() get wrapped
into another chunk by writeBodyStream(), so we end up with chunkedBody in
request handler although what we really want is plain body.
Deduplicate test and benchmark also.
* streaming: fix Read interface
It wasn't actually compatible with io.Reader as io.Reader _never_ returns n >
len(p) while this function easily did that for chunked payloads confusing its
users:
panic: runtime error: slice bounds out of range [:528] with capacity 512
goroutine 562 [running]:
io.ReadAll(0x9f4380, 0xc0003be1a0, 0xc0004fcd80, 0x0, 0x0, 0xc00086bc30, 0x46f99b)
/usr/lib64/go/1.16/src/io/io.go:634 +0x205
io/ioutil.ReadAll(...)
/usr/lib64/go/1.16/src/io/ioutil/ioutil.go:27
github.com/valyala/fasthttp.getChunkedTestEnv.func1(0xc001fdc680)
/home/rik/dev/fasthttp/streaming_test.go:108 +0x6c
github.com/valyala/fasthttp.(*Server).serveConn(0xc000416d80, 0xa034e8, 0xc0004da880, 0x0, 0x0)
/home/rik/dev/fasthttp/server.go:2219 +0x12ee
github.com/valyala/fasthttp.(*workerPool).workerFunc(0xc000148960, 0xc0003be160)
/home/rik/dev/fasthttp/workerpool.go:223 +0xba
github.com/valyala/fasthttp.(*workerPool).getCh.func1(0xc000148960, 0xc0003be160, 0x8b4ec0, 0xc0003be160)
/home/rik/dev/fasthttp/workerpool.go:195 +0x35
created by github.com/valyala/fasthttp.(*workerPool).getCh
/home/rik/dev/fasthttp/workerpool.go:194 +0x11f
It also returned len(p) in some cases where it read less than that.
- Allow DisablePreParseMultipartForm in combination with
StreamRequestBody.
- Support streaming into MultipartForm instead of reading the whole body
first.
- Support calling ctx.PostBody() when streaming is enabled.
New Functions:
CompressHandlerBrotliLevel(h RequestHandler, brotliLevel, otherLevel int) RequestHandler
Request.BodyUnbrotli() ([]byte, error)
Response.BodyUnbrotli() ([]byte, error)
AppendBrotliBytesLevel(dst, src []byte, level int) []byte
WriteBrotliLevel(w io.Writer, p []byte, level int) (int, error)
WriteBrotli(w io.Writer, p []byte) (int, error)
AppendBrotliBytes(dst, src []byte) []byte
WriteUnbrotli(w io.Writer, p []byte) (int, error)
AppendUnbrotliBytes(dst, src []byte) ([]byte, error)
New Constants:
CompressBrotliNoCompression
CompressBrotliBestSpeed
CompressBrotliBestCompression
CompressBrotliDefaultCompression
Brotli compression levels are different from gzip/flate. Because of this we have separate level constants and CompressHandlerBrotliLevel takes 2 levels.
I didn't add Brotli support to CompressHandler as this could cause a spike in CPU usage when users upgrade fasthttp.
fasthttp.CompressBrotliDefaultCompression is not the same as
brotli.DefaultCompression. brotli.DefaultCompression is more than twice
as slow as fasthttp.CompressBrotliDefaultCompression which I thought was
unreasonable as default.
Oleksandr Redko
Abe Saiovici
Anthony-Dong
Moritz Poldrack
Oleksandr Redko
tyltr
kinggo
Andy Pan
brian-armstrong-discord
nick9822
byene0923
Aoang
mathew
Sergey Ponomarev
Erik Dubbelboer
zzzzwc
ArminBTVS
Rennbon
Tolyar
ichx
Shivansh Vij
Evgenii
Meng
MoreFreeze
Roman Khimov
Daniel Firsht
Vladimir Shteinman
Kirill Danshin
kiyon
Mohammad Alian
So-chiru