* Reduce sizeof ResponseHeader and RequestHeader
+ Reduce ResponseHeader from 320 to 312 bytes
+ Reduce RequestHeader from 360 to 352 bytes
+ In the benchmark tests, although there is no significant performance improvement, it theoretically reduces memory usage by 2.2% to 2.5%.
* Remove redundant comment
* The StreamRequestBody should not read content beyond what is required.
The StreamRequestBody feature on the server side should not read content that does not belong to the current request body.This is more logical and consistent with the result of not using the StreamRequestBody feature.Fixes: https://github.com/valyala/fasthttp/issues/1816.
* Update server_test.go
Co-authored-by: Erik Dubbelboer <erik@dubbelboer.com>
* Update http.go
Co-authored-by: Erik Dubbelboer <erik@dubbelboer.com>
---------
Co-authored-by: Erik Dubbelboer <erik@dubbelboer.com>
1. Reduce RequestHeader from 368 bytes to 360 bytes
2. Reduce Request from 816 bytes to 800 bytes
3. Reduce Response from 432 bytes to 416 bytes
4. Reduce Client from 312 bytes to 288 bytes
5. Reduce HostClient from 416 bytes to 392 bytes
6. Reduce PipelineClient from 176 bytes to 168 bytes
7. Reduce pipelineConnClient from 216 bytes to 208 bytes
8. Reduce Cookie from 232 bytes to 224 bytes
9. Reduce FS from 184 bytes to 160 bytes
10. Reduce fsHandler from 168 bytes to 160 bytes
11. Reduce ResponseHeader from 328 bytes to 320 bytes
12. Reduce headerScanner from 128 bytes to 120 bytes
13. Reduce TCPDialer from 104 bytes to 96 bytes
14. Reduce workerPool from 152 btyes to 144 btyes
* fix: propagate body stream error to close function (#1743)
* fix: http test
* fix: close body stream with error in encoding functions
* fix: lint
---------
Co-authored-by: Max Denushev <denushev@tochka.com>
* feat:support zstd compress and uncompressed
* fix:real & stackless write using different pool to avoid get stackless.writer
* fix:zstd normalize compress level
* Change empty string checks to be more idiomatic (#1684)
* chore:lint fix and rebase with master
* chore:remove 1.18 test & upgrade compress version
* fix:error default compress level
* Fix lint
---------
Co-authored-by: Erik Dubbelboer <erik@dubbelboer.com>
or an *io.LimitedReader of an *os.File (because that's also supported by
https://cs.opensource.google/go/go/+/refs/tags/go1.21.4:src/bufio/bufio.go;l=784)
I think that having to flush less often outweighs the overhead of the
extra check. The appended data is known to be large, but it might still
save us a syscall by allowing it to buffer more.
These are the same statements at the beginning of io.CopyBuffer, but
by doing them ourselves first we trade off a little cpu for not holding
the 4kb buffer during the write.
* Allow redirect URI path to not be normalized.
* Introduce DisableRedirectPathNormalizing field to Request
* Use field name as start of comment.
Co-authored-by: Erik Dubbelboer <erik@dubbelboer.com>
---------
Co-authored-by: Erik Dubbelboer <erik@dubbelboer.com>
* feat: allow connection close for custom streams
* fix: avoid req access since might already be released
* fix: fix aloc test fails
* fix: race condition when clossing body stream
* Auto add 'Vary' header after compression
Add config `SetAddVaryHeaderForCompression` to enable
'Vary: Accept-Encoding' header when compression is used.
* feat: always set the Vary header
* create and use `ResponseHeader.AddVaryBytes`
* not export 'AddVaryBytes'
* client: simplify (*HostClient).do()
Remove an allocation in favour of deferring a call to release the
response.
* client: remove panic in dialAddr
Return an error instead of panicking if the user supplied a nonsensical
DialFunc.
* compression: remove panic on invalid compression level
If a compression level exceeding gzip's boundaries is provided, fasthttp
will panic. Instead it would be better to handle this error for them by
limiting it to the minimum or maximum value, depending on the direction
the user has exceeded the limits.
Clamp the value of gzip to always be between gzip.BestSpeed and
gzip.BestCompression.
* peripconn: remove panic on negative connection count
When a negative count is reached when unregistering a connection, a
panic is caused even though data-integrity is not at risk.
Replace the panic() with a simple clamp on the value to ensure the
value does not exceed it's expected lower bounds.
References: #1504
* compress: remove error on failed nonblocking writes
Since there is no way of handling or even logging non-critical errors in
stateless non-blocking writecalls, just drop them and hope the user
notices and tries again.
* workerPool: remove panic on redundant Start and Stop calls
Instead of panicking for invalid behaviour, it's preferable to just turn
the function into a noop.
* http: remove panic on invalid form boundary
* http: remove panic on negative reads
Since bufio already panics on negative reads, it is not necessary to do
so as well. If the length is zero and for some reason no error is
returned, readBodyIdentity and appendBodyFixedSize now errors in these
cases.
Link: https://github.com/golang/go/blob/851f6fd61425c810959c7ab51e6dc86f8a63c970/src/bufio/bufio.go#L246
* fs: remove panic on negative reader count
When a negative count is reached when unregistering a reader, a panic is
thrown even though data-integrity is not at risk.
Replace the panic() with a simple clamp on the value to ensure the
value does not exceed it's expected lower bounds.
* server: remove panic in favour of a segfault
Panicking with "BUG: " obscures the error. As the segfault causes a
panic anyway, just let the chaos unfold.
* server: remove panic in favour of returning an error
Writing on a timed-out response is not endangering data integrity and
just fails.
* chore: add comments to all panics
* chore: fix minor typo
* Request.SetTimeout
This functionally works the same as e.g. Client.DoTimeout(), but we can
also use it for Client.DoRedirect(). There is no way as far as I can
tell to set a timeout on a DoRedirect call, so this makes it possible.
* tests
* docs
* Update Go Version to Go1.19.x And add cache
* Fix CI Line endings
* Update test CI Go Version to Go1.19.x And add cache
* Update Gosec Security Scanner CI to securego/gosec@v2.12.0
* Format comment
Go 1.19 adds support for links, lists, and clearer headings in doc comments. As part of this change, gofmt now reformats doc comments to make their rendered meaning clearer. See “Go Doc Comments” for syntax details and descriptions of common mistakes now highlighted by gofmt. As another part of this change, the new package go/doc/comment provides parsing and reformatting of doc comments as well as support for rendering them to HTML, Markdown, and text.
ref: https://tip.golang.org/doc/go1.19
ref: https://tip.golang.org/doc/comment
* Fix doc structure
* header.go ContentEncoding() getter and setters
For Response the CE header is stored into a separate field because compressed responses are often used.
But for the Request let's just peek and store it from headers map
* http.go: New BodyUncompressed() method for request and responses
The new method returns a body and uncompress if it's gzipped
* Response.ContentEncoding(): store as field
The CE is not so often used for plain APIs responses and even not so often used for static files and on the fly compression.
But still it should be checked each time.
Also having a dedicated field getter and setter simplifies code
* header.go Use shorter Response.setNonSpecial() and Request.setNonSpecial() methods instead of SetCanonical()
The change should improve performance because the setSpecialHeader() call is omitted.
As a downside on adding a new basic header field all putHeader() must be replaced with a direct getter and setter.
* Read response when client closes connection #1232
* Fix edge case were client responds with invalid header
* Follow linter suggestions for tests
* Changes after review
* Reafactor error check after review
* Handle connection reset on windows
* Remove format string from test where not needed
* Run connection reset tests not on Windows
* feature: add request and response body size limit, it prevents the large body from slowly stretching the memory of the entire service
* fix: http.go fmt
* refact: optimize code naming
* Update http.go
Co-authored-by: Erik Dubbelboer <erik@dubbelboer.com>
* Update http.go
Co-authored-by: Erik Dubbelboer <erik@dubbelboer.com>
* Update http.go
Co-authored-by: Erik Dubbelboer <erik@dubbelboer.com>
* Update http.go
Co-authored-by: Erik Dubbelboer <erik@dubbelboer.com>
* Update http.go
Co-authored-by: Erik Dubbelboer <erik@dubbelboer.com>
Co-authored-by: Erik Dubbelboer <erik@dubbelboer.com>
* Allow to set Host header for Client
* Allow to change Host header without tests violation
* Rename AllowToChangeHostHeader and add tests.
* Allow to use empty uri.Host() when req.Header.Host() does not empty
Erik Dubbelboer
Kashiwa
newacorn
Juan Calderon-Perez
Max Denushev
Oleksandr Redko
Co1a
Oleksandr Redko
Jille Timmermans
Scott Kidder
Armin Becher
AutumnSun
Abe Saiovici
Anthony-Dong
Moritz Poldrack
tyltr
kinggo
Andy Pan
brian-armstrong-discord
nick9822
byene0923
Aoang
mathew
Sergey Ponomarev
zzzzwc
ArminBTVS
Rennbon
Tolyar