Reject space and tab between the chunk-size and chunk-extension separator while
preserving net/http-compatible trailing OWS before CRLF.
This avoids parser divergence for chunk lines such as "3 ;ext\r\n" without
breaking the existing acceptance of padded chunk-size lines like "3 \r\n".
Add parser regression coverage for both accepted and rejected forms.
* Add WithLimit methods for uncompression
The current uncompress methods don't enforce a memory limit and are
susceptible to things like zip bombs. This pull introduces new methods
so retain backwards compatibility. The old methods might be deprecated
in the future.
* Fix suggestion
This change updates header parsing to match the behavior of net/http more closely.
**Breaking change**: headers delimited by `\n` (instead of `\r\n`) are no longer supported.
Previously, fasthttp accepted `\n` as a delimiter, which is not spec compliant.
This made it difficult to correctly parse headers containing both `\n` and `\r\n`.
* refact: Eliminate duplication in Request/Response headers via struct embedding
* revert: SetMultipartFormBoundaryBytes
* refact: rename the params of SetProtocol and SetProtocolBytes
In theory this can optimize some code paths where a string first needs
to be converted to a []byte to use the normal Write method. By
implementing WriteString this extra copy isn't needed. Internall we
don't do the copy and just use s2b instead.
The current client implementation does not immediately return when encountering an RST packet while sending a request, but instead ignores it. This behavior is inconsistent with the net/http package and does not make logical sense.
* Reduce sizeof ResponseHeader and RequestHeader
+ Reduce ResponseHeader from 320 to 312 bytes
+ Reduce RequestHeader from 360 to 352 bytes
+ In the benchmark tests, although there is no significant performance improvement, it theoretically reduces memory usage by 2.2% to 2.5%.
* Remove redundant comment
* The StreamRequestBody should not read content beyond what is required.
The StreamRequestBody feature on the server side should not read content that does not belong to the current request body.This is more logical and consistent with the result of not using the StreamRequestBody feature.Fixes: https://github.com/valyala/fasthttp/issues/1816.
* Update server_test.go
Co-authored-by: Erik Dubbelboer <erik@dubbelboer.com>
* Update http.go
Co-authored-by: Erik Dubbelboer <erik@dubbelboer.com>
---------
Co-authored-by: Erik Dubbelboer <erik@dubbelboer.com>
1. Reduce RequestHeader from 368 bytes to 360 bytes
2. Reduce Request from 816 bytes to 800 bytes
3. Reduce Response from 432 bytes to 416 bytes
4. Reduce Client from 312 bytes to 288 bytes
5. Reduce HostClient from 416 bytes to 392 bytes
6. Reduce PipelineClient from 176 bytes to 168 bytes
7. Reduce pipelineConnClient from 216 bytes to 208 bytes
8. Reduce Cookie from 232 bytes to 224 bytes
9. Reduce FS from 184 bytes to 160 bytes
10. Reduce fsHandler from 168 bytes to 160 bytes
11. Reduce ResponseHeader from 328 bytes to 320 bytes
12. Reduce headerScanner from 128 bytes to 120 bytes
13. Reduce TCPDialer from 104 bytes to 96 bytes
14. Reduce workerPool from 152 btyes to 144 btyes
* fix: propagate body stream error to close function (#1743)
* fix: http test
* fix: close body stream with error in encoding functions
* fix: lint
---------
Co-authored-by: Max Denushev <denushev@tochka.com>
* feat:support zstd compress and uncompressed
* fix:real & stackless write using different pool to avoid get stackless.writer
* fix:zstd normalize compress level
* Change empty string checks to be more idiomatic (#1684)
* chore:lint fix and rebase with master
* chore:remove 1.18 test & upgrade compress version
* fix:error default compress level
* Fix lint
---------
Co-authored-by: Erik Dubbelboer <erik@dubbelboer.com>
or an *io.LimitedReader of an *os.File (because that's also supported by
https://cs.opensource.google/go/go/+/refs/tags/go1.21.4:src/bufio/bufio.go;l=784)
I think that having to flush less often outweighs the overhead of the
extra check. The appended data is known to be large, but it might still
save us a syscall by allowing it to buffer more.
These are the same statements at the beginning of io.CopyBuffer, but
by doing them ourselves first we trade off a little cpu for not holding
the 4kb buffer during the write.
* Allow redirect URI path to not be normalized.
* Introduce DisableRedirectPathNormalizing field to Request
* Use field name as start of comment.
Co-authored-by: Erik Dubbelboer <erik@dubbelboer.com>
---------
Co-authored-by: Erik Dubbelboer <erik@dubbelboer.com>
* feat: allow connection close for custom streams
* fix: avoid req access since might already be released
* fix: fix aloc test fails
* fix: race condition when clossing body stream
* Auto add 'Vary' header after compression
Add config `SetAddVaryHeaderForCompression` to enable
'Vary: Accept-Encoding' header when compression is used.
* feat: always set the Vary header
* create and use `ResponseHeader.AddVaryBytes`
* not export 'AddVaryBytes'
* client: simplify (*HostClient).do()
Remove an allocation in favour of deferring a call to release the
response.
* client: remove panic in dialAddr
Return an error instead of panicking if the user supplied a nonsensical
DialFunc.
* compression: remove panic on invalid compression level
If a compression level exceeding gzip's boundaries is provided, fasthttp
will panic. Instead it would be better to handle this error for them by
limiting it to the minimum or maximum value, depending on the direction
the user has exceeded the limits.
Clamp the value of gzip to always be between gzip.BestSpeed and
gzip.BestCompression.
* peripconn: remove panic on negative connection count
When a negative count is reached when unregistering a connection, a
panic is caused even though data-integrity is not at risk.
Replace the panic() with a simple clamp on the value to ensure the
value does not exceed it's expected lower bounds.
References: #1504
* compress: remove error on failed nonblocking writes
Since there is no way of handling or even logging non-critical errors in
stateless non-blocking writecalls, just drop them and hope the user
notices and tries again.
* workerPool: remove panic on redundant Start and Stop calls
Instead of panicking for invalid behaviour, it's preferable to just turn
the function into a noop.
* http: remove panic on invalid form boundary
* http: remove panic on negative reads
Since bufio already panics on negative reads, it is not necessary to do
so as well. If the length is zero and for some reason no error is
returned, readBodyIdentity and appendBodyFixedSize now errors in these
cases.
Link: https://github.com/golang/go/blob/851f6fd61425c810959c7ab51e6dc86f8a63c970/src/bufio/bufio.go#L246
* fs: remove panic on negative reader count
When a negative count is reached when unregistering a reader, a panic is
thrown even though data-integrity is not at risk.
Replace the panic() with a simple clamp on the value to ensure the
value does not exceed it's expected lower bounds.
* server: remove panic in favour of a segfault
Panicking with "BUG: " obscures the error. As the segfault causes a
panic anyway, just let the chaos unfold.
* server: remove panic in favour of returning an error
Writing on a timed-out response is not endangering data integrity and
just fails.
* chore: add comments to all panics
* chore: fix minor typo
Erik Dubbelboer
Kashiwa
Max Denushev
osxtest
zhangyongding
Jeppe Bonde Weikop
Oleksandr Redko
Agustin Arce
newacorn
Juan Calderon-Perez
Oleksandr Redko
Co1a
Jille Timmermans
Scott Kidder
Armin Becher
AutumnSun
Abe Saiovici
Anthony-Dong
Moritz Poldrack