- I noticed that fasthttp was taking up 1.8MB of heap memory, even
though it wasn't being used. This turned out to be the stackless
function: 1.80MB github.com/valyala/fasthttp/stackless.NewFunc
- Lazy load the stackless functions with sync.Once, given this a simple
atomic read, it shouldn't affect performance for the fast-path (I
haven't seen benchmarks with compression enabled).
* substitute *os.File by fs.File
* refactor error handling by using the new recommended form
* finish implementation
* substitute seek(offset,0) by seek(offset, io.SeekStart)
* add unit test
* use io.SeekStart on Seek method
* client: simplify (*HostClient).do()
Remove an allocation in favour of deferring a call to release the
response.
* client: remove panic in dialAddr
Return an error instead of panicking if the user supplied a nonsensical
DialFunc.
* compression: remove panic on invalid compression level
If a compression level exceeding gzip's boundaries is provided, fasthttp
will panic. Instead it would be better to handle this error for them by
limiting it to the minimum or maximum value, depending on the direction
the user has exceeded the limits.
Clamp the value of gzip to always be between gzip.BestSpeed and
gzip.BestCompression.
* peripconn: remove panic on negative connection count
When a negative count is reached when unregistering a connection, a
panic is caused even though data-integrity is not at risk.
Replace the panic() with a simple clamp on the value to ensure the
value does not exceed it's expected lower bounds.
References: #1504
* compress: remove error on failed nonblocking writes
Since there is no way of handling or even logging non-critical errors in
stateless non-blocking writecalls, just drop them and hope the user
notices and tries again.
* workerPool: remove panic on redundant Start and Stop calls
Instead of panicking for invalid behaviour, it's preferable to just turn
the function into a noop.
* http: remove panic on invalid form boundary
* http: remove panic on negative reads
Since bufio already panics on negative reads, it is not necessary to do
so as well. If the length is zero and for some reason no error is
returned, readBodyIdentity and appendBodyFixedSize now errors in these
cases.
Link: https://github.com/golang/go/blob/851f6fd61425c810959c7ab51e6dc86f8a63c970/src/bufio/bufio.go#L246
* fs: remove panic on negative reader count
When a negative count is reached when unregistering a reader, a panic is
thrown even though data-integrity is not at risk.
Replace the panic() with a simple clamp on the value to ensure the
value does not exceed it's expected lower bounds.
* server: remove panic in favour of a segfault
Panicking with "BUG: " obscures the error. As the segfault causes a
panic anyway, just let the chaos unfold.
* server: remove panic in favour of returning an error
Writing on a timed-out response is not endangering data integrity and
just fails.
* chore: add comments to all panics
* chore: fix minor typo
* Update Go Version to Go1.19.x And add cache
* Fix CI Line endings
* Update test CI Go Version to Go1.19.x And add cache
* Update Gosec Security Scanner CI to securego/gosec@v2.12.0
* Format comment
Go 1.19 adds support for links, lists, and clearer headings in doc comments. As part of this change, gofmt now reformats doc comments to make their rendered meaning clearer. See “Go Doc Comments” for syntax details and descriptions of common mistakes now highlighted by gofmt. As another part of this change, the new package go/doc/comment provides parsing and reformatting of doc comments as well as support for rendering them to HTML, Markdown, and text.
ref: https://tip.golang.org/doc/go1.19
ref: https://tip.golang.org/doc/comment
* Fix doc structure
Previously each concurrent compression could allocate huge compression state
with the size up to 1Mb each. So 10K concurrent connections could result in
10Gb of compression state in the heap.
This CL limits the number of compression states among concurrent requests
when {Append,Write}{Gzip,Deflate}* functions are called to O(GOMAXPROCS).
These functions are used by CompressHandler* for non-streaming responses,
i.e. it should cover the majority of use cases.
Memory usage for 10K concurrent connections that compress responses drops
from 10Gb to 200Mb after this CL.
Oleksandr Redko
Gusted
Tiago Peczenyj
Moritz Poldrack
Oleksandr Redko
Aoang
Erik Dubbelboer
Mike Faraponov
Aliaksandr Valialkin
Sebastian Schepens