Commit Graph

230 Commits

Author SHA1 Message Date
Erik Dubbelboer e6ce5f0b11 Remove invalid documentation
The Timeout functions don't use separate goroutines anymore and use
normaly deadlines. Because of that requests are terminated and don't
keep running in the background as before.
2023-05-22 16:27:56 +02:00
byte0o 9bc8e480c4 Request timeout settings for the same domain name are reused (#1558)
* Update client.go

fix client http SetReadDeadline/SetWriteDeadline Deadline is reused

* delete Deadline comments  fix test singleEchoConn implement SetWriteDeadline/SetReadDeadline

* fix test SetReadDeadline/SetWriteDeadline none implement

---------

Co-authored-by: gaoping <gaoping1@wps.cn>
2023-05-14 12:55:02 +02:00
Zhengkai Wang d2f97fc426 fix the problem: HostClient maybe delete when pendingClientRequests > 0 (#1562)
* add the functions to get host clients

* add concurrency for client's HostClient map

* delete test code

* add lock in once block

* fix the problem: HostClient maybe delete when pendingClientRequests > 0

* fix the problem: HostClient maybe delete when pendingClientRequests > 0

---------

Co-authored-by: wangzhengkai.wzk <wangzhengkai.wzk@alibaba-inc.com>
2023-05-12 09:49:37 +02:00
Zhengkai Wang 7ea3b6330e add concurrency for client's HostClient map (#1550)
* add the functions to get host clients

* add concurrency for client's HostClient map

* delete test code

* add lock in once block

---------

Co-authored-by: wangzhengkai.wzk <wangzhengkai.wzk@alibaba-inc.com>
2023-05-11 10:07:58 +02:00
kukayiyi 87cb886157 client:Fix DoTimeout timeout failure by setting temporary dial (#1535) 2023-04-15 13:39:32 +02:00
Anthony-Dong 6b958c2c22 support response body stream (#1414)
* feat: support response body stream

* style: add header interface

* Update http.go

Co-authored-by: Erik Dubbelboer <erik@dubbelboer.com>

* feat: support request、response、client stream

* fix: close reader bug

---------

Co-authored-by: fanhaodong.516 <fanhaodong.516@bytedance.com>
Co-authored-by: Erik Dubbelboer <erik@dubbelboer.com>
2023-04-05 18:56:31 +02:00
LeoSun 239cce45b1 format : update some codes style (#1533)
* update some codes style

* update adaptor.go and client.go

* Update fasthttpproxy/http.go

Co-authored-by: Erik Dubbelboer <erik@dubbelboer.com>

---------

Co-authored-by: Erik Dubbelboer <erik@dubbelboer.com>
2023-04-04 13:48:48 -07:00
Moritz Poldrack d0f2727a4d get rid of some panics (#1526)
* client: simplify (*HostClient).do()

Remove an allocation in favour of deferring a call to release the
response.

* client: remove panic in dialAddr

Return an error instead of panicking if the user supplied a nonsensical
DialFunc.

* compression: remove panic on invalid compression level

If a compression level exceeding gzip's boundaries is provided, fasthttp
will panic. Instead it would be better to handle this error for them by
limiting it to the minimum or maximum value, depending on the direction
the user has exceeded the limits.

Clamp the value of gzip to always be between gzip.BestSpeed and
gzip.BestCompression.

* peripconn: remove panic on negative connection count

When a negative count is reached when unregistering a connection, a
panic is caused even though data-integrity is not at risk.

Replace the panic() with a simple clamp on the value to ensure the
value does not exceed it's expected lower bounds.

References: #1504

* compress: remove error on failed nonblocking writes

Since there is no way of handling or even logging non-critical errors in
stateless non-blocking writecalls, just drop them and hope the user
notices and tries again.

* workerPool: remove panic on redundant Start and Stop calls

Instead of panicking for invalid behaviour, it's preferable to just turn
the function into a noop.

* http: remove panic on invalid form boundary

* http: remove panic on negative reads

Since bufio already panics on negative reads, it is not necessary to do
so as well. If the length is zero and for some reason no error is
returned, readBodyIdentity and appendBodyFixedSize now errors in these
cases.

Link: https://github.com/golang/go/blob/851f6fd61425c810959c7ab51e6dc86f8a63c970/src/bufio/bufio.go#L246

* fs: remove panic on negative reader count

When a negative count is reached when unregistering a reader, a panic is
thrown even though data-integrity is not at risk.

Replace the panic() with a simple clamp on the value to ensure the
value does not exceed it's expected lower bounds.

* server: remove panic in favour of a segfault

Panicking with "BUG: " obscures the error. As the segfault causes a
panic anyway, just let the chaos unfold.

* server: remove panic in favour of returning an error

Writing on a timed-out response is not endangering data integrity and
just fails.

* chore: add comments to all panics

* chore: fix minor typo
2023-03-30 03:38:28 +02:00
Oleksandr Redko a281f1d21f client: HostClient.Do hangs out when ErrConnPoolStrategyNotImpl (#1515) 2023-03-14 22:36:38 +01:00
Oleksandr Redko 7846101dc6 client: fix Do hangs when configure host client fails (#1514) 2023-03-11 17:48:33 +01:00
Oleksandr Redko a9d57a9461 client: remove unused go:build comment (#1511) 2023-03-06 17:08:11 +01:00
Igor Menshenin 74a050705b Immediately return ErrTimeout if deadline is already reached. (#1497)
* fix: Immediately return ErrTimeout if deadline is already reached.

* test: Added tests for negative timeout

---------

Co-authored-by: Igor Menshenin <igor@native.rent>
2023-03-03 21:25:39 +01:00
cui fliter ac631aa4eb fix functiom name (#1505)
Signed-off-by: cui fliter <imcusg@gmail.com>
2023-03-03 17:43:44 +01:00
Oleksandr Redko e15a810a98 docs: fix grammar issues and typos in comments (#1492) 2023-02-13 10:43:44 +08:00
Oleksandr Redko 934f04e330 Refactor golangci-lint config and remove redundant nolints (#1486)
* Refactor golangci-lint config

- Use golangci-lint-action for GitHub workflow.
- Add additional golangci-lint run options.
- Remove unused nolint directives.

* Revert exclude-use-default option
2023-02-11 15:35:15 +08:00
Oleksandr Redko f84e2346ba Rename unexported funcs, vars to match common Go (#1488)
See https://github.com/golang/go/wiki/CodeReviewComments#initialisms
and https://go.dev/doc/effective_go#mixed-caps
2023-02-10 21:34:49 +08:00
Oleksandr Redko c0c5f9cd38 doc,test: correct typos (#1484) 2023-02-09 09:29:28 +01:00
Sergey Ponomarev 51048b0620 client.go Simplify default UA logic (#1466)
The getClientName() checks if !NoDefaultUserAgentHeader then returns the Client.Name field.
But it also saves it to atomic field clientName. This is not needed and logic can be simplified.
Previously the clientName vas a byte slice that was copied from c.Name and cached. See 02e0722fb7

Fix #1458
2023-01-02 12:51:39 +08:00
Xuran e46afc3ae3 fix: client may remove hostclient incorrectly (#1461) 2022-12-29 19:18:27 +08:00
Sergey Ponomarev c50de95952 client.go fix addMissingPort() (#1444)
* client.go Make addMissingPort() public

It's needed for those who creates the instance of the HostClient manually.

* client.go fix AddMissingPort()

Previously for IPv6 addresses the default port wasn't added.
The fix adding a test and optimization that should avoid itoa() call and reduce a memory usage
2022-11-28 08:06:09 +01:00
Andy Pan 8f434434e7 Wait for the response of pipelineWork in background and return it to pool (#1436) 2022-11-17 06:31:03 +01:00
Andy Pan c367454ffe Fix some potential pool leaks (#1433) 2022-11-16 06:46:28 +02:00
Andy Pan b32a3dda13 Use time.Until(deadline) instead of -time.Since(deadline) (#1434) 2022-11-16 06:13:29 +02:00
Erik Dubbelboer 2f1e949d91 Improve isTLSAlready check
All conn implementations having a Handshake() error method are now seen
as a TLS conn and aren't wrapped in another tls.Client.
2022-09-03 11:02:32 +02:00
Erik Dubbelboer 79ccfff72b Don't use tls ClientSessionCache
net/http doesn't use it either. Some servers have issues with this
preventing fasthttp from working:

https://github.com/valyala/fasthttp/issues/1364
https://github.com/valyala/fasthttp/issues/1296
https://github.com/valyala/fasthttp/issues/1335
https://github.com/valyala/fasthttp/issues/984

Also removed code that benchmarks crypto/tls as that has nothing to do
with fasthttp.
2022-08-28 10:13:03 +02:00
Aoang ea6052464e Add Go 1.19 Support (#1355)
* Update Go Version to Go1.19.x And add cache

* Fix CI Line endings

* Update test CI Go Version to Go1.19.x And add cache

* Update Gosec Security Scanner CI to securego/gosec@v2.12.0

* Format comment

Go 1.19 adds support for links, lists, and clearer headings in doc comments. As part of this change, gofmt now reformats doc comments to make their rendered meaning clearer. See “Go Doc Comments” for syntax details and descriptions of common mistakes now highlighted by gofmt. As another part of this change, the new package go/doc/comment provides parsing and reformatting of doc comments as well as support for rendering them to HTML, Markdown, and text.

ref: https://tip.golang.org/doc/go1.19
ref: https://tip.golang.org/doc/comment

* Fix doc structure
2022-08-14 11:31:57 +02:00
Erik Dubbelboer a5f448fc97 Improve Client timeout (#1346)
Don't run requests in a separate Goroutine anymore. Instead use proper
conn deadlines to enforce timeouts.

- Also contains some linting fixes.
2022-07-29 19:03:15 +02:00
fare83 c94be05897 use timeout insteadof read/writetimeout when timeout lower than read/… (#1336)
* use timeout insteadof read/writetimeout when timeout lower than read/writetimeout

* use deadtime; fix test timeout;

Co-authored-by: 徐焱 <xuyan4@staff.sina.com.cn>
2022-07-06 14:03:50 +02:00
Thearas 8a320890c0 Add ConnPoolStrategy field to client (#1317) 2022-06-07 09:21:26 +02:00
Mikhail Faraponov b40b5a4ca3 Update tlsClientHandshake (#1263)
* Update tlsClientHandshake

* Update client.go

* Update client.go

* Update client.go

* Update client.go

* Changes according to the review
2022-04-10 01:21:37 +02:00
Erik Dubbelboer 7a5afddf5b Use %v for errors and %q for strings (#1262)
Mostly in tests.
2022-04-01 18:11:16 +02:00
Aoang f3bce3aa6e Add Go 1.18 support (#1253)
* Add Go 1.18 support

* fix Gosec Security Scanner

https://github.com/valyala/fasthttp/runs/5595618634?check_suite_focus=true

* fix https://github.com/securego/gosec/issues/469#issuecomment-1070608395 Gosec Github Action Doesn't Work at Go 1.18

* fix https://github.com/golangci/golangci-lint/pull/2438 golangci/golangci-lint Doesn't Work at Go 1.18

* fix golint unused

* fix golint: SA1019: netErr.Temporary is deprecated

* fix https://github.com/valyala/fasthttp/issues/1256
2022-03-20 14:34:31 +01:00
tyltr 2044e1e998 reduce unnessary type assart (#1254) 2022-03-18 08:19:34 +01:00
Y.Horie 8f5e51f0e9 Add connection pool queuing strategies in HostClient. (#1238)
* At first, we implement LIFO(conventional) and FIFO which indicate in #1236
* Change default strategy FIFO
2022-03-15 09:40:26 +01:00
ArminBTVS 1a5f2f40c6 Read response when client closes connection #1232 (#1233)
* Read response when client closes connection #1232

* Fix edge case were client responds with invalid header

* Follow linter suggestions for tests

* Changes after review

* Reafactor error check after review

* Handle connection reset on windows

* Remove format string from test where not needed

* Run connection reset tests not on Windows
2022-03-14 10:53:16 +01:00
lin longhjui c94581c005 support configure HostClient (#1214)
Co-authored-by: linlonghui <linlonghui@chandashi.com>
2022-02-15 08:03:54 +01:00
ArminBTVS 8d7953eda7 Fix scheme check for not yet parsed requests (#1203)
* Fix scheme check for not yet parsed requests

At this point the request might not be parsed yet and set. In that case uri is empty and isHttps() returns always false. I don't expect this is intended?

Otherwise URL() must be called before actually passing the request to client.Do()

* Add test

* Please linter
2022-01-31 22:02:58 +08:00
Erik Dubbelboer f7c354c760 Fix race condition in Client.mCleaner
This could result in HostClients being removed that were still in use.
2021-11-11 16:59:56 +01:00
Sergey Ponomarev 3ff6aaa591 uri: isHttps() and isHttp() (#1150)
* uri: isHttps() and isHttp()

Use them instead of manual schema comparison

* uri: use SetSchemeBytes()
2021-11-08 13:35:26 +01:00
Erik Dubbelboer c15e642a16 Don't run all race tests on windows (#1143)
* Don't run all race tests on windows

It's too slow and gives a lot of false positives in our tests.

* No FS tests on windows
2021-10-31 22:09:40 +01:00
tyltr f0a21893b9 feat: improve IsMethod (#1088)
* feat: improve bytesEqual

* benchmark

* nolint:unused

* remove unused  code

* Update client.go

Co-authored-by: Erik Dubbelboer <erik@dubbelboer.com>

Co-authored-by: Erik Dubbelboer <erik@dubbelboer.com>
2021-09-06 12:06:21 +02:00
Erik Dubbelboer a6f9c8a442 Fix Client doc and mCleaner (#1076)
- Reduce mCleaner goroutine time.
- Add comment about changing Client fields while in use.
2021-08-20 14:52:06 +02:00
Darío be13b504ab Defined Transport for the client (#1045)
* Defined Transport for the client

* Added Transport test
2021-06-16 12:55:56 +02:00
Daniel Firsht 2a6f7db5bb Adding support for securing error logs (#1001)
Co-authored-by: Daniel Firsht <firsht@amazon.com>
2021-03-26 10:46:57 +01:00
Mikhail Faraponov e61c9d5d3c Improve Client/HostClient pooling when many HostClient structs created and removed during program execution (#1000)
* Update client.go

* Update client.go
2021-03-25 22:47:39 +01:00
blanet 860c345f10 Fix unexpected panic when calling Do of a PipelineClient (#997)
* fix: Unexpected panic for PipelineClient

PipelineClient would panic when calling `Do` with a nil Response as
the second parm

This commit fixes the unexpected panic by checking nil first before
setting fields for Response

* Add tests to ensure nil resp is valid for PipelineClient
2021-03-17 21:26:56 +01:00
kiyon 02e0722fb7 Add PipelineClient name (#994)
* Improve documentation about DelClientCookie which related with #951.

* Add pipeline name
2021-03-15 17:46:16 +01:00
kiyon 34fa9a603f Add DisableHeaderNamesNormalizing to PipelineClient (#991)
* Improve documentation about DelClientCookie which related with #951.

* Add DisableHeaderNamesNormalizing to PipelineClient
2021-03-05 17:38:46 +01:00
kiyon c3cd5e1554 Export HostClient.connsCount (#981)
Export HostClient.connsCount
2021-02-25 13:57:48 +01:00
Seva Maltsev a4b0703152 Implemented DisablePathNormalizing in PipelineClient (#977) 2021-02-24 20:48:34 +01:00