Commit Graph

111 Commits

Author SHA1 Message Date
byte0o 9bc8e480c4 Request timeout settings for the same domain name are reused (#1558)
* Update client.go

fix client http SetReadDeadline/SetWriteDeadline Deadline is reused

* delete Deadline comments  fix test singleEchoConn implement SetWriteDeadline/SetReadDeadline

* fix test SetReadDeadline/SetWriteDeadline none implement

---------

Co-authored-by: gaoping <gaoping1@wps.cn>
2023-05-14 12:55:02 +02:00
Erik Dubbelboer fa72f3cc39 Fix tests (#1552) 2023-05-07 14:32:14 +02:00
Oleksandr Redko a281f1d21f client: HostClient.Do hangs out when ErrConnPoolStrategyNotImpl (#1515) 2023-03-14 22:36:38 +01:00
Oleksandr Redko 7846101dc6 client: fix Do hangs when configure host client fails (#1514) 2023-03-11 17:48:33 +01:00
Igor Menshenin 74a050705b Immediately return ErrTimeout if deadline is already reached. (#1497)
* fix: Immediately return ErrTimeout if deadline is already reached.

* test: Added tests for negative timeout

---------

Co-authored-by: Igor Menshenin <igor@native.rent>
2023-03-03 21:25:39 +01:00
Sergey Ponomarev 51048b0620 client.go Simplify default UA logic (#1466)
The getClientName() checks if !NoDefaultUserAgentHeader then returns the Client.Name field.
But it also saves it to atomic field clientName. This is not needed and logic can be simplified.
Previously the clientName vas a byte slice that was copied from c.Name and cached. See 02e0722fb7

Fix #1458
2023-01-02 12:51:39 +08:00
Sergey Ponomarev c50de95952 client.go fix addMissingPort() (#1444)
* client.go Make addMissingPort() public

It's needed for those who creates the instance of the HostClient manually.

* client.go fix AddMissingPort()

Previously for IPv6 addresses the default port wasn't added.
The fix adding a test and optimization that should avoid itoa() call and reduce a memory usage
2022-11-28 08:06:09 +01:00
brian-armstrong-discord f095481f00 Request.SetTimeout (#1415)
* Request.SetTimeout

This functionally works the same as e.g. Client.DoTimeout(), but we can
also use it for Client.DoRedirect(). There is no way as far as I can
tell to set a timeout on a DoRedirect call, so this makes it possible.

* tests

* docs
2022-11-03 21:25:19 +01:00
Erik Dubbelboer a5f448fc97 Improve Client timeout (#1346)
Don't run requests in a separate Goroutine anymore. Instead use proper
conn deadlines to enforce timeouts.

- Also contains some linting fixes.
2022-07-29 19:03:15 +02:00
fare83 c94be05897 use timeout insteadof read/writetimeout when timeout lower than read/… (#1336)
* use timeout insteadof read/writetimeout when timeout lower than read/writetimeout

* use deadtime; fix test timeout;

Co-authored-by: 徐焱 <xuyan4@staff.sina.com.cn>
2022-07-06 14:03:50 +02:00
Erik Dubbelboer 7a5afddf5b Use %v for errors and %q for strings (#1262)
Mostly in tests.
2022-04-01 18:11:16 +02:00
Erik Dubbelboer c6742630ef Fix race conditions in tests 2022-03-18 08:28:53 +01:00
ArminBTVS 1a5f2f40c6 Read response when client closes connection #1232 (#1233)
* Read response when client closes connection #1232

* Fix edge case were client responds with invalid header

* Follow linter suggestions for tests

* Changes after review

* Reafactor error check after review

* Handle connection reset on windows

* Remove format string from test where not needed

* Run connection reset tests not on Windows
2022-03-14 10:53:16 +01:00
Sergey Ponomarev 6a3cc2325b uri_test.go use example.com for clearness (#1212)
* uri_test.go replace xxx.com with example.com

* uri_test.go replace foobar.com with example.com

* uri_test.go use example.com inside of testURIUpdate()

* uri_test.go use example.com instead of google.com

* uri_test.go use example.com instead of google.com

* uri_test.go testURIUpdate() host with port
2022-02-09 10:19:34 +01:00
ArminBTVS 8d7953eda7 Fix scheme check for not yet parsed requests (#1203)
* Fix scheme check for not yet parsed requests

At this point the request might not be parsed yet and set. In that case uri is empty and isHttps() returns always false. I don't expect this is intended?

Otherwise URL() must be called before actually passing the request to client.Do()

* Add test

* Please linter
2022-01-31 22:02:58 +08:00
Erik Dubbelboer e9db537178 Use %w to wrap errors (#1175) 2021-12-13 09:41:34 +01:00
Erik Dubbelboer a50f59be52 Increase various test timeouts
Github workers on windows are slow.
2021-08-24 11:18:22 +02:00
Erik Dubbelboer 1504a84ed7 Increase TestHostClientMaxConnWaitTimeoutSuccess timeout 2021-06-29 09:36:30 +02:00
Erik Dubbelboer 874c8cad76 Increase timeouts for Windows github actions 2021-06-18 15:46:40 +02:00
Erik Dubbelboer 410bde6009 Fix race condition in TestPipelineClientIssue832 2021-06-18 15:28:58 +02:00
Erik Dubbelboer 9f2c63676d Lower go test time 2021-06-18 14:57:18 +02:00
Darío be13b504ab Defined Transport for the client (#1045)
* Defined Transport for the client

* Added Transport test
2021-06-16 12:55:56 +02:00
blanet 860c345f10 Fix unexpected panic when calling Do of a PipelineClient (#997)
* fix: Unexpected panic for PipelineClient

PipelineClient would panic when calling `Do` with a nil Response as
the second parm

This commit fixes the unexpected panic by checking nil first before
setting fields for Response

* Add tests to ensure nil resp is valid for PipelineClient
2021-03-17 21:26:56 +01:00
kiyon 02e0722fb7 Add PipelineClient name (#994)
* Improve documentation about DelClientCookie which related with #951.

* Add pipeline name
2021-03-15 17:46:16 +01:00
kiyon 34fa9a603f Add DisableHeaderNamesNormalizing to PipelineClient (#991)
* Improve documentation about DelClientCookie which related with #951.

* Add DisableHeaderNamesNormalizing to PipelineClient
2021-03-05 17:38:46 +01:00
Erik Dubbelboer fbe6a2d470 Add fasthttp.GenerateTestCertificate and use in tests
Remove ssl-cert-snakeoil so our tests don't fail in 2025.
2021-02-06 11:47:11 +01:00
Erik Dubbelboer df87e7089a Fix race condition in TestCloseIdleConnections
On really slow machines. when the test takes longer than 10 seconds,
mCleaner can already delete the connection from the map before we
check it again.
2020-11-06 12:00:12 +01:00
Erik Dubbelboer ae8b65fa62 Add Client.CloseIdleConnections() 2020-09-13 13:58:13 +02:00
Erik Dubbelboer 34a61fe63f Update linting (#851) 2020-07-17 14:22:28 +02:00
Erik Dubbelboer 9dd7979b2e Restart PipelineClient worker on error (#834) 2020-06-16 11:58:07 +02:00
Vural Kap 77dcda39ae Add custom retry decider for fail http requests 2020-05-27 20:40:04 +03:00
Erik Dubbelboer 123f6a8cee Fix memory reusage bug with auth
Fixes #814
2020-05-25 20:27:15 +02:00
Erik Dubbelboer d22782d3f7 HostClient can't switch between protocols (#800) 2020-05-18 18:30:29 +02:00
Erik Dubbelboer 05d4602a68 Fix race condition in test 2020-05-16 11:54:57 +02:00
Erik Dubbelboer dacd0353f9 HostClient can't switch between protocols 2020-05-02 21:01:52 +02:00
Erik Dubbelboer 079f39bddc Don't allow ASCII control character in URLs (#790)
* Don't allow ASCII control character in URLs

* Add tests
2020-04-25 20:54:59 +02:00
Daniel Qian 70b1d3bce7 feat: make client to wait when no free connections (#764)
* feat: make client to wait when no free connections

* feat: make client to wait when no free connections

use AcquireTimer to do timeout instead of using context

* feat: make client to wait when no free connections

Add BenchmarkClientGetEndToEndWaitConn* to test heap allocation
in waiting for free connection situation

* feat: make client to wait when no free connections

Add BenchmarkHTTPClientGetEndToEndWaitConn* to test heap allocation
in waiting for free connection situation

* feat: make client to wait when no free connections

fix bug in BenchmarkHTTPClientGetEndToEndWaitConn*

* feat: make client to wait when no free connections

fix bug in TestHostClientMaxConnWaitTimeoutSuccess make it wait
longer to avoid ErrNoFreeConns on travis-ci

* feat: make client to wait when no free connections

fix do not compile benchmark(NetHTTP?)ClientGetEndToEndWaitConn
if go version < 1.11.x

* feat: make client to wait when no free connections

fix the bug that if deadline is earlier than MaxConnWaitTimeout,
still wait MaxConnWaitTimeout which later than deadline.

* feat: make client to wait when no free connections

fix race condition in TestHostClientMaxConnWaitTimeoutError

* feat: make client to wait when no free connections

fix bug in TestHostClientMaxConnWaitTimeoutWithEarlierDeadline
2020-04-03 17:12:50 +02:00
Erik Dubbelboer 0ba26f7b42 Don't retry requests with a BodyStream
We won't be able to re-read the body stream. So instead return an error.

Fixes #766
2020-03-30 22:33:31 +02:00
Tomás Senart 75c6008129 client: Implement DoRedirects (#765)
This commit adds a `DoRedirects` method to both `HostClient` and
`Client` as well as top level convenience function of the same name that
is called with the package level `defaultClient`.

Re-implementing this redirect logic in user code is harder than
necessary.
2020-03-25 17:55:13 +01:00
Erik Dubbelboer 69d5c3721a Client should return ErrTimeout on timeout (#736)
Not ErrConnectionClosed which is incorrect.

Fixes: https://github.com/valyala/fasthttp/issues/355
2020-01-31 22:21:00 +01:00
Andy Pan 59b28fe0e5 Resolve code issues from goreportcard.com (#725) 2020-01-12 12:57:42 +01:00
Erik Dubbelboer 39dd1045bb Allow a body for GET requests (#703)
This means we can't skip parsing headers for GET requests anymore. This
can be seen as good as it also allows us to reject malformed GET
requests, something we didn't do before this. Performance also isn't
affect much:

benchmark                                            old ns/op     new ns/op     delta
BenchmarkClientGetEndToEnd1Inmemory-16               640           641           +0.16%
BenchmarkClientGetEndToEnd10Inmemory-16              713           710           -0.42%
BenchmarkClientGetEndToEnd100Inmemory-16             732           749           +2.32%
BenchmarkClientGetEndToEnd1000Inmemory-16            759           774           +1.98%
BenchmarkClientGetEndToEnd10KInmemory-16             785           808           +2.93%
BenchmarkNetHTTPClientGetEndToEnd1Inmemory-16        5045          4954          -1.80%
BenchmarkNetHTTPClientGetEndToEnd10Inmemory-16       5806          6225          +7.22%
BenchmarkNetHTTPClientGetEndToEnd100Inmemory-16      7877          7998          +1.54%
BenchmarkNetHTTPClientGetEndToEnd1000Inmemory-16     16603         16559         -0.27%
2019-12-01 09:44:11 +01:00
Erik Dubbelboer 32793db72d Run golangci-lint using a Github Action 2019-11-16 18:09:28 +01:00
Erik Dubbelboer ec2e3965a4 Don't run TestClientNilResp in parallel 2019-10-19 22:34:56 +02:00
Erik Dubbelboer d428e1b05e Fix race condition in tests, fix URI bug
- Some tests can't be run in parallel.
- `URI` had a pointer to `RequestHeader` which was updated with
`RequestHeader.CopyTo` which resulted in the URI pointing to the wrong
`RequestHeader` causing bugs and race conditions.

The only reason `URI` contained a pointer to `RequestHeader` was to delay the
call to `RequestHeader.Host()` until really needed. But these days instead
of parsing all headers, `RequestHeader.Host()` uses
`RequestHeader.peekRawHeader()` which is rather fast. So we can remove the
pointer in `URI` and completely decouple the two structs improving code
quality and fixing the bug.

For some reason this results in faster code on average as well:
benchmark                                          old ns/op     new ns/op     delta
BenchmarkClientGetEndToEnd1Inmemory-8              1189          1369          +15.14%
BenchmarkClientGetEndToEnd10Inmemory-8             1143          1161          +1.57%
BenchmarkClientGetEndToEnd100Inmemory-8            1228          1236          +0.65%
BenchmarkClientGetEndToEnd1000Inmemory-8           1213          1213          +0.00%
BenchmarkClientGetEndToEnd10KInmemory-8            1362          1350          -0.88%
BenchmarkClientEndToEndBigResponse1Inmemory-8      139967        130070        -7.07%
BenchmarkClientEndToEndBigResponse10Inmemory-8     142233        131809        -7.33%
BenchmarkServerGet1ReqPerConn-8                    1726          1593          -7.71%
BenchmarkServerGet2ReqPerConn-8                    882           927           +5.10%
BenchmarkServerGet10ReqPerConn-8                   440           436           -0.91%
BenchmarkServerGet10KReqPerConn-8                  341           339           -0.59%
BenchmarkServerPost1ReqPerConn-8                   1728          1706          -1.27%
BenchmarkServerPost2ReqPerConn-8                   968           963           -0.52%
BenchmarkServerPost10ReqPerConn-8                  506           505           -0.20%
BenchmarkServerPost10KReqPerConn-8                 424           420           -0.94%
BenchmarkServerGet1ReqPerConn10KClients-8          1117          1051          -5.91%
BenchmarkServerGet2ReqPerConn10KClients-8          565           514           -9.03%
BenchmarkServerGet10ReqPerConn10KClients-8         390           387           -0.77%
BenchmarkServerGet100ReqPerConn10KClients-8        355           348           -1.97%
BenchmarkServerHijack-8                            339           348           +2.65%
BenchmarkServerMaxConnsPerIP-8                     326           325           -0.31%
BenchmarkServerTimeoutError-8                      24355         24180         -0.72%
2019-10-19 21:02:31 +02:00
Erik Dubbelboer c3d82ca3a4 Speed up testing by running tests in parallel 2019-10-16 10:20:13 +02:00
Erik Dubbelboer 9dbe5fc77c Don't allow spaces in request header keys
See: https://github.com/golang/go/commit/6e6f4aaf70c8b1cc81e65a26332aa9409de03ad8

Reject any non GET or HEAD requests with a 400.

We can't reject GET or HEAD requests with bad headers as we delay
parsing of these headers until the user asks for one. So in this case we
just ignore the header and don't return a value for it.
2019-10-16 10:20:13 +02:00
Ian Leue ffcdb753b1 Allow client to disable path normalization (#672) 2019-10-11 14:20:29 +08:00
Erik Dubbelboer 1d6a7e0493 The Authorization header should include the Basic keyword
Thanks to https://github.com/paween1980
2019-09-27 17:29:44 +02:00
Mike Faraponov ce02b85a9c Add ability to set timeout for handshake (#631)
* Fixed issue with handshake timeout
2019-08-18 14:34:47 +02:00