Documentation
¶
Overview ¶
h1client.go
Index ¶
- Constants
- Variables
- func SerializeRequest(req *http.Request) ([]byte, error)
- type ClientConfig
- type ConnectionPool
- type CredentialsProvider
- type CustomClient
- type H1Client
- func (c *H1Client) Close() error
- func (c *H1Client) Connect(ctx context.Context) error
- func (c *H1Client) Do(ctx context.Context, req *http.Request) (*http.Response, error)
- func (c *H1Client) IsIdle(timeout time.Duration) bool
- func (c *H1Client) ReadPipelinedResponses(ctx context.Context, expectedCount int) ([]*http.Response, error)
- func (c *H1Client) SendRaw(ctx context.Context, data []byte) error
- type H2Client
- func (c *H2Client) Close() error
- func (c *H2Client) Connect(ctx context.Context) error
- func (c *H2Client) Do(ctx context.Context, req *http.Request) (*http.Response, error)
- func (c *H2Client) IsIdle(timeout time.Duration) bool
- func (c *H2Client) PrepareRequest(ctx context.Context, req *http.Request) (*H2StreamHandle, error)
- func (c *H2Client) ReleaseBody(handle *H2StreamHandle) error
- func (c *H2Client) WaitResponse(ctx context.Context, handle *H2StreamHandle) (*http.Response, error)
- type H2Settings
- type H2StreamHandle
- type H2StreamResetError
- type H3Client
- type H3Settings
- type PaddingStrategy
- type RetryPolicy
Constants ¶
const ( // MaxResponseBodyBytes defines a reasonable limit for response bodies. MaxResponseBodyBytes = 32 * 1024 * 1024 // 32 MB // H2 specific constants (RFC 9113) DefaultH2InitialWindowSize = 65535 DefaultH2MaxFrameSize = 16384 // Define target receive window sizes for better throughput. TargetH2ConnWindowSize = 8 * 1024 * 1024 // 8 MB TargetH2StreamWindowSize = 4 * 1024 * 1024 // 4 MB )
const MaxReplayableBodyBytes = 2 * 1024 * 1024 // 2 MB
Define a reasonable limit for buffering request bodies in memory for replayability. This prevents excessive memory usage when handling large uploads that might need retries/redirects.
Variables ¶
var ErrCredentialsNotFound = errors.New("credentials not found")
Sentinel error returned when credentials are not available for a given host and realm.
Signals to the client that authentication cannot be handled and the original 401/407 response should be returned.
Functions ¶
func SerializeRequest ¶
SerializeRequest converts an `http.Request` object into its raw HTTP/1.1 wire format as a byte slice. It correctly handles the request line, headers, and body, ensuring the `Content-Length` is set and the `Connection: keep-alive` header is present for persistent connections.
This utility is essential for manual request sending, such as in HTTP pipelining.
Types ¶
type ClientConfig ¶
type ClientConfig struct {
// Low-level configuration for establishing TCP and TLS connections, including
// proxy settings.
DialerConfig *network.DialerConfig
// Specifies the cookie jar for the client. If nil, cookies are not managed.
CookieJar http.CookieJar
// Sets the timeout for a single HTTP request attempt.
RequestTimeout time.Duration
// Maximum duration a connection can remain idle in the pool before it is closed
// and evicted.
IdleConnTimeout time.Duration
// Controls whether to skip TLS certificate verification.
InsecureSkipVerify bool
/* Provides a function to define a custom redirect policy. If nil, the client's
default policy is used.*/
CheckRedirect func(req *http.Request, via []*http.Request) error
// Defines the rules for retrying failed requests.
RetryPolicy *RetryPolicy
// Interface for dynamically supplying credentials for HTTP authentication.
CredentialsProvider CredentialsProvider
// Settings specific to HTTP/2 connections.
H2Config H2Settings
// Settings specific to HTTP/3 connections.
H3Config H3Settings
/* Controls the injection of padding bytes into HTTP/2 frames. If nil, no
padding is applied. */
PaddingStrategy PaddingStrategy
}
The primary configuration struct for a CustomClient. Aggregates all configurable
aspects of the client, including dialing, cookies, timeouts, redirection, retries, authentication, and protocol-specific settings. *
func NewBrowserClientConfig ¶
func NewBrowserClientConfig() *ClientConfig
Creates a new configuration with defaults optimized to emulate the behavior of a modern web browser. Includes a pre-configured cookie jar, sensible timeouts, a default retry policy, and standard HTTP/2 and HTTP/3 settings. *
func (*ClientConfig) SetProxy ¶
func (c *ClientConfig) SetProxy(proxyURL *url.URL)
A convenience method to configure an HTTP/HTTPS proxy. Sets the `ProxyURL` field in the underlying `DialerConfig`. *
type ConnectionPool ¶
type ConnectionPool interface {
// Close immediately closes all connections in the pool.
Close() error
// IsIdle returns true if the pool has been idle for at least the specified duration.
IsIdle(timeout time.Duration) bool
}
ConnectionPool defines a minimal interface for a connection pool, used by the CustomClient's connection evictor to manage and close idle connections.
type CredentialsProvider ¶
type CredentialsProvider interface {
/** Called when the client receives a 401 (Unauthorized) or 407 (Proxy Authentication Required)
response.
Parameters:
- host: The host (e.g., "example.com:443") that issued the challenge.
- realm: The authentication realm specified in the WWW-Authenticate header.
Should return the username, password, and nil on success. If credentials
are not available, it must return ErrCredentialsNotFound. Other errors
will halt the request. */
GetCredentials(host string, realm string) (username string, password string, err error)
}
Defines an interface for dynamically supplying credentials in response to an
HTTP authentication challenge (e.g., Basic Auth). Allows the client to support authentication without hardcoding usernames and passwords. *
type CustomClient ¶
type CustomClient struct {
Config *ClientConfig
Logger *zap.Logger
// MaxRedirects specifies the maximum number of redirects to follow for a single request.
MaxRedirects int
// contains filtered or unexported fields
}
CustomClient is a sophisticated, low level HTTP client designed for fine grained control over HTTP/1.1, HTTP/2 and HTTP/3 connections. It is the core of the browser's networking stack, managing persistent connections on a per-host basis and handling the full request lifecycle, including cookies, redirects, retries, and authentication.
It maintains separate pools of `H1Client`, `H2Client`, and `H3Client` instances, allowing it to transparently handle different protocol versions. A background goroutine periodically evicts idle connections to conserve resources.
func NewCustomClient ¶
func NewCustomClient(config *ClientConfig, logger *zap.Logger) *CustomClient
NewCustomClient creates and initializes a new CustomClient with the given configuration. It also starts a background goroutine to periodically close idle connections from its pools.
func (*CustomClient) CloseAll ¶
func (c *CustomClient) CloseAll()
CloseAll shuts down the client, closing all active and idle connections.
func (*CustomClient) ConnectionCount ¶
func (c *CustomClient) ConnectionCount() int
ConnectionCount returns the total number of active connections (H1 + H2 + H3). This method is thread-safe and primarily intended for testing purposes.
func (*CustomClient) Do ¶
Do executes a single HTTP request and returns the response. This is the main entry point for the client. It orchestrates the entire request lifecycle, including:
1. Making the request body replayable for retries and redirects. 2. Adding cookies from the client's cookie jar. 3. Executing the request with a configurable retry policy for transient errors. 4. Storing cookies from the response. 5. Handling authentication challenges (e.g., HTTP Basic Auth). 6. Following HTTP redirects up to a configured limit.
Parameters:
- ctx: The context for the entire request operation, including all retries and redirects.
- req: The HTTP request to send.
Returns the final HTTP response or an error if the request could not be completed.
type H1Client ¶
type H1Client struct {
Conn net.Conn
Config *ClientConfig
TargetURL *url.URL
Address string
Logger *zap.Logger
// contains filtered or unexported fields
}
H1Client manages a single, persistent HTTP/1.1 connection to a specific host. It is designed to emulate the behavior of a browser's connection handling for HTTP/1.1, including support for Keep-Alive (reusing a single TCP/TLS connection for multiple sequential requests) and offering primitives for manual HTTP pipelining.
The client establishes its connection lazily (on the first request) and is thread-safe, ensuring that requests over the single connection are properly serialized.
func NewH1Client ¶
NewH1Client creates a new, un-connected H1Client for a given target URL and configuration. The actual network connection is established lazily when the first request is made.
func (*H1Client) Close ¶
Close terminates the client's network connection. This method is thread-safe.
func (*H1Client) Connect ¶
Connect establishes the underlying TCP and TLS connection to the target host if it is not already connected. It is called automatically by `Do` and other methods and is idempotent.
For HTTPS connections, it uses ALPN to explicitly negotiate HTTP/1.1, ensuring that servers that support both H2 and H1.1 will select the correct protocol for this client.
func (*H1Client) Do ¶
Do sends a single HTTP request over the persistent connection and waits for the response. It serializes access to the underlying connection, ensuring that multiple goroutines calling `Do` will have their requests sent sequentially, respecting HTTP/1.1 Keep-Alive semantics.
This method handles the entire request-response cycle, including serializing the request, writing it to the wire, parsing the response, and handling context cancellation during I/O operations.
func (*H1Client) IsIdle ¶
IsIdle determines if the connection has been idle for a duration longer than the specified timeout. This method is used by the `CustomClient`'s connection evictor to manage the connection pool and implements the `ConnectionPool` interface.
func (*H1Client) ReadPipelinedResponses ¶
func (c *H1Client) ReadPipelinedResponses(ctx context.Context, expectedCount int) ([]*http.Response, error)
ReadPipelinedResponses reads a specified number of HTTP responses from the connection's buffered reader. This method is the counterpart to `SendRaw` and is used to retrieve the results of a pipelined request sequence.
Parameters:
- ctx: The context for the read operation.
- expectedCount: The number of responses to parse from the stream.
Returns a slice of the parsed `http.Response` objects or an error if parsing fails or the connection is closed prematurely.
func (*H1Client) SendRaw ¶
SendRaw provides a low level mechanism to send arbitrary raw bytes over the connection. This is primarily intended for implementing manual HTTP/1.1 pipelining, where multiple serialized requests are sent without waiting for individual responses.
The caller is responsible for ensuring the data is a valid sequence of HTTP requests. Access to the connection is serialized. NOTE: SendRaw/ReadPipelinedResponses rely on the deadlines set based on the context/config and do not implement the active context cancellation monitoring used in Do().
type H2Client ¶
type H2Client struct {
Conn net.Conn
Config *ClientConfig
TargetURL *url.URL
Address string
Logger *zap.Logger
Framer *http2.Framer
HPEncoder *hpack.Encoder
HPDecoder *hpack.Decoder
// contains filtered or unexported fields
}
H2Client manages a single, persistent HTTP/2 connection to a specific host.
func NewH2Client ¶
NewH2Client creates a new, un-connected H2Client.
func (*H2Client) PrepareRequest ¶ added in v0.1.7
PrepareRequest sends the HEADERS frame immediately but strictly holds the DATA frame.
func (*H2Client) ReleaseBody ¶ added in v0.1.7
func (c *H2Client) ReleaseBody(handle *H2StreamHandle) error
ReleaseBody sends the DATA frame for a prepared stream.
func (*H2Client) WaitResponse ¶ added in v0.1.7
func (c *H2Client) WaitResponse(ctx context.Context, handle *H2StreamHandle) (*http.Response, error)
WaitResponse allows the caller to wait for the result of a specific stream handle.
type H2Settings ¶
type H2Settings struct {
// Time between sending PING frames to the server to check for connection liveness.
PingInterval time.Duration
// Maximum time to wait for a PING acknowledgment before considering the
// connection dead and closing it.
PingTimeout time.Duration
}
Defines configuration parameters specific to HTTP/2 connections.
func DefaultH2Settings ¶
func DefaultH2Settings() H2Settings
Returns a default configuration with a 30-second ping interval and a 5-second ping timeout.
type H2StreamHandle ¶ added in v0.1.7
type H2StreamHandle struct {
// contains filtered or unexported fields
}
H2StreamHandle is an opaque reference required to trigger the second phase of an SPA attack.
type H2StreamResetError ¶
H2StreamResetError is returned when an H2 stream is reset by the server.
func (H2StreamResetError) Error ¶
func (e H2StreamResetError) Error() string
type H3Client ¶ added in v0.1.7
type H3Client struct {
Config *ClientConfig
TargetURL *url.URL
Logger *zap.Logger
// contains filtered or unexported fields
}
H3Client manages a persistent HTTP/3 (QUIC) connection. Unlike TCP, QUIC connections are sessions managed by the RoundTripper.
func NewH3Client ¶ added in v0.1.7
func (*H3Client) Connect ¶ added in v0.1.7
Connect is a no-op for H3 because quic-go connects lazily on the first request.
type H3Settings ¶ added in v0.1.7
type H3Settings struct {
// Frequency of keep-alive packets sent to maintain the QUIC connection and
// prevent NAT timeouts.
KeepAlivePeriod time.Duration
// Maximum duration the connection can remain idle before being closed by the
// QUIC layer.
MaxIdleTimeout time.Duration
}
Defines configuration parameters specific to HTTP/3 (QUIC) connections.
func DefaultH3Settings ¶ added in v0.1.7
func DefaultH3Settings() H3Settings
Returns standard QUIC parameters optimized for robustness.
type PaddingStrategy ¶ added in v0.1.7
type PaddingStrategy interface {
// CalculatePadding returns the number of padding bytes to add for a given frame type and payload length.
// req: The specific request associated with this frame.
// frameType: http2.FrameHeaders or http2.FrameData.
// payloadLen: The size of the data/header block being sent.
CalculatePadding(req *http.Request, frameType http2.FrameType, payloadLen int) uint8
}
PaddingStrategy defines how to calculate padding for specific frames. It accepts the http.Request object, allowing the strategy to make decisions based on headers, context values, or other request-specific metadata.
type RetryPolicy ¶
type RetryPolicy struct {
// Maximum number of retry attempts after the initial request fails.
MaxRetries int
// Base duration to wait before the first retry.
InitialBackoff time.Duration
// Upper limit for the backoff duration, preventing excessively long waits.
MaxBackoff time.Duration
// Multiplier for the exponential backoff calculation (e.g., 2.0).
BackoffFactor float64
// If true, adds a random factor to the backoff duration to prevent multiple
// clients from retrying in synchronized waves (thundering herd problem).
Jitter bool
// Set of HTTP status codes that should trigger a retry for idempotent requests
// (e.g., GET, PUT, DELETE).
RetryableStatusCodes map[int]bool
}
Encapsulates the rules for retrying failed or transient HTTP requests. Supports
exponential backoff with jitter to prevent overwhelming a server that is temporarily unavailable. *
func NewDefaultRetryPolicy ¶
func NewDefaultRetryPolicy() *RetryPolicy
Creates and returns a policy with sensible defaults, such as 3 max retries, exponential backoff starting at 500ms, and retries for common transient server errors like 502, 503, and 504.
Source Files