proxy

package
v0.0.15 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 20, 2026 License: AGPL-3.0 Imports: 43 Imported by: 0

Documentation

Overview

Package proxy implements the HTTP proxy server for cc-relay.

Package proxy implements the HTTP proxy server for cc-relay.

Package proxy implements the HTTP proxy server for cc-relay.

Package proxy implements the HTTP reverse proxy for Claude Code.

Package proxy implements the HTTP reverse proxy for Claude Code.

Package proxy implements the HTTP reverse proxy for Claude Code.

Package proxy implements the HTTP proxy server for cc-relay.

Package proxy implements the HTTP proxy server for cc-relay.

Package proxy implements the HTTP proxy server for cc-relay.

Package proxy implements the HTTP proxy server for cc-relay.

Package proxy implements the HTTP proxy server for cc-relay.

Package proxy implements the HTTP proxy server for cc-relay.

Package proxy implements the HTTP proxy server for cc-relay.

Package proxy provides HTTP proxy functionality for cc-relay. This file provides SSE (Server-Sent Events) streaming utilities using samber/ro.

SSE streaming utilities provide reactive stream processing for SSE events. They are designed to work alongside the existing handler.go implementation, providing an alternative approach for future refactoring.

Current handler.go uses direct streaming (which is performant). These utilities can be used when reactive stream processing is beneficial:

  • Transforming SSE events during streaming
  • Filtering or aggregating events
  • Composing multiple event streams
  • Testing SSE processing in isolation

When to use SSE stream utilities:

  • Building custom SSE processing pipelines
  • Unit testing SSE transformations
  • Implementing SSE middleware

When to use direct streaming (current approach):

  • Simple passthrough proxying (handler.go)
  • Maximum performance requirements
  • Minimal transformation needed

Package proxy implements the HTTP proxy server for cc-relay.

Package proxy implements the HTTP proxy server for cc-relay.

Index

Constants

View Source 
const (
	HeaderRelayKeyID     = "X-CC-Relay-Key-ID"         // Selected key ID (first 8 chars)
	HeaderRelayCapacity  = "X-CC-Relay-Capacity"       // Remaining capacity %
	HeaderRelayKeysTotal = "X-CC-Relay-Keys-Total"     // Total keys in pool
	HeaderRelayKeysAvail = "X-CC-Relay-Keys-Available" // Available keys
)

Custom header constants for relay metadata.

View Source 
const (
	// SignatureCacheTTL is the TTL for cached signatures (3 hours, matching CLIProxyAPI).
	SignatureCacheTTL = 3 * time.Hour

	// SignatureHashLen is the number of hex characters to use from SHA256 hash.
	SignatureHashLen = 16

	// MinSignatureLen is the minimum length for a valid signature.
	MinSignatureLen = 50

	// GeminiSignatureSentinel is a special sentinel value for Gemini models.
	GeminiSignatureSentinel = "skip_thought_signature_validator"
)
View Source 
const RequestIDKey ctxKey = "request_id"

RequestIDKey is the context key for request IDs.

Variables

View Source 
var ErrNotFlushable = errors.New("sse: ResponseWriter does not implement http.Flusher")

ErrNotFlushable is returned when the ResponseWriter doesn't support flushing.

View Source 
var ErrStreamClosed = errors.New("sse: stream is closed")

ErrStreamClosed is returned when attempting to write to a closed stream.

Functions

func AddRequestID 

func AddRequestID(ctx context.Context, requestID string) context.Context

AddRequestID adds or extracts request ID from request headers and adds it to the context. If X-Request-ID header exists, use it. Otherwise, generate a new UUID.

func AttachTLSTrace 

func AttachTLSTrace(ctx context.Context, _ *http.Request) (newCtx context.Context, getMetrics func() TLSMetrics)

AttachTLSTrace attaches httptrace to request for TLS metric collection. Returns updated context with trace and a function to retrieve metrics.

func AuthMiddleware 

func AuthMiddleware(expectedAPIKey string) func(http.Handler) http.Handler

AuthMiddleware creates middleware that validates x-api-key header. Uses constant-time comparison to prevent timing attacks.

Security note: SHA-256 is appropriate for API key hashing because: - API keys are high-entropy secrets (32+ random characters), not passwords - SHA-256 provides sufficient pre-image resistance for high-entropy inputs - Pre-hashing at middleware creation prevents per-request hash computation - Constant-time comparison (subtle.ConstantTimeCompare) prevents timing attacks.

func CacheModelInContext added in v0.0.8 

func CacheModelInContext(ctx context.Context, model string) context.Context

CacheModelInContext stores the extracted model name in the request context. This avoids re-reading the body when the model is needed again (e.g., for rewriting).

func CacheThinkingAffinityInContext added in v0.0.8 

func CacheThinkingAffinityInContext(ctx context.Context, hasThinking bool) context.Context

CacheThinkingAffinityInContext stores the thinking affinity detection result in context.

func ConcurrencyMiddleware added in v0.0.12 

func ConcurrencyMiddleware(limiter *ConcurrencyLimiter) func(http.Handler) http.Handler

ConcurrencyMiddleware creates middleware that enforces a global concurrency limit. Uses the provided ConcurrencyLimiter which supports hot-reload via SetLimit.

func CountEvents added in v0.0.5 

func CountEvents() func(ro.Observable[SSEEvent]) ro.Observable[int64]

CountEvents creates an operator that counts events and emits the running total. Useful for monitoring stream progress.

func ExtractModelFromRequest added in v0.0.8 

func ExtractModelFromRequest(r *http.Request) mo.Option[string]

ExtractModelFromRequest reads the model field from the request body. Returns mo.None if body is missing, malformed, or has no model field. Returns mo.Some with the model name if extraction succeeds. The request body is restored for subsequent reads.

If the body exceeds max_body_bytes limit (set via http.MaxBytesReader), returns mo.None. Use ExtractModelWithBodyCheck for explicit error detection.

func ExtractModelWithBodyCheck added in v0.0.12 

func ExtractModelWithBodyCheck(request *http.Request) (model mo.Option[string], bodyTooLarge bool)

ExtractModelWithBodyCheck reads the model field and reports body size errors. Returns:

  • mo.Some[string] if model extraction succeeds
  • mo.None[string] if body is missing, malformed, or has no model field
  • bodyTooLarge=true if reading failed due to http.MaxBytesReader limit

The request body is always restored for downstream use.

func FilterEvents added in v0.0.5 

func FilterEvents(eventType string) func(ro.Observable[SSEEvent]) ro.Observable[SSEEvent]

FilterEvents creates an operator that filters SSE events by type. Events with a matching Event field are passed through, others are dropped.

Example: 

// Only keep message_delta events
filtered := ro.Pipe1(events, FilterEvents("message_delta"))

func FilterEventsByPrefix added in v0.0.5 

func FilterEventsByPrefix(prefix string) func(ro.Observable[SSEEvent]) ro.Observable[SSEEvent]

FilterEventsByPrefix creates an operator that filters SSE events by type prefix.

Example: 

// Keep all content_block_* events
filtered := ro.Pipe1(events, FilterEventsByPrefix("content_block_"))

func FilterProvidersByModel added in v0.0.8 

func FilterProvidersByModel(
	model string,
	providers []router.ProviderInfo,
	modelMapping map[string]string,
	defaultProviderName string,
) []router.ProviderInfo

FilterProvidersByModel returns providers that can serve the given model. Uses prefix matching on model names (e.g., "claude-opus" matches "claude-opus-4").

Parameters:

  • model: The model name from the request (e.g., "claude-opus-4")
  • providers: All available providers
  • modelMapping: Map of model prefix to provider name (e.g., "claude-opus" -> "anthropic")
  • defaultProviderName: Fallback provider if no mapping matches

Returns:

  • Filtered providers that match the model, or all providers if no filtering applies

Behavior:

  • If model is empty or modelMapping is empty, returns all providers
  • Uses longest prefix match for specificity
  • Falls back to defaultProviderName if no prefix matches
  • Returns all providers if neither match nor default found

func FormatSignature added in v0.0.8 

func FormatSignature(modelName, signature string) string

FormatSignature adds modelGroup prefix: "claude#abc123...".

func ForwardSSE added in v0.0.5 

func ForwardSSE(events ro.Observable[SSEEvent], writer http.ResponseWriter) error

ForwardSSE pipes SSE events from an Observable to an http.ResponseWriter. Sets appropriate SSE headers and flushes after each event. Blocks until the observable completes or errors.

Returns ErrNotFlushable if the ResponseWriter doesn't support flushing. Returns any error that occurs during streaming.

Example: 

events := StreamSSE(upstreamResp.Body)
err := ForwardSSE(events, writer)
if err != nil {
    // Handle error
}

func GetModelFromContext added in v0.0.8 

func GetModelFromContext(ctx context.Context) (string, bool)

GetModelFromContext retrieves the cached model name from the context. Returns the model and true if found, empty string and false otherwise.

func GetModelGroup added in v0.0.8 

func GetModelGroup(modelName string) string

GetModelGroup returns the model group for signature sharing. Models in the same group share signatures (e.g., claude-sonnet-4, claude-3-opus → "claude").

func GetModelNameFromContext added in v0.0.8 

func GetModelNameFromContext(ctx context.Context) string

GetModelNameFromContext retrieves the model name from context.

func GetRequestID 

func GetRequestID(ctx context.Context) string

GetRequestID retrieves the request ID from context.

func GetThinkingAffinityFromContext added in v0.0.8 

func GetThinkingAffinityFromContext(ctx context.Context) bool

GetThinkingAffinityFromContext retrieves the cached thinking affinity result from context. Returns false if not cached.

func HasThinkingBlocks added in v0.0.8 

func HasThinkingBlocks(body []byte) bool

HasThinkingBlocks performs fast detection without JSON parsing. Returns true if the body likely contains thinking blocks with signatures. Uses bytes.Contains which is 10-100x faster than JSON parsing.

func HasThinkingSignature added in v0.0.8 

func HasThinkingSignature(request *http.Request) bool

HasThinkingSignature checks if the request body contains thinking signatures in assistant messages. This indicates a conversation with extended thinking enabled, which requires sticky provider routing to avoid signature validation errors.

When extended thinking is enabled, providers return a thinking content block with a provider-specific signature. On subsequent turns, this signature must be validated by the same provider. If requests are routed to a different provider (e.g., via round-robin), the signature validation fails.

The request body is restored for subsequent reads.

func IsBodyTooLargeError added in v0.0.12 

func IsBodyTooLargeError(err error) bool

IsBodyTooLargeError checks if an error is from http.MaxBytesReader.

func IsStreamingRequest 

func IsStreamingRequest(body []byte) bool

IsStreamingRequest checks if request body contains "stream": true. Returns false if the body is invalid JSON or stream field is missing/false.

func IsValidSignature added in v0.0.8 

func IsValidSignature(modelName, signature string) bool

IsValidSignature checks if a signature is valid (non-empty and long enough). Special case: "skip_thought_signature_validator" is valid only for Gemini models.

func LiveAuthMiddleware added in v0.0.11 

func LiveAuthMiddleware(cfgProvider config.RuntimeConfigGetter) func(http.Handler) http.Handler

LiveAuthMiddleware creates middleware that enforces auth based on live config. It rebuilds the authenticator chain when auth-related config values change.

func LogProxyMetrics 

func LogProxyMetrics(ctx context.Context, metrics Metrics, _ config.DebugOptions)

LogProxyMetrics logs proxy-level performance metrics in debug mode.

func LogRequestDetails 

func LogRequestDetails(ctx context.Context, req *http.Request, opts config.DebugOptions)

LogRequestDetails logs request body and headers in debug mode. Respects DebugOptions.LogRequestBody and MaxBodyLogSize.

func LogResponseDetails 

func LogResponseDetails(
	ctx context.Context,
	headers http.Header,
	statusCode, eventCount int,
	opts config.DebugOptions,
)

LogResponseDetails logs response headers and streaming event count in debug mode.

func LogTLSMetrics 

func LogTLSMetrics(ctx context.Context, metrics TLSMetrics, opts config.DebugOptions)

LogTLSMetrics logs TLS connection metrics in debug mode.

func LoggingMiddleware 

func LoggingMiddleware(debugOpts config.DebugOptions) func(http.Handler) http.Handler

LoggingMiddleware logs each request with method, path, and duration. If debugOpts has debug logging enabled, logs additional request/response details.

func LoggingMiddlewareWithProvider added in v0.0.11 

func LoggingMiddlewareWithProvider(provider DebugOptionsProvider) func(http.Handler) http.Handler

LoggingMiddlewareWithProvider logs each request using live debug options.

func MapEventData added in v0.0.5 

func MapEventData(mapper func([]byte) []byte) func(ro.Observable[SSEEvent]) ro.Observable[SSEEvent]

MapEventData transforms the data field of each SSE event.

Example: 

// Add prefix to all event data
transformed := ro.Pipe1(events, MapEventData(func(data []byte) []byte {
    return append([]byte("prefix:"), data...)
}))

func MaxBodyBytesMiddleware added in v0.0.12 

func MaxBodyBytesMiddleware(limitProvider func() int64) func(http.Handler) http.Handler

MaxBodyBytesMiddleware creates middleware that limits request body size. Uses http.MaxBytesReader to enforce the limit efficiently. The limitProvider is called per-request to support hot-reload.

func MultiAuthMiddleware 

func MultiAuthMiddleware(authConfig *config.AuthConfig) func(http.Handler) http.Handler

MultiAuthMiddleware creates middleware supporting multiple authentication methods. Supports both x-api-key and Authorization: Bearer token authentication. If authConfig has no methods enabled, all requests pass through.

func NewLogger 

func NewLogger(cfg config.LoggingConfig) (zerolog.Logger, error)

NewLogger creates a zerolog.Logger from LoggingConfig. Returns a configured logger ready for use as global logger.

func ParseSignature added in v0.0.8 

func ParseSignature(prefixed string) (modelGroup, signature string, ok bool)

ParseSignature extracts modelGroup and raw signature from prefixed format. Returns modelGroup, signature, ok.

func ProcessNonStreamingResponse added in v0.0.8 

func ProcessNonStreamingResponse(
	ctx context.Context,
	body []byte,
	modelName string,
	cache *SignatureCache,
) []byte

ProcessNonStreamingResponse processes thinking blocks in a non-streaming response. Extracts and caches signatures, adds modelGroup prefix to signatures.

func ProcessResponseSignature added in v0.0.8 

func ProcessResponseSignature(
	ctx context.Context,
	eventData []byte,
	thinkingText string,
	modelName string,
	cache *SignatureCache,
) []byte

ProcessResponseSignature handles signature_delta events from upstream. Extracts signature, caches it, and transforms to include modelGroup prefix. Returns the modified event data with prefixed signature.

func ReplaceContentWithPlaceholder added in v0.0.15 

func ReplaceContentWithPlaceholder(body []byte, msgIndex int64) []byte

ReplaceContentWithPlaceholder replaces the content of an assistant message with a single text block, preserving the message structure so that adjacent tool_result blocks are not orphaned.

func RequestIDMiddleware 

func RequestIDMiddleware() func(http.Handler) http.Handler

RequestIDMiddleware adds X-Request-ID header and logger with request ID to context.

func SetSSEHeaders 

func SetSSEHeaders(h http.Header)

SetSSEHeaders sets required headers for SSE streaming. These headers MUST be set for proper streaming through nginx/CDN:

  • Content-Type: text/event-stream - SSE format
  • Cache-Control: no-cache, no-transform - prevent caching
  • X-Accel-Buffering: no - CRITICAL: disable nginx/Cloudflare buffering
  • Connection: keep-alive - maintain streaming connection

func SetupRoutes 

func SetupRoutes(
	cfg *config.Config,
	provider providers.Provider,
	providerKey string,
	pool *keypool.KeyPool,
) (http.Handler, error)

SetupRoutes creates the HTTP handler with all routes configured. Routes:

  • POST /v1/messages - Proxy to backend provider (with auth if configured)
  • GET /v1/models - List available models (no auth required)
  • GET /health - Health check endpoint (no auth required)

This is a convenience wrapper that calls SetupRoutesWithProviders with nil for pool and allProviders.

func SetupRoutesWithLiveKeyPools added in v0.0.11 

func SetupRoutesWithLiveKeyPools(opts *RoutesOptions) (http.Handler, error)

SetupRoutesWithLiveKeyPools creates the HTTP handler with full hot-reload support. Extends SetupRoutesWithRouterLive with live key pool accessors, enabling: - Hot-reloadable provider info (enabled/disabled, weights, priorities) - Hot-reloadable routing strategy and timeout - Hot-reloadable key pools (newly enabled providers get keys immediately) Routes:

  • POST /v1/messages - Proxy to backend provider with router-based selection
  • GET /v1/models - List available models from all providers (no auth required)
  • GET /v1/providers - List active providers with metadata (no auth required)
  • GET /health - Health check endpoint (no auth required)

func SetupRoutesWithProviders 

func SetupRoutesWithProviders(
	cfg *config.Config,
	provider providers.Provider,
	providerKey string,
	pool *keypool.KeyPool,
	allProviders []providers.Provider,
) (http.Handler, error)

SetupRoutesWithProviders creates the HTTP handler with all routes configured. Routes:

  • POST /v1/messages - Proxy to backend provider (with auth if configured)
  • GET /v1/models - List available models from all providers (no auth required)
  • GET /v1/providers - List active providers with metadata (no auth required)
  • GET /health - Health check endpoint (no auth required)

The allProviders parameter is used for the /v1/models and /v1/providers endpoints to list models and providers from all configured providers. If nil, only the primary provider's models are listed.

func SetupRoutesWithRouter added in v0.0.6 

func SetupRoutesWithRouter(cfg *config.Config, opts *RoutesOptions) (http.Handler, error)

SetupRoutesWithRouter creates the HTTP handler with all routes configured and router support. This is the DI-friendly version that accepts a ProviderRouter for multi-provider routing. Routes:

  • POST /v1/messages - Proxy to backend provider with router-based selection
  • GET /v1/models - List available models from all providers (no auth required)
  • GET /v1/providers - List active providers with metadata (no auth required)
  • GET /health - Health check endpoint (no auth required)

func SetupRoutesWithRouterLive added in v0.0.11 

func SetupRoutesWithRouterLive(opts *RoutesOptions) (http.Handler, error)

SetupRoutesWithRouterLive creates the HTTP handler with hot-reloadable provider info and router. This is the DI-friendly version that accepts functions for dynamic provider/router access. ProviderInfosFunc is called per-request to get current provider routing information, allowing changes to enabled/disabled, weights, and priorities to take effect without restart. Routes:

  • POST /v1/messages - Proxy to backend provider with router-based selection
  • GET /v1/models - List available models from all providers (no auth required)
  • GET /v1/providers - List active providers with metadata (no auth required)
  • GET /health - Health check endpoint (no auth required)

func StreamSSE added in v0.0.5 

func StreamSSE(body io.Reader) ro.Observable[SSEEvent]

StreamSSE creates an Observable from an SSE response body. Events are parsed according to the SSE specification and emitted as they arrive. The stream completes when the response body is fully read or EOF is reached. The stream errors if parsing fails or the body read encounters an error.

Note: The caller is responsible for closing the response body after the observable completes or errors.

Example: 

resp, _ := http.Get("https://api.example.com/events")
events := StreamSSE(resp.Body)
events.Subscribe(ro.NewObserver(
    func(e SSEEvent) { process(e) },
    func(err error) { handleError(err) },
    func() { resp.Body.Close() },
))

func WouldOrphanToolResults added in v0.0.15 

func WouldOrphanToolResults(body []byte, msgIndex int64) bool

WouldOrphanToolResults checks if dropping the assistant message at msgIndex would leave orphaned tool_result blocks in adjacent user messages. The Anthropic API requires every tool_result to have a corresponding tool_use in the immediately preceding assistant message. Dropping an assistant message can violate this invariant.

func WriteBodyTooLargeError added in v0.0.12 

func WriteBodyTooLargeError(w http.ResponseWriter)

WriteBodyTooLargeError writes a 413 Request Entity Too Large response.

func WriteError 

func WriteError(writer http.ResponseWriter, statusCode int, errorType, message string)

WriteError writes a JSON error response in Anthropic API format.

func WriteRateLimitError added in v0.0.3 

func WriteRateLimitError(writer http.ResponseWriter, retryAfter time.Duration)

WriteRateLimitError writes a 429 Too Many Requests response in Anthropic format. The retryAfter parameter specifies when capacity will be available.

func WriteSSEEvent added in v0.0.5 

func WriteSSEEvent(writer http.ResponseWriter, event SSEEvent) error

WriteSSEEvent writes a single SSE event to an http.ResponseWriter. Returns an error if the write fails or the writer doesn't support flushing.

This is a convenience function for writing individual events without creating an observable stream.

Example: 

event := SSEEvent{Event: "message", Data: []byte("Hello")}
if err := WriteSSEEvent(writer, event); err != nil {
    // Handle error
}

Types

type ConcurrencyLimiter added in v0.0.12 

type ConcurrencyLimiter struct {
	// contains filtered or unexported fields
}

ConcurrencyLimiter enforces a global maximum number of concurrent requests. It uses an atomic counter with a configurable limit that supports hot-reload. When the limit is reached, new requests receive 503 Service Unavailable.

func NewConcurrencyLimiter added in v0.0.12 

func NewConcurrencyLimiter(maxLimit int64) *ConcurrencyLimiter

NewConcurrencyLimiter creates a new concurrency limiter with the given max limit. A limit of 0 or negative means unlimited.

func (*ConcurrencyLimiter) CurrentInFlight added in v0.0.12 

func (l *ConcurrencyLimiter) CurrentInFlight() int64

CurrentInFlight returns the current number of in-flight requests.

func (*ConcurrencyLimiter) GetLimit added in v0.0.12 

func (l *ConcurrencyLimiter) GetLimit() int64

GetLimit returns the current configured limit.

func (*ConcurrencyLimiter) Release added in v0.0.12 

func (l *ConcurrencyLimiter) Release()

Release releases a slot after request completion. Must be called after a successful TryAcquire.

func (*ConcurrencyLimiter) SetLimit added in v0.0.12 

func (l *ConcurrencyLimiter) SetLimit(maxLimit int64)

SetLimit updates the concurrency limit for hot-reload support. A limit of 0 or negative means unlimited.

func (*ConcurrencyLimiter) TryAcquire added in v0.0.12 

func (l *ConcurrencyLimiter) TryAcquire() bool

TryAcquire attempts to acquire a slot for a request. Returns true if the request can proceed, false if the limit is reached. If limit is 0 or negative, always returns true (unlimited).

type DebugOptionsProvider added in v0.0.11 

type DebugOptionsProvider func() config.DebugOptions

DebugOptionsProvider returns current debug options for live-config logging.

type ErrorDetail 

type ErrorDetail struct {
	Type    string `json:"type"`
	Message string `json:"message"`
}

ErrorDetail contains the error type and message.

type ErrorResponse 

type ErrorResponse struct {
	Type  string      `json:"type"`
	Error ErrorDetail `json:"error"`
}

ErrorResponse matches Anthropic's error response format exactly.

type Handler 

type Handler struct {
	// contains filtered or unexported fields
}

Handler proxies requests to a backend provider.

func NewHandler 

func NewHandler(opts *HandlerOptions) (*Handler, error)

NewHandler creates a new proxy handler. If ProviderRouter is provided, it will be used for provider selection. If ProviderRouter is nil, Provider is used directly (single provider mode). ProviderPools maps provider names to their key pools (may be nil for providers without pooling). ProviderKeys maps provider names to their fallback API keys. RoutingConfig contains model-based routing configuration (may be nil). If HealthTracker is provided, success/failure will be reported to circuit breakers. If SignatureCache is provided, thinking signatures are cached for cross-provider reuse.

For hot-reloadable provider inputs, set ProviderInfosFunc. Otherwise, ProviderInfos is used.

func NewHandlerWithLiveKeyPools added in v0.0.11 

func NewHandlerWithLiveKeyPools(opts *HandlerOptions) (*Handler, error)

NewHandlerWithLiveKeyPools creates a new proxy handler with hot-reloadable key pools. When providers are enabled via config reload, their keys and pools are available immediately.

func NewHandlerWithLiveProviders added in v0.0.11 

func NewHandlerWithLiveProviders(opts *HandlerOptions) (*Handler, error)

NewHandlerWithLiveProviders creates a new proxy handler with hot-reloadable provider info. ProviderInfosFunc is called per-request to get current provider routing information.

func (*Handler) ServeHTTP 

func (h *Handler) ServeHTTP(writer http.ResponseWriter, request *http.Request)

ServeHTTP handles the proxy request.

func (*Handler) SetRuntimeConfigGetter added in v0.0.11 

func (h *Handler) SetRuntimeConfigGetter(cfg config.RuntimeConfigGetter)

SetRuntimeConfigGetter sets a live config provider for dynamic routing/debug settings. When set, the handler prefers this over static routingConfig/debugOpts/routingDebug.

type HandlerOptions added in v0.0.11 

type HandlerOptions struct {
	ProviderRouter    router.ProviderRouter
	Provider          providers.Provider
	ProviderPools     map[string]*keypool.KeyPool
	ProviderInfosFunc ProviderInfoFunc
	Pool              *keypool.KeyPool
	ProviderKeys      map[string]string
	GetProviderPools  KeyPoolsFunc
	GetProviderKeys   KeysFunc
	RoutingConfig     *config.RoutingConfig
	HealthTracker     *health.Tracker
	SignatureCache    *SignatureCache
	APIKey            string `json:"-"`
	ProviderInfos     []router.ProviderInfo
	DebugOptions      config.DebugOptions
	RoutingDebug      bool
}

HandlerOptions configures handler construction.

type KeyPoolsFunc added in v0.0.11 

type KeyPoolsFunc func() map[string]*keypool.KeyPool

KeyPoolsFunc returns the current key pools map for hot-reload support.

type KeysFunc added in v0.0.11 

type KeysFunc func() map[string]string

KeysFunc returns the current fallback keys map for hot-reload support.

type Metrics 

type Metrics struct {
	BackendTime     time.Duration
	TotalTime       time.Duration
	BytesSent       int64
	BytesReceived   int64
	StreamingEvents int
}

Metrics holds proxy-level performance metrics.

type ModelRewriter added in v0.0.8 

type ModelRewriter struct {
	// contains filtered or unexported fields
}

ModelRewriter handles model name rewriting in request bodies.

func NewModelRewriter added in v0.0.8 

func NewModelRewriter(mapping map[string]string) *ModelRewriter

NewModelRewriter creates a new model rewriter with the given mapping. If mapping is nil or empty, the rewriter will pass through all models unchanged.

func (*ModelRewriter) HasMapping added in v0.0.8 

func (r *ModelRewriter) HasMapping() bool

HasMapping returns true if the rewriter has any mappings configured.

func (*ModelRewriter) RewriteModel added in v0.0.8 

func (r *ModelRewriter) RewriteModel(model string) string

RewriteModel maps a model name using the configured mapping. Returns the mapped name if found, otherwise returns the original unchanged.

func (*ModelRewriter) RewriteRequest added in v0.0.8 

func (r *ModelRewriter) RewriteRequest(req *http.Request, logger *zerolog.Logger) error

RewriteRequest rewrites the model field in the request body if a mapping exists. Returns the modified request with updated body if rewriting occurred. The original model name is logged for debugging purposes. Uses mo.Result for railway-oriented error handling with centralized body restoration.

type ModelsHandler 

type ModelsHandler struct {
	// contains filtered or unexported fields
}

ModelsHandler handles requests to /v1/models endpoint.

func NewModelsHandler 

func NewModelsHandler(providerList []providers.Provider) *ModelsHandler

NewModelsHandler creates a new models handler with the given providers.

func NewModelsHandlerWithProviderFunc added in v0.0.12 

func NewModelsHandlerWithProviderFunc(getProviders ProvidersGetter) *ModelsHandler

NewModelsHandlerWithProviderFunc creates a models handler with a live provider accessor.

func (*ModelsHandler) ServeHTTP 

func (h *ModelsHandler) ServeHTTP(writer http.ResponseWriter, _ *http.Request)

ServeHTTP handles GET /v1/models requests.

type ModelsResponse 

type ModelsResponse struct {
	Object string            `json:"object"`
	Data   []providers.Model `json:"data"`
}

ModelsResponse represents the response format for /v1/models endpoint. This matches the Anthropic/OpenAI model list response format.

type ModifyResponseFunc added in v0.0.8 

type ModifyResponseFunc func(resp *http.Response) error

ModifyResponseFunc is a callback for additional response processing.

type ProviderInfo 

type ProviderInfo struct {
	Name    string   `json:"name"`
	Type    string   `json:"type"`
	BaseURL string   `json:"base_url"`
	Models  []string `json:"models"`
	Active  bool     `json:"active"`
}

ProviderInfo represents provider information in the API response.

type ProviderInfoFunc added in v0.0.11 

type ProviderInfoFunc func() []router.ProviderInfo

ProviderInfoFunc is a function that returns current provider routing information. This enables hot-reload of provider inputs (enabled/disabled, weights, priorities) without recreating the handler.

type ProviderProxy added in v0.0.8 

type ProviderProxy struct {
	Provider providers.Provider
	Proxy    *httputil.ReverseProxy
	KeyPool  *keypool.KeyPool

	APIKey string `json:"-"`
	// contains filtered or unexported fields
}

ProviderProxy bundles a provider with its dedicated reverse proxy. Each proxy has the provider's URL and auth baked in at creation time, ensuring requests are routed to the correct backend with correct authentication.

func NewProviderProxy added in v0.0.8 

func NewProviderProxy(
	provider providers.Provider,
	apiKey string,
	pool *keypool.KeyPool,
	debugOpts config.DebugOptions,
	modifyResponseHook ModifyResponseFunc,
) (*ProviderProxy, error)

NewProviderProxy creates a provider-specific proxy with correct URL and auth. The proxy is configured to use this provider's BaseURL for all requests. The modifyResponseHook is called after SSE header handling for additional processing.

func (*ProviderProxy) GetTargetURL added in v0.0.8 

func (pp *ProviderProxy) GetTargetURL() *url.URL

GetTargetURL returns the target URL for this provider's proxy. Useful for testing and debugging.

type ProvidersGetter added in v0.0.12 

type ProvidersGetter func() []providers.Provider

ProvidersGetter returns the current provider list for live updates.

type ProvidersHandler 

type ProvidersHandler struct {
	// contains filtered or unexported fields
}

ProvidersHandler handles requests to /v1/providers endpoint.

func NewProvidersHandler 

func NewProvidersHandler(providerList []providers.Provider) *ProvidersHandler

NewProvidersHandler creates a new providers handler with the given providers.

func NewProvidersHandlerWithProviderFunc added in v0.0.12 

func NewProvidersHandlerWithProviderFunc(getProviders ProvidersGetter) *ProvidersHandler

NewProvidersHandlerWithProviderFunc creates a providers handler with a live provider accessor.

func (*ProvidersHandler) ServeHTTP 

func (h *ProvidersHandler) ServeHTTP(writer http.ResponseWriter, _ *http.Request)

ServeHTTP handles GET /v1/providers requests.

type ProvidersResponse 

type ProvidersResponse struct {
	Object string         `json:"object"`
	Data   []ProviderInfo `json:"data"`
}

ProvidersResponse represents the response format for /v1/providers endpoint.

type RoutesOptions added in v0.0.11 

type RoutesOptions struct {
	ProviderRouter     router.ProviderRouter
	Provider           providers.Provider
	ConfigProvider     config.RuntimeConfigGetter
	Pool               *keypool.KeyPool
	ProviderInfosFunc  ProviderInfoFunc
	ProviderPools      map[string]*keypool.KeyPool
	ProviderKeys       map[string]string
	GetProviderPools   KeyPoolsFunc
	GetProviderKeys    KeysFunc
	GetAllProviders    ProvidersGetter
	HealthTracker      *health.Tracker
	SignatureCache     *SignatureCache
	ConcurrencyLimiter *ConcurrencyLimiter
	ProviderKey        string
	ProviderInfos      []router.ProviderInfo
	AllProviders       []providers.Provider
}

RoutesOptions configures route setup with optional hot-reload support.

type SSEEvent added in v0.0.5 

type SSEEvent struct {
	Event string
	ID    string
	Data  []byte
	Retry int
}

SSEEvent represents a Server-Sent Event. Fields match the SSE specification: https://html.spec.whatwg.org/multipage/server-sent-events.html

func (SSEEvent) Bytes added in v0.0.5 

func (e SSEEvent) Bytes() []byte

Bytes returns the SSE wire format representation as bytes.

func (SSEEvent) String added in v0.0.5 

func (e SSEEvent) String() string

String returns the SSE wire format representation of the event.

type SSESignatureProcessor added in v0.0.8 

type SSESignatureProcessor struct {
	// contains filtered or unexported fields
}

SSESignatureProcessor handles signature processing for SSE events. Accumulates thinking text and caches signatures as they stream.

func NewSSESignatureProcessor added in v0.0.8 

func NewSSESignatureProcessor(cache *SignatureCache, modelName string) *SSESignatureProcessor

NewSSESignatureProcessor creates a new SSE signature processor.

func (*SSESignatureProcessor) GetCurrentSignature added in v0.0.8 

func (p *SSESignatureProcessor) GetCurrentSignature() string

GetCurrentSignature returns the last processed signature.

func (*SSESignatureProcessor) ProcessEvent added in v0.0.8 

func (p *SSESignatureProcessor) ProcessEvent(ctx context.Context, eventData []byte) []byte

ProcessEvent processes a single SSE event line. Accumulates thinking text from thinking_delta events. Caches and transforms signatures from signature_delta events. Returns the potentially modified event data.

type Server 

type Server struct {
	// contains filtered or unexported fields
}

Server wraps http.Server with cc-relay configuration.

func NewServer 

func NewServer(addr string, handler http.Handler, enableHTTP2 bool) *Server

NewServer creates a new Server with proper timeouts for streaming. Timeout rationale:

  • ReadTimeout: 10s - protect against slowloris attacks
  • WriteTimeout: 600s - Claude Code operations can stream for 10+ minutes
  • IdleTimeout: 120s - reasonable keep-alive

If enableHTTP2 is true, enables HTTP/2 cleartext (h2c) support for non-TLS connections. HTTP/2 provides better multiplexing and performance for Claude Code's concurrent tool calls.

func (*Server) ListenAndServe 

func (s *Server) ListenAndServe() error

ListenAndServe starts the server (blocks).

func (*Server) Shutdown 

func (s *Server) Shutdown(ctx context.Context) error

Shutdown gracefully stops the server.

type SignatureCache added in v0.0.8 

type SignatureCache struct {
	// contains filtered or unexported fields
}

SignatureCache provides thread-safe caching of thinking block signatures. Uses cc-relay's cache.Cache interface for storage.

func NewSignatureCache added in v0.0.8 

func NewSignatureCache(c cache.Cache) *SignatureCache

NewSignatureCache creates a new signature cache using the provided cache backend. Returns nil if the cache is nil (no-op mode).

func (*SignatureCache) Get added in v0.0.8 

func (sc *SignatureCache) Get(ctx context.Context, modelName, text string) string

Get retrieves a cached signature for the given model and text. Returns empty string on cache miss or error.

func (*SignatureCache) Set added in v0.0.8 

func (sc *SignatureCache) Set(ctx context.Context, modelName, text, signature string)

Set caches a signature for the given model and text. Skips caching if signature is too short or cache is nil.

type TLSMetrics 

type TLSMetrics struct {
	Version     string
	DNSTime     time.Duration
	ConnectTime time.Duration
	TLSTime     time.Duration
	Reused      bool
	HasMetrics  bool
}

TLSMetrics holds TLS connection timing and metadata.

type ThinkingContext added in v0.0.8 

type ThinkingContext struct {
	CurrentSignature        string
	AccumulatedThinkingText strings.Builder
	DroppedBlocks           int
	ReorderedBlocks         bool
}

ThinkingContext holds state for processing thinking blocks in a message.

func GetThinkingContextFromContext added in v0.0.8 

func GetThinkingContextFromContext(ctx context.Context) *ThinkingContext

GetThinkingContextFromContext retrieves the thinking context from context.

func ProcessRequestThinking added in v0.0.8 

func ProcessRequestThinking(
	ctx context.Context,
	body []byte,
	modelName string,
	cache *SignatureCache,
) ([]byte, *ThinkingContext, error)

ProcessRequestThinking processes thinking blocks in the request body: 1. Looks up cached signatures for thinking blocks 2. Falls back to client-provided signature (validating format) 3. Drops unsigned thinking blocks 4. Tracks signature for tool_use inheritance 5. Reorders blocks so thinking comes first

Returns modified body and context, or error.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.