Runtime View

Request-Response Roundtrip

Any incoming TCP connection is wrapped into a Netty channel (io.netty.channel.Channel) and incoming bytes are handled by a chain of ChannelInboundHandlers. For connections resulting in bytes written back to the channel, the write attempts accordingly are passed along a chain of ChannelOutboundHandlers. To gain a deeper understanding about Netty’s asynchronous non-blocking I/O model, please refer to the documentation at https://netty.io/.

Inbound TCP streams that contain HTTP requests, are effectively handled by Zuul, where they are passed along chains of inbound filters and outbound filters. Between both chains, the final request is actually processed, usually by performing a proxy request against an upstream/origin server. This processing is performed by an endpoint filter.

Proxy requests are performed using Netty again on the HTTP and network layers.

request-response flow

Server Request Processing

This section describes, how incoming HTTP requests are processed by handlers of the underlying Zuul and Netty frameworks and handlers added by OX. It describes the various io.netty.channel.ChannelHandler implementations that perform actual work during this process. The term Channel denotes a low-level socket connection between client and server. A ChannelHandler reacts to events that happen on such channels. The most relevant events are:

  • inbound data is available for reading

  • outbound data was written

  • a user-defined event was fired

Channel handlers are implemented as chains, where each handler optionally performs its work based on an event and then calls the next handler in the chain to perform its own work. To preserve state between different handlers, multiple invocations of the same handler or different events of the same handler, a ChannelContext instance is passed along with the actual event object.

Server Channel Handlers

The following chain of inbound and outbound channel handlers is processed for each incoming HTTP request. The yellow boxes denote plain Netty handlers while the red ones denote Zuul handlers.

server channel handlers

IdleStateHandler

Schedules tasks to periodically check whether a configured idle time value was exceeded, resulting in an event.

Inbound: Tracks time of last complete message read event on the channel.

Outbound: Tracks time of last channel write event.

Triggered Events:

  • io.netty.handler.timeout.IdleStateEvent

Handled Events: –

CloseOnIdleStateHandler

Closes idle channels.

Inbound: –

Outbound: –

Triggered Events: –

Handled Events:

  • io.netty.handler.timeout.IdleStateEvent

PassportStateServerHandler

Passport maintenance to track channel lifecycles. See https://github.com/Netflix/zuul/wiki/Core-Features#request-passport.

Inbound: Tracked states:

  • SERVER_CH_ACTIVE

  • SERVER_CH_INACTIVE

  • SERVER_CH_EXCEPTION

Outbound: Tracked states:

  • SERVER_CH_CLOSE

  • SERVER_CH_DISCONNECT

  • SERVER_CH_EXCEPTION

Triggered Events: –

Handled Events:

  • io.netty.handler.timeout.IdleStateEvent: Tracked as SERVER_CH_IDLE_TIMEOUT

SourceAddressChannelHandler

Stores actual IP address and port of client and server sides of the channel within the channel context.

Inbound: –

Outbound: –

Triggered Events: –

Handled Events: –

ServerChannelMetrics

Collects monitoring metrics on the channel level:

  • server.connections.current

  • server.connections.connect

  • server.connections.errors

  • server.connections.close

  • server.connections.idle.timeout

  • server.connections.throttled

Inbound: –

Outbound: –

Triggered Events: –

Handled Events:

  • MaxInboundConnectionsHandler.CONNECTION_THROTTLED_EVENT

  • io.netty.handler.timeout.IdleStateEvent

PerEventLoopMetricsChannelHandler

Collects monitoring metrics on the event loop level:

  • server.eventloop.http.requests.current

  • server.eventloop.connections.current

Inbound: –

Outbound: –

Triggered Events: –

Handled Events:

  • HttpLifecycleChannelHandler.StartEvent: increment current HTTP requests

  • HttpLifecycleChannelHandler.CompleteEvent: decrement current HTTP requests

ElbProxyProtocolChannelHandler

Decodes PROXY protocol and if applicable stores the contained client and server IP addresses and ports within the channel context. This effectively overrides the values stored by SourceAddressChannelHandler.

Inbound: Check for PROXY protocol and process its metadata.

Outbound: –

Triggered Events: –

Handled Events: –

MaxInboundConnectionsHandler

Closes any incoming new connections if current count is above a configured threshold. For throttled connections, this is stored within the passport.

Inbound: –

Outbound: –

Triggered Events:

  • MaxInboundConnectionsHandler.CONNECTION_THROTTLED_EVENT

Handled Events: –

SslHandler

Performs TLS handshake on incoming connections and ensures according wrapping of input/output byte buffers. See JavaDoc of io.netty.handler.ssl.SslHandler for details.

Inbound: –

Outbound: –

Triggered Events:

  • SslHandshakeCompletionEvent

Handled Events: –

SslHandshakeInfoHandler

Stores info about the client and server’s SSL certificates in the context, after a successful handshake.

Inbound: –

Outbound: –

Triggered Events: –

Handled Events:

  • SslHandshakeCompletionEvent

HttpServerCodec

De-/encodes the in-/outbound byte streams to/from HTTP domain objects.

Inbound: Decode byte stream into io.netty.handler.codec.http.HttpRequest.

Outbound: Encode io.netty.handler.codec.http.HttpResponse into byte stream.

Triggered Events: –

Handled Events: –

Http1ConnectionCloseHandler

Manages HTTP keep-alive lifecycle of inbound connections.

Inbound: –

Outbound: Adds Connection: close response header if channel is supposed to be closed after the current request was handled. In addition it actively closes the channel after the last packet has been sent.

Triggered Events: –

Handled Events: –

PassportStateHttpServerHandler

Passport maintenance to track HTTP request lifecycle events. See https://github.com/Netflix/zuul/wiki/Core-Features#request-passport.

Inbound: Tracked states:

  • IN_REQ_HEADERS_RECEIVED

  • IN_REQ_CONTENT_RECEIVED

  • IN_REQ_LAST_CONTENT_RECEIVED

Outbound: Tracked states:

  • OUT_RESP_HEADERS_SENDING

  • OUT_RESP_HEADERS_SENT

  • OUT_RESP_HEADERS_ERROR_SENDING

  • OUT_RESP_CONTENT_SENDING

  • OUT_RESP_CONTENT_SENT

  • OUT_RESP_CONTENT_ERROR_SENDING

  • OUT_RESP_LAST_CONTENT_SENDING

  • OUT_RESP_LAST_CONTENT_SENT

  • OUT_RESP_LAST_CONTENT_ERROR_SENDING

Triggered Events: –

Handled Events:

  • HttpLifecycleChannelHandler.CompleteEvent: Reset channel passport

HttpRequestReadTimeoutHandler

Fires timeout event if configured read timeout is exceeded while waiting for data of an incoming HTTP message. The handler does NOT close a timed out connection!

Sets IN_REQ_READ_TIMEOUT state in passport for timed out requests. Also increments server.http.request.read.timeout monitoring counter.

Inbound: Tracks time since last chunk of data was read.

Outbound: –

Triggered Events:

  • HttpRequestReadTimeoutEvent

Handled Events: –

HttpServerLifecycleChannelHandler

Triggers HTTP request lifecycle events for the Zuul layer.

Inbound: Fires event as soon as an incoming HTTP request along with all headers is available.

Outbound: Fires event as soon as an HTTP response was fully sent.

Triggered Events:

  • com.netflix.netty.common.HttpLifecycleChannelHandler.StartEvent

  • com.netflix.netty.common.HttpLifecycleChannelHandler.CompleteEvent

Handled Events: –

HttpBodySizeRecordingChannelHandler

Records body sizes of HTTP requests and responses and stores them as attributes within the channel context.

Inbound: Records request body size.

Outbound: Records response body size.

Triggered Events: –

Handled Events:

  • HttpLifecycleChannelHandler.CompleteEvent: Resets channel context

HttpMetricsChannelHandler

Collects monitoring metrics on the HTTP request level:

  • server.http.requests.current

  • server.http.requests.pipelining.dropped

Inbound: –

Outbound: –

Triggered Events: –

Handled Events:

  • HttpServerLifecycleChannelHandler.StartEvent

  • HttpServerLifecycleChannelHandler.CompleteEventEvent

  • HttpServerLifecycleChannelHandler.RejectedPipeliningEvent

AccessLogChannelHandler

Performs HTTP access logging for handled requests.

Inbound: Tracks start time (as soon as request headers are parsed) and request body size and stores it in channel context.

Outbound: Tracks response body size and stores it in channel context.

Triggered Events: –

Handled Events:

  • HttpServerLifecycleChannelHandler.CompleteEvent: Combines tracked data with other channel attributes and writes access log entry.

StripUntrustedProxyHeadersHandler

Based on configuration removes X-Forwarded-* headers from incoming requests.

Inbound: Strips headers.

Outbound: –

Triggered Events: –

Handled Events: –

LoggingHandler

Netty debug handler that logs any channel event according to a configured log level threshold.

Inbound: –

Outbound: –

Triggered Events: –

Handled Events: –

ClientRequestReceiver

Converts Netty HTTP request object into a Zuul domain object and passes it further down the channel pipeline.

Inbound: As soon as an io.netty.handler.codec.http.HttpRequest instance is available, it is converted into a com.netflix.zuul.message.http.HttpRequestMessage instance.

Outbound: Tracks I/O errors during response writing

Triggered Events: –

Handled Events:

  • HttpServerLifecycleChannelHandler.CompleteEventEvent

PassportLoggingHandler

Logs the passport (https://github.com/Netflix/zuul/wiki/Core-Features#request-passport) for completed requests. Typically this results in DEBUG logs. But if request or response processing took longer than a configured time, this is logged on INFO level.

Inbound: –

Outbound: –

Triggered Events: –

Handled Events:

  • HttpLifecycleChannelHandler.CompleteEvent

ZuulFilterChainHandler

This is the actual bridge between Netty events and the Zuul proxy engine.

Inbound: com.netflix.zuul.message.http.HttpRequestMessage are taken and handed over to the Zuul request filter chain.

Outbound: –

Triggered Events: –

Handled Events:

  • HttpLifecycleChannelHandler.CompleteEvent: Reports processing completion to proxy endpoint.

  • HttpRequestReadTimeoutEvent: Responds with HTTP 408

  • io.netty.handler.timeout.IdleStateEvent: Responds with HTTP 504

  • RequestCancelledEvent: Track cancelled status

  • HttpLifecycleChannelHandler.RejectedPipeliningEvent: Responds with HTTP 400

ClientResponseWriter

Reacts to Zuul HTTP responses passed along the channel pipeline and converts them into Netty domain objects.

Inbound: If the Zuul proxy engine is done with a request and has a response ready in the form of com.netflix.zuul.message.http.HttpResponseMessage, it is actually passed along the Netty channel pipeline as a read event. This handler converts those instances into io.netty.handler.codec.http.HttpResponse instances. It then effectively interrupts the read handler chain for this request and instead initiates the write handler chain by writing the Netty HTTP response object onto it.

Outbound: –

Triggered Events: –

Handled Events:

  • HttpServerLifecycleChannelHandler.StartEvent

  • HttpServerLifecycleChannelHandler.CompleteEventEvent: If the channel is to be closed because of a Connection: close response header, this handler does so. Otherwise it restarts the read handler chain on this channel to process the next request.

ServerStatusHeaderHandler

Does nothing currently.

Http1ConnectionExpiryHandler

Tracks the number of requests per channel so that it can be marked as closed and be effectively closed by Http1ConnectionCloseHandler and ClientResponseWriter.

Inbound: –

Outbound: Marks channel to be closed if max. number of requests per channel was reached.

Triggered Events: –

Handled Events: –

HTTP Request Handling

Any HTTP request that successfully passed the inbound channel handlers is finally available as a com.netflix.zuul.message.http.HttpRequestMessage object. This object is passed on to the Zuul request filter chain, where it is handled. Handling means, the request is either forwarded (“proxied”) to an origin server or handled by an endpoint filter directly. In any case it leads to the creation of an com.netflix.zuul.message.http.HttpResponseMessage object, which is then passed forward to the server channel outbound handlers.

http request handling

App Suite Proxy Engine

TODO

Client Request Processing

This section describes, how HTTP requests are

  • transformed and forwarded to an origin server and its response then transformed and written to the client or

  • handled by App Suite Proxy directly, resulting in a generated HTTP response

Client Connection Handling

This subsection touches the concepts of origins

TODO