.. _arc42_runtime_view: ************ Runtime View ************ Request-Response Roundtrip ========================== Any incoming TCP connection is wrapped into a `Netty `_ channel (``io.netty.channel.Channel``) and incoming bytes are handled by a chain of ``ChannelInboundHandlers``. For connections resulting in bytes written back to the channel, the write attempts accordingly are passed along a chain of ``ChannelOutboundHandlers``. To gain a deeper understanding about Netty's asynchronous non-blocking I/O model, please refer to the documentation at https://netty.io/. Inbound TCP streams that contain HTTP requests, are effectively handled by Zuul, where they are passed along chains of *inbound filters* and *outbound filters*. Between both chains, the final request is actually processed, usually by performing a proxy request against an upstream/origin server. This processing is performed by an *endpoint filter*. Proxy requests are performed using Netty again on the HTTP and network layers. |request-response flow| Server Request Processing ========================= This section describes, how incoming HTTP requests are processed by handlers of the underlying Zuul and Netty frameworks and handlers added by OX. It describes the various ``io.netty.channel.ChannelHandler`` implementations that perform actual work during this process. The term ``Channel`` denotes a low-level socket connection between client and server. A ``ChannelHandler`` reacts to events that happen on such channels. The most relevant events are: - inbound data is available for reading - outbound data was written - a user-defined event was fired Channel handlers are implemented as chains, where each handler optionally performs its work based on an event and then calls the next handler in the chain to perform its own work. To preserve state between different handlers, multiple invocations of the same handler or different events of the same handler, a ``ChannelContext`` instance is passed along with the actual event object. Server Channel Handlers ----------------------- The following chain of inbound and outbound channel handlers is processed for each incoming HTTP request. The yellow boxes denote plain Netty handlers while the red ones denote Zuul handlers. |server channel handlers| **IdleStateHandler** Schedules tasks to periodically check whether a configured idle time value was exceeded, resulting in an event. *Inbound:* Tracks time of last complete message read event on the channel. *Outbound:* Tracks time of last channel write event. *Triggered Events:* - ``io.netty.handler.timeout.IdleStateEvent`` *Handled Events:* -- **CloseOnIdleStateHandler** Closes idle channels. *Inbound:* -- *Outbound:* -- *Triggered Events:* -- *Handled Events:* - ``io.netty.handler.timeout.IdleStateEvent`` **PassportStateServerHandler** Passport maintenance to track channel lifecycles. See https://github.com/Netflix/zuul/wiki/Core-Features#request-passport. *Inbound:* Tracked states: - ``SERVER_CH_ACTIVE`` - ``SERVER_CH_INACTIVE`` - ``SERVER_CH_EXCEPTION`` *Outbound:* Tracked states: - ``SERVER_CH_CLOSE`` - ``SERVER_CH_DISCONNECT`` - ``SERVER_CH_EXCEPTION`` *Triggered Events:* -- *Handled Events:* - ``io.netty.handler.timeout.IdleStateEvent``: Tracked as ``SERVER_CH_IDLE_TIMEOUT`` **SourceAddressChannelHandler** Stores actual IP address and port of client and server sides of the channel within the channel context. *Inbound:* -- *Outbound:* -- *Triggered Events:* -- *Handled Events:* -- **ServerChannelMetrics** Collects monitoring metrics on the channel level: - ``server.connections.current`` - ``server.connections.connect`` - ``server.connections.errors`` - ``server.connections.close`` - ``server.connections.idle.timeout`` - ``server.connections.throttled`` *Inbound:* -- *Outbound:* -- *Triggered Events:* -- *Handled Events:* - ``MaxInboundConnectionsHandler.CONNECTION_THROTTLED_EVENT`` - ``io.netty.handler.timeout.IdleStateEvent`` **PerEventLoopMetricsChannelHandler** Collects monitoring metrics on the event loop level: - ``server.eventloop.http.requests.current`` - ``server.eventloop.connections.current`` *Inbound:* -- *Outbound:* -- *Triggered Events:* -- *Handled Events:* - ``HttpLifecycleChannelHandler.StartEvent``: increment current HTTP requests - ``HttpLifecycleChannelHandler.CompleteEvent``: decrement current HTTP requests **ElbProxyProtocolChannelHandler** Decodes PROXY protocol and if applicable stores the contained client and server IP addresses and ports within the channel context. This effectively overrides the values stored by ``SourceAddressChannelHandler``. *Inbound:* Check for PROXY protocol and process its metadata. *Outbound:* -- *Triggered Events:* -- *Handled Events:* -- **MaxInboundConnectionsHandler** Closes any incoming new connections if current count is above a configured threshold. For throttled connections, this is stored within the passport. *Inbound:* -- *Outbound:* -- *Triggered Events:* - ``MaxInboundConnectionsHandler.CONNECTION_THROTTLED_EVENT`` *Handled Events:* -- **SslHandler** Performs TLS handshake on incoming connections and ensures according wrapping of input/output byte buffers. See JavaDoc of ``io.netty.handler.ssl.SslHandler`` for details. *Inbound:* -- *Outbound:* -- *Triggered Events:* - ``SslHandshakeCompletionEvent`` *Handled Events:* -- **SslHandshakeInfoHandler** Stores info about the client and server's SSL certificates in the context, after a successful handshake. *Inbound:* -- *Outbound:* -- *Triggered Events:* -- *Handled Events:* - ``SslHandshakeCompletionEvent`` **HttpServerCodec** De-/encodes the in-/outbound byte streams to/from HTTP domain objects. *Inbound:* Decode byte stream into ``io.netty.handler.codec.http.HttpRequest``. *Outbound:* Encode ``io.netty.handler.codec.http.HttpResponse`` into byte stream. *Triggered Events:* -- *Handled Events:* -- **Http1ConnectionCloseHandler** Manages HTTP keep-alive lifecycle of inbound connections. *Inbound:* -- *Outbound:* Adds ``Connection: close`` response header if channel is supposed to be closed after the current request was handled. In addition it actively closes the channel after the last packet has been sent. *Triggered Events:* -- *Handled Events:* -- **PassportStateHttpServerHandler** Passport maintenance to track HTTP request lifecycle events. See https://github.com/Netflix/zuul/wiki/Core-Features#request-passport. *Inbound:* Tracked states: - ``IN_REQ_HEADERS_RECEIVED`` - ``IN_REQ_CONTENT_RECEIVED`` - ``IN_REQ_LAST_CONTENT_RECEIVED`` *Outbound:* Tracked states: - ``OUT_RESP_HEADERS_SENDING`` - ``OUT_RESP_HEADERS_SENT`` - ``OUT_RESP_HEADERS_ERROR_SENDING`` - ``OUT_RESP_CONTENT_SENDING`` - ``OUT_RESP_CONTENT_SENT`` - ``OUT_RESP_CONTENT_ERROR_SENDING`` - ``OUT_RESP_LAST_CONTENT_SENDING`` - ``OUT_RESP_LAST_CONTENT_SENT`` - ``OUT_RESP_LAST_CONTENT_ERROR_SENDING`` *Triggered Events:* -- *Handled Events:* - ``HttpLifecycleChannelHandler.CompleteEvent``: Reset channel passport **HttpRequestReadTimeoutHandler** Fires timeout event if configured read timeout is exceeded while waiting for data of an incoming HTTP message. The handler does NOT close a timed out connection! Sets ``IN_REQ_READ_TIMEOUT`` state in passport for timed out requests. Also increments ``server.http.request.read.timeout`` monitoring counter. *Inbound:* Tracks time since last chunk of data was read. *Outbound:* -- *Triggered Events:* - ``HttpRequestReadTimeoutEvent`` *Handled Events:* -- **HttpServerLifecycleChannelHandler** Triggers HTTP request lifecycle events for the Zuul layer. *Inbound:* Fires event as soon as an incoming HTTP request along with all headers is available. *Outbound:* Fires event as soon as an HTTP response was fully sent. *Triggered Events:* - ``com.netflix.netty.common.HttpLifecycleChannelHandler.StartEvent`` - ``com.netflix.netty.common.HttpLifecycleChannelHandler.CompleteEvent`` *Handled Events:* -- **HttpBodySizeRecordingChannelHandler** Records body sizes of HTTP requests and responses and stores them as attributes within the channel context. *Inbound:* Records request body size. *Outbound:* Records response body size. *Triggered Events:* -- *Handled Events:* - ``HttpLifecycleChannelHandler.CompleteEvent``: Resets channel context **HttpMetricsChannelHandler** Collects monitoring metrics on the HTTP request level: - ``server.http.requests.current`` - ``server.http.requests.pipelining.dropped`` *Inbound:* -- *Outbound:* -- *Triggered Events:* -- *Handled Events:* - ``HttpServerLifecycleChannelHandler.StartEvent`` - ``HttpServerLifecycleChannelHandler.CompleteEventEvent`` - ``HttpServerLifecycleChannelHandler.RejectedPipeliningEvent`` **AccessLogChannelHandler** Performs HTTP access logging for handled requests. *Inbound:* Tracks start time (as soon as request headers are parsed) and request body size and stores it in channel context. *Outbound:* Tracks response body size and stores it in channel context. *Triggered Events:* -- *Handled Events:* - ``HttpServerLifecycleChannelHandler.CompleteEvent``: Combines tracked data with other channel attributes and writes access log entry. **StripUntrustedProxyHeadersHandler** Based on configuration removes ``X-Forwarded-*`` headers from incoming requests. *Inbound:* Strips headers. *Outbound:* -- *Triggered Events:* -- *Handled Events:* -- **LoggingHandler** Netty debug handler that logs any channel event according to a configured log level threshold. *Inbound:* -- *Outbound:* -- *Triggered Events:* -- *Handled Events:* -- **ClientRequestReceiver** Converts Netty HTTP request object into a Zuul domain object and passes it further down the channel pipeline. *Inbound:* As soon as an ``io.netty.handler.codec.http.HttpRequest`` instance is available, it is converted into a ``com.netflix.zuul.message.http.HttpRequestMessage`` instance. *Outbound:* Tracks I/O errors during response writing *Triggered Events:* -- *Handled Events:* - ``HttpServerLifecycleChannelHandler.CompleteEventEvent`` **PassportLoggingHandler** Logs the passport (https://github.com/Netflix/zuul/wiki/Core-Features#request-passport) for completed requests. Typically this results in ``DEBUG`` logs. But if request or response processing took longer than a configured time, this is logged on ``INFO`` level. *Inbound:* -- *Outbound:* -- *Triggered Events:* -- *Handled Events:* - ``HttpLifecycleChannelHandler.CompleteEvent`` **ZuulFilterChainHandler** This is the actual bridge between Netty events and the Zuul proxy engine. *Inbound:* ``com.netflix.zuul.message.http.HttpRequestMessage`` are taken and handed over to the Zuul request filter chain. *Outbound:* -- *Triggered Events:* -- *Handled Events:* - ``HttpLifecycleChannelHandler.CompleteEvent``: Reports processing completion to proxy endpoint. - ``HttpRequestReadTimeoutEvent``: Responds with HTTP 408 - ``io.netty.handler.timeout.IdleStateEvent``: Responds with HTTP 504 - ``RequestCancelledEvent``: Track cancelled status - ``HttpLifecycleChannelHandler.RejectedPipeliningEvent``: Responds with HTTP 400 **ClientResponseWriter** Reacts to Zuul HTTP responses passed along the channel pipeline and converts them into Netty domain objects. *Inbound:* If the Zuul proxy engine is done with a request and has a response ready in the form of ``com.netflix.zuul.message.http.HttpResponseMessage``, it is actually passed along the Netty channel pipeline as a read event. This handler converts those instances into ``io.netty.handler.codec.http.HttpResponse`` instances. It then effectively interrupts the read handler chain for this request and instead initiates the write handler chain by writing the Netty HTTP response object onto it. *Outbound:* -- *Triggered Events:* -- *Handled Events:* - ``HttpServerLifecycleChannelHandler.StartEvent`` - ``HttpServerLifecycleChannelHandler.CompleteEventEvent``: If the channel is to be closed because of a ``Connection: close`` response header, this handler does so. Otherwise it restarts the read handler chain on this channel to process the next request. **ServerStatusHeaderHandler** Does nothing currently. **Http1ConnectionExpiryHandler** Tracks the number of requests per channel so that it can be marked as closed and be effectively closed by ``Http1ConnectionCloseHandler`` and ``ClientResponseWriter``. *Inbound:* -- *Outbound:* Marks channel to be closed if max. number of requests per channel was reached. *Triggered Events:* -- *Handled Events:* -- HTTP Request Handling --------------------- Any HTTP request that successfully passed the inbound channel handlers is finally available as a ``com.netflix.zuul.message.http.HttpRequestMessage`` object. This object is passed on to the Zuul request filter chain, where it is handled. Handling means, the request is either forwarded ("proxied") to an origin server or handled by an *endpoint filter* directly. In any case it leads to the creation of an ``com.netflix.zuul.message.http.HttpResponseMessage`` object, which is then passed forward to the server channel outbound handlers. |http request handling| App Suite Proxy Engine ---------------------- TODO Client Request Processing ========================= This section describes, how HTTP requests are - transformed and forwarded to an origin server and its response then transformed and written to the client or - handled by App Suite Proxy directly, resulting in a generated HTTP response Client Connection Handling -------------------------- This subsection touches the concepts of origins TODO .. |request-response flow| image:: images/06_runtime_view-request_response_flow.png .. |server channel handlers| image:: images/06_runtime_view-server_channel_handlers.png .. |http request handling| image:: images/06_runtime_view-http_request_handling.png