my web 0.2 website

Ambiguity in HTTP/1.1

Frustrating does not being to describe a bug I had to work through today.

HTTP 1.1 supports request multiplexing. This technology allows you to send multiple HTTP requests through a single connection – when one http request is answered, the connection remains open to allow user-agents (“browsers”) to send more requests without having to go through the pains of tearing down the connection, and opening a new one. RFC 2616 says “persistent connections are the default behavior of any HTTP connection“.

RFC 2616 also implements a system to ensure user-agents can understand when a http reply has been fully communicated. In fact, this is a requirement, “….all messages on the connection MUST have a self-defined message length“.

Microsoft Internet Explorer claims to support HTTP 1.1.

Given a set of well known internet standards, and such commonly used software, you’d expect no issues with decisions on ‘where a reply boundary’ exists’ and ‘whether to close the connection’.

Well, you’d be wrong. Firstly, HTTP 1.1 also allows the server-side of the connection to signal that the message must not be multiplexed; where the signal wants to indicate that the connection will be torn down following the reply, the server should send the ‘Connection’ header and populate it with the token ‘close’. Sounds easy ? Well, HTTP 1.1 also says : “The Connection general-header field allows the sender to specify options that are desired for that particular connection and MUST NOT be communicated by proxies over further connections“. What about when the proxy is actually implemented at the application/web server end, as a reverse proxy ?

This distinction, I presume, is designed to allow the client side to maintain the persistent connection to the proxy. But in this example, the proxy is actually a reverse proxy that is close to the web-server infrastructure and not the browser. What if this is important to the application ?

You may wonder, why should it matter ? A proxy is expressly forbidden from changing the Content-Length header, because this header is used to establish where the boundary between different requests in the multiplexed http 1.1 connection exist. Can anyone therefore explain the following :

• Internet Explorer loads a page with an ‘option/select’ dropdown menu, and an empty textarea field.
• A javascript code block describes a relationship between the two form elements – when an item on the drop down menu is selected, an AJAX call which corresponds to the drop-down option is made to the web server that pulls an XML document describing how to populate the textarea field.
• With Internet Explorer, when the request is delivered directly to a web-server, the code works perfectly – the textarea is populated.
• With Internet Explorer, when the request is delivered via a Redline reverse proxy, the textarea field is never populated.
• With Firefox and other HTTP 1.1 browsers, the correct behaviour is noticed whether the proxy is used or not.

A packet sniffer was used to identify what was happening. On both occasions, the XML document containing the data which should be dropped into the textarea was downloaded. When passed through the proxy the ‘Connection:’ header was stripped as per http 1.1. The Connection header exists when we contact the server directly. The Content-Length was identical for both replies.

MSIE refused to use the Content-Length header to verify where the reply boundary exists. Firefox works just fine.

The fix was to break the effectiveness of the proxy and

• place a 1:1 relationship between connections from end-user browsers and requests to the server.
• honour the Connection headers in breach of RFC 2616

Neither of which I wanted to do.