Re: Fwd: [luasocket] http GET not working on luasocket 3.0 (#102)

[Diego Nehab <diego@...>]

I am all for being pragmatic. But shouldn't we draw the line somewhere? Perhaps a good place for the line is disobeying a strict requirement from an RFC and checking the server header to decide how *not* to follow the standard. This type of shenanigan may prevent server writers from ever realizing they are non-compliant. Soon there are mutually inconsistent servers and you have to resort to the server header. :) The solution I suggested is conformant and works with the server in question (which, by the way, is broken in other ways).

On Tue, Jul 15, 2014 at 5:05 PM, William Ahern <william@25thandclement.com> wrote:

On Tue, Jul 15, 2014 at 02:48:32PM -0300, Diego Nehab wrote:
> Pierre Chapuis sent us the clarification from RFC7230

> <http://tools.ietf.org/html/rfc7230#section-5.4> that requires us to use

> the "authority" in the URI. This is the solution I favor. Do you think we
> need to check the "scheme" and "port" and add the logic to exclude the
> default port even if it was included in the URI, thereby going against the
> RFC? Otherwise, I favor the fix that uses the exact "authority" in the URI,
> excluding the authentication information.
>

Using absolute URLs in the request line won't fly, either. Broken servers
which can't handle a port in the Host field are also unlikely to be able to
handle absolute URLs in the request. The issue isn't standards compliance,
it's dealing with broken servers.

I maintain a C library which implements both HTTP and RTSP, client and
server. RTSP requires absolute URLs in the request line because it has no
analog to the Host field. But I found out quickly that when sending requests
in HTTP mode I could only use relative URLs (path?query#anchor) because a
ton of services across the Internet wouldn't work properly. I can guarantee
you that using absolute URLs in the request line will cause problems,
perhaps more than including the port in the Host field.

To be widely compatible you have to follow the old mantra of "be
conservative in what you send and liberal in what you accept". There are
only a few compliant HTTP 1.1 servers (in terms of projects/vendors, not
necessarily in terms of usage). If at all. I'd be shocked if any custom,
embedded HTTP server software (other than maybe Google) was fully HTTP 1.1
compliant (including all the MIME standards, like RFC 2047, etc). Almost all
of them, for example, just use regular expressions to grok fields, which is
insufficient to parse the MIME grammar. Heck, even Nginx has holes regarding
mixtures of chunked transfers, multipart MIME entities, and/or a missing
Content-Length header.

The safe bet is to use Wireshark and take note of how the browsers operate.