Protocol Headers and Extensions

The Library provides a few powerful mechanisms to handle document metainformation and how to generate and parse additional header information coming across the network. This section describes how to handle metainformation and headers and how this can be used to experiment with existing protocols by means of additional headers. Headers are registered in a Response Object but are not parsed until they are needed. If we want to cache the object then the headers are moved to an HTAnchor Object but the headers are still not parsed before we need to.

Generating Headers

Outgoing metainformation describing preferences in requests or entities to be sent to a remote server is handled in two ways: The Library supports a "native" set (called known headers of headers which can be manipulated directly, but it also provides support for header extensions defined by the application. This section describes how both the existing set of headers and the extensions can be handled.

Generating Known Headers

The Library manages a "native" set of protocol headers which we will introduce in this section. The default behavior for the Library is to use a representative set of headers on each request but all headers can be explicitly enabled or disabled on a per request basic by the application. Here we will mainly describe the set of native headers but leave the description of how to manipulate them for the section on managing Request objects. The native set of headers fall into the following three categories:

General Headers

There are a few header fields which have general applicability for both request and response messages, but which do not apply to the communication parties or the entity being transferred. This mask enables and disables these headers. If the bit is not turned on they are not sent. All headers are optional and the default value is not to use any of these headers at all.

Request Headers

The request header fields allow the client to pass additional information about the request (and about the client itself) to the server. All headers are optional but the default behavior is to use all request headers except From and Pragma. The reason is that the former in general requires permission by the user and the latter has special meanings for proxy servers.

Entity Headers

The entity headers contain information about the object sent in the HTTP transaction. This flag defines which headers are to be sent in a request together with an entity body. All headers are optional but the default value is to use as many as possible.

As mentioned, the set of native headers are equivalent to the set of header defined by the HTTP/1.1 protocol specification. Libwww also provides functionality for registering additional headers which we will have a look at in the next section.

Generating Extension Headers

Each time a request is to be generated, the Library looks to see if a list of callback functions has been registered to provide additional metainformation to send along with the request. If this is the case then each of these callback functions will be called in turn and the resulting request is then the sum of the original response and the information provided by the callback functions.

Registrering Header Generators

Parsing Headers

The MIME parser stream parses MIME metainformation, for example generated by MIME-like protocols, such as HTTP, and NNTP. For HTTP it handles all headers as defined in HTTP/1.1 of the specification. When a header is parsed, the information is first stored in an HTResponse object.

Parsing Known Headers

The set of headers directly handled by the internal MIME parser is the reader is referred to the actual implementation in order to see the exact list. However, some of the more special headers are:

Allow

Builds a list of allowed methods for this entity

Content-Language

Builds a list of natural languages

Content-Length

This parameter is now passed

Content-Type

The Content-Type header now support the charset parameter and the level parameter, however none of them are used by the HTML parser

Date, Expires, Retry-After, and LastModified

All date and time headers are parsed understanding the following formats: RFC 1123, RFC 850, ANSI C's asctime(), and delta time. The latter is a non-negative integer indicating seconds after the message was received. Note, that it is always for the application to issue a new request as a function of any of the date and time headers..

DerivedFrom, Version

For handling version control when managing collaborative works using HTTP.

Parsing Extension Headers

In many cases, if you have registered an extra set of headers to be generated, you are also in a situation where you would like to handle the result that is returned by the remote server. As we will describe in this section, the Library provides a very similar interface to the one presented above for generating extra headers.

Each time a request is received, and a unknown header is encountered by the internal MIME parser, libwww looks to see if a list of callback functions has been registered to parse additional metainformation. In case a parser is found for this particular header, the call back is called with the header and all parameters that might follow it. As MIME headers can contain line wrappings, the MIME parser canonicalizes the header line before the callback function is called which makes the job easier for the callback function.

Registrering Header Parsers

PEP Extensions

The PEP Manager is a registry for PEP Protocols that follow the generic syntax defined by the HTTP PEP protocol headers. All PEP Protocols are registered at run-time in form of a PEP Module.

Registrering PEP Extensions