Copyright © 2009 W3C® (MIT, ERCIM, Keio), All Rights Reserved. W3C liability, trademark and document use rules apply.
This document is an editors' copy that has no official standing.
This section describes the status of this document at the time of its publication. Other documents may supersede this document. A list of current W3C publications and the latest revision of this technical report can be found in the W3C technical reports index at http://www.w3.org/TR/.
This document reflects group resolutions on comments received on the previous Last Call Working Draft.
Publication as a Group Working Draft of a proposed normative Recommendation does not imply endorsement by the W3C Membership. This is a draft document and may be updated, replaced or obsoleted by other documents at any time. It is inappropriate to cite this document as other than work in progress.
This document has been produced by the Mobile Web Best Practices Working Group as part of the Mobile Web Initiative . Please send comments on this document to the Working Group's public email list public-bpwg-ct@w3.org, a publicly archived mailing list .
This document was produced under the 5 February 2004 W3C Patent Policy. W3C maintains a public list of patent disclosures made in connection with this document; that page also includes instructions for disclosing a patent. An individual who has actual knowledge of a patent which the individual believes contains Essential Claim(s) with respect to this specification must disclose the information in accordance with section 6 of the W3C Patent Policy.
1 Introduction (Non-Normative)
1.1 Purpose
1.2 Audience
1.3 Scope
1.4 Principles
1.4.1 IAB Considerations
1.4.2 Priority of Intention
2 Terminology (Normative)
2.1 Types of Proxy
2.2 Types of Transformation
3 Conformance (Normative)
3.1 Classes of Product
3.2 Normative and Informative Parts
3.3 Normative Language for Conformance Requirements
3.4 Transformation Deployment Conformance
4 Behavior of Components (Normative)
4.1 Proxy Forwarding of Request
4.1.1 Applicable HTTP Methods
4.1.2 no-transform directive in Request
4.1.3 Treatment of Requesters that are not Web browsers
4.1.4 Serving Cached Responses
4.1.5 Alteration of HTTP Header Field Values
4.1.5.1 Content Tasting
4.1.5.2 Avoiding "Request Unacceptable" Responses
4.1.5.3 User Selection of Restructured Experience
4.1.5.4 Sequence of Requests
4.1.5.5 Original Header Fields
4.1.6 Additional HTTP Header Fields
4.1.6.1 Proxy Treatment of Via Header Field
4.2 Proxy Forwarding of Response to User Agent
4.2.1 Applicable Responses
4.2.2 User Preferences
4.2.3 Receipt of Cache-Control: no-transform
4.2.4 Use of Cache-Control: no-transform
4.2.5 Server Rejection of HTTP Request
4.2.6 Receipt of Vary HTTP Header Field
4.2.7 Link to "handheld" Representation
4.2.8 WML Content
4.2.9 Proxy Decision to Transform
4.2.9.1 Alteration of Response
4.2.9.2 Link Rewriting
4.2.9.3 HTTPS Link Rewriting
5 Testing (Normative)
A References
B Conformance Statement
C Internet Content Types associated with Mobile Content
D DOCTYPEs Associated with Mobile Content
E URI Patterns Associated with Mobile Web Sites
F Example Transformation Interactions (Non-Normative)
F.1 Basic Content Tasting by Proxy
F.2 Optimization based on Previous Server Interaction
F.3 Optimization based on Previous Server Interaction, Server has Changed its
Operation
F.4 Server Response Indicating that this Representation is Intended for the Target
Device
F.5 Server Response Indicating that another Representation is Intended for the
Target Device
G Informative Guidance for Origin Servers (Non-Normative)
G.1 Server Response to Proxy
G.1.1 Use of HTTP 406 Status
G.1.2 Use of HTTP 403 Status
G.1.3 Server Origination of Cache-Control: no-transform
G.1.4 Varying Representations
G.1.4.1 Use of Vary HTTP Header Field
G.1.4.2 Indication of Intended Presentation Media Type of Representation
H Applicability to Transforming Solutions which are Out of Scope (Non-Normative)
I Scope for Future Work (Non-Normative)
I.1 POWDER
I.2 link HTTP Header Field
I.3 Sources of Device Information
I.4 Inter Proxy Communication
I.5 Amendment to and Refinement of HTTP
J Acknowledgments (Non-Normative)
The overall objective of this document is to provide a means, as far as is practical, for users to be provided with at least a "functional user experience" [Device Independence Glossary] of the Web, when mobile, taking into account the fact that an increasing number of content providers create experiences specially tailored to the mobile context which they do not wish to be altered by third parties. Equally it takes into account the fact that there remain a very large number of Web sites that do not provide a functional user experience when perceived on many mobile devices.
The W3C Mobile Web Best Practices Working Group (BPWG) is not chartered to create new technology - its role is to advise on best practice for use of existing technology. In satisfying Content Transformation requirements, existing HTTP header fields, directives and behaviors must be respected, and as far as is practical, no extensions to [RFC 2616 HTTP] are to be used.
The recommendations in this document refer to interactions of a proxy and do not refer to any presumed aspects of the internal operation of the proxy. For this reason, the document does not discuss use of "allow" and "disallow" lists (though it does discuss behavior that is induced by the implementation of such lists). In addition it does not discuss details of how transformation is carried out except if this is reflected in interoperability. For this reason, it does not discuss the insertion or insertion of headers and footers or any other specific behaviors (though it does discuss the need for essential user interaction of some form).
The BPWG made reference to Internet Architecture Board (IAB) work on "Open Pluggable Edge Services" [RFC 3238 OPES] for various principles that underlie behavior of proxies. In this work the IAB expressed its concerns about privacy, control, monitoring, and accountability of such services.
Alteration of HTTP requests and responses is not prohibited by HTTP other than in the circumstances referred to in [RFC 2616 HTTP] Section 13.5.2 and Section 14.9.5.
HTTP defines two types of proxy: transparent proxies and non-transparent proxies. As discussed in [RFC 2616 HTTP] Section 1.3, Terminology:
"A transparent proxy is a proxy that does not modify the request or response beyond what is required for proxy authentication and identification. A non-transparent proxy is a proxy that modifies the request or response in order to provide some added service to the user agent, such as group annotation services, media type transformation, protocol reduction, or anonymity filtering. Except where either transparent or non-transparent behavior is explicitly stated, the HTTP proxy requirements apply to both types of proxies."
This document elaborates the behavior of non-transparent proxies, when used for Content Transformation in the context discussed in [CT Landscape].
There are three classes of operation on responses:
Restructuring content is a process whereby the original layout is altered so that content is added or removed or where the spatial or navigational relationship of parts of content is altered, e.g. linearization (i.e. reordering presentation elements, especially tables, so that they fit on a narrow display and can be traversed without horizontal scrolling) or pagination (i.e. splitting a document too large to be stored in or transmitted to the terminal in one piece, so that it can be nevertheless accessed by browsing through a succession of smaller interlinked documents). It also includes rewriting URIs so that subsequent requests are routed via the proxy handling the response.
Recoding content
Optimizing content
The Content Transformation Guidelines specification has one class of products:
A Transformation Deployment is the provision of non-transparent components in the path of HTTP requests and responses. Provisions that are applicable to a Transformation Deployment are identified in this document by use of the term "transforming proxy" or "proxy" in the singular or plural.
The key words must, must not, required, shall, shall not, should, should not, recommended, not recommended, may, and optional in this Recommendation have the meaning defined in [RFC 2119].
A Transformation Deployment conforms to these guidelines if it follows the statements in 4.1 Proxy Forwarding of Request, 4.2 Proxy Forwarding of Response to User Agent and 5 Testing (Normative).
A Transformation Deployment that wishes to claim conformance must make available a conformance statement B Conformance Statement that specifies the reasons for non-compliance with any clauses containing the key words should and should not, recommended and not recommended.
Proxies should not intervene in methods other than GET, POST, HEAD.
If the HTTP method is altered from HEAD to GET, proxies should (providing such action is in accordance with normal HTTP caching rules) cache the response so that a second GET request for the same content is not required (see also 4.1.4 Serving Cached Responses).
Other than to convert between HEAD and GET proxies must not alter request methods.
no-transform
directive in RequestIf the request
contains a Cache-Control: no-transform
directive, proxies must not
alter the request other than to comply with transparent HTTP
behavior defined in [RFC 2616 HTTP] sections section 14.9.5 and section 13.5.2 and to
add header fields as described in 4.1.6 Additional HTTP Header Fields below.
Note:
An example of the use of Cache-Control: no-transform
is the
issuing of asynchronous HTTP requests, perhaps by means of
XMLHttpRequest [XHR], which may include such a
directive in order to prevent transformation of both the request and the
response.
Aside from the usual caching procedures defined in [RFC 2616 HTTP], in some circumstances, proxies may paginate responses and where this is the case a request may be for a subsequent page of a previously requested resource. In this case proxies may for the sake of consistency of representation serve stale data but when doing so should notify the user that this is the case and must provide a simple means of retrieving a fresh copy.
Aside from the usual procedures defined in [RFC 2616 HTTP] proxies should not modify the values of header fields other than
the User-Agent
, Accept
, Accept-Charset
, Accept-Encoding
, and Accept-Language
header fields
and must not delete header fields.
It must be possible for the server
to reconstruct the original User Agent originated header
fields by copying directly from the corresponding
X-Device header field values (see 4.1.5.5 Original Header Fields).
Other than to comply with transparent HTTP operation, proxies should not modify any request header fields unless:
the user would be prohibited from accessing content as a result of the server responding that the request is "unacceptable" (see 4.2.5 Server Rejection of HTTP Request);
the user has specifically requested a restructured desktop experience (see 4.1.5.3 User Selection of Restructured Experience);
the request is part of a sequence of requests to the same Web site and either it is technically infeasible not to adjust the request because of earlier interaction, or because doing so preserves consistency of user experience.
These circumstances are detailed in the following sections.
Note:
In this section, the concept of "Web site" is used (rather than "origin server") as some origin servers host many different Web sites. Since the concept of "Web site" is not strictly defined, proxies should use heuristics including comparisons of domain name to assess whether resources form part of the same "Web site".
Note:
The heuristics discussed in 4.2.9 Proxy Decision to Transform relating to URI patterns are not part of the decision to alter HTTP Header Field values.
A proxy may reissue a request with altered HTTP header field values if a previous request with unaltered values resulted in the origin server rejecting the request as "unacceptable" (see 4.2.5 Server Rejection of HTTP Request). A proxy may apply heuristics of various kinds to assess, in advance of sending unaltered header field values, whether the request is likely to cause a "request unacceptable" response. If it determines that this is likely then it may alter header field values without sending unaltered values in advance, providing that it subsequently assesses the response as described under 4.2.6 Receipt of Vary HTTP Header Field below, and is prepared to reissue the request with unaltered header fields, and alter its subsequent behavior in respect of the Web site so that unaltered header fields are sent.
A proxy must not reissue a POST request with altered header fields when the response to the unaltered POST request has HTTP status code 200 (in other words, it may only send the altered request for a POST/PUT request when the unaltered one resulted in an HTTP 406 response, and not a "request unacceptable" response).
Proxies may offer users an option to choose to view a restructured experience even when a Web site offers a choice of user experience. If a user has made such a choice then proxies may alter header field values when requesting resources in order to reflect that choice, but must, on receipt of an indication from a Web site that it offers alternative representations (see G.1.4.2 Indication of Intended Presentation Media Type of Representation), inform the user of that and allow them to select an alternative representation.
Proxies should assume that by default users will wish to receive a representation prepared by the Web site. Proxies must assess whether a user's expressed preference for a restructured representation is still valid if a Web site changes its choice of representations (see 4.2.6 Receipt of Vary HTTP Header Field).
When requesting resources that are included resources (e.g. style sheets, images), proxies should
make the request for such resources with the same User-Agent
header field as the request
for the resource from which they are referenced.
For the purpose of consistency of representation, proxies
may request linked resources (e.g. those referenced
using the a
element) that form part of the same Web site as
a previously requested resource with the same header fields as the resource
from which they are referenced.
When requesting linked resources that do not form part of the same Web site as the resource from which they are linked, proxies should not base their choice of header fields on a consistency of presentation premise.
Specifically the following mapping must be used:
Original | Replacement | Ref |
---|---|---|
User-Agent | X-Device-User-Agent | RFC2616 Section 14.43 |
Accept | X-Device-Accept | RFC2616 Section 14.1 |
Accept-Charset | X-Device-Accept-Charset | RFC2616 Section 14.2 |
Accept-Encoding | X-Device-Accept-Encoding | RFC2616 Section 14.3 |
Accept-Language | X-Device-Accept-Language | RFC2616 Section 14.4 |
The X-Device-
prefix was chosen primarily on the basis
that this is a already existing convention. It is noted that the
values encoded in such header fields may not ultimately derive from a
device, they are merely received fields. The treatment of received
X-Device
header fields, which may happen where there are
multiple transforming proxies, is undefined (see I Scope for Future Work).
Irrespective of the presence of a no-transform
directive:
Via
Header FieldWhen forwarding Via
header fields, proxies should
not alter them by removing comments from them.
According to [RFC 2616 HTTP]
Section 14.45
Via
header field comments "may be removed
by any recipient prior to forwarding the message". However, the
justification for removing such comments is based on memory
limitations of early proxies. Most modern proxies do not suffer such
limitations.
Proxies should not intervene in the response if the request method was not HEAD, GET or POST.
Proxies must provide a means for users to express preferences for inhibiting content transformation. Those preferences must be maintained on a user by user and Web site by Web site basis. Proxies must solicit re-expression of preferences in respect of a server if the server starts to indicate that it offers varying responses as discussed under 4.2.6 Receipt of Vary HTTP Header Field.
Cache-Control: no-transform
If the response includes a Cache-Control: no-transform
directive
then proxies must not alter it other than to comply with
transparent HTTP behavior as described in [RFC 2616 HTTP] Section 13.5.2 and Section 14.9.5 and
other than as follows.
If a proxy determines that a resource as currently represented is likely to cause serious misoperation of the user agent then it may advise the user that this is the case and must provide the option for the user to continue with unaltered content.
Cache-Control: no-transform
Proxies may use Cache-Control: no-transform
to inhibit transformation by further proxies.
Vary
HTTP Header FieldA proxy may not be carrying out content tasting as described under 4.1.5.2 Avoiding "Request Unacceptable" Responses if it anticipates receiving a "request unacceptable" response. However, if it makes a request with altered header fields in these circumstances, and receives a response
containing a Vary
header field referring to one of the altered
header fields then it should request the resource again with
unaltered header fields. It should also update whatever heuristics
it uses so that unaltered header fields are presented first in subsequent requests
for this resource.
If the content is WML proxies should act in a transparent manner.
Note:
This does not affect the operation of proxies that are also WAP Gateways.
the content is HTML and contains <link rel="alternate" media="handheld" href=""/>
the DOCTYPE
of the content (if it has one) indicates XHTML-MP, XHTML Basic, WML or iMode as listed in D DOCTYPEs Associated with Mobile Content.
the Content-Type
has a value listed in C Internet Content Types associated with Mobile Content.
the URI of the response (following redirection or as indicated by the
Content-Location
HTTP header field) matches a pattern listed in E URI Patterns Associated with Mobile Web Sites.
a claim of mobileOK Basic [mobileOK Basic Tests] conformance is indicated (see [mobileOK Scheme] for how such a claim may be indicated);
Other factors that a proxy may take into account:
The Web site (see note) has previously shown that it is contextually aware, even if the present response does not indicate this;
the user agent has features (such as linearization or zoom) that allow it to present the content unaltered;
the response contains client side scripts that may misoperate if the resource is restructured;
the response is an HTML response and it includes
<link>
elements specifying
alternatives according to presentation media type.
Other than as noted in this section the nature of restructuring that is carried out, any character encoding alterations and what is omitted and what is inserted is, as discussed in 1.3 Scope, out of scope of this document.
If a proxy alters the response then:
It must add a Warning 214 Transformation
Applied
HTTP header field;
The altered content should validate according to an appropriate published formal grammar and if XML must be well-formed;
It should indicate to the user that the content has been transformed for mobile presentation and provide an option to view the original, unmodified content.
Proxies must not rewrite links when content transformation is prohibited.
Editorial Note: 1r: are we clear that it's just cookies and scripts?
Editorial Note: 1s: await outcome of Chaals's ACTION-969 here
The practice of intercepting HTTPS links is strongly NOT RECOMMENDED.
If a proxy rewrites HTTPS links, replacement links
must have the scheme https
.
When forwarding requests originating from HTTPS links proxies must include a Via
header field as discussed under 4.1.6.1 Proxy Treatment of Via Header Field.
When forwarding responses from servers proxies must notify the user of invalid server certificates.
See example conformance statement from Francois (link below) and his covering note
See http://www.w3.org/2005/MWI/BPWG/Group/TaskForces/CT/editors-drafts/Guidelines/ics-081107
Request resource with original header fields
If the response is a 406 response:
If the response contains Cache-Control:
no-transform
, forward it
Otherwise request again with altered header fields
If the response is a 200 response:
Otherwise assess whether the 200 response is a form of "Request Unacceptable"
Proxy receives a request for resource P that it has not encountered before
Response is a desktop oriented representation of the resource
Proxy transforms this response into content that the user agent can display well and forwards it
Proxy receives a further request for the resource P
Response is a desktop oriented representation of the resource
Proxy transforms this response into content that the user agent can display well and forwards it
Proxy receives a request for resource P, that it has previously encountered as in F.2 Optimization based on Previous Server Interaction
Proxy forwards request with altered header fields
Response is 200 OK containing a Vary: User-Agent
header field
Proxy notices that behavior has changed and reissues the request with original header fields
Response is 200 OK and proxy forwards it
Content providers may wish to follow these procedures in order to improve interoperability.
Vary
HTTP Header FieldIf a server varies its representation according to examination of
received HTTP header fields then [RFC 2616 HTTP] describes how to use the Vary
header field to indicate this.
Servers that are aware of the presence of a transforming proxy, as identified by a
Via
HTTP Header field might alter their responses according to their knowledge of specific proxy behavior. When doing so it is good practice to make sure that the
Internet content type for a response is correct for the actual content (e.g. a server should
not choose Content-Type: application/vnd.wap.xhtml+xml
because it suspects that proxies will not transform
content of this type, if its content is not valid XHTML-MP).
If a server has distinct representations that vary according to the
target presentation media type, it can inhibit
transformation of the response by including a Cache-Control:
no-transform
directive (see G.1.3 Server Origination of Cache-Control: no-transform).
In addition, in HTML content it can indicate the medium for
which the representation is intended by including a link
element identifying in its media
attribute the target
presentation media types of this representation and setting the
href
attribute to "Same-Document Reference" (see [RFC 3986] section 4.4) and in particular an empty href
attribute is a "Same Document Reference".
In addition it is good practice to include link
elements identifying the target presentation media types of other
available representations in a similar manner.
If content for more than one presentation media type is served from the same URI, it is better not to use a link element identifying the presentation media types as the URI will appear to be a "same document reference", indicating to a client that this representation is suitable for all the named presentation media types. Instead, use a Vary
HTTP header field indicating that the response varies according to the received User-Agent
HTTP header field.
Note:
Some examples of the use of the link
element are
included above in F Example Transformation Interactions.
The BPWG believes that POWDER will represent a powerful mechanism by which a server may express transformation preferences. Future work in this area may recommend the use of POWDER to provide a mechanism for origin servers to indicate more precisely which alternatives they have and what transformation they are willing to allow on them, and in addition to provide for Content Transformation proxies to indicate which services they are able to perform.
At present HTTP does not provide a mechanism for communicating original header field values. The scheme based on X-Device prefixed fields described under 4.1.5 Alteration of HTTP Header Field Values records and clarifies an approach used to achieve this effect by some content transformation proxies. This scheme relies upon non-standard HTTP fields, which are identified by their prefix as experimental according to IETF standards (notably RFC 822 and RFC 2076), and are not included in the IANA registry of HTTP header fields. While the mechanism defined in that section, based on current practice, applies to conforming transformation proxy deployments, it is possible that in future, in collaboration with the IETF, this approach will be reconsidered. This implies that the specified X-Device prefixed fields may, at some time, become deprecated in favor of new equivalent fields, or that an entirely different approach will be taken to representing such values.
A number of mechanisms exist in HTTP which might be exploited given more precise
definition of their operation - for example the OPTIONS
method and
the HTTP 300 (Multiple Choices) Status.
The editor acknowledges contributions of various kinds from members of the Mobile Web Best Practices Working Group Content Transformation Task Force.
The editor acknowledges significant written contributions from: