URI Schemes and Web Protocols

3.1 Scenario: an existing http resource adds video capability

An http resource named http://example.org/todaysNews is deployed using HTTP. It is a popular source of news, and is widely bookmarked by users of the Web. HTTP is used to provide representations of the news reports in one or more traditional media types such as application/xhtml+xml, text/plain, application/pdf, etc. The video industry creates a new VIDEO protocol for managing high quality video on the Web. It provides streaming and frame-rate control capabilities beyond those available with HTTP, and is in any case a completely different protocol. The owner of http://example.org/todaysNews wishes to provide a high quality video feed of the news resource, while continuing to support HTTP access for the traditional media types. At least three approaches are licensed by this finding:

Approach 1: a new "video" scheme is created by the inventors of the VIDEO protocol. The owner of http://example.org/todaysNews creates a second URI such as video://example.org/todaysNews, which is used for references to the video feed (see G4. Serve using the protocol associated with the URI scheme). Browsers follow the common practice of choosing protocol handlers based on the scheme (see G8. Access using the protocol associated with the URI scheme), correctly choosing HTTP for the original URI and the VIDEO protocol for the new one. Unfortunately, existing bookmarks do not successfully access the video feed, emails pointing to the news feed might require two links, semantic web statements about one resource do not in general apply to the other, and so on.

Approach 2: the VIDEO protocol supports operations on existing URI schemes including http (see G5. Deploy new protocols using existing schemes and G6. Provide protocol support for the http scheme). Browsers have the option to dereference the existing bookmarks using either protocol, but no standard means is established for choosing which protocol to try first (see R7. Try any protocol for any resource). A given browser might set a fixed choice using configuration options, might provide a per-link selection in the user interface, or might after an initial successful interaction default to using the same protocol for subsequent references. This approach has the advantage that there is only one URI for the resource, so existing bookmarks and Semantic Web statements work. It has the disadvantage of complicating the configuration of browsers, and of failing to honor the tradition that URI schemes designate protocols or protocol families.

Approach 3: an initial request is in all cases made using the HTTP protocol. The result is either a representation of the resource in some traditional media type such as application/xhtml+xml, or a video deployment descriptor in some new media type such as application/videoDeploymentDescriptor+xml. As is customary, the browser triggers a handler based on the media type, and the handler for application/videoDeploymentDescriptor+xml in turn accesses the video stream itself using the VIDEO protocol. (This approach is discussed in G6. Provide protocol support for the http scheme). Like approach 2, this one also has the key advantage that existing links work and each resource can be referenced with a single URI. It has the further advantage that the means by which protocols are activated is covered by normative specifications, rather than by unspecified logic that might vary from one user agent to another, and it preserves the convention that the http scheme is directly associated only with protocols in the HTTP family (currently HTTP 1.0 and HTTP 1.1). The principle drawbacks of approach 3 are that it requires the availability of an HTTP implementation even to access the video stream, and more importantly, that it requires a round-trip to the HTTP server before even attempting to access the VIDEO server.

The rules and guidelines in this finding license all three approaches. In examples such as the one described here, approach 3 is preferred whenever the need for HTTP access and the associated round-trip delays are acceptable. Approach 1 may be desirable in situations where the uses of the new protocol and the old mostly for distinct resources. In such cases, approach 1 preserves the tradition that both user agents and users can infer the protocol from the URI scheme.

3.2 Scenario: Accessing http resources using a peer-to-peer protocol

A government committee is appointed to investigate a leading political figure, and the committee's report is to be published on the web as an XHTML document named with a URI in the http scheme. Past experience suggests that the servers will be subject to millions of "hits" during the first few hours after publication, a load beyond the capability of the available servers using HTTP. Accordingly, the document is also to be published using a scalable peer-to-peer protocol.

Rule R1. Serve using any protocol makes clear that an http resource can in principle be published using protocols other than HTTP, but rule R2. Serve faithfully requires that good faith efforts be made to serve resources accurately and reliably. In other words, the publisher has the responsibility to ensure that the protocols used will in fact serve correct representations of the intended resource. Many peer-to-peer protocols have the characteristic that the content they serve is based on a distributed agreement among those using the network. For example, in some networks the representations served for a resource may be determined by voting among all representations that have been offered by any user of the network. In the case of a controversial document such as the investigative report, there might be a risk that erroneous or "spoofed" copies would be inserted to corrupt the system. Thus, the P2P protocol is appropriate only to the extent that the publisher can in good faith rely on it to faithfully serve the resource.

3.3 Scenario: Accessing https resources using a peer-to-peer protocol

Consider the scenario just discussed, but modified so that the investigative report is to be named with an https URI. To a significant degree, the considerations remain the same, but R3. URI scheme sets expectations for integrity of access explains that the scheme chosen can set expectations for the integrity of access to resources. In this regard, the two scenarios are different.

The https scheme is a URI space in which resolutions must be authenticable (is this a word??), typically using Public Key Infrastructure. The P2P protocol is thus suitable only insofar as it enforces such integrity checking in a manner that is true to the spirit of https resource resolution. In principle, the same problem exists for the http-based scenario, but the expectations for http integrity are lower. Note that in some cases, checksums or other out-of-band mechanisms can be used to ensure the integrity of accesses to resources even over otherwise unreliable networks; the checksums do not guarantee that correct results will be received, but they can greatly reduce the chances that incorrect information will be trusted.

3.4 Scenario: Configuring browsers to use the right protocol

The scenarios above make clear that multiple protocols may on occasion be used to access resources from a given URI scheme. This scenario involves configuring a browser or other user agent to use the correct protocol(s) for accessing resources.

Q.If the scheme does not uniquely determine a protocol, then how is it that user agents know what protocol to use?

A.Guideline G4. Serve using the protocol associated with the URI scheme makes clear that the tradition of associating a scheme with at most one protocol, or failing that with a set of versions of the same protocol, is important. Thus G8. Access using the protocol associated with the URI scheme establishes the heuristic that, unless otherwise specified, user agents should use (a suitable version of) the protocol associated with a given scheme. Thus, the default for accessing an ftp resource should be to use the FTP protocol, and so on.

The means by which a user agent selects among multiple versions from the same family is not specified by Web architecture. Indeed it is common, if not entirely appealing, to find expert-level user agent switches to control whether HTTP 1.0 or HTTP 1.1 should be used for http resources. Similarly, as discussed above, configuration controls are likely to be needed in cases where protocols from different families are used to access resources in the same scheme.

3.5 Scenario: A new scheme to launch a media player

A provider of multimedia on the web wishes to use its own software for "playing" content in user agents. The content is delivered using the HTTP protocol, but named using a new media: scheme. Browser plugins are registered to handle the new scheme, to render the desired user interface, and to use HTTP to retrieve content as necessary.

This is an inappropriate use of a new scheme. As discussed in [AWWW], scheme names, especially those that are short, easy to remember, and easy to type, are a scarce resource on the web. Having been allocated for the convenience of the example media player, the media: scheme is unavailable for any other use. Schemes are properly introduced to enable Web access to new information spaces and sometimes to enable new network protocols; new schemes should not be allocated merely to facilitate invocation of user-agent software. As discussed in 4.2.3 G6. Provide protocol support for the http scheme, a better approach would be to name the media resources using the http scheme, and to return either the media files or descriptors of them using an identifiable media type such as application/mediaDescriptor+xml. User agent software is generally capable of dispatching handlers based on the media type returned from an HTTP GET.

3.6 Scenario: Configuring gateway proxies

Although the HTTP protocol is primarily used for http resources, it can be and often is used with others as well. For example, HTTP is commonly used as a gateway protocol to access ftp resources. Often, an intermediate server known as a gateway proxy is used to convert between the two protocols:

Q.Should the ftp URI be used as the HTTP Request-URI, or should an http-scheme URI be created for each ftp resource to be accessed?

A.R1. Serve using any protocol makes clear that an ftp resource can in principle be served using HTTP, and indeed HTTP provides for ftp- or other-scheme URIs to be carried as Request-URIs. Using the same URI on both "hops" avoids the need to create two names for the same resource, but with the drawback that user agents may require special configuration to access ftp resources using HTTP. Rule R7. Try any protocol for any resource establishes that user agents have permission to access an ftp resource using HTTP, but there is no standard means of establishing when to do so.

In contrast, allocating a second http URI for each resource preserves the illusion of an "all http" world for the many software tools supporting that scheme, and in particular user agents can use the traditional heuristic of choosing a protocol based on the URI scheme (see G8. Access using the protocol associated with the URI scheme).

The gateway can thus be implemented using either of these conventions. In general, the [AWWW] admonition in to avoid multiple names is strong, suggesting that when in doubt the first option should be chosen.

3.7 Scenario: Web services gateway scenario

[To be supplied - this will be another gateway scenario, similar in physical arrangement to the one above, but the role of ftp will be played by a robust queuing system such as WebSphere MQ. We assume for the scenario that the queuing service has its own scheme queue: for the resources that it manages. A web service (on the far right) will "mint" an Endpoint Reference to a resource stored in the server, but clients on the left will send messages to that resource using HTTP. The Web Services gateway in the middle will translate from http to the queuing system. The scenario will explore the question of whether the [address] property in the endpoint reference MAY/MUST/SHOULD/MUSTNOT be in the queue scheme. In practice, the reasoning is no different than for the FTP gateway, but the applicability to Web Services may be interesting. I'll add it later if the rest of the finding meets with approval -- Noah]

4.1 Rules for Servers

4.2 Guidelines for Servers

The following guidelines should be considered when deploying Web servers:

4.3 Rules for User Agents

The following rule applies to user agents

4.4 Guidelines for User Agents

The following are guidelines for user agents:

Note that rule R7 and guideline G8 are not contradictory. The rule specifies that it is safe for a user agent to attempt to manipulate resources using any protocol that has appropriate characteristics; the guideline suggests that the common practice of associating schemes with protocols should be exploited wherever practical.

5.1 The relationship of operations to schemes and protocols

The operations available on a resource are determined by the protocol(s) used for access, not by the URI scheme employed. In 3.1 Scenario: an existing http resource adds video capability, for example, an http resource is accessible through both the HTTP protocol and/or the VIDEO protocol. When the VIDEO protocol is used, the operations available are the ones provided by that protocol. Guideline G4. Serve using the protocol associated with the URI scheme suggests that, where practical, resources be served using the protocol associated with the scheme. In such cases, because the scheme determines the protocol, the suite of available operations can indeed be inferred from the scheme. Thus, http resources frequently (but not always) support the HTTP operations of GET, PUT, POST, etc.

5.2 Consistency of operations across protocols

User agents often are called upon to provide consistent user interfaces for resources accessed using a variety of protocols. Although each protocol can in principle support its own operations, consistency is easier to maintain when protocols support the same or similar operations. It is highly desirable, for example, that any new protocols used for information retrieval support operations as close as possible to HTTP GET. This is an example of the Principle of Orthogonality (see [AWWW]). So, a high definition streaming video protocol may not support exactly an HTTP "Get", but if it supports something similar then a browser can provide a consistent navigation experience as one goes from a web page to a movie and back. Similarly, if a peer-to-peer protocol supports retrieval of media typed octet streams, then browsers can use existing renderers, caches, etc.

5.3 Protocol operations vs. API operations

In HTTP, GET is visible both as a browser operation and on the wire. When a link is clicked, the browser typically invokes a local operation that results in an HTTP "GET" message being transmitted to the server. The distinction between local and wire-level operations becomes important when new sorts of protocols are integrated into the Web. Many peer-to-peer networks support information retrieval that is very close to the service provided by HTTP GET, but the resulting wire-level messages are often very different. In a peer-to-peer network, the messages resulting from such a retrieval may be sent to multiple servers, and may or may not directly relate to the information being retrieved. For example, in some peer systems, the messages will have to do with negotiating caching policies, bandwidth commitments, etc. The data actually received may multiplex information from multiple resources. Thus, in considering the compatibility goal set out in 5.2 Consistency of operations across protocols, it is often the abstract API-level operations for which consistency is important. In most cases, the messages transmitted "on the wire" are private to the protocol.