Authoring Techniques for XHTML & HTML Internationalization: Specifying the language of content 1.0

How to: Declare the text-processing language of the page using the lang and/or xml:lang attributes on the html tag.. Example 5 declares an HTML document to be in Canadian French:

For details of which language attribute to use, see Technique 4: Should I use the lang or xml:lang attribute?.

For details of how to use language codes, see Section 7: How to choose language values.

Discussion: Declaring the text-processing language in the html tag sets the default text-processing language for the whole document. It can be overridden for portions of the document as required. This is already important for applications such as accessibility and searching, but many other possible applications for this information may emerge over time.

For this reason you should try to always declare the text-processing language in the html tag. It is usually very easy to do when creating the content, but more difficult to retrofit when you want to take advantage of language-related features.

Note that language declarations in the HTTP header or the Content-Language meta tag should be used to describe the primary language, ie. metadata about the document as a whole, rather than the default text-processing language.

For a comparison of 'primary language' and 'text-processing language', see Section 3: Important concepts.

Most documents have one primary language, but where there are more it may not be appropriate to declare a single default text-processing language in the html tag. The relevance will depend on the structure used for the document. See Technique 2: html declarations for multilingual docs.

How to: See Technique 1: Always declare language in the html tag.

Discussion:See the definition of primary language. Documents with more than one primary language are rare. A document does not have multiple primary languages if it contains small amounts of text in another language. We are talking here about documents where the basic content is repeated and the document is simultaneously aimed at more than one linguistic audience.

The language attribute on the html tag should be used to declare the default text-processing language for the document. Given that only one language can be defined at a time as the text-processing language, there may appear to be little point in using an attribute on html if all primary languages are used in a completely unbiased way in the document. It may be more appropriate to begin labelling the language on lower level elements.

If, however, the page header information or navigation is in one particular language, or there is a bias of some other kind towards one particular language, you may still want to use a language attribute on the html tag.

[Ed. note: Should we mention the use of 'mul' as a language code? If we do, should we recommend it or not?]

How to: Where the language of the text is different from the language declared in the html tag, you should indicate this using the lang or xml:lang attributes. For example, in HTML you would write:

The lang attribute can be used on all HTML elements except applet, base, basefont, br, frame, frameset, iframe, param and script. (Note, by the way, that this means that you could use language attributes on things like bitmaps and audio files that are language specific. Such information may be particularly useful for script-based processing of documents.)

If there is no markup around the text in a different language, use a span element to delimit the boundaries. Here is an example in XHTML 1.0 served as text/html:

For details of which language attribute to use, see Technique 4: Should I use the lang or xml:lang attribute?.

For details of how to use language codes, see Section 7: How to choose language values.

How to: When serving HTML you should use the lang attribute to declare the language of the document or a range of text. For example, the following declares a document to be in Canadian French:

When serving XHTML as text/html, you should use both the lang attribute and the xml:lang attribute. The xml:lang attribute is the standard way to identify language information in XML. Example 9 shows how you would mark up the previous example for XHTML 1.0 served as text/html.

The xml:lang attribute is not actually useful for handling the file as HTML, but takes over from the lang attribute any time you treat the document as XML for, say, scripting or validation.

If you are serving XHTML 1.0 pages as XML (ie. using a MIME type such as application/xhtml+xml), or serving pages as XHTML 1.1, you do not need the lang attribute, since lang is part of the HTML language. The xml:lang attribute alone will suffice (see Example 10).

How to: Use HTTP headers and Content-Language meta elements to refer to primary language, but language attributes on the html tag to indicate the default text-processing language.

Discussion: There is generally a lot of confusion about the difference between declaring language information using Content-Language in HTTP or Content-Language meta elements, and using language attributes. In particular, much of the informal advice on the Web about how to declare the language of a document tells you to use the Content-Language meta tag to declare the language of the document. At least one popular authoring tool automatically inserts into the Content-Language meta element language information that you declare in the page properties dialog box. Unfortunately, we have yet to identify any user agent or application that recognizes information declared in this way for text-processing, whereas language information declared in the html tag is consistently recognized.

Techniques in this document recommend that Content-Language be used for describing primary language metadata, and that attributes be used for describing the default text-processing language of the document. (In fact, this question only arises when describing the language of a document at the highest level, since the language of fragments in a document can only be expressed using attributes.)

It is easy to see the rationale here when dealing with documents with multiple primary languages. The language attribute can only declare a single language at a time. Content-Language declarations, however, can declare a list of languages. Also, Content-Language declarations that declare a list of languages are not specific enough to indicate the default text-processing language.

Furthermore, it is strangely inconsistent not to use attributes to declare the default text-processing language when they have to be used for all fragments of text in a document.

The HTML specification recommends that, in the absence of a language attribute, the HTTP information be used to establish the default text-processing language. (Note that there is no mention of the Content-Language meta element in the HTML specification.)

In practise, the information contained in HTTP Content-Language headers is rarely used by mainstream browsers for language-dependent processing, and such implementation as there is is inconsistent. The behaviour of mainstream browsers also varies when multiple languages are declared in the HTTP header. The information in the Content-Language meta tag is typically not recognized at all by current user agents in a processing context.

There are still some unknowns related to the use of language information due to the currently low level of exploitation of this information. This may change in the future, particularly as the larger search engines take an increasing interest in language. For example, we may in the future see systematic use of in-document declarations of primary language using the Content-Language meta element. It may also be acceptable to infer primary language from the language attribute on the html element for documents with a single primary language. Discussion amongst various stakeholders needs to take place, however, before this can be known.

In the meantime, we recommend that you use HTTP headers and Content-Language meta elements to refer to primary language, and language attributes on the html tag to indicate the default text-processing language.

How to: See Technique 1: Always declare language in the html tag.

Discussion: The html element is the highest level element in the document, and is therefore most appropriate for declaring the default text-processing language of the document. All elements within the document will inherit that value.

The body tag is usually the wrong place to express this information because it only refers to a portion of the text in the document. For example, the text in the title element is natural language text that should also inherit the language information. If language is declared in the body element, however, this is not the case.

The only time it would make sense is when the content of the head and body elements are in different languages.

Problem: You may come across a situation where the language of the text in an attribute and the element content are in different languages. For example, the top right corner of pages on the W3C Internationalization site show links to translated alternatives (see Figure 1). The name of the language is given in the language of the target page, but a title attribute contains the name in the language of the current page:

Figure 1: An example of a scenario where the content and attribute value of an element could be in different languages.

If you create the code as shown in Example 12 below, the language attributes would actually be saying that not only the content but also the title attribute text is in Swedish. This is obviously incorrect.

How to: A better approach would involve moving the title attribute up a level in the hierarchy, since in this example the p tag inherits the default en setting of the html tag.

The markup in Example 13 lends itself easily to this approach. In other cases you may need to add a span element.

For details of which language attribute to use, see Technique 4: Should I use the lang or xml:lang attribute?.

For details of how to use language codes, see Section 7: How to choose language values.

How to: Content-Language information sent in the HTTP header is defined on the server.

Example 14 shows how the language would be declared in a Content-Language meta tag inside a document:

Discussion: The Content-Language declaration, whether it is used in the HTTP header or a Content-Language meta tag, can be useful for expressing metadata about the primary language(s) of the document being served.

Note that this is different from expressing the default language of content for text-processing, which must be done using a language attribute on the html tag.

The extent to which applications use metadata information in the HTTP header or a Content-Language meta tag, or which of the two is preferred, is not clear at this point.

Using the HTTP Content-Language header entails potential issues related to the maintenance and use of server-side information. Many authors may find it difficult to access server settings, particularly when dealing with an ISP. Also, pages may not always be located on servers. So this approach is not a solution that is always available.

For a comparison of 'primary language' and 'text-processing language', see Section 3: Important concepts.

How to: RFC 3066 is the IETF document that defines how to use language tags to identify languages. It obsoletes the RFC 1766 referred to by earlier specifications.

For an introduction to the RFC3066 rules for language codes, see Language tags in HTML and XML.

Discussion: RFC 3066 merely expands and clarifies the possibilities for specifying languages. If you have been using RFC 1766 you typically do not need to make any changes to your code in order to start using RFC 3066.

A proposed successor to RFC 3066 is currently being developed, but it aims to retain backwards compatibility with tags created using RFC 3066.

Note also that lang and xml:lang attributes only take a single language value (unlike HTTP Content-language headers).

How to: Pick the two-letter codes from ISO 639: Codes for the Representation of Names of Languages where both two- and three-letter codes exist.

Discussion: RFC3066 specifies that the two letter codes should be used where available, since this aids interoperability by ensuring that a single code is used everywhere to refer to a particular language.

This also avoids the question of which 3-letter code to use for those languages that have two 3-letter codes, since all such languages have a 2-letter code also.

How to: The IANA registry now makes available the codes zh-Hans and zh-Hant for Simplified and Traditional Chinese, respectively. The following two examples illustrate the use of these tags.

Discussion: RFC3066 specifies how to identify a language. Simplified vs. Traditional Chinese is a distinction based on script. In the past zh-CN (Chinese spoken in Mainland China) was commonly used to label Simplified Chinese, and zh-TW (Chinese spoken in Taiwan) was commonly used for Traditional Chinese. Apart from the fact that this is mislabeled, you could not guarantee that others would recognize these conventions, or even follow them. For example, some people used zh-HK to represent Traditional Chinese.

It is expected that these tags will persist for the foreseeable future, so on the one hand it would be good to use them as soon as possible in order to improve interoperability sooner rather than later.

On the other hand, you need to assess the impact of changing the tags. This is not really an issue for self-describing usage, such as with :lang for application of language-based styling. It may be more of an issue where external applications are looking for tags related to Chinese but are unaware of the zh-Hans and zh-Hant variants.

Pros: May help the reader avoid wasted time linking to pages they can't read.

Cons: May become out-of-date and so give incorrect information.

Discussion: If you add some text or graphic to a link indicating that the target document is in another language, it may allow the reader to decide in advance whether or not to follow the link, according to their language skill. If the user has to waste time following the link to find out that they cannot read the target document, this introduces fatigue, and they may lack confidence when faced with links that do go to readable pages.

There are, however, potential problems with this approach.

For example, a newly translated version may become available. Assume, for example that a French page has used this approach some time ago to point to a document which at that time was only in English. Later, the document is translated into French and language negotiation is put in place. Unless the French page referred to earlier is updated, it will now be incorrectly warning French readers that the document is in English, and possibly discouraging them from following a link to what is actually a perfectly legible document.

Pros: May help the reader avoid wasted time linking to pages they can't read; saves the author time and effort if hreflang is used consistently.

Cons: May become out-of-date and so give incorrect information; not all user agents support the necessary CSS; problematic when linking to language negotiated sites.

How to: This approach relies on CSS selectors that detect the value of the hreflang attribute and use the CSS content property to display an indicator of the language.

For example, the following link points to a page in Swedish.

The code to enable this in CSS may be something like:

This says, "For each a element with an hreflang attribute, add the value of that attribute in square parentheses after the link". You could just as easily append text or even a graphic after the link.

The markup would read as follows:

Discussion: In HTML, the hreflang attribute on an a element indicates the language of the document at the other end of the link. In practice, hreflang is typically not used by mainstream browsers. Besides that it is much better to ensure that the target document uses the language attribute in the html tag, so that this information is not needed.

A common alternative use for this attribute is to generate a visible marker attached to link text that indicates the language of the destination page for the reader. The idea is to allow the reader to decide in advance whether or not to follow the link, according to their language skill.

There are some usability-related pros and cons to this approach that are discussed in Technique 14: Pros and cons of identifying the language.

There are, also, potential technical problems with this approach.

1. Not all user agents support the CSS required to enable it (see the test results page). Internet Explorer does not support :after.

2. Note that if a resource is available in multiple languages (say you are linking from an English overview to detailed descriptions that are available in multiple languages) it is not possible to express that, since the hreflang attribute accepts only a single language as its value.

How to: A much better approach is to use text. In Technique 15: Using hreflang with CSS, Example 19 uses the actual attribute value, since these two-letter codes are typically recognizable by speakers of the language.

Discussion: Flags represent countries, not languages. There are many countries that use the same language, and numerous countries that have more than one official language.