[ contents ]

W3C

Internationalization Best Practices: Specifying Language in XHTML & HTML Content

W3C Working Draft 21 July 2006

This version:
http://www.w3.org/TR/2006/WD-i18n-html-tech-lang-20060721/
Latest version:
http://www.w3.org/TR/i18n-html-tech-lang/
Previous version:
http://www.w3.org/TR/2005/WD-i18n-html-tech-lang-20050224/
Editor:
Richard Ishida, W3C

Abstract

Specifying the language of content is useful for a wide number of applications, from linguistically sensitive searching to applying language-specific display properties. In some cases the potential applications for language information are still waiting for implementations to catch up, whereas in others, such as detection of language by voice browsers, it is a necessity today. On the other hand, adding markup for language information to content is something that can and should be done today. Without it, it will not be possible to take advantage of any future developments.

This document is one of a series of documents providing HTML authors with best practices for developing internationalized HTML using XHTML 1.0 or HTML 4.01, supported by CSS1, CSS2 and some aspects of CSS3. It focuses specifically on advice about specifying the language of content. It is produced by the Internationalization GEO (Guidelines, Education & Outreach) Working Group of the W3C Internationalization Activity.

Status of this Document

This section describes the status of this document at the time of its publication. Other documents may supersede this document. A list of current W3C publications and the latest revision of this technical report can be found in the W3C technical reports index at http://www.w3.org/TR/.

This is a Working Draft of a document produced by the Internationalization GEO (Guidelines, Education & Outreach) Working Group, part of the W3C Internationalization Activity. This is a draft document that may not fully represent the consensus of the group at this time. The Working Group expects to advance this Working Draft to Working Group Note

The document provides practical best practices related to specifying the language of content that HTML content authors can use to ensure that their HTML is easily adaptable for an international audience. These are best practices that are best addressed from the start of content development if unnecessary costs and resource issues are to be avoided later on.

A new version of the Working Draft has been published at this time to enable wide review and feedback before moving the document to Working Group Note status.

The Task Force encourages feedback about the content of this document (as well as participation in the development of the best practices by people who have experience creating Web content that conforms to internationalization needs). Send comments about this document to www-international@w3.org. The archives for this list are publicly available.

The Internationalization Working Group will not allow early implementation to constrain its ability to make changes to this specification prior to final release. Publication as a Working Draft does not imply endorsement by the W3C Membership. This is a draft document and may be updated, replaced or obsoleted by other documents at any time. It is inappropriate to cite this document as other than work in progress.

Table of Contents

Appendix

A Acknowledgements

Go to the table of contents.1 Introduction

Go to the table of contents.1.1 Who should use this document

All HTML content authors working with XHTML 1.0, HTML 4.01, XHTML 1.1, and CSS.

The term 'author' is used in the sense described by the HTML 4.01 specification, ie. as a person or program that writes or generates HTML documents.

This document provides guidance for developers of HTML that enables support for international deployment. Enabling international deployment is the responsibility of all content authors, not just localization groups or vendors, and is relevant from the very start of development. Ignoring the advice in this document, or relegating it to a later phase in the development process, will only add unnecessary costs and resource issues at a later date.

It is assumed that readers of this document are proficient in developing HTML and XHTML pages - this document limits itself to providing advice specifically related to internationalization.

Go to the table of contents.1.2 How to use this document

This document is one of several documents relating to best practices for the design of Web content using W3C technologies.

If you are new to this topic you may wish to read the document from end to end. It is, however, expected that you will typically use the document for reference purposes - dipping in to a particular section to find out how to perform a specific task with internationalization in mind. In order to support this kind of usage, an effort has been made to make each best practice stand alone, or to point to relevant cross-references. In some cases this leads to a small amount of repetitiveness.

Cross-references and additional resources are pointed to by each best practice. To check whether new resources have become available since the publication of this document, follow the links at the end of the resource sections to the techniques and topic indexes provided on the Internationalization section of the W3C site.

Editorial notes have been left in this version of the document. [Ed. note: These are marked like this].

You will also find information associated with each best practice about the applicability of recommendations to user agents (see Section 1.4: User agent support).

Go to the table of contents.1.3 Technologies addressed

This document provides best practices for developing pages using HTML 4.01, XHTML 1.0 and XHTML 1.1 with CSS.

XHTML 1.0 can be served as XML (using MIME types application/xhtml+xml, application/xml or text/xml) or HTML (using the MIME type text/html).

It is very common for XHTML 1.0 to be served as HTML, following the compatibility guidelines in Appendix C of the XHTML 1.0 specification. This allows authors to produce valid XML code. HTML represented as valid XML code lends itself to processing with such things as scripting or XSLT, but is also well supported for display by most mainstream browsers. (XHTML served as application/xhtml+xml is not well supported for browser display at the moment.)

In this document we wish to reflect practical reality for content authors, so we cover XHTML served as text/html in the best practices. Indeed we encourage the use of XHTML, and all the examples (unless trying to make a specific point about HTML 4.01) are written in XHTML.

For XHTML served as XML, this document limits its advice to documents served as application/xhtml+xml. Note that user agent support for XHTML served as XML is still patchy.

Go to the table of contents.1.4 User agent support

We try to ensure that, whenever we recommend a particular best practice, you have information immediately to hand about whether there are difficulties in implementing that recommendation for a given user agent. User agents, in the current version of this document, means a number of mainstream browsers. (The scope may grow as resources and test results become available for other user agents.)[Ed. note: Note that this version of the Working Draft is not yet completely up to date in this area.]

Where a browser operates in both standards- and quirks-mode, standards-mode is assumed (ie. you should use a DOCTYPE statement).

Generally, the best practices described will be applicable for use in current mainstream user agents. However we may also recommend things that are not yet widely supported, but are described by the standards, and hopefully will be supported given a little time. Where this is the case, or where there are other issues related to user agent support, this informatioin will be flagged by small graphics immediately after the initial summary of the best practice. We will then describe any issues in the detailed text that follows. We will assess the applicability of the best practices against the latest version of the user agent available at the time of publication.

The user agents tested, the versions, and the icons used are as follows:

  • Internet Explorer 6 (Windows) Internet Explorer icon

  • Firefox 1.5.03 Firefox icon

  • Opera 8.53 Opera icon

  • Mozilla 1.7.12 Mozilla icon

  • Netscape Navigator 8.1 Mozilla icon

  • Safari 2.0 Safari icon

Summaries for such best practices will also be worded more cautiously. For example, "Consider doing X".

If more testing is needed to ascertain whether there are issues with a particular user agent, the user agent icon will be followed by a question mark.

Detailed information may also be provided from time to time about behavior of a user agent in another version than the base or current versions.

Go to the table of contents.2 Why specify language?

Applications exist that can use information about the natural language of content to deliver to users the most relevant information, based on their language preferences. The more content is tagged and tagged correctly, the more useful and pervasive such applications will become.

Applications for language information are found in such things as authoring tools, translation tools, accessibility, font selection, page rendering, search, and scripting.

Language information should be specified for the page as a whole, and wherever language changes within the page.

There are existing applications that currently require language information, such as voice browsers in the accessibility world. In the future there will be other applications for language information, driven by developments in technology. For example, implementations of the CSS3 :first-letter pseudo-element will need language information to apply correct styling.However, we are currently faced with a circular problem. People who don't see the application of language information do not provide information about their content, and language-related applications are slow to be deployed until this information is widely available. This cycle can be broken by content authors taking steps now to declare language information. This is usually very easy to do, and carries no penalties.

Go to the table of contents.3 Important concepts

Go to the table of contents.3.1 The language of the intended audience

Metadata about the language of the intended audience is about the document as a whole. Such metadata may be used for searching, serving the right language version, classification, etc. It is not specific enough to indicate the language of a particular run of text in the document for text-processing - for example, in a way that would be needed for the application of text-to-speech, styling, automatic font assignment, etc.

The language of the intended audience does not include every language used in a document. Many documents on the Web contain embedded fragments of content in different languages, whereas the page is clearly aimed at speakers of one particular language. For example, a German city-guide for Beijing may contain useful phrases in Chinese, but it is aimed at a German-speaking audience, not a Chinese one.

It is also possible to imagine a situation where a document contains the same or parallel content in more than one language. For example, a Web page may welcome Canadian readers with French content in the left column, and the same content in English in the right-hand column. Here the document is equally targeted at speakers of both languages, so there are two audience languages. This situation is not as common on the Web as in printed material since it is easy to link to separate pages on the Web for different audiences, but it does occur where there are multilingual communities. Another use case is a blog or a news page aimed at a multilingual community, where some articles on a page are in one language and some in another.

There are also pages where the navigational information, including the page title, is in one language but the real content of the page is in another. While this is not necessarily good practice, it doesn't change the fact that the language of the intended audience is usually that of the content, regardless of the language at the top of the document source.

Metadata about the language of the intended audience is usually best declared outside the document in the HTTP Content-Language header, although there may be situations where an internal declaration using the meta element is appropriate.

Go to the table of contents.3.2 The text-processing language

When specifying the text-processing language you are declaring the language in which a specific range of text is actually written, so that user agents or applications that manipulate the text, such as voice browsers, spell checkers, or style processors can effectively handle the text in question. So we are, by necessity, talking about associating a single language with a specified range of text.

This is much more specific than the language of the intended audience.

The text-processing language is usually best declared using attributes on elements. Enclosed elements inherit the declared value, but you can, of course, override an initial declaration by specifying a different language on embedded elements where the language changes, eg. a French word in an English paragraph (see Section 5: Using attributes to declare the text-processing language).

Go to the table of contents.3.3 Relationships between language, character encoding and directionality

Language declarations in HTML and XHTML do not, and should not, provide information about character encoding or the direction of text.

'Character encoding' refers to the sequence of bytes that are used to represent characters in text. It is important to declare what encoding is being used for your document, but this is a separate issue from declaring language.

Some people think that information about language can be inferred from the character encoding, but this is not true. There must be a one-to-one mapping between encoding and language for this to work, and there isn't. A single character encoding such as ISO 8859-1 (Latin1), could encode both French and English, as well as a great many other languages. In addition, different character encodings can be used for a single language, eg, Arabic could be encoded with 'Windows-1256' or 'ISO 8859-6' or 'UTF-8'.

In some scripts, such as Arabic and Hebrew, text runs predominantly from right to left. Within that flow, numbers and text from other scripts run from left to right. It is important to adequately specify the intended directionality of text in a document.

Similarly, there is not always a one-to-one mapping between language and script, and therefore directionality. For example, Azerbaijani can be written using both right-to-left and left-to-right scripts, and the language code az can be relevant for either. In addition, when using directional markup inline, the various required values of that markup cannot be simulated using language declarations.

There are separate mechanisms for declaring character encoding and directionality in HTML and XHTML, and these ideas should not be confused with mechanisms for declaring language.

Additional techniques at the W3C Internationalization site describe how to declare character encoding and text direction.

Go to the table of contents.4 Mechanisms for declaring language in HTML

Go to the table of contents.4.1 Possible approaches

The HTML and XHTML specifications define a number of places where you can declare language. In this section we will simply show examples of the alternatives available. The next section will discuss in detail which you should use, and when.

One method is to use the lang and xml:lang attributes on an XHTML element. To set the language of a whole document, you can use this attribute on the html tag. This value will be inherited by the whole document, unless overridden by a later declaration.

Example 1: 

<html lang="en" xml:lang="en" xmlns= "http://www.w3.org/1999/xhtml">

Alternatively, you may find documents that put language information in a meta element with http-equiv set to Content-Language.

Example 2: 

<meta http-equiv="Content-Language" content="en" />

Since the meta element puts few limits on what you can say, it would also be possible, though not very common, to express language information using Dublin Core notation.

Example 3: 

<meta name="dc.language" content="en" />

Language information may also be found in the HTTP header that is sent with a document (see the last line in the following example of an HTTP header).

Example 4: 
HTTP/1.1 200 OK
Date: Wed, 05 Nov 2003 10:46:04 GMT
Server: Apache/1.3.28 (Unix) PHP/4.2.3
Content-Location: CSS2-REC.en.html
Vary: negotiate,accept-language,accept-charset
TCN: choice
P3P: policyref=http://www.w3.org/2001/05/P3P/p3p.xml
Cache-Control: max-age=21600
Expires: Wed, 05 Nov 2003 16:46:04 GMT
Last-Modified: Tue, 12 May 1998 22:18:49 GMT
ETag: "3558cac9;36f99e2b"
Accept-Ranges: bytes
Content-Length: 10734
Connection: close
Content-Type: text/html; charset=iso-8859-1
Content-Language: en

It is also worth noting that the meta element with Content-Language and the HTTP header both allow you to supply a list of values. The example below declares the languages of the intended audience of the document to be (in equal measure) German, French and Italian.

Example 5: 

<meta http-equiv="Content-Language" content="de, fr, it"/>

It is not possible to declare the language of text in CSS declarations.

The next section addresses the question of which approach is the best in what situation.

Go to the table of contents.4.2 Attributes or metadata?

There is generally a lot of confusion about the difference between declaring language information using the Content-Language field in the HTTP header or meta elements, and using a language attribute on the html element. In particular, much of the informal advice on the Web about how to declare the language of a document tells you to use the meta tag to declare the language of the document. At least one popular authoring tool automatically inserts language information that you declare in the page properties dialog box into a meta element.

Best practices in this document recommend that HTTP and the meta element be used for describing metadata about the language of the intended audience only, and that attributes be used for describing the default text-processing language of the document.

Reasons for making this distinction include:

  1. HTTP and meta declarations allow you to specify more than one language value. This is inappropriate for labelling the text-processing language, which must be done one language at a time. On the other hand, multiple language values are appropriate when declaring language for documents that are aimed at speakers of more than one language. Attribute-based language declarations can only specify one language at a time, so they are less appropriate for specifying the language of the intended audience, but they are perfect for labelling the text-processing language for text.)

  2. The language information contained in HTTP headers is rarely used by mainstream browsers for text-processing applications, and such implementation as there is is inconsistent (see the test results). Unfortunately, we have yet to identify any user agent or application that recognizes information declared in a meta tag when it comes to text-processing. On the other hand, language information declared in the html tag is consistently recognized.

  3. Since changes in the text-processing language within the document can only be done using attributes, it promotes consistency to use attributes on the html element to express the default text-processing language of the document.

  4. It is important to always declare the default text-processsing language for the document, but if the document is not read from a server, the HTTP content header will not be available.

There are still some unknowns surrounding the use of HTTP headers or meta elements to declare the language of the intended audience, due to the currently low level of exploitation of this information. This may change in the future, particularly if libraries and similar users take an increasing interest in language metadata.

When it comes to choosing between the HTTP header or the meta element for expressing information about the intended audience, there is also a lack of information on which to base any advice. In some ways the meta element may appeal, because it is an in-document declaration. This avoids potential issues if authors cannot access server settings, particularly if dealing with an ISP, or if the document is to be read from a CD or other non-HTTP source. Until more practical use cases arise, however, this is just theory.

If, in the future, we see systematic use of in-document declarations of audience language using the meta element. It may also become acceptable to infer the language of the intended audience from the language attribute on the html element for documents with a monolingual audience. Discussion amongst various stakeholders needs to take place, however, before this can be decided.

In the meantime, we recommend that you use HTTP headers and meta elements to provide document metadata about the language of the intended audience(s), and language attributes on the html tag to indicate the default text-processing language. Furthermore, we recommend that you always declare the default text-processing language.

Go to the table of contents.5 Using attributes to declare the text-processing language

Always declare the default text-processing language of the page using attributes on the html tag, unless the intended audience speaks multiple languages.
No UA applicability issues.

How to: Declare the default text-processing language of the page using the lang and/or xml:lang attributes on the html tag. Example 6 declares an HTML document to be in Canadian French:

Example 6: 

<html lang="fr-CA">

For details of which language attribute to use, see Best Practice 4: Should I use the lang or xml:lang attribute?.

For details of how to use language codes, see Section 7: Choosing language values.

Discussion: Declaring the default text-processing language is already important for applications such as accessibility and searching, but many other possible applications for this information may emerge over time.

Declaring the text-processing language in the html tag sets the default text-processing language for the whole document. It can be overridden for portions of the document as required. For this reason you should try to always declare a language in the html tag. It is usually very easy to do when creating the content, but more difficult to retrofit later in order to take advantage of language-related features.

Note that language declarations related to the intended language of the audience, ie. metadata about the document as a whole, should use the HTTP header or the meta tag, rather than an attribute on the html element. (See Section 4: Mechanisms for declaring language in HTML.)

Most documents contain content aimed at speakers of a single language, but where the intended audience is expected to read content in more than one language (eg. a multilingual blog, or a page aimed at more than one language community) it may make more sense to declare the default text-processing language lower down in the document than in the html tag. The relevance will depend on the structure used for the document. See Best Practice 2: html-based declarations for multilingual audiences.

Resources:

Background information

Reference links

Sources

More resources

Technique index - Topic index
Where a document contains content aimed at speakers of more than one language, decide whether you want to declare one text-processing language in the html tag, or leave the text-processing languages undefined until later.
No UA applicability issues.

How to: Example 7 shows a very simple document containing content aimed at multiple linguistic audiences. In this case, the document is split in two right after the body element, and the author has delayed the declaration of the text-processing language until then.

Example 7: 
<html xmlns= "http://www.w3.org/1999/xhtml"> 
<head> 
  <meta http-equiv="Content-Type" content="text/html; charset=utf-8"/> 
	 <title>Welcome - Bienvenue</title> 
	 </head> 
<body> 
	 <div lang="en" xml:lang="en">
     <h1>Welcome!</h1> 
	    <p>Lots of text in English...</p>
     </div>
  <div lang="fr" xml:lang="fr">
     <h1>Bienvenue !</h1> 
	    <p>Beaucoup de texte en français...</p>
     </div>
  </body> 
</html> 

Note: There is a problem when dealing with multilingual title elements. Only one language can be declared for this element in HTML 4.01, since the only content allowed is character data. There is currently no adequate solution for this problem. In XHTML 2.0 this problem will disappear.

For details of which language attribute to use, see Best Practice 4: Should I use the lang or xml:lang attribute?.

For details of how to use language codes, see Section 7: Choosing language values.

Discussion: See the definition of the intended language of the audience. Documents containing content aimed at an audience in more than one language are rare. A document is not aimed at a multilingual audience if it contains small amounts of text in another language. We are talking here about the languages the intended audience speaks.

Although we would normally recommended to declare the default text-processing language in the html tag, given that only one language can be defined at a time as the text-processing language, if a document has separate content to support more than one linguistic audience there may appear to be little point in doing so. It may be more appropriate to begin labelling the language on lower level elements, where the actual text is in one language or another.

If, however, the page header information or navigation is in one particular language, or there is a bias of some other kind towards one particular language in the early part of the document, you may still want to use a language attribute on the html tag, and then override it in the appropriate lower level elements.

[Ed. note: Should we mention the use of 'mul' as a language code? If we do, should we recommend it or not?]

Resources:

Background information

More resources

Technique index - Topic index
Use the lang and/or xml:lang attributes around text to indicate any changes in language.
No UA applicability issues.

How to: Where the language of the text is different from the language declared in the html tag, you should indicate this using the lang or xml:lang attributes. For example, in HTML you would write:

Example 8: 

<p>The French for <em>Cat</em> is <em lang="fr">chat</em>.</p>

The lang attribute can be used on all HTML elements except applet, base, basefont, br, frame, frameset, iframe, param and script. (Note, by the way, that this means that you could use language attributes on things like bitmaps and audio files that are language specific. Such information may be particularly useful for script-based processing of documents.)

If there is no markup around the text in a different language, use a span element to delimit the boundaries. Here is an example in XHTML 1.0 served as text/html:

Example 9: 

<p>The title in Chinese is <span lang="zh-Hans" xml:lang="zh-Hans">中国科学院文献情报中心</span>.</p>

For details of which language attribute to use, see Best Practice 4: Should I use the lang or xml:lang attribute?.

For details of how to use language codes, see Section 7: Choosing language values.

Resources:

Background information

Reference links

Sources

More resources

Technique index - Topic index
For HTML use the lang attribute only, for XHTML 1.0 served as text/html use the lang and xml:lang attributes, and for XHTML served as XML use the xml:lang attribute only.
No UA applicability issues.

How to: When serving HTML you should use the lang attribute to declare the language of the document or a range of text. For example, the following declares a document to be in Canadian French:

Example 10: 

<html lang="fr-CA">

When serving XHTML as text/html, you should use both the lang attribute and the xml:lang attribute. The xml:lang attribute is the standard way to identify language information in XML, but the browser only recognizes the lang attribute. On the other hand, when processing the document as XML, the xml:lang will be the most useful. Example 11 shows how you would mark up the previous example for XHTML 1.0 served as text/html.

Example 11: 

<html lang="fr-CA" xml:lang="fr-CA" xmlns ="http://www.w3.org/1999/xhtml">

The xml:lang attribute is not actually useful for handling the file as HTML, but takes over from the lang attribute any time you treat the document as XML for, say, scripting or validation.

If you are serving XHTML 1.0 pages as XML (ie. using a MIME type such as application/xhtml+xml), for instance serving pages as XHTML 1.1, you do not need the lang attribute, since lang is part of the HTML language. The xml:lang attribute alone will suffice (see Example 12).

Example 12: 

<html xml:lang="fr-CA" xmlns ="http://www.w3.org/1999/xhtml">

Resources:

Sources

More resources

Technique index - Topic index
Use language attributes rather than HTTP or meta elements to declare the default text-processing language.
No UA applicability issues.

How to: Declare the default text-processing language of the page using the lang and/or xml:lang attributes on the html tag. Example 13 declares an HTML document to be in Canadian French:

Example 13: 

<html lang="fr-CA">

Discussion: The basic reason is that current user agents rarely use information in the HTTP header or meta element for text-processing features, and such implementations as there are are inconsistent (see the test results).

This is explained fully in Section 4: Mechanisms for declaring language in HTML.

Resources:

Background information

Sources

More resources

Technique index - Topic index
Do not declare the default text-processing language of a document in the body element, use the html element.
No UA applicability issues.

How to: Declare the default text-processing language of the page using the lang and/or xml:lang attributes on the html tag. Example 14 declares an HTML document to be in Canadian French:

Example 14: 

<html lang="fr-CA">

Discussion: The html element is the highest level element in the document, and is therefore most appropriate for declaring the default text-processing language of the document. All elements within the document will inherit that value.

The body tag is usually the wrong place to express this information because it only refers to a portion of the text in the document. For example, the text in the title element is natural language text that should also inherit the language information. If language is declared in the body element, however, this is not the case.

The only time it would make sense is when the content of the head and body elements are in different languages.

If the text in attribute values and element content is in different languages, consider using a nested approach.
No UA applicability issues.

Problem: You may come across a situation where the language of the text in an attribute and the element content are in different languages. For example, the top right corner of pages on the W3C Internationalization site show links to translated versions of the page (see Figure 1). The name of the language is given in the language of the target page, but a title attribute contains the name in the language of the current page:

If you create the code as shown in Example 15 below, the language attributes would actually indicate that not only the content but also the title attribute text is in Swedish. This is obviously incorrect.

Example 15: An inappropriate way to label language when the attribute value and element content differ.

<p><img src="ra.png" alt="translations" /> <a xml:lang="sv" lang="sv" title="Swedish" href="index.sv.html">svenska</a></p>

How to: A better approach would be to move the title attribute up a level in the hierarchy, since in this example the p tag inherits the default en setting of the html tag.

Example 16: A better way to label language when the attribute value and element content differ.

<p title="Swedish"><img src="ra.png" alt="translations" /> <a xml:lang="sv" lang="sv" href="index.sv.html">svenska</a></p>

The markup in Example 16 lends itself easily to this approach. In other cases you may need to add a span element, to have somewhere to attach the title attribute.

For details of which language attribute to use, see Best Practice 4: Should I use the lang or xml:lang attribute?.

For details of how to use language codes, see Section 7: Choosing language values.

Resources:

Sources

More resources

Technique index - Topic index

Go to the table of contents.6 Using metadata to specify the language of the intended audience

Consider using a Content-Language declaration in the HTTP header or a meta tag to declare metadata about the language(s) of the intended audience of a document.
No UA applicability issues.

How to: Content-Language information sent in the HTTP header is defined on the server. The method for setting that up is server-specific and is not discussed here.

Alternatively, you can add a Content-Language declaration in a meta element to the head of your document, as shown in Example 17).

Example 17: 

<meta http-equiv="Content-Language" content="en"/>

Discussion: The Content-Language declaration, whether it is used in the HTTP header or a Content-Language meta tag, can be useful for expressing metadata about the language(s) of the intended audience of the document being served.

Note that this is different from expressing the default language of content for text-processing, which must be done using a language attribute on the html tag (see Best Practice 1: Always declare language in the html tag).

The extent to which applications use metadata information in the HTTP header or a meta tag, or which of the two is preferred, is not clear at this point.

Using Content-Language in the HTTP header entails potential issues related to the maintenance and use of server-side information. Many authors may find it difficult to access server settings, particularly when dealing with an ISP. Also, pages may not always be located on servers. So this approach is not a solution that is always available.

Sometimes a server has been set up to automatically serve a language-specific version of a resource based on the user's browser settings (content negotiation). In this case, your server is likely to send language information in the Content-Language header.

For further discussion of this topic, see Section 3: Important concepts and Section 4: Mechanisms for declaring language in HTML.

Resources:

Reference links

Sources

More resources

Technique index - Topic index

Go to the table of contents.6.1 Documents with multilingual audiences

It is not common to find pages on the Web aimed at a multilingual audience. One reason is that it is easy to link to alternative pages instead. On the other hand, such pages do exist, particularly when the target audience is bilingual.

For documents with content aimed at a multilingual audience, use Content-Language with a comma-separated list of language tags.
No UA applicability issues.

How to: Content-Language information sent in the HTTP header is defined on the server. The HTTP specification provides for more than one language to be expressed as the value of the Content-Language header.

Example 18 shows part of the HTTP header sent from the server and declares a document to be aimed at speakers of three languages: German, French and Italian:

Example 18: 

Content-Language: de,fr,it

The in-document Content-Language meta element provides a similar possibility (see Example 19):

Example 19: 

<meta http-equiv="Content-Language" content="de,fr,it"/>

Resources:

Sources

More resources

Technique index - Topic index
For documents aimed at a multilingual audience, try to divide the document at the highest possible level, and declare the appropriate text-processing language in those blocks.
No UA applicability issues.

Dividing parallel text at the highest possible level, can simplify the process of guiding users to the text via searching, links, etc. It also reduces the work of labeling the language of document fragments.

For details of how to use language attributes, see the section Section 7: Choosing language values.

Go to the table of contents.7 Choosing language values

Follow the guidelines in the IETF's RFC3066bis or its successors for language attribute values.
No UA applicability issues.

[Ed. note: This best practice refers to RFC3066bis, since it is now in force. When the IETF assigns RFC3066bis its own number, that will be used here.]

How to: RFC3066bis, Tags for the Identification of Languages, is the IETF document that defines how to use language tags to identify languages.

For an introduction to the RFC3066bis rules for composing language codes, see Language tags in HTML and XML. [Ed. note: This will shortly be updated to reflect the changes in RFC3066bis.]

Note that lang and xml:lang attributes only take a single language value (unlike HTTP Content-language headers).

Discussion: Using RFC3066bis as a common reference for defining language tags ensures that your tags will be recognized widely.

Note: Each new RFC has a number which is typically not related to the number of any RFC it replaces and obsoletes. The original IETF specification that described values for language tags was RFC 1766. This was then obsoleted by RFC 3066. A new version of the specification, RFC3066bis, has, however, already been approved by the IETF. RFC3066bis is currently awaiting publication and assignment of a number. Nevertheless, it and its associated Language Subtag Registry are now in force, and should be used for language values.

The HTML specification still recommends the use of RFC 1766 for identifying language. However, there is a planned erratum in place for the HTML specification, so despite what the HTML specification currently says, you should use RFC 3066bis or its successor when that is published.

RFC 3066bis merely expands and clarifies the possibilities for specifying languages. If you have been using RFC 1766 or RFC 3066 you do not need to make any changes to your code in order to start using RFC 3066bis. Successors to RFC 3066bis will also retain backwards compatibility with tags created using RFC 3066bis.

Resources:

How to's

Reference links

More resources

Technique index - Topic index
Use the shortest possible language tag values.
No UA applicability issues.

How to: The golden rule when creating language tags is to keep the tag as short as possible. Avoid region, script or other subtags except where they add useful distinguishing information. For instance, use ja for Japanese and not ja-JP, unless there is a particular reason that you need to say that this is Japanese as spoken in Japan.

Similarly, do not use script or variant codes unless they are needed to correctly distinguish your content from something else. Although RFC 3066bis introduces script tags, as RFC 3066bis co-author, Addison Phillips, writes, "For virtually any content that does not use a script tag today, it remains the best practice not to use one in the future".

In the past, people tended to wonder which ISO language code to choose, since there are often 2-letter and 3-letter alternatives for the same language (and sometimes two 3-letter alternatives). Although there were clear rules about this in RFC3066, this question is now moot because you should only use language tags specified in the IANALanguage Subtag Registry, and only one subtag exists per language in that registry (the shortest one).

Resources:

How to's

Reference links

More resources

Technique index - Topic index
Where possible, use the codes zh-Hans and zh-Hant to refer to Simplified and Traditional Chinese, respectively.
UA applicability issues for:   Internet Explorer (Windows)  

How to: Use zh-Hans and zh-Hant for Simplified and Traditional Chinese, respectively, in language attribute values, and possibly also for Content-Language values. These codes are available from the IANA Language Subtag Registry.

Example 20: Simplified Chinese:

<p lang="zh-Hans" xml:lang="zh-Hans">当世界需要沟通时,请用统一码!</p>

Example 21: Traditional Chinese:

<p lang="zh-Hant" xml:lang="zh-Hant">當世界需要溝通時,請用統一碼!</p>

Discussion: Simplified vs. Traditional Chinese is a distinction based on script. Until recently there was no provision for using script information in language tags, so zh-CN (Chinese spoken in Mainland China) was commonly used to label Simplified Chinese writing, and zh-TW (Chinese spoken in Taiwan) was commonly used for Traditional Chinese writing. Apart from the fact that this is mislabeled, you could not guarantee that others would recognize these conventions, or even follow them. For example, some people used zh-HK to represent Traditional Chinese.

You should start using the new tags as soon as possible in order to introduce widespread interoperability as soon as possible. There is already substantial use of these codes.

On the other hand, you may need to assess the impact of changing the tags. This is not really an issue for self-describing usage, such as with :lang for application of language-based styling. It may be more of an issue where external applications are looking for tags related to Chinese but are unaware of the zh-Hans and zh-Hant variants.

UA issues: There is one particular area where this may be an issue for the display of text on a user agent. Some user agents use language information to automatically choose a font for CJK ideographic text. Note that this assumes that the following conditions hold:

  1. you have appropriate fonts set in your preferences,

  2. the document styling does not apply a font, and that

  3. the user agent supports this behavior (not all do).

So this is a fairly limited scenario.

The following summarizes support for this feature in the user agents tested for this document at the time of writing. See the test results page for more details and latest results.

Opera and Safari don't support this automatic font assignment. Firefox, Mozilla and Netscape handle zh-Hans and zh-Hant as you would expect. IE6, however, applies the default font, which is Japanese. (It is expected that IE7 will support these codes, however).

Note that Firefox, Mozilla and Netscape also allow you to set a different setting for Traditional Chinese in Taiwan and Hong Kong. They use the Taiwan font for zh-Hans and zh-TW. They use the Hong Kong font setting for zh-HK.

Resources:

How to's

Reference links

Test data

More resources

Technique index - Topic index

Go to the table of contents.8 Indicating the language of a link destination

When pointing to a resource in another language, consider the pros and cons of indicating the language of the target document.
No UA applicability issues.

Pros: May help the reader avoid wasted time linking to pages they can't read.

Cons: May become out-of-date and so give incorrect information.

Discussion: If you add some text or graphic to a link indicating that the target document is in another language, it may allow the reader to decide in advance whether or not to follow the link, according to their language skill. If the user has to waste time following the link to find out that they cannot read the target document, this introduces fatigue, and they may lack confidence when faced with links that actually do go to readable pages.

There are, however, potential problems with this approach if a newly translated version becomes available. Assume, for example that a French page has used this approach some time ago to point to a document which at that time was only in English. Later, the document is translated into French and language negotiation is put in place. Unless the French page referred to earlier is updated, it will now be incorrectly warning French readers that the document is in English, and possibly discouraging them from following a link to what is actually a perfectly legible document.

If you want to indicate that the target document of an a element is in another language, consider the pros and cons of using hreflang with CSS.
UA applicability issues for:   Internet Explorer (Windows)  

Pros: May help the reader avoid wasted time linking to pages they can't read; saves the author time and effort if hreflang is used consistently.

Cons: May become out-of-date and so give incorrect information; not all user agents support the necessary CSS; problematic when linking to language negotiated sites.

How to: This approach relies on CSS selectors that detect the value of the hreflang attribute and use the CSS content property to display an indicator of the language.

For example, the following link points to a page in Swedish.

Example 22: 

There is also a page describing why a DOCTYPE is useful [sv].

The code to enable this in CSS may be something like:

Example 23: 

a[hreflang]:after { content: " [" attr(hreflang) "] "; }

This says, "For each a element with an hreflang attribute, add the value of that attribute in square parentheses after the link". You could just as easily append text or even a graphic after the link by associating it with the content property, rather than the attr(hreflang).

The markup would read as follows:

Example 24: 

<p>There is also a page describing <a href="swedish-doc.html" hreflang="sv">why a DOCTYPE is useful</a>.</p>

Discussion: In HTML, the hreflang attribute on an a element indicates the language of the document at the other end of the link. In practice, hreflang is typically not used by mainstream browsers. Besides that it is much better to ensure that the target document uses the language attribute in the html tag, so that this information is not needed.

It is perhaps (slightly) more common to use this attribute to generate a visible marker attached to link text that indicates the language of the destination page for the reader. The idea is to allow the reader to decide in advance whether or not to follow the link, according to their language skill.

There are some usability-related pros and cons to this approach that are discussed in Best Practice 14: Pros and cons of identifying the language of a target document.

There are, also, potential technical problems with this approach when using Internet Explorer (see below). The fact that IE doesn't support this is unfortunate, given its market share. It doesn't break the page, however, on IE. The user simply doesn't see this information. This means that as long as the information is not critical for the user, you can still use this technique and it will provide an enhanced user experience on the other browsers.

Note also that if a resource is available in multiple languages (say you are linking from an English overview to detailed descriptions that are available in multiple languages) it is not possible to express that, since the hreflang attribute accepts only a single language as its value.

UA issues: The following summarizes support for this feature in the user agents tested for this document at the time of writing. See the test results page for more details and latest results.

Internet Explorer does not support the :before, :after selectors, or the content property.

The approach works fine for all other user agents.

Resources:

How to's

Sources

Test data

More resources

Technique index - Topic index
Do not use flag icons to indicate languages.
No UA applicability issues.

How to: A much better approach is to use text. In Best Practice 15: Using hreflang with CSS, Example 22 uses the actual attribute value, on the assumption that these two-letter codes are typically recognizable by speakers of the language.

Discussion: Flags represent countries, not languages. Numerous countries use the same language as another country, and numerous countries have more than one official language.

Go to the table of contents.A Acknowledgements

The following members of the GEO Working Group and the former GEO Task Force have contributed their time and valuable comments to shaping these best practices:

Phil Arko (Siemens), Steve Billings, Deborah Cawkwell (BBC World Service), Wendy Chisholm (W3C WAI), Andrew Cunningham (State Library of Victoria), Martin Dürst (Aoyama Gakuin University), Lloyd Honomichl, Susan K. Miller (Boeing), Russ Rolfe (Microsoft), Peter Sigrist, Tex Texin (Yahoo), Najib Tounsi (Ecole Mohammadia d'Ingénieurs)