Latin Script Gap Analysis

Abstract

This document describes and prioritises gaps for the support of the Latin script on the Web and in eBooks. In particular, it is concerned with text layout. It checks that needed features are supported in W3C specifications, such as HTML and CSS and those relating to digital publications. It also checks whether the features have been implemented in browsers and ereaders.

This document was created by Richard Ishida.

Other contributors can be found in the GitHub contributors list.

The W3C needs to make sure that the needs of scripts and languages around the world are built in to technologies such as HTML, CSS, SVG, etc. so that Web pages and eBooks can look and behave as people expect around the world.

This page documents difficulties that people encounter when trying to use languages written in the Latin script on the Web.

Having identified an issue, it investigates the current status with regards to web specifications and implementations by user agents (browsers, e-readers, etc.), and attempts to prioritise the severity of the issue for web users.

This document not only describes gaps, it also attempts to prioritise them in terms of the impact on the local user. The prioritisation is indicated by colour.

Key:

It is important to note that these colours do not indicate to what extent a particular feature is broken. They indicate the impact of a broken or missing feature on the content author or end user.

Basic styling is the level that would be generally accepted as sufficient for most Web pages. Advanced level support would include additional features one might expect to include in ebooks or other advanced typographic formats. There may be features of a script or language that are not supported on the Web, but that are not generally regarded as necessary (usually archaic or obscure features). In this case, the feature can be described here, but the status should be marked as OK.

The decision as to what priority level is assigned to a described gap is down to the experts doing the gap analysis. It may not always be straightforward to decide. If a given section in this document refers to more than one feature that is broken, each with different impacts on Web users, the priority for the section should be the lowest denominator.

A cell can be scored as OK if the feature in question is specified in an appropriate specification, and is supported by user agents. A specification that is in CR or later and has two implementations in 'major' browsers will count. This means that the feature may not be supported in all browsers yet. (At some point in the future we may try to distinguish, visually, whether support is available in a specification but still pending in major browsers or applications.)

See also General page layout & progression for features such as column layout, page turning direction, etc. that are affected by text direction.

Are the script requirements for vertically oriented text met? What about if you mix vertical text with scripts that are normally only horizontal? Do you need a switch to use different characters in vertical vs. horizontal text? Does the browser support short runs of horizontal text in vertical lines (tate-chu-yoko in Japanese) as expected? Is the orientation of characters and the directional ordering of characters supported as needed?

GitHub issue #11

Languages: el fr de hu ka nl ca no ru uk uz bg sr az-cyrl 2

This issue is applicable to most languages.

Text that is normally written horizontally may need to be written vertically for special effects (the spine of a book, table column headings, etc). Typographically, it is simply horizontal text that is rotated.

The CSS feature defined in Writing Modes Level 4, which is in CR, provides a solution using sideways-lr|sideways-rl values of the writing-mode property.

There is no way to do this effectively until browsers support the new CSS properties.

writing-mode:vertical-lr/rl is not appropriate for achieving this, because of things such as wrong locations for line start and end, and incorrect direction of line wrapping, not to mention the implications for glyph rotation.

Only Gecko supports these values. Blink and WebKit do not.

Support is needed sooner rather than later, since many people keep writing tutorials or explainers that show how to do this using the currently supported writing-modes values. However, the currently supported values only work properly for CJK languages, and produce incorrect results for normally horizontal scripts unless used in the simplest way possible (for example, line breaking and text alignment break things).

The sideways values were moved from CSS Writing Modes 3 to level 4 because implementation support was there for the values needed for CJK. However, CSS Writing Modes 4 went to CR almost immediately with the expectation that support for the sideways features would follow soon.

interactive test, sideways-rl is supported in browsers

interactive test, sideways-lr is supported in browsers

Chromium • WebKit

tbd

If this script runs right-to-left, are there any issues when handling that? Is bidirectional text adequately supported? What about numbers and expressions? Do the Unicode bidi controls and HTML markup provide the support needed? Is isolation of directional runs problematic?

Do the standard fallback fonts used in browsers (eg. serif, sans-serif, cursive, etc.) match expectations? Are special font or OpenType features needed for this script that are not available?

Does the script in question require additional user control features to support alterations to the position or shape of glyphs, for example adjusting the distance between the base text and diacritics, or changing the glyphs used in a systematic way? Do you need to be able to compose/decompose conjuncts, or show characters that are otherwise hidden, etc?

This covers ways of modifying the glyphs, such as for italicisation, bolding, oblique, etc. Do italic fonts lean in the right direction? Is synthesised italicisation problematic? Are there other problems relating to bolding or italicisation - perhaps relating to generalised assumptions of applicability?

If this script is cursive (eg. Arabic, N’Ko, Syriac, etc), are there problems or needed features related to the handling of cursive text? Do cursive links break if parts of a word are marked up or styled? Do Unicode joiner and non-joiner characters behave as expected?

Does your script need special text transforms that are not supported? Does your script convert letters to uppercase, capitalised and lowercase alternatives according to your typographic needs? Do you need to to convert between half-width and full-width presentation forms?

GitHub issue #32

Languages: ca 3

This issue is applicable to Catalan.

The sequence <l·l> in Catalan orthography signals a geminated /l/ sound. The midpoint dot should not lead to capitalisation of the second l. For example, a capitalisation of cancel·lar should produce Cancel·lar and not Cancel·Lar.

Wikipedia, Punt volat (middot)

This punctuation mark is also used in a similar way for other languages, though between different letters. See examples.

Blink and Webkit both do the right thing here, but Gecko produces Cancel·Lar.

Incorrect text-transform capitalize behavior for Catalan geminate Ls #8987

This is a significant problem for Catalan users wanting to apply text-transform:capitalize. However, that is not something users do particularly frequently. Therefore marking this as Advanced.

Interactive test, text-transform:capitalize doesn't uppercase a word-medial long L in Catalan, whether written as 2 or 3 characters

Interactive test, text-transform:capitalize doesn't uppercase a word-medial letter following middle dot in words from various languages.

Gecko • dup Gecko

Gecko fixed this in version 116.

GitHub issue #2

Languages: nl 1

This issue is specific to Dutch.

Capitalisation should convert 'ij' at the start of a word to 'IJ', for example 'ijsland' should become 'IJsland'.

Tests & results: i18n test suite, Tailoring, Dutch.

When the text is labelled as Dutch, Gecko passes, but Blink and Webkit fail.

Browser bug reports: Blink • Webkit

Priority: This is a basic issue for Dutch.

Are there specific problems related to punctuation or the interaction of the text with punctuation (for example separation of punctuation from previous text, but allowing no line break between)? Are there issues related to handling of abbreviation, ellipsis, or iteration? Are there problems related to bracketing information or demarcating things such as proper nouns, etc?

Are there any issues when dealing with quotations marks, especially when nested? Should block quotes be indented or handled specially?

GitHub issue #22

Languages: el fr de hu ka nl ca no ru uk uz bg sr az-cyrl 2

This issue is common across all languages that use the q element.

When an English page contains a quotation in another language, the quotation marks used around that quotation (and inside it for embedded quotes) should be the English ones – not those of the language of the quotation. The same applies for other languages.

Currently, if the language of the quotation is declared on the q tag in HTML and that tag has a lang attribute, browsers instead set the quotation marks based on the language of the quote.

Quotations work fine in a sentence that is all in the same language. For example, the markup for this Georgian text:

<span lang="ka">ერთი <q>ორი <q>სამი</q></q></span>

will result in:

ერთი „ორი «სამი»“

However, if the quote is in English and lang="en" is added to the first q tag, the result becomes:

ერთი “two ‘three’”

whereas it should be:

ერთი „two «three»“

This is the case for Gecko, Blink, and WebKit.

Interactive test, When an embedded quote is in a different language, the quotation marks should be those of the main body, even if the language of the quote is declared using a lang attribute.

i18n test suite, Multilingual nesting.

This incorrect behaviour was initially dictated by the HTML specification. issue 3636 was raised to change the spec. In the end the entire section was removed from the HTML spec, and HTML now relies on CSS for this behaviour.

css-content says that If a quotation is in a different language than the surrounding text, it is customary to quote the text with the quote marks of the language of the surrounding text, not the language of the quotation itself., however it is non-normative text.

Issue 5478 Open, requests that this be made normative, and has been agreed by the CSS WG.

Browser bug reports: Gecko • Blink • Webkit

tbd

Marking this as advanced because it's possible, though not always as convenient, to use Unicode characters instead of the q element.

How are emphasis and highlighting achieved? If lines or marks are drawn alongside, over or through the text, do they need to be a special distance from the text itself? Is it important to skip characters when underlining, etc? How do things change for vertically set text?

What characters or other methods are used to indicate abbreviation, ellipsis & repetition?

The ruby spec currently specifies an initial subset of requirements for fine-tuning the typography of phonetic and semantic annotations of East Asian text, including furigana, pinyin and zhuyin fuhao systems. Is is adequate for what it sets out to do? What other controls will be needed in the future? What about other types of inline annotation, such as warichu? (For referent-type notes such as footnotes, see below.)

This is about ways of marking text (see also specific sections dedicated to quotations and inline notes/annotations). Is it possible to express emphasis or highlight content as expected? Bold, italic and under-/over-lines are not always appropriate, and some scripts have their own unique ways of doing things, that are not in the Western tradition at all. Text delimiters mark certain items or sections off from the main text, such as book names in Chinese, quotations, head markers in Tibetan, etc, and often involve the use of punctuation. Is there any behaviour that isn't well supported, such as overlines for numeric digits in Syriac? Are there issues about the positioning or use of underlines? Some aspects related to the drawing of lines alongside or through text involve local typographic considerations. Do underlines need to be broken in special ways for this script? Do you need support for additional line shapes or widths? Does the distance or position of the lines relative to the text need to vary in ways that are not achievable? Are lines correctly drawn relative to vertical text?

If the script has its own set of number digits, are there any issues in how they are used? Does the script or language use special format patterns that are problematic (eg. 12,34,000 in India)? What about date/time formats and selection - and are non-Gregorian calendars needed? Do percent signs and other symbols associated with number work correctly, and do numbers need special decorations, (like in Ethiopic or Syriac)? How about the management of personal names, addresses, etc. in web pages: are there issues?

Does the browser capture the rules about the way text in your script wraps when it hits the end of a line? Does line-breaking wrap whole 'words' at a time, or characters, or something else (such as syllables in Tibetan and Javanese)? What characters should not appear at the end or start of a line, and what should be done to prevent that?

GitHub issue #13

Languages: hu 2

Hungarian needs hyphenation. Hyphenation has some more complex rules (handling compound words, digraphs, rules on repeated letters expressing emphasis-like longer sounds, etc.). Hungarian does not use break opportunities when digraphs are duplicated. E.g., in "Összeg" the "ssz" is a shortened form for "szsz", i.e., a double "sz" digraph; when hyphenating, one could do a "Ösz-szeg" which Gecko doesn't do. That being said, it is a bit of an edge case and people often do what Gecko does.

Tests & results: Hungarian hyphenation is supported by Gecko, but not by Blink or Webkit.

More systematic tests are needed to ascertain whether Gecko handles everything (such as the digraphs mentioned above).

Priority: Marking this as advanced, since although it could be argued that hyphenation is actually a basic need.

When text in a paragraph needs to have flush lines down both sides, does it follow the rules for your script? Does the script need assistance to conform to a grid pattern? Does your script allow punctuation to hang outside the text box at the start or end of a line? Where adjustments are need to make a line flush, how is that done? Do you shrink/stretch space between words and/or letters? Are word baselines stretched, as in Arabic? What about paragraph indents, or the need for logical alignment keywords, such as start/end, rather than left/right?

GitHub issue #4

Languages: nl 2

The (rarely used) hanging punctuation is not supported. Some newspapers allow letter spacing to help with justification, which is not supported either.

Some scripts create emphasis or other effects by spacing out the words, letters or syllables in a word. Are there requirements for this script/language that are unsupported? (For justification related spacing, see below.)

GitHub issue #23

Languages: el fr de hu ka nl ca no ru uk uz bg sr az-cyrl 2

This issue applies to all languages that use letter-spacing.

Various orthographies use letter-spacing to stretch words or phrases. This may be for a purely visual effect, but may also carry semantics, for example in languages such as German and Hebrew it is a way of emphasizing text.

When letter-spacing is applied, authors expect the spaces to occur only within the text to which it is applied.

Currently browsers that apply letter-spacing do so by adding a space after every letter in the text that is tracked. This results in a superfluous space at the end of the range, which creates an inappropriate gap before the following text. Letter spacing at the end of a line makes the line look misaligned in justified or right-justified text. It also has implications for text that has other styling, such as an outline or a coloured background, at the same time as being stretched.

Example in German: 87327736-5c89f780-c52c-11ea-8d22-7b994925d61a

Gecko, Chrome, and Webkit put the letter spacing after a letter even when it is at the end of a line, which makes the line look misaligned in justified or right-justified text.

For now, content authors have to work around the problem by also applying a negative margin to remove the trailing space. This is problematic not only because it requires additional work on the part of the author every time letter-spacing is used, but it is quite likely that the author is unaware that they need to do this, or indeed how to do it.

Interactive test Letter-spaced text should create no space on the outside of the selection

CSS issue 1518 The css-text specification requires letter-spacing to only insert space within the selection, not outside, so it is the browser implementations that need to be fixed. However, there is discussion around how to manage backward compatibility if browsers change their implementations.

tbd

Given that a workaround is possible, though not really acceptable, this is prioritised as an advanced issue.

Does the browser support requirements for baseline alignment between mixed scripts and in general?

The CSS Counter Styles specification describes a limited set of simple and complex styles for counters to be used in list numbering, chapter heading numbering, etc.The rules plus more counter styles (totalling around 120 for over 30 scripts) are listed in the document Ready-made Counter Styles. Do these cover your needs? Are the details correct? Are there other aspects related to counters and lists that need to be addressed?

Does the browser or ereader correctly handle special styling of the initial letter of a line or paragraph, such as for drop caps or similar? How about the size relationship between the large letter and the lines alongide? where does the large letter anchor relative to the lines alongside? is it normal to include initial quote marks in the large letter? is the large letter really a syllable? etc. Are all of these things working as expected?

GitHub issue #21

Languages: fr no 2

This applies to all languages that add a space between initial punctuation and the text that follows.

When a paragraph begins with a quotation mark or other punctuation and there is a space between it and the first letter, both the punctuation and the space separator should be included in the ::first-letter pseudo element.

Just below is an example in French. There is a tendency to prefer NNBSP (Narrow no-break space) because it is about the right width and prevents incorrect breaking. This should, however, also work when a normal space or the Non-break Space is used.

Screenshot 2021-01-25 at 13 00 35

A similar requirement occurs in Norwegian, although a different punctuation mark is used to indicate the start of the quotation.

Screenshot 2021-01-25 at 13 26 59

For «, Blink highlights the punctuation alone when NNBSP used, but there is no highlight at all when SP or NBSP intervene; Webkit highlights only the punctuation before NNBSP, but it works as expected for SP and NBSP.

For en-dash and em-dash, Blink and Webkit only highlight the dash punctuation.

Gecko fails to highlight anything at all.

issue 5154 Closed.

Marking this as advanced because initial-letter styling is not essential. It is, however, common enough to need some attention.

Interactive test Initial-letter selections capture punctuation and initial letter when separated by space

Blink • Webkit • Mozilla

css-pseudo-4 text edited to include any intervening space separators. browser bugs tbd

GitHub issue #6

Languages: nl 1

When 'IJ' appear together at the beginning of a paragraph (eg. in IJzertoren is een toren en vredesmonument in de West-Vlaamse stad Diksmuide.), both the first two letters should be highlighted.

On the other hand, 'Ij' should not be converted to 'IJ', if it occurs, since it may relate to (rare) nouns such as Ijad, Ijara, Ijaw, Ijaz, Ijebu, Ijenda, Ijeoma, Ijewere, Ijima, Ijja, Ijlal, Ijo, Ijobalit, Ijumaa and probably others. (see https://github.com/w3c/line_paragraph_tests/issues/56#issuecomment-1560126673)

This should only work if the language of the text is set to Dutch.

Even with the language of the text set to Dutch, neither Blink nor Webkit select 'ij' or 'IJ' together.

This has been fixed, however, for Gecko if the language of the content is set to Dutch.

However, Gecko also selects 'Ij', which is not correct.

This is a Basic error for Dutch users, and needs to be fixed.

Interactive test Dutch IJ are selected as a single grapheme by ::first-letter

Interactive test Ij is non ambiguously not Dutch IJ nor Dutch ij and should not be selected as a single grapheme for ::first-letter.

The CSS specification specifically mentions that the IJ combination should be treated as a single letter.

Blink • Webkit • Gecko

Gecko fixed selection of 'ij' and 'IJ', if the language of the content is set to Dutch.

Latin Script Gap Analysis

Abstract

Status of This Document

1. Introduction

1.1 Contributors

1.2 About this document

1.3 Prioritization

1.4 Related information

2. Text direction

2.1 Vertical text

#11 CSS Writing-modes sideways values not implemented

The GAP

Priority

Tests & results

Action taken

Outcomes

2.2 Bidirectional text

3. Glyph shaping & positioning

3.1 Fonts & font styles

3.2 Context-based shaping and positioning

3.3 Letterform slopes, weights, & italics

3.4 Cursive text

3.5 Case & other character transforms

#32 Catalan geminated L is not capitalised correctly

The GAP

Priority

Tests & results

Action taken

Outcomes

#2 IJ case conversion

4. Typographic units

4.1 Characters & encoding

4.2 Grapheme/word segmentation & selection

5. Punctuation & inline features

5.1 Phrase & section boundaries

5.2 Quotations & citations

#22 q element produces incorrect quotation marks when language changes

The GAP

Tests & results

Action taken

Outcomes

Priority

5.3 Emphasis & highlighting

5.4 Abbreviation, ellipsis & repetition

5.5 Inline notes & annotations

5.6 Text decoration & other inline features

5.7 Data formats & numbers

6. Line and paragraph layout

6.1 Line breaking & hyphenation

#13 Hungarian hyphenation details poorly supported

6.2 Text alignment & justification

#4 Hanging punctuation not supported

6.3 Text spacing

#23 Browsers apply extraneous spaces when letter-spacing

The GAP

Tests & results

Action taken

Outcomes

Priority

6.4 Baselines, line-height, etc

6.5 Lists, counters, etc.

6.6 Styling initials

#21 ::first-letter doesn't capture opening-quote+space+letter

The GAP

Priority

Tests

Action taken

Outcomes

#6 Dutch IJ not selected as a single grapheme by ::first-letter

The Gap:

Priority:

Tests & results:

Spec status:

Action taken:

Outcomes

7. Page & book layout

7.1 General page layout & progression

7.2 Grids & tables

7.3 Footnotes, endnotes, etc.

7.4 Page headers, footers, etc.