Meeting minutes
layout dependent feature subsetting raised by John.
John: I'm happy with the answer given in email so far.
John: I mentioned it would be good to have some test cases around this.
John: just to confirm things are working as expected.
John: some examples that had vertical layout of chinese. Situations in which features associated with vertical layout are applied. Make sure the correct glyphs are coming through.
John: same for bidi. Test using bidi control characters + a font with directional variants.
Skef: do browsers have layout like this?
John: I'm not sure. Unicode has a bunch of formatting control characters related to bidi.
John: in theory if those occur they should effect layout at the harfbuzz level.
John: less familiar with CJK
Skef: cases where a unicode codepoint will reference non-default features?
John: yes, harfbuzz recognizes particular fraction slash character. If it occurs between numerals harfbuzz will apply the numerator/demon feature. So it's turning on a display of fraction
John: same for directional layout. Should be triggered by directional control characters.
John: should trigger right to left alternate forms.
Skef: what were talking about in email is if the layout engine of a program uses features in particular cases it would identify them and request them.
Skef: now we're talking about is the shaper turning on non default features.
Skef: should the subsetter effect what features are included inthe subset. Based on certain codepoints. That implicitly turn on these features.
Skef: now we're talking about something at the shaper level.
John: my understanding is that the subsetter provides glyph coverage for the features which are active. Some of that has to come from the shaper.
Skef: you pass a set of features and set of codepoints. Will hb subset if you pass one of the control codepoints will it include related features.
Garret: I believe implementations will be sending off the request pre-shaping. We could ask the shaper what features it might need in order to inform the request.
Skef: another option is to make this open. If there are this set of codepoints the server side must include to be safe these additional features.
Skef: I'm a little bit inclined the second route. Seems like this is a hole in subsetting technology. If you ask for a codepoint that will require a feature the subetter should handle this.
Skef: seems better to add this to the idea of subsetting.
Garret: the subsetter definition in the spec requires it render the same, so it makes this a subsetter issue.
John: the on by default features in my list include the RTL control characters if you have RTL text.
Garret: confirmed those are currently listed as on by default.
John: CJK vertical writing forms are also on by default, but I'm not an expert. We may want to double check I got that right.
Garret: the vertical features are also listed as on by default.
John: forms specific to japanese standards are off by default, presume they get turned on by some higher level protocol. Would need to figure out where they get set in the process.
John: ensure you get the appropriate forms.
Garret: should check chrome and see if they handle these.
Skef: if you have a codepoint that activates one of these features. If you activate it globally will the right thing happen?
John: example unicode 2044 is the fraction slash. In the unicode spec there's a suggestion that if this occurs between numerals then that constitutes a fraction.
John: harfbuzz if it sees that it applies the numerator and denominator those features are triggered (normally off by default)
John: subsetter would need to be aware of that.
Garret: dnom is included by default.
Skef: what we've done is made a list of included by default.
John: airing on the side of inclusion.
Skef: is the use of these things clear enough in the shaper that the subsetter could know which codepoints are triggering in the shaper can optionally omit them.
Garret: we could have something inthe spec that records features that need to be included given some conditions.
Skef: something like these could be omitted based on the codepoints.
John: or the layout. eg. a typical greek document won't need the RTL variants.
Skef: won't the codepoints decide that?
John: yes... if you have a font dealing with archaic greek you have RTL and LTR text. Could be triggered by unicode directional control characters. The presence of those is a signal. There's probably also a way in CSS.
John: not sure, but may be a way in CSS to set that.
John: should have that info prior to shaping.
Skef: sounds like we have to make conservative list
Vlad: I'd rather have us reference opentype spec and specify criteria for which feature needs. On by default features should be included.
John: there's a set of inclusions that can be depedent instead of default.
Vlad: cases render with full font vs with the subset which must provide the same rendering.
Vlad: users shouldn't be able to tell. That implies that a requirement for subsetter that any feature which is on by default or conditional must be included as part of the subset.
Vlad: optimizations would be up to implementations.
Skef: optimizations aren't currently possible.
Skef: we're throwing inthe kitchen sink since shapers use features as they feel like.
Skef: and that could change too.
Skef: we need global knowledge of what shapers poke around in.
Vlad: should be clear with terminology, on by default (at the shaper) and subsetted by default.
Skef: the set of features that are not on by default, but the shaper will reference itself as it chooses.
Vlad: if it doesn't effect layout I'm fine with that.
Skef: the way the current list was developed is based on shaper behaviour.
Garret: not sure that the font format spec covers these.
John: that's right, some of descriptions do imply but it's not standardized.
John: that's one of the reasons I compiled my list.
Skef: the list of features on by default (eg. liga) that list is fairly clear.
John: yes, I think we might need a slightly finer clarification. There's on by default applied to all text. If it finds a match it applies the subst.
Skef: the microsoft feature list has a notation for those.
John: there's features that are on by default but applied selectively by the shaping engine.
John: based on character sequences.
Skef: is that list shaper specific?
John: it is. There are features that are specific indic shapers or specific to cjk shapers. The universal shaping engine has a generous model. It includes more of them.
Vlad: so you're talking about script specific, not platform specific shapers.
Skef: whether something converts a fraction automatically is a matter of opinion.
John: yeah, don't think all shapers do that.
Skef: if someone registers a new feature and Behdad looks at it and decides to apply it in a specific context then you have a new one.
Garret: I think you ultimately end up needing to consult the shaper for what features might be needed.
Skef: so I think you can have short list + consult with shaper or big list that tries to get everything.
<jhudson> [Quick test indicates that Apple Core Text engine does not treat U+2044 fraction slash as productive of fraction shaping.]
John: could we present both as options. Here's the conservative approach or it's up to you to query the shaper.
Skef: I think in that case our list becomes more advisory.
Skef: we've done a reasonable effort to be complete, but be aware.
Vlad: I'm not sure we should include this list in our spec instead of referencing it somewhere else.
https://
Bianca: I think we have to provide this list because it's not available elsewhere.
Vlad: not comfortable with us defining this list.
John: over the years I've complained to microsoft about the feature registry and have suggested it needs to be overhauled. No ones come up with the budget to do it.
John: if we're looking for a resource clarifying status and something that's easily machine parsable. Then we could also look at doing the same thing with regards to subsettting status.
John: locating it the open type registry.
Skef: the problem is that a lot of this isn't objective. A lot of this rest on the shaper implementations.
Skef: it seems better to put the onus on the shaper.
Skef: maybe we can reference the appendix as a starting point.
Skef: but ultimately we give our codepoints to the shaper and it gives the list of features.
John: that sounds ideal it will give you the most optimal set.
Vlad: would a shaper look at the IFT spec?
Skef: no, the just produce the list of features they need.
Skef: I think there is still some value in the protocol in having the list.
Skef: if the feature is on by default then the list is fairly objective and we can point to the normal spec. These features which are globally applied should be included in the subset.
Vlad: would love to have that, but not in the IFT spec.
Vlad: because it would be looked at by many eyes.
Skef: we don't want the meaning of default to have as serious of consequences. Unless the documentation indicates it should be globally applied.
Vlad: it's a typical arrangement to not duplicate information.
Vlad: I would seek ways that we can make it normative and have this list defined where it's most important.
Garret: the list that we have is very specific to IFT.
Garret: it's about assigning how the protocol communicates features.
John: there's always been a question about the status of the registries in the specification.
John: the font format spec defines the data structures.
John: but it's up to the shapers on how to implement it.
Skef: let's take the things currently marked default and give them a 2 byte encoding and have them be included unless you list them and the other not included unless you list them.
Garret: yes, that sounds good to me.
Skef: sounds like we also want the interaction with the shaper.
Skef: shaper should also be consulted.
Skef: consequence is there isn't a universal polyfill.
Garret: makes sense.
Vlad: appendix can be normative, but needs to be explicit.
Vlad: not comfortable copying the list from elsewhere. What happens when new features get registered?
Skef: if the feature is not on the list it's communicated with it's 4 byte tag.
Skef: the list is a set of indices for compact communication.
Skef: if there's a set of new features that would benefit, they look like they've become common by giving them an index.
Vlad: as an example Ken Lunde suggested a new set of features added in 2022.
Vlad: these changes happen regularily.
Vlad: so need two changes mark appendices as normative and second add the reference to the source the list was compiled.
Skef: if we have liga in the included unless turned off, it's generally a bad idea to turn it off.
Vlad: you create a subset that included certain ligatures the shaper still has the some option to not apply them. Why do you need to communicate it to subsetter.
Garret: reached agreement on changing the feature list in the IFT spec to have an exclusion mechanism.
Skef: probably want to have the included by default at higher numbers to leave space for the indices to grow.
Vlad: Ok need three things: Make section normative, reference where the data came from, and explain why it's normative (protocol efficiency)
Vlad: there's the microsoft spec and there's identical text published by ISO.
Vlad: the list of features in both documents.
Vlad: iso is actually registered.
Garret: should probably update references in the IFT to the OFF spec.
Skef: is registry versioned in OFF?
Vlad: both documents have version numbers.
John: historically there have been additions to the registry without spec updates.
<Vlad> Next WG meeting will be July 11.