Meeting minutes
This meeting
group: [discussion of zoom access, URLs, the ability to use W3C zoom]
Atsushi: I will send the Chairs an email about W3C zoom
Nigel: Thank you
Nigel: Today we have DAPT, a couple of points on IMSC-HRM, and if there's a status update on the
… charter FO Council, we can take that too.
… Is there any other business?
Pierre: On IMSC-HRM I would like to cover CR Exit criteria - I created a Pull Request.
Nigel: OK, thank you
group: [no other business]
Rechartering Formal Objection Council update
Nigel: Anything to report?
Atsushi: I just know that the Council is set up and they're working to coordinate on the time of the meeting
… so nothing further has happened.
Nigel: Thank you. Any questions?
IMSC-HRM
Plan to reference IMSC-HRM from future revisions of IMSC?
Nigel: This comes from a comment in w3c/imsc-hrm#51 where Pierre said he thought we
<Github> https://
Nigel: should not reference IMSC-HRM from IMSC.
… First question: are you saying we should remove the normative requirement to meet IMSC-HRM from IMSC?
Pierre: Yes, that's correct. It would remove that from conformance in future editions of IMSC.
Nigel: The argument is that it's not used?
Pierre: Two arguments. A) to avoid the circular reference.
… B) It won't have a significant difference because organisations that want to enforce
… the HRM have called it out separately.
… I don't think it will cause an impact in practice.
… We can always have a note in IMSC that points to IMSC-HRM, e.g.
<atsushi> +1 for informative note to inform, consumer can choose to apply both or only imsc
Pierre: saying "this spec does not impose any constraints on presentation complexity, here's where to find it".
Nigel: This raises the question of if IMSC-HRM needs to be a Rec track document?
Pierre: To the extent that other orgs would use it for conformance I think they would appreciate it being
… a Rec track document. To a large extent it depends on those external organisations.
… I know some that would be reluctant to reference a WG Note for instance.
Nigel: And the positive reason for removing from IMSC - is it only to avoid the circular reference?
Cyril: Why is it a problem to have circular references? Other specs do that a lot - almost every standard in ISO!
Nigel: We've always tried to be clear about the chain of dependencies from spec to spec, and avoided
… circular references.
… That was my next question too, is this the only way to avoid circular references?
Atsushi: I'm curious too - I believe this is something that humans can check, for circularity,
… e.g. if A defines in terms of B and B is defined in terms of A. That can even be in one specification.
… It cannot easily be detected automatically.
Nigel: That's right, there are ways to deal with this. Also, can IMSC-HRM stand on its own, or can
… it be used only for IMSC.
Pierre: It's a constraint on IMSC, the dependency is pretty obvious that IMSC-HRM depends on IMSC.
… For better or worse, for many implementations, renderers have implemented IMSC but not the HRM validation.
Pierre: I think circular references are bad simply because they're circular.
… I don't want this to be an obstacle. If there's a desire to have a circular reference I'm not going to
… no-vote the document.
… Imagine: circular refs are bad because if IMSC gets updated without updating IMSC-HRM, then you don't
… go back to where you started if you follow the reference. Sometimes that's harmless, sometimes harmful.
… Being serious about conformance. which some orgs are, then we would have a separate conformance
… document.
Nigel: That would be the IMSC spec though?
Pierre: Not always. I'm simply advocating having IMSC-HRM reference IMSC, and have an undated
… reference from IMSC to IMSC-HRM.
… Maybe later we'll have a spec on readability complexity for Western European languages, as a separate spec.
Nigel: I'd like two things to happen:
… 1. We arrange it so that the decision about how to reference IMSC-HRM from IMSC is made separately,
… which means:
… 2. We word IMSC-HRM so that it can be referenced normatively from IMSC, if that's what we some time want.
… An easy way to deal with this circular reference is to duplicate the definition.
Pierre: That is an editorial nightmare - I'd object to that.
Nigel: Is it though? Is it worse?
Pierre: Yes, it causes work.
Nigel: My argument is that a change to one doc doesn't then require a change to the other, because
… the definition has document-level scope only.
Pierre: I'd prefer circular references than defining the same term twice in two different specifications.
… I've never seen that end well.
Nigel: I'm thinking that carefully arranged circular references might be the best option here.
… My original issue was that there should be no circular references _that cannot be resolved_, so if we
… can check that, it should be fine.
Pierre: I think the references are undated now, so I think it will work.
Nigel: Let's say we make some tweak to a defined term in a future version of IMSC, can we make
… it so that the reference to that term in IMSC-HRM is to the one for the version of IMSC that applies?
Pierre: We should try to avoid doing that and if we do, we should do it very carefully.
… One could argue that we should reference a particular version of IMSC.
… Nowadays in practice people look at the latest version.
… The responsibility would be on us, if we make a substantive change to a definition or something
… depended upon in the spec, then we better warn people, it could be disruptive.
… Like changing glyph buffer Gn to glyph back buffer is really editorial, but it we were to change
… Presented Region that would have a negative impact and we might want to define a new term instead.
Github: https://
Pierre: Not to be philosophical, let's not make a new version of IMSC that's incompatible with an old one!
… I've seen that in other specs.
SUMMARY: Keep the references circular for the time being, and check that they can all be resolved successfully.
Pierre: We can defer this until we revise IMSC to reference IMSC-HRM.
Nigel: No, I want us to make sure we don't put something in IMSC-HRM that gives us a problem later.
Pierre: I'm not aware of anything that gives us that problem right now.
Nigel: That may well be the case.
SUMMARY: Nigel to verify that the current text will not cause any issues.
github-bot, end topic
CR1 exit criteria w3c/imsc-hrm#56
<Github> https://
github: https://
Nigel: This is a draft PR, maybe that's why I did not notice it ahead of the call.
Pierre: Walking through this.
Nigel: [shares https://
Pierre: I thought adding a MD doc was a good way to kick around ideas.
… This lays out how we will demonstrate Implementation Experience.
… The idea is that each IMSC document is accompanied by an assertion that says if it passes or fails.
… The idea is that each document is tested against each implementation and we determine that it
… meets the assertion.
… Features at risk: it occurred to me that the current implementation of imsc-hrm does not support
… image profile, because nobody has asked for it. Unless we can find actual image profile documents,
… from people who use it at scale, we should just remove that feature from IMSC-HRM
… Then there's a bunch of synthetic tests to test the boundaries and features of the HRM.
… If we are happy with this structure, then I'll add more synthetic tests.
… That's my proposal for exit criteria.
… Atsushi asked if we want exit criteria to be different for each CR version.
… In the past I have found that CR exit tests are not necessarily useful for the community, long term.
… Right now the synthetic tests are just in a directory.
Cyril: Two questions.
… First, the collection of sample IMSC documents.
… I'm wondering, e.g. for Netflix, how we can contribute them, for IP reasons.
Pierre: Thanks for the offer! I would expect Netflix to run the implementation internally and just report
… on the results, I would be satisfied with that.
Cyril: Okay, that makes sense.
… Second, do you have a list of features that you want to test?
Pierre: Not yet. There are different ways the HRM can be exceeded,
… maximum glyph buffer size, maximum rendering time.
… You'd want a test for each of those.
… There's also IPD, a maximum render time, to test.
… We should also test failure on the first ISD of a sequence, because it's somewhat of a special case.
… We should test the ideograms as having a different rendering rate from non-ideograms.
… We should have test documentations that fail if the glyph buffer is not being used.
… I am planning to clean up the tests for the implementation and make them available.
Cyril: I'm wondering if there's a list of those features?
Pierre: Yes, we could add a feature test column.
Cyril: In other standard bodies we're working on having explicit references from the spec to the tests.
… Tools like bikeshed or respec help, they can generate assert ids, which you add with special markup
… in the spec text. Then you can use those in the naming of the tests.
… Respec has a way to show you the tests, if you cross-reference them properly.
… It may be worth investigating.
Pierre: Absolutely, I'll look at that, thanks.
Nigel: Elephant in the room - this approach is going to be dependent on the result of the FO Council.
… I think we cannot finalise this approach until we have the FO Council outcome.
Atsushi: Usually they have a single call to arrive at their conclusions, so we may hear something at the
… end of this month, at least. Of course it make take time to generate the formal document to be published.
… In any case I believe we can get a decision this month if the pattern follows previous FO Councils.
Pierre: By definition the deliberation does not stop our work. Anyway end of month works for
… timing for requesting transition to CR.
Nigel: For the collections of documents, you want an assertion for each document whether it passes or fails?
Pierre: For the synthetic ones, its our responsibility to generate those.
… For third parties, e.g. Netflix, BBC, it's up to them to say whether they think a document should pass
… or fail, and report whether the implementation agreed with their expectation.
… If it agrees, there's nothing more to do.
… If it does not agree, then we have to determine what to do.
… It could be the content is invalid, an error in the spec, an error in the implementation.
Atsushi: One concern with building a test suite for HRM. It needs to have at least
… two generated documents from different implementations.
… Should there be some example of presentation?
… And IMSC-HRM implementation will go over that and generate a pass/fail result.
… If I am correct I think we need to have some descriptive note for our test case?
Pierre: My proposal is that we ask folks like BBC, like Netflix and others to test their commercial content.
… That would be their criteria. Content that they use commercially or plan to use commercially.
… We cannot specify any particular style, it's different for every provider.
… That's what's most useful for the industry - it will tell us if the HRM is useful, and right.
Atsushi: Usually, just seeing it built against each API call or value definition is ideal. I'm not sure how
… to convince readers of the implementation report that it covers all defined functionality in the HRM.
… Like which test is related to each definition, etc. It's difficult to regenerate the cases.
Pierre: I agree it's different than other specs, e.g. API based specs, for sure.
… We cannot control what people create commercially which ultimately is what people care about.
… It does not fit the template of API specs.
Nigel: If we were to be able to show that the HRM is successful at selecting between content that plays
… successfully on some player and content that does not, that would be really useful.
Pierre: We can fairly easily architect a synthetic test that presents alright but is likely to cause most players
… to have a problem, and is rejected by the HRM.
… Then we could invite folks with players to try it. We could do that too.
Nigel: That's the big unknown to me, how well the HRM tracks real world playability - that's really hard to grasp.
Pierre: It would be helpful actually to generate synthetic documents.
… Any objection to the overall approach, before I start working on it?
Cyril: No, this is fine, but maybe we should wait for the FO Council?
… You might waste effort.
Pierre: I think we can address their conclusions later.
SUMMARY: No immediate objections, Pierre to proceed. TTWG to continue reviewing.
github-bot, end topic
Pierre: Thanks for the feedback.
DAPT
Cyril: Suggest scheduling an editor's session
Nigel: +1 let's arrange offline
Cyril: OK
Meeting close
Nigel: We're over-time - thanks everyone. Next call in 2 weeks. [rrsagent, make minutes]