DID WG Telco — Minutes

Date: 2021-02-23

See also the Agenda and the IRC Log

Attendees

Present: Adrian Gropper, Daniel Burnett, Shigeya Suzuki, Brent Zundel, Justin Richer, Geun-Hyung Kim, Markus Sabadello, Wayne Chang, Manu Sporny, Orie Steele, Ted Thibodeau Jr., Drummond Reed, Dmitri Zagidulin, Kaliya Young

Regrets:

Guests:

Chair: Brent Zundel, Daniel Burnett

Scribe(s): Justin Richer, Daniel Burnett, Manu Sporny

Content:

• 1. Agenda Review, Introductions, Re-introductions
• 2. Special Topic Call
• 3. Date for Transition to CR
• 4. Substantive PRs
• 5. Tests for Normative Statements
• 6. test suite discussion

1. Agenda Review, Introductions, Re-introductions

Daniel Burnett: agenda for today is to briefly talk about special topic call
… plan for transition to CR, including date target
… expressly time-boxed substantive PRs

2. Special Topic Call

Daniel Burnett: special topic call is this thursday @ noon US ET
… topic is PRs and issues
… if you need help with an issue and PR this is where

3. Date for Transition to CR

Daniel Burnett: intent is to have spec ready for march 2 (week from today)
… week for people to look at “the version”
… look now; group will have 1 week before vote to publish
… editors still have work to do, but the more we can do now the better

Manu Sporny: no changes to timeline, things have improved since last we talked
… reminder, we have all the normative language sections (through security considerations) with multiple reviews and big substantive PRs are in
… it’s a great place to be
… document in editor’s draft space is the latest; review now
… (also doc in TR space)
… all additional stuff needs editorial work; it’s rough and we expect to do it during CR
… shouldn’t affect the core of the spec
… if you haven’t done a review and you promised, now is the time
… needed: 👀
… “TR Space” is “technical report” space in w3c

Manu Sporny: This is an example of Technical Report space: https://www.w3.org/TR/did-core/

Manu Sporny: it’s where the final document goes, and includes the snapshots

Daniel Burnett: right now latest version is there, after CR the latest version might not be there (we’ll talk when that happens)
… we have to point to a fixed version (unchanging)
… once we say “this is the version” we expect not to make changes
… chairs will request publication after vote and let you know how it goes

4. Substantive PRs

Daniel Burnett: machine representability of tests and representation of production/consumption

Manu Sporny: concerned there would be last-minute conversations about last-minute changes in representations
… there’s a new map for rep-specific data, take a close look at it
… not expecting any more substantive PRs to come in
… markus_sabadello will update diagrams, ivan will update data model buckets
… diagrams will be merged, might be tweaked
… (thanks to shigeya)
… only remaining concerns are around test suite and testability

Daniel Burnett: this is it for comments

Markus Sabadello: confirming, fine with data model and representation-related content
… have worked on an updated diagram

Daniel Burnett: markus_sabadello’s diagrams are the best

Manu Sporny: +1 to peacekeeper diagrams being the best :)

Justin Richer: (the above statement needs no consensus call)

5. Tests for Normative Statements

Manu Sporny: have made multiple passes on normative statements, there’s a new button to generate requirements list on editor’s draft
… we’ve taken the list and made a list of machine-testable statements

Daniel Burnett: thank you to Orie who’s done a massive amount of work on test harness and test suite

Manu Sporny: See List of normative testable statements that we need volunteers for

Drummond Reed: +1 to Orie’s massive work on the test harness

Orie Steele: If you want to convert that list to json, you can use this: https://github.com/transmute-industries/did-core/tree/main/packages/did-core-tests#generate-json-test-cases

Manu Sporny: this (list) is the next major thing, we need volunteers
… bunched into topical groups
… we need volunteers to create tests
… if we don’t get volunteers to write the tests, that section is not going to make it
… amy’s done a fantastic job of getting a list of all resolutions and cross-referencing it with spec text
… to make sure resolutions resulted in spec text changes that survive to this day

Manu Sporny: See Amy’s list of cross-checking resolutions to spec text

Manu Sporny: make sure the things we’ve crossed out are actually in the spec, feel free to provide input on items that aren’t crossed out
… we’re also in good shape for testable normative statements
… there are two normative statements suggested for removal
… we need volunteers, people to sign up

Daniel Burnett: we’ll do that in this meeting by making people uncomfortable

Manu Sporny: that sounds good

Orie Steele: we should cover what people are signing up for before people sign up

Daniel Burnett: I like the plan of teasing the content, but want to give people an option to choose one
… we’ll let people speak up and then go one by one

Manu Sporny: https://github.com/w3c/did-test-suite/issues?q=is%3Aissue+is%3Aopen+label%3A%22volunteer+needed%22

Orie Steele: these are the major sections, will give a summary
… these normative statements are divided by section and we believe they’re testable
… what you agree to do when you volunteer is you show in the test suite that this section is testable
… this is separate from proving conformance
… we’re still waiting for people to figure out how to test some of these, there were some strong opinions
… not sure what additional changes will be needed to the test suite; each section might need a unique approach
… you should volunteer for a section that you feel comfortable writing tests for

Daniel Burnett: editors and test writers are happy to help you understand what’s needed to write tests

Orie Steele: one issue that markus_sabadello and I try to resolve is difference between “your implementation conforms” vs. “testing normative statement”
… this past weekend I took the approach of testing the statement w/o any DID method
… it was easy but I expect we’ll have some issues when testing specific methods

Manu Sporny: reiterate we’re spreading the load out, this is a moving target
… what we need are people who can write code

Daniel Burnett: any volunteers?

Wayne Chang: volunteering for two sections; doesn’t matter which ones except not CBOR

Daniel Burnett: before we discuss how tests are done, do we have to discuss that before pushing for volunteers?

Justin Richer: Not wanting to get into specifics of writing those tests, just to remind people, different philosophies on what to test and how to expose sections to test harness. Some of these will be in DID Core spec, some of them will just be handshaking w/ test harness.
… People may need to write adapters, etc. Keep in mind it needs to be adapted to different platforms/etc… you might not be calling things directly.

Manu Sporny: “the tests will be the tests”
… we just need people to volunteer
… all of us will be writing the same kinda class of tests

Daniel Burnett: https://github.com/w3c/did-test-suite/issues/31

Shigeya Suzuki: I volunteer to do that

Daniel Burnett: https://github.com/w3c/did-test-suite/issues/30

Daniel Burnett: did url dereferencing

Manu Sporny: this is more complex, not clear how we’ll test it

Daniel Burnett: what are the most complex/challenging ones to give to wayne?

Manu Sporny: did resolution and did url dereferencing

Daniel Burnett: would you be comfortable doing that, wayne ?

Daniel Burnett: https://github.com/w3c/did-test-suite/issues/29

Daniel Burnett: we will give these two to wayne b/c he offered earlier

Manu Sporny: I’m slightly concerned about that but we can take it up
… I’d hope that people involved with resolution would do this

Markus Sabadello: I’m not great at JS but I can work w/wayne

Daniel Burnett: that suggests it would be better to take one of these sections and create tests

Daniel Burnett: https://github.com/w3c/did-test-suite/issues/28

Daniel Burnett: cbor production and consumption

Manu Sporny: this requires deep cbor knowledge

Drummond Reed: and we don’t have jonathan on the call

Daniel Burnett: willing to skip for now unless someone has that experience

Orie Steele: will take this one

Daniel Burnett: https://github.com/w3c/did-test-suite/issues/26

Daniel Burnett: json production and consumption

Manu Sporny: this is what the “json-only people” were asking for so I hoped they’d want to be there

Orie Steele: this is 99% of the production rules for the spec

Daniel Burnett: this is precisely the kind of test group that daniel buchner should take
… since MS had asked for this very strongly
… Orie you convince Daniel to do it instead of you, or share it

Manu Sporny: would prefer Orie focus on test suite architecture, give the rest to minions

Daniel Burnett: https://github.com/w3c/did-test-suite/issues/25

Daniel Burnett: conforming consumer

Manu Sporny: this is weird! don’t know how we actually test this
… w/o writing linters

Orie Steele: hard to do; I had implement an abstract data model class to assert properties on the model

Daniel Burnett: the three remaining are consumer, producer, and did params

Manu Sporny: production is easiest, if we pass all the other tests we have passed the production test
… wayne should do did parameters

Daniel Burnett: still need someone for conforming consumer

Manu Sporny: who wants to write the equivalent to what Orie just wrote?
… consumer should probably go to wayne

Daniel Burnett: looks like we’re going to leave one or two open, which would you rather leave open?

Orie Steele: production/consumption are more important than deref/resolution

Markus Sabadello: they are complex and difficult but they are similar to each other
… i can figure it out w/help from someone

Daniel Burnett: tests are such that things are similar enough, once you figure out how to do one you’ve got an idea to do others
… assign 29/30 to markus
… assign 25/22 to wayne

Manu Sporny: volunteer for producer

Daniel Burnett: thank you [we have volunteers for everything]
… publication of CR doesn’t depend on these being written
… these are needed for implementers to implement and report to us, to exit CR
… start to talk about test writing

6. test suite discussion

Orie Steele: our first thoughts were “wouldn’t it be great to have static test fixtures to adapt your did method / implementation and run them through the suite”
… there are concerns on testing some things with static fixtures
… what’s the alternative?

Manu Sporny: this is an opinion “just for now”
… there are three basic levels of testing w/a spec
… 1) test to see the spec is internally consistent
… not testing implementations
… 2) static test vectors, you get implementers submitting input
… suite determines whether or not submitted content is good/valid
… more useful than just testing spec, gets input from implementors
… 3) testing dynamically against implementations
… “generate a did for me, register it on a ledger, resolve it”
… “generate a did:key, take that and shove it into another library
… totally different
… test suite is exercising a bunch of libraries against each other
… more useful than static
… always like to see the last one
… if you have that, you can put up an ecosystem scoreboard
… you can be sure that “we passed a test suite” tests the right things
… static vectors don’t get updated

Justin Richer: (scribe’s note, that’s not true but ok…)

Manu Sporny: we’re shooting for the last thing in the list to make sure everything’s up to date
… and that everything’s always testable
… don’t think we’ll get there for DID Core
… count it as “useful but a failure” if all we can do is level (1)

Daniel Burnett: only requirement is level 1

Orie Steele: agree w/manu, intermediate step between 2 and 3
… testing implementation that isn’t method-specific
… for ex, implement production/consumption w/o talking to blockchain
… you need an implementation and let things register dynamic stuff, or having implementation inside of test suite

Markus Sabadello: +1 to orie, not all functionality defined by the spec is method-specific

Orie Steele: testing of test suite w/static inputs w/o looking at blockchain, we’ll never succeed at that
… implementation is actually being tested, you’re calling an implementation

Markus Sabadello: e.g. DID (URL) syntax parsing

Justin Richer: Agree with Orie about the nature of a lot of the testing here – there is a lot of variability between the posts that manu has laid out, there’s a lot of space in there that’s definable. I want to point to OpenID test suite, which do test live implementations, but test live tests implementations, in that test suite has a whole bunch of functions that it calls, OpenID is network protocol, tests network protocol, test suite does a whole bunch of actions in protocol definition, within specification, respond to positive and negative tests in appropriate and detectable ways.
… Look for error here, or error there, but some of these tests is to display something to user, test suite has to account for that, screen shot of interaction w/ test harness, error that is displayed, not all of this is fully automatable, that’s ok, it’s fine that we define what integration points are in test suite, not all are going to come from DID Core – comes into whole notion of – I can test implementation, but here are actions I need to take, provide a layer/endpoint that I can call to do the action.

Daniel Burnett: topic will continue in the special-topics call

Justin Richer: Personally, that’s what I think we should be aiming for… long way from testing production endpoints… we can do test endpoints, head in that direction.