DID WG Topical Call on DID Resolution contracts — Minutes

Date: 2020-05-21

See also the Agenda and the IRC Log

Attendees

Present: Brent Zundel, Ivan Herman, Dave Longley, Justin Richer, Daniel Burnett, Markus Sabadello, Manu Sporny, Orie Steele, Dmitri Zagidulin, Drummond Reed, Kaliya Young

Regrets:

Guests:

Chair:

Scribe(s): Manu Sporny, Brent Zundel

Content:

• 1. Justin Walks Us Through PRs

Markus Sabadello: This is a special topic call about DID Resolution Contracts - again, I think we have a number of open PRs for DID Resolution, DID Resolution Contract, and related to that PRs for dereferencing and metadata. For Resolution contract, there are two PRs, one by Justin, based on my original PR (to some extent), and another by Manu.
… We’re going to discuss the nature/details of those contracts, wrt. testing.
… If we add DID Resolution contract to spec, how are we going to test statements.
… I have a diagram
… In this diagram, we have a client invoking a resolver.
… There is a resolve function, input options, resolver options, and output document, document ematdata, resolution metadata.
… I don’t think there is disagreement on what goes in and what comes out.
… the resolver invoked the read operation of the DID Method, that’s out of scope for DID Core.
… The details of the resolution function is out of scope for DID Core, we are not defining how you invoke a resolver.
… We’re not talking about metadata being part of DID Document, protocol headers, etc. Also out of scope.
… What I tried to do is look up a few example/normative statements in different PRs… the input DID MUST conform to the ‘did’ rule of the Generic DID sytnax”
… Saying things like “DID Document metadata may be empty. key maps must not be repeated, normative statements.
… Then we have a number of statements, if the didDocument does not conform, an ‘invalid did document error MUST be returned’… ‘content-tyupe resolution should match input metadata.
… One of my questions, that I want to ask, underlying question/issue – how far does the contract go, what exactly do we want to define in DID Core.
… Do we want to define concrete error codes? Do we want to specify what input should match what output?
… Are we saying that the return DID Document must have an ID field that has input DID?
… or does that go too far?
… So what’s in scope, what’s out of scope?

Dmitri Zagidulin: Question to Markus, very much like this whole model, agree with everything you’ve said… Why the accept option?
… Why is that optional, we have multiple MIME Types, like JSON-LD, or JWT (or whatever ends up happening), do we not need to specify? Is there a default?

Justin Richer: I could jump in…

Markus Sabadello: Just one meta comment, they’re in PRs, not in spec yet, question wasn’t if we agree on statements, which one of those statements belong in DID Core and which ones are already out of scope?
… do we even want to talk about headers/options/error codes?

Manu Sporny: We are here to debate the options, and lots is up for debate

Orie Steele: What goes in resolution metadata? Came up in sidetree - want to note that with 3 places where you can put critical data that developer accesses, for interop, vendors that chose to put data in one bucket and not the DID Document bucket, have the opportunity to make DID Document contain nothing useful.
… Just want to note that we should be careful about providing guidance around did document metadata, resolution metadata, if a popular vendor can put stuff in certain buckets, it can force the market in ways that are dangerous.

Dave Longley: I feel like we agree that we’re defining the API/abstract function that takes inputs and has outputs, we need to understand for those inputs what the meaningful output is going to be. Every one of these statements, generally speaking, would fall under that. We’re going to provide parameters, based on input, we expect certain outputs.
… If we want an abstract interface, we need to specify that, every single one of those statements belongs in DID Core.

Justin Richer: To reiterate, the main purpose of today’s call is where we want to draw the line, and what that line means. The different PRs draw the line in significantly different ways. What actions do we expect to happen inside the resolve function, and what do we expect to happen outside of resolve() function.
… At present, there are disagreements on what exactly those different actions should be… where those actions should occur, that’s one of the things I was planning to do today when going over different PRs, where we want to draw that line.
… I agree that all of these could make sense, but where is that line drawn? I disagree that all of these are part of the resolve() function, but we need to decide that as a WG.

Markus Sabadello: To what Dave just said, all of these are potentially in scope, that we should define if there are certain inptus, then what are the expected outputs, my understanding of F2F meeting, that’s already beyond just a contract, don’t have strong opinion, thought contract would define what are valid inputs, what are valid outputs, but not define what outputs you get when you provide given inputs. That exists in DID Resolution spec today. Again,

Manu Sporny: I’m fine w/ having that in DID Core.

Markus Sabadello: My understanding, on screen, first is in scope, second is in scope, third statement, certain keys… but 5, 6, and 7 is out of scope, implementation detail… ‘content-type’ given as input, the output must be X.
… This is the resolution contract, the next is dereferncing, that one is potentially more complex, given certain DID Parameters, what is the expected output.

Dave Longley: We are defining a function, we won’t say what happens inside, but there are certain constraints on what comes out… it’s not a useful thing to say… it’s not a useful contract if implementer just pops out shapes that we recognize.

1. Justin Walks Us Through PRs

Drummond Reed: +1

Justin Richer: I’m going to be showing stuff on Github on screen.

Dave Longley: (there’s no point in naming a function “resolve” if it can do anything at all)

Justin Richer: A bit of history - there have been 4 attempted PRs for this… Markus wrote the first one, I wrote the second one, Manu adapted it into two other PRs, some editorial stuff that needs tweaking, mostly same text, then 286 was Manu’s take on a different approach of defining what resolution function was. It redoes how function is presented and also in a handful of ways redefines what the resolution function is and does.
… So, what do we mean by the resolution function – what we can start with was my intent for the resolution function.
… So, DID Resolution function, my thought here is that inputs that you got here were defined in terms of data structures, a string, a map of strings, with string keys and values, and outputs are a bytestream to be parsed, and two maps of string key-value pairs.
… This draws the line of parsing and understanding document itself outside of resolution function itself - you still have to parse/understand it before you have to do any procesisng with it.
… PR 286 takes a different approach - in this, it must ensure that the didDocument being returned is a conforming DID Document.

Orie Steele: big plus 1 to seperating resolution response from parsing…. this has and continues to cause issues in the universal resolver…

Justin Richer: So, it has to be parsed internally and processed internally, raises question on how much conformance is required in the document.

Orie Steele: in a world where controllers can edit their did document, its a huge problem to have the resolution do parsing.

Dave Longley: we’re going to need some flexibility here somehow – we don’t want to make it hard to write a resolver or force resolvers to reserialize

Dave Longley: +1 to defining resolve() in a DID-method independent way

Orie Steele: bug plus one to what justin is saying.

Justin Richer: There is a real reason where I put parsing outside of the resolve function, that’s that in my view, the resolve and dereference functions have to be defined in a method-independent way. Importantly, in a representation independent way. So I should be able to call resolve on any resolver and get back a DID Document in whatever format that resolver can get it in, using whatever method the resolver can do, the important thing here is that by splitting

Manu Sporny: this out, it limits what the resolve function actually is.

Justin Richer: it limits it to something that’s testable and interoperable.

Manu Sporny: I don’t think the intent was to change that. tit was more to ensure that at the end of the day we have a valid did document
… if the resolution process can kick out a document that is malformed, I’m concerned. I have great interest in where that happens.
… if we don’t do that we kick the can down the road. I think you’re asserting something that’s stronger than intended.
… I found no issue with anything you said, except for the open question of if we don’t describe it here, where do we do it?

Justin Richer: I understand that that wasn’t the intent, but that’s the result of language in 286 - it very clearly moves the line by requiring additional processing of the DID Document itself and also not specifying what the DID Document itself is that gets returned. It says it’s a DID Document, that doesn’t give us enough to make an interoperable function method.
… That’s basically telling me, that within the function itself, I have to parse and understand it before I hand it back, and I hand it back in some proprietary format that I can just make up and claim conformance.
… I absolutely agree that there needs to be conformance testing of that document somewhere in the system, but that is outside of the resolve function. In my view, for a very barebones system, you would call resolve, parse, validate and then move forward.

Orie Steele: I’m a huge fan of separation between resolving and parsing, I think we can define what a DID Document is and what default parsers for various representations can be and have confidence around that. Prior to F2F, took Universal Resolver, tested for conformance, mutated them, tested with additional terminology - and one of the problems I encountered, if the resolver is parsing them in JSON-LD then it drops things, can’t test from universal resolver can’t

Manu Sporny: know if it’s processed - give me a way to get raw response from method, let me understand if I need to mutate that.

Dave Longley: Orie: +1 to avoiding mutations in the resolver, however, we should be aware that a lot of tools (that are or would be used to implement resolvers) may automatically parse data such as JSON data

Orie Steele: Or maybe my custom rejection is the issue – doesn’t mean I can’t also get something that’s been processed.

Justin Richer: +1 my pull request was intented to be a pre-parser method

Dmitri Zagidulin: -1 to pre-parser method, from me.

Markus Sabadello: I do think we should say that resolve function should return conformant DID Document, we should not talk about resolver parsing a document.
… If we return a bad DID Document, parse it, errors happen, implementation details… shouldn’t talk about it in DID Core - if we say resolver must parse/validate the DID Document, that’s very confusing, the point of the resolver is to produce a DID Document, we have consumer/producer, resolver is a producer, it doesn’t parse/validate documents, it has to produce valid DID Document.

Orie Steele: Example of how being able to control parsing can be used to show that nobody is producing compliant did documents: https://identity.foundation/context/test-report.html

Dmitri Zagidulin: I too am slightly confused by requirement for resolver to return it unparsed, I don’t know what unparsed means. Take the BTCR method, for example, if I’m asking for BTCR DID, what am I expecting resolver to return? JSON-only representation, raw binary format that Bitcoin blockchain uses? If return from resolver, resolve function, stream of bytes, literally no developer is going to use it, very first thing that resolver library will do is define a

Manu Sporny: resolve+() function that also does the parsing, and return that.

Manu Sporny: the only thing I’m concerned about is I feel like that is too much of a developer-centric concept.
… does that mean the resolve function can return a raw stream of bytes, or with syntax errors, or should it clean up syntax errors?
… I’m concerned about the variability as that damages interop.

Orie Steele: almost every did document that exists today is invalid.

Daniel Burnett: manu/dlongley, how does a data model return an error?

Manu Sporny: a resolver should never return a did document that has errors. If you ask for a JSON did document and get back one with errors, I would kick that implementation to the curb.
… I want to consume valid DID documents and expect something conformant.
… that’s the purpose of the language there. The PR doesn’t have a strong opinion about where it happens in the resolution process, but that it needs to.

Dave Longley: burn, it doesn’t… but a function (which we’re discussing at least in the abstract) can throw/return one.

Manu Sporny: the core infrastructure needs to return valid documents.

Justin Richer: I wanted to point out, this is where Manu pulled the language in the new PR from…
… My PR does say, in the happy path, the DID Document is returned as a byte stream of a conformant representation.
… Doesn’t mean it isn’t a conformant DID Document - I’m getting back something that I know when I get it back

Dmitri Zagidulin: I was /only/ mentioning bundling resolve() and parse. NOT validate etc

Justin Richer: I think there will be libraries that bundle this up into convenience function, can be defined as chaining of smaller functions. A large part of what got us to this discussion in the first place, without some way to discuss “this is the document I have and this is the format it’s in”… you can’t parse anything until you know what format it is in.

Dmitri Zagidulin: I genuinely don’t understand why we can’t use the Accept: mechanism, for the resolver. You ask the resolver for a JSON document, it returns a json document.

Justin Richer: That’s why my PR doesn’t say anything about validating, why parsing is put outside, and after resolve function itself. It gives me all info I need to do that… without that, we are glossing over a very important step to ensure interop. Separation of representation from method to get it.

Daniel Burnett: I keep going back and forth on this statement… I know that we’re in this bind because we have a data model spec, but we have some execution semantics that we need to define. When we were working on VC spec, we had a case where we wanted to say what was at end of URL provided, couldn’t make normative statement, requires fetching URL. Can we reorder/rephrase that is less about the thing that “the thing that is returned is X”, to “if you get something

Manu Sporny: like X, it’s to be interpreted this way”. It’s different from a check that must occur.

Daniel Burnett: I think those two phrasings are different, some a priori item vs. if we get X, then it is to be interpreted as Y.

Justin Richer: +1, I think this was more in line with what I meant. You should be allowed to get something that fails the parser – and that’s a parse error, not a resolve error.

Justin Richer: +1 to Orie’s statement

Drummond Reed: There’s another representation format right there - what’s needed for NFC

Orie Steele: I recently implemented DID Resolution over NFC, one of the things I had to do is support taglength bytelength value… first thing I did after getting bytes from NFC, then I could construct well-formed valid DID Document. Saying the response is a byte string, saying byte string is of a particular format, helpful to build as separate layers. Most developers will ask for a lib to do all of this for them, we should formalize lower layers first.

Manu Sporny: to see representation that Justin is proposing as a start… then JSON, then CBOR, I think all of these are “I get DID Doc” in valid format can only be formalized if we get bottom layer formalized first.

Orie Steele: and get agreement around it.

Manu Sporny: I’m not hearing too much disagreement. Are we all agreeing?
… one thing that seems to be emerging is to specify multiple layers and to specify what happens at each.
… we’re not talking about the higher layers. One thing we could do is specify the layers.
… the process includes a resolution layer, but we believe there will also be a parsing layer and a validation layer, etc.
… let’s specify the layers appropriately, and maybe that will help us (just like defining the metadata buckets), to be able to say at what layer each thing happens.

Brent Zundel: I’m not hearing much disagreement.

Orie Steele: drummond: NFC is a transport, not a representation…. it supports any DID representation, just like HTTP.

Dave Longley: I agree with a lot of what Manu said, also not in disagreement w/ what Justin/Orie is saying – there are a number of tools, there is no point at which there is a byte array to represent a DID Document when it’s being resolved. For example, did:key - there is no byte array there. Literally no point at which a resolver for that method needs a byte array which is a serialized representation of the DID Document, I want to make sure we have flexibility

Manu Sporny: here to serialize things to a byte array.

Orie Steele: I would use this layer

Dmitri Zagidulin: I don’t understand the emphasis on parsing. Do we need to specify how a JSON parser works?

Dave Longley: It’s not like there are these lower layers, you literally wouldn’t use the lower layer in did:key - there are cases where it makes sense to chain functions internally to do these steps, there are other cases where the representation would never be used.
… We need to allow for that to happen, implement something that’s non-conformant, want us to be aware of that.

Orie Steele: dmitriz: you do if you say it happens for sure.

Justin Richer: q

Orie Steele: dmitriz: regarding json parsing

Markus Sabadello: Like Manu, I don’t think we’re disagreeing on much here - I understand that the way Manu described it, when you invoke resolution, you get a DID Document (a conformant one), that’s it. I also understand Justin’s approach, you get a byte stream back, tells you content type, valid way of looking at it. You could argue that in did:key there is a bytestream serialization of DID Document, even if it’s in memory.

Daniel Burnett: The key isn’t that we get back “a DID document”, it’s that we get back “a DID document supposedly in some format”

Markus Sabadello: That’s a detail we can figure out in the language, in the spec somewhere.
… One thing I’d like to decide if possible, in DID Core, do we want to have language that looks like an algorithm, that’s what’s in Manu’s PR, but not in my or Justin’s PR. To implement resolution function, here are steps. My understanding was precise understanding was out of scope, for dereferencing it’ll be much more complex, more discussion

Dave Longley: the fact that some object is in memory in the heap based on language internals is not the same thing as serializing a DID document to a byte stream in application space (nor would that internal byte stream likely be parsable/in a valid representation)

Markus Sabadello: I’d rather not have an algorithm, but have as much as possible, normative statements about inputs and outputs, that’s good.
… step by step algorithm, don’t know if that’s something we want.

Justin Richer: I wanted to agree w/ Markus - it’s also my stance that we don’t want to do a detailed algorithm, puts too much prescription on implementations.
… Wanted to respond to optimizing away layers - you can’t optimize things away until you define what layers are - this is trying to define layers, your implementation does not call or use this function, you could see a DID or make something up in memory to return that as an object, you are not doing the resolve function, but you are compliant with rest of DID framework.

Dave Longley: “you’re not doing a resolve function then” … i think that’s a problem.

Dave Longley: resolve perhaps needs to be even further broken down if that’s the case.

Orie Steele: +1 to Justin’s point, without defining the layers, we won’t be able to ensure the higher level interfaces at all…. that maybe more developers care about, than the lower layers.

Justin Richer: What I’m trying to do here, and I think Manu hit on this, define a very narrow layer, possible that there are other functions, including parse and validate, that we can define within DID Core - I was thinking of parse as pointing to “read representation consumption section” that’s parsing… validation is “go do whatever needs to be done for section 6 and 7 once you’ve parsed everything”

Dave Longley: at the end of the day, implementers won’t actually be implementing “resolve” as we’re specifying it for one or more DID methods that exist today.

Justin Richer: All of that can be represented as a higher level function, this is the result of calling this and this and this, all in a row, in order to actually get the result. That will help clear up where to put pieces of data.

Manu Sporny: the PR that I put in, 286, there’s a section where we use well-trodden weasel wording. “an equivalent result”
… which means an implementor can do whatever they want to as long as you meet the normative statements.
… I don’t want to make it seem like having an algorithm that prescribes everything is necessary.
… the other way of doing this is to specify the input and output options.
… what I found that there was a process, even in Justin’s PR.
… there are functions and steps,. the algorithm just makes that clear.
… most of the steps just say ensure X or Y.
… it doesn’t get specific (translating byte strings).

Justin Richer: -1, these “ensure” statements imply specific normative actions

Manu Sporny: the algorithm is just a set of ensure statements
… the intent is it’s doing the same thing as Justin’s text
… we’ve been talking about functions and processes, we almost always end up writing them as steps to follow.

Dave Longley: The algorithm can be specified as entirely informative (to help implementers) and the inputs/outputs (and potential errors) as normative.

Manu Sporny: that’s the reason for the algorithm, it’s easier to read.

Drummond Reed: +1 to a generic algorithm

Dave Longley: I’d recommend the algorithm being informative – and then there’s all the leeway anyone needs.

Justin Richer: +1 to any deeper algorithm being outside

Drummond Reed: I’m okay with the algorithm being informative

Justin Richer: functions as normative, implementation as informative

Markus Sabadello: If we do it in the form of an algorithm, then it has to be generic and extensible… at least three things that resolver has to do that’s not in this algorithm… resulting DID Document that has input did, what happens when DID is deactivated, additional proofs, does resolver have to do that… with dereferencing it gets more complicated… algorithm is bigger and has to be extensible and formulated in a generic way according to details in resolution spec.

Drummond Reed: +1 to Justin