Skip ⬇

Declarative Shadow DOM

Facilitator: Mason Freed

Minutes (including discussions that were not audio-recorded)

Previous: Parts and Template Instantiation All breakouts Next: CSS Module Scripts

Skip ⬇

Skip ⬇

Transcript

I have a presentation to go through and a few points in it I would like to have some discussions so we could potentially stop the recording during those, if you'd prefer.

So just quick introduction, I'm Mason Freed.

I'm the TL of the DOM team for Chrome at Google.

We're talking today about Declarative Shadow DOM, if you speak, we don't have time for introductions and things so if you speak, just try and introduce yourself really quickly so we know who you are.

Can we get somebody to scribe please?

I would like to have a log of what we talk about.

Ooh, ooh, me, me.

I'll do it.

Well, thank you for that.

I will start after you're done.

Great.

I don't think you have to cover what's in the presentation, but just the discussion part.

Thank you, John.

I appreciate it.

I guess with that, let me pull up the presentation.

So let me know if you can see that.

We can see it.

Sounds like we're good.

All right.

So like I said, Declarative Shadow DOM. I wanted to give a quick sort of background, just a few slides of what this feature is just to make sure everyone's on the same page and then we can get to the discussion, but first few slides.

So first, what is the motivation?

There are a few of them.

One is to allow the use of Shadow DOM in no JavaScript environments.

Examples of those are server side rendering, SEO, and some developer or design system environments that don't allow JavaScript.

Another is Ergonomics.

So, allowing access to particularly the style scoping feature of Shadow DOM without needing JavaScript to do that and just generally the ergonomics of allowing you to build a HTML markup that represents the full DOM tree, including this thing called Shadow DOM, which previously had no way to be represented in HTML and then the last one, which is a bit peripheral, but it is enabled by this feature previously serialization.

So using innerHTML was unable to return anything for shadow roots because there's no way to represent them.

So with the addition of a declarative shadow DOM syntax, you can also add features to allow serialization of a full tree, including its shadow roots.

So the basic proposal is to reuse the template element with a new attribute called shadow root, which when it takes on a value of either open or closed forms a declarative shadow root on the parent node containing that template.

So in the example, you see here, there is on the left side of the HTML containing template shadow root with some children.

Those are transformed into a shadow root on the DOM side, which contains the children of the template element.

So it's pretty, one-to-one, it's pretty transparent.

What's happening here about the only confusion which does get discussed, and we can discuss it today if you like, is the syntax.

Template shadow root is a little less ergonomic than just a shadow root element, but there are some reasons for that.

The other piece is serialization.

So, we can't just start using innerHTML to serialize shadow roots because that would break lots of things so a new method called getInnerHTML exists under this proposal with an options bag, including a bullying include shadow roots, which when true returns, a full tree including shadow roots and there are some options also, if you know about some closed shadow roots, you can also serialize those, but you have to have knowledge obviously of the closed shadow roots for that to be enabled.

Quick update.

We have discussed this in the past in particular in March, and there've been a number of improvements I just wanted to highlight since then, thanks to the great discussion on the issue and various other issues.

Most importantly, I think is issue 871 has been resolved and I'm working now to actually get that launched in Chrome in Chromium and then that is that element internals now contains a shadow root attribute, which gives access to the shadow root in the open or closed case.

So the idea here is your custom element wakes up and expect some declarative shadow DOM content, and you're using close shadow roots than you previously would have no way to access that declarative content with this attribute, you now do have that ability.

To do that, we also patched up a few security issues in particular attached internals that APA already existed, but was able to be called prior to a custom element constructor running and we've changed that behavior so that now it throws an exception before that custom element has had a chance to get first dibs on the element internals.

A few other small things, we've changed the behavior of attached shadow to improve some backwards compatibility.

So previously, if you had a custom element that didn't have any knowledge of declarative shadow Dom, and would happily call attached shadow expecting no exceptions, it might have run into problems before if there was declarative content there with a declarative shadow root that would have previously thrown an exception.

So we've modified the behavior of attached shadow, such that if it finds a declarative shadow root, so this is important, if it finds an imperative shadow root, like before it will now still throw an exception, but if it finds a declarative shadow root, it will retrieve that shadow root, empty out all the contents and return it so that the custom element sort of gets the same behavior that it had before and it won't break.

Another change for sort of security or to avoid leaking close shadow roots is that during parsing, when this template is being parsed, but hasn't yet closed, the content attribute is no and that's to keep, a mutation observer from being able to grab references to things inside of a shadow root.

And then the last one is again, the serialization syntax has changed.

So, previously I was trying to use innerHTML with some opt-in mechanism for each shadow root and now you just have a sort of easier to use, getInnerHTML method which will automatically serialize at least the open shadow roots in a tree and there's also as I mentioned a parameter for closed shadow root as well.

This, as I mentioned, has also been implemented in Chromium and at least on Chrome it's available through some Origin Trials.

So far the feedback, which is still pretty early, but feedback has been pretty good, straightforward, easy to use, unfortunately, no production data yet and anxiously awaiting that and there've been some supportive comments as well so I would qualify the feedback as cautiously optimistic.

So that's kind of the summary of the state of things in terms of open issues, As I think people who are following the issue are aware there has been the recent discovery recent in the last week or two of a potential cross-site scripting issue, which we definitely will need to discuss today.

My fear though, is that that discussion will take, could take, maybe I'm wrong the rest of the remaining time in this meeting.

So what I wanted to do right here was to see if there are other issues.

So from my point of view, other than the cross site scripting, which is again a big that all the other issues that have been discussed in my opinion, at least have been resolved.

So what I wanted to do is open up the floor and see if there were other things that people wanted to discuss today about this feature in general and I suppose we could stop the recording now if you wanna open it up for discussion.

Go ahead.

Let me know when you have. So I have a few slides, there are a few points at which we could discuss, but I wanted to at least sort of summarize the situation as I understand it.

This is a very recent thing in the last week or two, essentially a few cross-site scripting, sanitizer bypasses were discovered that are specifically related or sort of enabled by Declarative Shadow DOM. A really big thanks to all the people commenting on the thread.

I think the thread has doubled in size in the last week, but, and it's up to me at least to understand, and kind of tease out what the real issues are here.

I'm gonna try and summarize my understanding of that, but let's discuss if I've missed something important.

First, I wanted to sort of set the stage or the context within which we should understand these issues.

So, there are a number of situations that are goals of this that we need to solve essentially, these are concerning situations that need attention and these are all related to cross site scripting sanitizers in particular.

So, one of those situations is a site that runs up-to-date sanitizers, but for which the sanitizer does not yet know about or understand Declarative Shadow DOM. That's a concern we need to work on it.

Another is older sites or servers that are running non updated sanitizer libraries, just stale links to stale libraries.

We should worry about those because they may have been secure before and they may not be secure now and we should make sure that they stay secure and then the third class of problems are sort of manually configured sanitizers where the settings were non default, but the end result was previously safe but now with declarative Shadow DOM those settings are no longer safe.

That's obviously a bad one that we need to worry about.

There are some also some non-goals, some things that I don't think at least to me should be a concern for this issue so the first sort of obvious one is sites that run up-to-date sanitizers that have safe defaults, those can be updated or are being updated to be safe in the context of declarative Shadow DOM I just wanna say that's a possible thing and so those are sort of okay already or those will be okay.

Another situation that I don't think should be a concern for this issue is a site that is running no sanitizers at all.

Those are either sites that don't use any entrusted inputs and those are already fine, or they're already hugely vulnerable and declarative Shadow DOM doesn't make that situation any worse than it already is.

And the last one is sort of similar to that site is running either a really bad sanitizer that doesn't work or a misconfigured sanitizer that allows through every element without being sanitized.

Those are already vulnerable as well, even without declarative Shadow DOM and so vulnerable is vulnerable we can't make it worse if that's already bad.

So let me go into we can talk about all that, by the way, when it gets through, I guess these four slides and we can sort of pause for another discussion break but the two situations that I know about or that I've sort of gleaned from the conversation fall into two categories.

So category number one, is a client side cross-site scripting issue and it is one where the sanitizer in use uses the browser's own parser so let me take you through that one.

This problem happens when sort of all of these six things are true.

Number one, youand the user are in a browser that supports the declarative Shadow DOM, you are browsing a site that uses an old client side sanitizer that doesn't yet it hasn't been updated for Declarative Shadow DOM yet.

That sanitizer uses, and this is the important part uses the browser's own HTML parser to convert whatever dirty HTML it's trying to sanitize into an isolated dirty DocumentFragment.

The difference here is that with declarative Shadow DOM, this can now create shadow roots in that DocumentFragment.

We're not unsafe yet, but it's a change in situation at this point.

Step four there, the sanitizer walks that tree looking for nodes that it wants to remove dangerous nodes and the problem here is that older sanitizers don't know anything about declarative Shadow DOM so they don't even know to look in that tree for shadow DOM at all, and even newer ones that are updated, won't be able to see by definition, anything inside of a closed shadow root.

So we begin to see the problem here.

Step five, the sanitizer, and this is important too, returns the DOM tree itself.

That DocumentFragment back to the user rather than a sanitized HTML string.

So many sanitizers have the defaults that return an HTML string, and they do that by innerHTML in the DocumentFragment and that would, that still is actually safe because innerHTML would strip out all the Shadow Dom so you'd be okay there, but in this problem case, the sanitizer is either configured to, or by default returns, the Dom tree itself.

And the last step is that sanitized DOM tree is taken by the user and attached or sorry, by the site and attached to the live document using adoptNode or appendChild or something other than importNode, which it's so important it clones the tree and removes all shadow roots.

Those, so at this point now that document with the shadow root content is live and the exploits will execute.

That's the bad situation.

That's problem number one, problem number two, sounds somewhat similar, but it's different at about step four.

So again, excuse me, you're using a browser that supports declarative Shadow DOM and you're on a site that uses an old either in this case server side or client side sanitizer that doesn't know about declarative Shadow DOM. In this case, though, the sanitizer uses its own parser.

So it's not using a browser's parser.

It's not using the Dom parser, it's using its own parser to build its own representation of the DOM and here this step is sort of, okay, at this point, it won't create any shadow DOM in quotes because it doesn't know about it, but it might keep around any template nodes that it finds and it might keep the shadow root attribute on those template nodes.

So step four now, either just by default or because of some configuration that sanitizer, this is the important step passes, when it finds a template node, it passes through all of the children of that node without sanitizing them at all, because it assumes that it's a template and therefore everything is inert.

It also must preserve the attributes on that template node, including the shadow root and then that obviously leads to step five, which is that sanitized HTML is sent to the browser and parsed and again, we're in a browser that supports declarative Shadow DOM so those template nodes with shadow roots become live and exploits executed at that point.

Those are the two classes of problems that I've seen.

I have seen several others being discussed so I wanted to bring them up as well to mention them as non-issues, at least in my opinion, there are three of those, one, is that any client side sanitizer that does use the browsers parser, but returns HTML instead of DOM should be okay because those use innerHTML, which strips out all the shadow DOM there should be no declarative Shadow DOM content left here.

The second one is, there may be server-side sanitizers, I haven't seen these, but they may exist where the server-side sanitizer uses a DOM based parser, like for example, using puppeteer to parse the tree and provide the DOM tree, but since these are server-side, they need to eventually serialize their sanitized HTML back to HTML to send it to the client, and that again is okay because innerHTML strips the Shadow DOM and the last one that I also heard was it's possible that some sanitizers might wrap potentially dangerous markup with a template element because the template element is inert and renders all of its contents inert and this behavior is actually still okay, because these sanitizers aren't going to begin for some reason wrapping in a template shadow root, so they will continue wrapping things in a template, which will still render everything, including declarative Shadow DOM inert.

So these, that situation should also be okay.

I wanted to pause right here.

This is the situation as I understand it, but it's definitely possible I missed a case that we should be worried about.

So I don't know if we wanna pause the presentation and just see if there's any further input or other situations that we should talk about.

Let me know when I, looks like we're okay.

All right.

So I just wanted, I have two slides basically talking about problem one and problem two and the sort of implications.

So, for problem number one, again, the client side sanitizer that uses the browser's own parser, this one, at least to me, and maybe if I'm reading between the lines, the people on the issue thread, it seems like the most pressing of the issues, there are a number of clients-side sanitizers they are kept up to date and security best practices that you keep your security stuff up-to-date all the time, but it's pretty common.

I think it sounds like to not keep up to the very latest of client-side sanitizer libraries, and potentially also to have configuration options that are not the default and so that, this is for me, at least the most worrying of them.

What I would like to do, I haven't had a chance to yet is try and do some digging in HTMLArchive and other places to see how prevalent that is.

So I can look for old versions of libraries there and I can look for manual configurations that are risky, particularly the option to return the DOM tree instead of HTML, that's the one that really triggers the behavior I think, and I will go and do that.

But assuming that this is an issue that we need to mitigate, I sort of, there has been some discussion about possible mitigations and importantly, we're talking about sanitizers for this issue, we're talking about sanitizers and sanitizers necessarily have to parse some string HTML into DOM using one of the DOM's own methods and there are only a handful of them really it's two, there's Dom parser, which is sort of the best practice DOMParser.parseFromString and the other one of a couple of ways is to use innerHTML on some isolated DocumentFragment created a few different ways.

So one potential fix that stands out there would be to, this is a hindrance on the feature so this does limit the usefulness, or it makes it a little less ergonomic for the feature for the declarative Shadow DOM feature but perhaps it's not so bad and that is to make declarative Shadow DOM opt-in through options on either innerHTML, which is a bit more magical or on DOMParser, which seems pretty straightforward.

So you could, for example, add, there are no options on DOMParser today, but we could add an options bag there with an option that's, opt-in to allowed declarative Shadow DOM false by default so that if older sanitizer libraries are using DOMParser and they parsed some malicious content with declarative Shadow DOM in it by default, those get stripped out by the browser.

That's a little more difficult to do for innerHTML one quick option, a few, there are some options, but you could on a document or shadow root, you could add a property for allowing declarative Shadow DOM within that document.

I'm sure the smart people on this thread can figure out the best API shape for opting in, but basically the fix that has come up a few times is to opt-in for all of the parser API.

Let me just get through the other, the next slide and then we can talk about both, I guess maybe that's the easiest way to do this.

So for problem two, at least for me in contrast to problem one, this seems like less of a concern to me at least.

The reason for that, the primary reason for that is that if this problem case exists so remember this is the either server side or client side sanitizer that uses its own DOM representation, its own parser and importantly is configured on some site to pass through all template elements that it finds and allow all their children to be, go through unsanitized completely.

The issue with that is that there are a number of browsers in the wild that don't support the template element and therefore the sanitizers are currently broken on all of those sites.

Sorry, on any users on any of those browsers.

looking at can I use that's about 3.7% of page loads on browsers today that use browsers that don't support template and for all of these browsers right now that behavior would be completely broken and those users would be completely vulnerable.

You could make the argument that maybe in this hypothetical situation where the sanitizer is being configured to pass through template elements, unsanitized there's also some UA sniffing on the server side or some feature detection and on the browser side, which is changing the behavior so that when a browser doesn't support template, the behavior of the sanitizer is to inspect and filter the whole contents of that template element but for browsers that do support template, they let through all the contents of the template, all the children of the template element unfiltered that would get around this 3.7% but I just find that I don't understand why that would be the configuration in any case.

It's definitely possible I don't understand, but I don't see it.

I would love to hear feedback on that.

I would really importantly like to find some sort of data if we can gather it for how prevalent can we find an example where this situation is true?

And, and the reason I'm saying that is that, there are potential fixes, I've listed two of them there are probably others at the bottom, but they're pretty draconian so one of them is adding an HTTP header, which ops the whole page in, that would mitigate this issue for sure but that is a big, that's a big deal, Remember that one of the use cases of this is to allow designers to use declarative Shadow DOM because they're not allowed to use Java script.

I really imagine that somebody who is not allowed to use JavaScript definitely can't change HTTP headers so that sort of kills a few of the use cases outright I think.

Another fix, which was suggested, which isn't a perfect fix, but does seem to get a little bit better would be to rename the attribute instead of shadow root to make it onShadowRoot and then assume that any sanitizer that's working will strip out any attribute that starts with on that would also probably fix most of this problem.

Both of these seem pretty bad, but let's discuss that.

I should say that is, those are the only two slides I have on the situation and what we should do about it, I would love to open it up for discussion at this point.