<Jennie> *Need a scribe for first 45 minutes Rachael?

<Jennie> scribe: Jennie

Rachael: We need a subscribe for the 2nd part of this meeting
... and please present then plus (without a space)
... 1st is Normative vs Informative
... Last night we had a good working session about this
... I will try to summarize
... Please jump in if you were there, and others ask questions
... Link to slides is above
... We tried to clarify what things are
... Functional Needs (reads from slide 22)

<scribe> ...(continues reading through that slide)

UNKNOWN_SPEAKER: We had conversations about headings that might have the same outcome
... (moving to slide 23)
... Then we discussed Methods and Tests
... Likely a many to 1
... Each method should be sufficient, meet the outcome
... Methods include the scope tested
... We talked about testing against pages in WCAG 2

<scribe> ...(continues reading from slide)

UNKNOWN_SPEAKER: Different methods for PDF, or because you have a different scope
... Then we talked about test sets - this is new
... (reads from slide)

<janina> +1 to the concept of test-sets!

<Rachael> https://jamboard.google.com/d/1b3MZ6ToJja5vbKiRREpWBpCnReFRz4BO37EtR03IlGo/viewer?f=1

UNKNOWN_SPEAKER: (moved to slide 24)
... There is a link to the jamboard we used (above)
... We then discussed which pieces are normative, which are informative
... (reads from slide 24)
... Principles, Functional Needs, and User Needs - were in the middle, and are bringing that back to the group today
... We will start with a straw poll

<Rachael> straw poll: Do you agree with guidelines, outcomes, and glossary items being normative

<Amanda> +1

<maryjom> +1

<jeanne> +1

<ShawnT> +1

<Wilco> +1

<alastairc> +1

<MichaelC> +.5

<janina> +1

<Lauriat> +1

+1

<MichaelC> I don´t think guidelines should be normative, except ¨accidentally¨ as part of doc structure

<jeanne> +1 to Michael

Rachael: (reads Michael's comment)

Michael: W3C specs - top level indicatives normative vs informative
... WCAG 2 is organized with a structure
... That includes this information
... I saw them as organizational rather than meaning
... I would like to see a way to see them not be normative without requirements

Wilco: I second that
... I have heard people arguing things from a guideline point of view, even though the criterion didn't specify it

<Zakim> alastairc, you wanted to ask about the testability / clarity of outcomes

Alastair: This seems quite reasonable
... The split that was created
... Was there a discussion around the testability of the outcomes?
... WCAG 2 - the success criteria - we read a lot into them
... Is that the same here?
... Or was there discussion around this?

Rachael: There was
... Could someone else talk to that?

Michael: The conformance section already says notes are not normative. We could add guidelines

Rachael: We talked about in WCAG2 it was difficult to make the tests normative
... Ideally, yes, we want them to be normative
... But because they change so frequently
... and the public needs to be able to add to the tests
... we had to make all of it informative in order to keep it up to date
... We need methods and tests to be informative in order to maintain the content
... But it seemed to work in WCAG 2
... That we had a normative, testable statement

<Zakim> alastairc, you wanted to ask whether people could create their own methods / tests to meet outomes?

Rachael: While not ideal, we would like repeat it

Alastair: A couple of complications ...1: if meeting outcomes with tests or methods we haven't provided
... I assume that is possible if meeting the normative outcome?
... How would some people say whether their test was critical or non-critical if we go down that path?

<Zakim> jeanne, you wanted to add Test Reliability Subgroup work last spring on improving the testability of Outcomes ...2: what if the author needs to do different things to meet an outcome, sometimes with content, sometimes with user agent?
... That may be problematic

Jeanne: That is definitely something we want to solve
... That is the situation we have today
... What we are talking about is following the model from today
... The methods being roughly equivalent to techniques
... Last night - the test reliability group did work on how to write good methods
... And how to improve the testability of the outcomes
... There is a lot of work already done in this area
... Work AGWG has seen already

Wilco: To Alastair's point
... I don't think critical errors can be part
... I don't think there is a way to do that
... In the test type subgroup: if we let org's create their own outcomes (like for protocols)
... You may want quality
... for outcomes
... That could be a direction?

<alastairc> outcomes or methods? I hadn't thought other people would create outcomes

Wilco: It is like how ACT did things too
... The ACT rules are informative
... But when we have a normative document that says what you need to do
... Even though the rules themselves are not normative

<Zakim> Lauriat, you wanted to speak to the technology differences: that works by design and allows us to do things like highlight opportunities for user agents et al to support outcomes

Shawn L: On the tech differences: that works on design

scribe: It allows us to highlight gaps
... And it can solve
... It can address an outcome at that level
... I also want to note on critical errors
... I don't think it is clearly indicated on the slide
... Tests themselves would cover critical errors
... The definition of critical error could be normative
... And this would include it in the conformance model
... But then how you decide if something is, then it would be in the informative

<Zakim> Rachael, you wanted to discuss emerging technology methods

Rachael: I added those to the slides
... When we worked on WCAG 3 we had good conversations, and others have discussed emerging technologies
... To address the situation for technology not covered by existing methods
... We wouldn't be able to write for more than HTML
... in a comprehensive way
... By doing this it would allow us to grow the spec

<Zakim> jeanne, you wanted to say Critical Errors are normative today in WCAG3

Rachael: I think this is something we should revisit (chair hat off)

Jeanne: Both the functional categories and critical errors are normative (today)

<Zakim> MichaelC, you wanted to -1 functional needs

MichaelC: I do not think functional needs should be normative

<Lauriat> +1 to MichaelC

MichaelC: They are too abstract, way too comprehensive

<alastairc> +1 should be informative, they are tags.

MichaelC: I am hoping they will be a W3C statement

<Amanda> +1 to Michael

MichaelC: But going to normative would be really (said with emphasis) difficult

Rachael: Going to a straw poll

<Rachael> strawpoll: Should we move guidelines to informative?

<jeanne> +1

<MichaelC> +1

<alastairc> +1

<ShawnT> +1

<janina> _1

+1

<Wilco> +1, I think. Guideline text only though. We need good language for it

<Lauriat> 0 I definitely need to think on the implications of this more, I missed the conversations leading up to this

Janina: I am worried about leaving the most general statement to be informative
... I can still be convinced
... I will change to zero

<janina> +0

Rachael: I will move them to informative with a question mark ...Next: should functional needs be informative

MichaelC: I said my peace

Amanda: Going back to guidelines - I didn't vote because I am not sure
... Does it take away from the gravitas?
... What we need it to represent?

<Zakim> jeanne, you wanted to say that Functional Categories are needed if we are going to set minimums by disability

Jeanne: I will defer to Michael first

MIchael: That's why I mentioned W3C statement - a new process
... Up to now we have recommendation track documents
... these go through both testing, committee, and both
... (correction, vote)
... The idea, but there hasn't been a statement published

Rachael: That is regarding functional needs

Jeanne: I want to disagree with Michael
... At least the functional categories, we will need as normative
... One of the solutions to having an equitable outcome for all disability groups is that
... We set a minimum by disability group
... Which I hope we do
... We will need the functional need categories as normative
... I don't want to vote that we will never use functional needs as normative if it will prevent that solution

MichaelC: It is an interesting thought
... I feel that functional categories has the same needs
... I hear the piece about equity
... You are probably right that we should have a normative way to ensure we have equity
... I am not sure this is the way to do it, but we should find a way to solve it
... Maybe we say functional categories with a question mark for now

<Wilco> +1 to leaving it open. Good point Jeanne

Rachael: OK, let's add it as an item for now
... To clarify, when you say functional category, do you mean visual?

Jeanne: An example would be use without vision or born without vision is the category vision and visual
... We have other categories

<Rachael> Document at: https://www.w3.org/WAI/GL/WCAG3/2020/functional-needs/

Jeanne: So it is more of the categorization of the individual functional needs

MichaelC: that is the highest level organization we have
... They are extremely broad
... I think if we share we will ensure cognitive is included, people will not appreciate that
... We could go down another level
... I think we need to figure out how to do it, but there is something here

<jeanne> I go back to my comment on MOnday that the organization and categorization will be critical to equity

Rachael: I will add Higher Level Functional Categories for now
... If I have categorized, it would be helpful to know

<Rachael> strawpoll: Higher level Functional Categories may be normative. Lower Level FUnctional needs are informative

<michael> I missed argument, but question functional needs as normative

<jeanne> +1

<MichaelC> +.5 just because it needs more refinement down the road

<michael> 0

<Jaunita_George> +1

<Wilco> 0

<janina> +1

<Lauriat> 0

<Amanda> +1 for now

.5 agreeing with MichaelC

<alastairc> +1, and we can refine later.

<ShawnT> 0

Rachael: We absolutely can refine later
... I want to go back to Amanda's concern - guidelines being informative as kind of our default
... If it has implications we are not thinking about
... Do we want to discuss this now?
... Or can we come back to it?

<Zakim> jeanne, you wanted to say that making guideline informative makdes them ignorable

Jeanne: We struggle with the number of people who take whatever we write for guidelines and make it into checklists
... Having the guidelines be normative is an argument that if it gets into a checklist, but I don't have a strong feeling either way

Alastair: When people work from the most abstract part
... Misunderstandings are more likely
... If people make a checklist the outcomes should be the starting point
... I think on balance it is more likely to prevent misunderstandings if they are informative

<alastairc> scribe+

<alastairc> Jennie: I'm agreeing with the checklist points, for people new we often teach at that guideline level. So useful for newbies to have that included.

<alastairc> ... how we inform people of those is key, where they see the quick ref and they see the phrase with the SCs.

Rachael: I do wonder if the problem with them being normative is not having conformance statements

<matatk> present_

Rachael: The content below is proving the guideline in some form (chair hat off)
... I will leave it as informative with a question mark for now
... Where are we on principles?
... Informative or normative

<michael> Informative

<Wilco> same as the guidelines IMO

<Jaunita_George> informative

MichaelC: I think the same thing about the principles
... I think they are hard to make normative
... I think principles have been recast as user needs in the new way of thinking

<Zakim> jeanne, you wanted to say they are tags, and then they won't go away

Jeanne: We plan to have them be part of the tagging
... People can tag for a principle, and then people can pull up related guidelines
... Many are related to multiple principles

<Zakim> alastairc, you wanted to say informative

Alastair: I would say informative because they are essentially tags
... and we can use these to structure.
... There may be better structures

<Rachael> strawpoll: Principles are informative

<alastairc> Jennie: I'd revise my previous statement, I meant my comment for principles not guildelines.

<Jaunita_George> +1

<jeanne> +1\

Rachael: a straw poll

<Amanda> +1

<alastairc> +1

<janina> +1

<ShawnT> +1

<Wilco> 0

<David-Clarke> +1

<MichaelC> actually right now the highest level user needs are POUPD - perceivable, operable, understandable, personalization, deceptive patterns

<MichaelC> +1

<Lauriat> +1

<michael> 0 I think they may be normative now

<michael> But likely should be informative

Rachael: I see 2 zeros
... Do you feel strongly to revisit or abstaining?

MichaelG: I am fine with discussion

Wilco: I can go with it
... I don't think these should change any more easily than outcomes should
... Informative things change more easily
... If being used in educational materials, they should not be frequently tweaked
... But we can do that with informative content too

MichaelG: We had a discussion about information architecture yesterday
... I wouldn't want contrainsts for those trying to help with information architecture
... Which is why I am leaning to leave it alone for now

Rachael: I will put it as informative with a ?
... Critical errors in the next one
... I heard a strong inclination for it being normative

<michael> No

Rachael: Did anyone want to speak to it being informative?

Wilco: 1 potential direction with critical errors
... if we want to use them in the conformance model
... We may say conformance still means every outcome fully met
... But if you want to go outside of that, there is a non normative method of scoring
... Which includes critical errors
... This may enable us to have less repeatability than we might need for normative conformance model

<michael> Nice consideration

Alastair: I am torn
... You want things like methods and tests to be informative
... To be quickly updated
... But if we want to use some form of critical errors as part of normative conformance
... I think they need to contribute, but I am not sure how to get it in

ShawnL: I need to think more on Wilco's proposal
... I think there is a differentiation we need to clarify ...1: definition of critical error
... How you arrive at something being a critical error needs to be in the tests, which is not normative

<Zakim> michael, you wanted to say we have some ...1: I think that can work

MichaelG: We have something hints of critical error right now
... Some SCs say
... If this fails, the page fails,
... Flashing content, some others have this
... We have this without applying critical error
... So I like Wilco's idea

Wilco: The reason I started thinking this direction: I am not convinced that deciding something
... As a critical error, can be done consistently enough

<alastairc> I like the idea of defining critical errors normatively, but not sure how that could then apply to tests if we need to get a panel of experts & PwD deciding what is (likely to be) critical

Wilco: It is a direction we could explore

*scribe change?

scribe: I think we explore the normative option first

<alastairc> +1 to exploring normative version first.

JonA: I wonder if that works for regulators
... If it changes, does that impact adoption?

<Rachael> strawpoll: Explore Critical errors as normative

*scribe change?

<Lauriat> -1

<Wilco> +1

<michael> +1

<David-Clarke> +1

<jeanne> +1 to JonA as regulators want to control change

<Ben_Tillyer> +1

<alastairc> +1

<ShawnT> +1

<Wilco> scribe+

<jeanne> +1

+1

<jon_avila> +.5

<Wilco> Shawn: I think we can only really have the definition of critical errors

<Wilco> ... I don't think we can include a critical error for each outcome / guideance

<Wilco> ... The critical errors themselves need to include the context.

<Wilco> ... There is too much that is so dependant on the text that I don't think it can be normative.

<Wilco> Alastair: From the subgroup working through issue severity, the approach we took was looking through the tests and voting.

<Wilco> ... Those that are likely blockers or severe. I don't know how you'd define that in a way you'd get consistent results.

<Wilco> Rachael: I do wonder if we can assign critical errors, but what the criteria for a critical error is at the normative level.

<Ben_Tillyer> +1 to Rachael

<Wilco> ... Tie it into the user needs, that could be the normative part.

<Wilco> +1 I think that sounds promising

<Lauriat> +1, I had that in mind of the definition

<Wilco> Rachael: What I'd like to do is to put it in both places.

<Rachael> straw poll: Put critical errors in both locations

<alastairc> +1, I'm skeptical but happy to expore defining critical errors.

<Wilco> Jennie: How would a person that may have a difficulty processing it identify its in both parts? Can we note that in brackets?

<Lauriat> +1

<Amanda_> +1

<Ben_Tillyer> +1

<Jaunita_George> +1

<Wilco> Rachael: I've put in the slides under normative, critical errors definition, and under informative critical errors details

<alastairc> +1

<jeanne> +1

<jeanne> +1

<David-Clarke> +1

<ShawnT> +1

+1

<alastairc> Wilco: I had a thought that was more fundamental. This is for allowing tolerances for minor issues. Is the number of issues a policy issue rather than something that goes into WCAG.

<Wilco> Rachael: I see three different things. What a critical error is, how you do it in a method, and how it fits into conformance.

<Wilco> ... How it fits into conformance is a different thing.

<Zakim> Lauriat, you wanted to restate the goal from my understanding of having conformance more aligned with the experience for people with disabilities

<Jaunita_George> A lot of minor issues could equal a major one

+1 to Jaunita

<Wilco> Lauriat: The goal of critical errors is giving us a way to have conformance better align with the experience, rather than have it be a goal of squashing issues.

<Wilco> ... It gives us a language to express severity.

<michael> +1

<Wilco> Rachael: The last one to talk about is user needs. I think I'd like to leave that as a question.

<Wilco> ... We'll come back to this later.

Outcome or User Need Statements

<Wilco> Rachael: A piece that evolved out of the discussion yesterday.

<Wilco> ... The concept of user need statement. There has been confusion expressed about what an outcome is.

<Wilco> ... One thing to do is to change the way outcomes are written.

<Wilco> ... User needs statements are not going to be at the same level. They are very generic.

<Wilco> Jeanne: The way this evolved is, in the meeting last night we had the idea that maybe functional needs could be normative.

<alastairc> Wilco: Sounds a bit like a previous thing on lang, feedback about it being a bit abstract. Should target what the actual outome of the user need rather than what the author does.

<Wilco> ... This morning I looked at some examples of what it'd look like.

<Wilco> ... The first thing I found is functional needs don't work, but the detailed user needs could.

<Wilco> ... We had set up a writing process on how to write guidelines for WCAG 3.

<Wilco> ... The error handling subgroup did a lot of work on this. I went to their examples of what this could look like.

<Wilco> ... The way they worked is it defines the tasks and the user need. Then outcomes, tests, methods, how-to.

<Wilco> ... Error prevention is currently in our working draft. I put in the list of main functional categories.

<Wilco> ... I also looked at the main user need from the FAST document.

<Wilco> ... If you look at the actual examples. Help user avoid errors. They had a number of user needs.

<Wilco> ... (reading from slides)

<Wilco> ... They had techniques and ACT rules.

<Wilco> ... I did a couple more where we had the user needs

<Wilco> ... It can be tested in G184, and easy checks.

<Wilco> ... I think this is a viable prototype we could explore more

<MichaelC> https://w3c.github.io/fast/#user-needs

<Wilco> Michael: The general idea of user needs as outcomes, I just merged a PR. We do have users can perceive content, controls, structure, etc.

<Wilco> ... They do sound like outcomes. This was a rewording done independently of this conversation.

<Wilco> ... Maybe this can work.

<mbgower> scribe+ if we went to another term, I would be happy for that

<alastairc> Wilco: I would't mind changing 'outcomes', the term is widely used for other things, happy to change it.

<mbgower> scribe-

<Wilco> Rachael: I like putting things in the perspective of the user, instead of the product perspective

<Wilco> ... I think that's an exciting change. It's more understandable. I think "outcome" has caused confusion.

<Wilco> ... I think it makes a lot of sense to use the word "user need"

<Wilco> Rachael: We have a few options. We could reject it, we could set up a subgroup, or we could work with it.

<Wilco> subgroup

<MichaelC> +1 to work with, or explore in subgroup

<Wilco> Alastair: Particularly on the example we're looking at, on first impression this seems very open. How do you know when you've done that.

<Jaunita_George> +1 to Alastair

<Wilco> ... It feels open-ended.

<Lauriat> +1 to subgroup

<Zakim> mbgower, you wanted to say I'm just getting my brain around it

<Wilco> Mike: I think that user needs are not normative, in the sense of data normalisation. You can have user needs that are the same for different circumstances.

<Wilco> ... I think you'll end up with a lot of overlap.

<Wilco> ... The less overlap we have, the clear testing and reporting will be. User needs makes that less separated.

<Wilco> Jeanne: Sharing a spreadsheet the error subgroup worked with. They had a column on outcomes.

<ShawnT> I think this is the spreadsheet: https://docs.google.com/spreadsheets/d/1BEzSKUsoMQZAwLY5uM5T4Jk7vqKruQT8EaheijQJ7nk/edit#gid=1269671282

<Wilco> ... In each case they took a user need, referenced it to the success criterion, the techniques, anything in easy checks, tutorials, and ACT rules.

<Wilco> ... Each of these user needs has a specific test.

<Wilco> Fazio: Some examples, in form fields a checkmark in green when its correct is an easy win.

<Wilco> Jon: Two things came to mind. This is not just important for testing, but also in fixing and designing.

<jeanne> +1

<Wilco> ... We want to make sure we meet the user needs with techniques. Not all techniques are equal.

<Wilco> ... Maybe you do need a combination of methods to meet all requirements.

<Wilco> ... Need to think about how you meet these, not just how to test these.

<Wilco> ... Second, there is such a large number of user needs, that's too many at a high level. It's important we need those.

<mbgower> +1 to Jon

<Wilco> ... We need something higher level you can show.

<Wilco> Mike: Looking at row 3 error message describes error. I think a larger rollup would be on accuracy maybe.

<Wilco> ... How much of an umbrella we want to report an issue against is how we can make things potentially better or worse for reporting on issues.

<Wilco> Jeanne:Another piece of the work the Errors group did. They took the full list of functional needs, going through each and saying which is not applicable and why.

<Wilco> ... They went through each of the functional needs and put in a need for the topic.

<Zakim> alastairc, you wanted to ask about the difference

<Rachael> +1 to the difference being in phrasing

<mbgower> In other words, we have a lot of qualitative parts of requirements that say "describes". Can we have an overall 'content accurately describes..." that could be applied to a bunch of situations? Is there value in that?

<Wilco> Alastair: The way I'd read outcomes is that they're about the product, and it's maybe just a difference in phrasing that that becomes a user need.

<Wilco> ... Maybe it's just a question of style?

<Wilco> Rachael: I agree, it's a question of style, it doesn't shift the granularity. It's more a question of does phrasing it this way make it easier for people?

<alastairc> Wilco: I think it does make a difference, in that if we use user-needs, it opens up the content to take the role of assistive tech. And that would be a way to pas.

<alastairc> ... that's the kind of thing you can do if you focus on user-need rather than intermediate stuff. But it make the outcome less clear to developers who need to know what AT people can use.

<Lauriat> +1

<alastairc> ... it creates a later of abstraction, but could be useful.

<Zakim> SuzanneTaylor, you wanted to say there might be a difference between "there must be captions" vs "the user needs to understand the narration" - the user need might open the door

<Wilco> Suzanne: I think this might be helpful for specific products to address different audiences.

<Wilco> ... I think an outcome that mentions captions, if the user is they need to understand narration, if the site if for K through 3 they might not use captions but maybe use arrows.

<Wilco> ... It allows for methods to address different audiences.

<Wilco> Jennie: Maybe the user need needs a broader lenze of description.

<alastairc> -1, we need flexiblity as the outcome/user-need might not require the developer to do something.

<SuzanneTaylor> +1 to trying to include both actors (user and developer) in the statements

Could the user need include 2 aspects: both what they need in terms of how to use it, but also

scribe: What they need from the person who is designing/creating the tech or content

<Wilco> Rachael: Chair hat off. That seems to me like a fantastic addition under the method. It would be an interesting way to put that in our guidance.

<Zakim> jeanne, you wanted to interesting idea, but hard to include in normative. Inflexible

<Wilco> Rachael: What is our next step? Have a subgroup try this out? Do we want to try this for a while and check in?

<Wilco> Michael: If we go with a subgroup we need to involve the APA functional needs group

<Rachael> DRAFT strawpoll: 1) To move this to a subgroup 2) To try using "user needs" and recheck at next major milestone

<MichaelC> 2 then 1 - think a subgroup will have more data if we let it ferment first

<Wilco> 1

<jon_avila> 1

<alastairc> 1, I think there's a reasonably big task to try a range of guidelines across the various areas.

<Rachael> 2

no preference

<shadi> 1

<Jaunita_George> 1 or 2 could work for me. I think there are unanswered questions

<Ben_Tillyer> either

<Lauriat> no preference

<Wilco> Mike: I think it would help to have more directions for the subgroup.

<David-Clarke> 2

<Wilco> ... We have clear words for instance. It's a user need that's cross-component in nature.

<Wilco> ... What I'm wondering about is if there's a user need at a higher level. In some way you link incidents of it, maybe by component, or whether we have to be specific about clear words on every scenario.

<Wilco> ... If that group can explore granularity of capturing user needs.

<MichaelC> granularity is the biggest challenge!

<jeanne> 2, then 1

<Wilco> Fazio: Wouldn't clear words be more of a method though?

<Wilco> ... Clear words would be a way to help people comprehend.

<Wilco> Rachael: A lot of times the best way to have this conversation if to have the hierarchy mapped out. I'm not ready to say if it's an outcome or method.

<Zakim> Rachael, you wanted to answer

<ShawnT> 1

<mbgower> 1

<alastairc> I think we'll need to tot up the number of sub-groups we've talked about creating this week...

<Wilco> Rachael: If we have space for a subgroup we'll consider adding it, if not we'll bring it back to the group.

<wendyreid> scribe+

Working Definitions & Category Exercise

<Rachael> slides: https://docs.google.com/presentation/d/18W8M21acEeb5Z1ARdafTQc01_mzOvvjsS184_Qoma0c/edit#slide=id.g15a9271844c_1_109

<wendyreid> [slide 30]

<wendyreid> Rachael: This is goign back to our topic from earlier

<wendyreid> ... not all of the right people are in the room, scheduling mistake

<wendyreid> ... there is still value in having this conversation

<wendyreid> ... [slide 31]

<wendyreid> ... definition for a11y now is whether people with disabilities can use content

<wendyreid> ... maps to user needs

<wendyreid> ... conformance is an important part of following the guidelines

<wendyreid> ... i put voluntarily at the front, conformance is voluntary and compliance is mandatory

<Zakim> MichaelC, you wanted to noun and verb

<wendyreid> ... how do we craft this into working definitions

<wendyreid> MichaelC: Conformance is both a noun and verb

<wendyreid> ... as a verb its an action, they look very different

<wendyreid> [editing the slide]

<wendyreid> MichaelC: Rules we write to define how to meet the guidelines and record it

<wendyreid> Rachael: A starting point

<alastairc> Isn't conformance more of a state than a set of rules?

<wendyreid> MichaelC: Verb is the act of bringing a website into a state of conformance

<mbgower> Is reporting part of conformance?

<wendyreid> ... the act of following the rules

<wendyreid> ... which includes the development, testing, and reporting

<mbgower> or is it separate? Or should it be separated?

<alastairc> Conformance (noun) - A state of meeting the guidelines (rules)

<wendyreid> Rachael: Alastair has mentioned whether conformance is more of a state than set of rules

<wendyreid> mbgower: I conform, I am reporting my conformance

<wendyreid> ... adding these clarifications to the definitions

<wendyreid> ... instead of having to say conformant, conforming

Conformance, in wikipedia: "How well something, such as a product, service or a system, meets a specified standard..."

<wendyreid> ... then you don't have to worry about nuances of the same word

<wendyreid> Rachael: [more slide editing]

<wendyreid> mbgower: Test for conformance, report on testing

*Note, not always a fan of wikipedia, but can have plain language definitions sometimes

<wendyreid> Rachael: I'm not sure conformance is a verb, writing a sentence to make it work

<wendyreid> MichaelC: To conform

<wendyreid> mbgower: We want our website to conform

<wendyreid> Rachael: I do like instead of a state, I like Jennie's suggestion about how well content meets WCAG

<wendyreid> ... can we be more specific in these

<wendyreid> Jennie: Is it content or is it a technology

<wendyreid> ... a solution, a technology element, etc.

<mbgower> conformance level, conformance outcome?

<wendyreid> mbgower: I hesitate to use a level or outcome, the state of meeting the guidelines

<wendyreid> Rachael: Status?

<wendyreid> mbgower: That's meeting conformance, we should be careful about whether conformance being a thing you meet or report the success of achieving

<wendyreid> ... there are a lot of nuances

<Zakim> Lauriat, you wanted to suggest noun 1: "conformance model" to differentiate from noun 2?

<wendyreid> Lauriat: Was thinking, in order to differentiate

<wendyreid> ... for the first noun, the rules we write

<wendyreid> ... I tend to refer to as conformance model

<wendyreid> Jennie: A good way to test the definition we're using, writing the opposite as a sub-bullet and see if it still holds true

<mbgower> S/he did not conform to the requirement. They are not conformant.

<wendyreid> ... if we take the conformance noun, if the state of meeting the guidelines, or a state of failing to meet

<wendyreid> Rachael: Non-conformance

<wendyreid> mbgower: Thank you, Jennie

<wendyreid> ... thank you for stating things so clearly, so helpful

<wendyreid> ... is conformance a binary

<Rachael> +1

<wendyreid> ... is it binary, or is it a level

<wendyreid> ... we've had discussions about that, right now we have levels

<wendyreid> Jennie: Thanks mbgower, to add, if I think about WCAG 2.0 compliance, we often say full compliance vs partial

<wendyreid> ... but I wonder if that breaks down level even further

<wendyreid> ... the word of conformance on its own, a single rule conformance or the whole kit and caboodle

<wendyreid> ... a tricky word

*: )

<wendyreid> ... we have to get to that

<wendyreid> Rachael: Hopping between conformance and compliance, want to make sure we focus on conformance

<wendyreid> MichaelC: I'm intrigued by the opposite of conformance

<wendyreid> ... opposite of a website is no website

<wendyreid> ... opposite of conformance is 0 accessibility

<wendyreid> ... whereas a site that doesn't meet compliance requirements could pass most tests

<wendyreid> ... could largely be in a state of conformance with some exceptions

Merriam webster opposites to conformance (near antonyms): contrast, discrepancy, disparity

<wendyreid> ... hate to go into levels, but this is where my brain is taking me

scribe: unlikeness, deviance

<Zakim> Lauriat, you wanted to ask about W3C's definition of conformance? I can't find it by searching w3.org

<wendyreid> Lauriat: W3C's definition? Is there an overarching idea of conformance?

<wendyreid> ... can we refer to that

<wendyreid> ... all the pages I found were owned by WAI

<shadi> https://www.w3.org/WAI/fundamentals/accessibility-usability-inclusion/

<wendyreid> Shadi: As far as I remember, there isn't a single definition, there is a page called Accessibility Usability Inclusion, its a description

<Rachael> This is our definition from the spec: Conformance Satisfying all the requirements of the guidelines. Conformance is an important part of following the guidelines even when not making a formal Conformance Claim.

<Zakim> ShawnT, you wanted to say conformance versus conformance level

<shadi> https://www.w3.org/WAI/fundamentals/accessibility-intro/

<wendyreid> ShawnT: I just wanted to say, myself personally, conformance is a binary, and then we talk about levels

<shadi> "Web accessibility means that websites, tools, and technologies are designed and developed so that people with disabilities can use them"

<wendyreid> ... we add level to modify conformance

<MichaelC> https://www.w3.org/TR/2005/REC-qaframe-spec-20050817/#specifying-conformance

<wendyreid> MichaelC: At least historically, the W3C definition comes from the QA framework

<wendyreid> ... this document was influential in the model of WCAG 2.0

<wendyreid> ... this guidance is no longer actively recommended

<wendyreid> ... but is not rescinded

<wendyreid> ... we've generally followed it

<wendyreid> ... the fact that it's mostly WAI pages, other W3C groups don't see a need to worry about it as much as we do

<wendyreid> Rachael: Perspective differs between groups

<wendyreid> shadi: Many other standards are also not adopted by policies

<wendyreid> Rachael: Do we want to leave this here as capturing our ideas?

<wendyreid> ... do we want to wordsmith this down to tighter definitions

<Ben_Tillyer> +1 to leave here

<jeanne> +1 that's it's good enough for now.

<wendyreid> shadi: As others are +1ing, I would look forward to moving ahead, big picture the discussion were really helpful

<wendyreid> ... where do things fit in

<wendyreid> Rachael: Testing, we talked about these

<wendyreid> ... [slide 32]

<wendyreid> ... any additions to these working definitions?

<wendyreid> mbgower: I don't love mechanism

<wendyreid> ... but I can't think of an alternative

<wendyreid> wendyreid: Criteria?

* "a way" ?

<wendyreid> mbgower: Mechanism implies that there can be a test harness, but the test is much simpler than that

<Lauriat> +1 on both counts and +1 to Jennie's suggestion

<wendyreid> Jennie: Criteria could have other implications

<wendyreid> ... a "way", no pun intended

<wendyreid> mbgower: Way is probably clearer than mechanism

<mbgower> thanks

<wendyreid> Rachael: We will come back to this conversation

<wendyreid> ... [slide 33]

<wendyreid> ... Regulatory Environment

<wendyreid> ... I have tried to pull the content from the email list

<wendyreid> ... Policy, public or organizational

<wendyreid> ... public referring to government

<wendyreid> ... organizational to guide operations

<wendyreid> ... Regulation, the simpler version

<wendyreid> ... a written rule how the ideas of a law are to be implemented

<wendyreid> mbgower: Can we make it to "whom"

<wendyreid> Rachael: other alternative, any standard, statement or procedure that is adopted

<wendyreid> shadi: Looking from the discussion we've had, there are many sensitivities here

<wendyreid> ... backing up, this discussion emerged from wanting to look at enforcement more generally

<wendyreid> ... there is a standard and conformance which is voluntary, but then there is compliance

<wendyreid> ... we may want to step away from trying to define these terms, step back to say the enforcement of a standard

<wendyreid> Jennie: A request, when we're closer to publishing, review for passive voice and plain language

<Zakim> mbgower, you wanted to say I like the shortened one better

<wendyreid> ... "to be implemented" is an example

<wendyreid> mbgower: Of course things written like policies will be passive but we don't have to write them that way

<wendyreid> ... I don't think we're losing much by trying not to cover every base

<wendyreid> ... I've worked with the BC legislature, the law is very broad, with regulations within

<wendyreid> ... the first one is very accurate

<jenniferS> +1 to Shadi's suggestion to sidestep defining policy & regulation by focusing on 'enforcement'

<wendyreid> jaunita_george_: If we want to flush this out more, it might help to work with an attorney

<wendyreid> ... one who works with a jurisdiction like Japan or EU, where this is more in the civil law space

<wendyreid> ... in NA regulation has a different application

<wendyreid> ... we want to make sure we're using a definition that can be understood by anyone

<jenniferS> +1 to Jaunita re getting input from other global jurisdictions for applicability

<wendyreid> ... I'd be happy to work on rewriting for plain language

<mbgower> nice point, Jaunita. ONe question: is this for internal guidance or for exposing in the standard?

<Zakim> Rachael, you wanted to suggest way forward

<wendyreid> Rachael: Wanted to suggest a way forward, what if we capture compliance

<wendyreid> ... make that the top level

<wendyreid> ... capture the sensitivity and i18n issues

<mbgower> nice Rachael!

<wendyreid> ... note that these need work, and then move on

<Ben_Tillyer> +1

<shadi> +1 to Rachael

<alastairc> +1

+

<mbgower> +1

+1

<ShawnT> +1

<jenniferS> +1

<jaunita_george_> @mbgower I would say that policy is internal to the organization, but regulation would be an external requirement, but that's really North America specific I think

<wendyreid> Rachael: These are working definitions, not things that will go in the standard

<wendyreid> ... it's worth wordsmithing at some point, but not today

<alastairc> I wonder if there is a place we can put these so they are easy to refer to.

<jaunita_george_> +1

<wendyreid> mbgower: Classic IBM, if people are really analytic, they put the key information at the top, the details below, it just makes everything else flow from it

<wendyreid> ... if we can keep that in mind

<wendyreid> ... putting the simplest conclusion up front then elaborating

+1 to mbgower

<jaunita_george_> keep

<mbgower> lose it

<Ben_Tillyer> -1

<jaunita_george_> +1

<wendyreid> Rachael: Compliance, should we remove mandatory?

<alastairc> +1, keep

with the ? ?

<jenniferS> +1 to lose

<alastairc> as conformance is the non-manditory

+1 to keep with the ?

<wendyreid> ... would remove the ?

<ShawnT> -1

<mbgower> yes

<jenniferS> yes

<wendyreid> ... I am going to remove it here, but make a note that compliance is normally mandatory, and conformance is voluntary

<ShawnT> yes

yes

<wendyreid> Rachael: I think we can move on

<wendyreid> ... anything to add?

<wendyreid> ... [reviewing slide 33]

<wendyreid> ... [slide 34]

<wendyreid> ... can someone please speak to these visualizations

<wendyreid> jenniferS: My understanding is that MichaelC wanted a temp check to get thoughts on these, but AWK actually

<wendyreid> ... Andrew and Michael and Ben and I tried to better clarify what we all thought

<wendyreid> ... using these drawings as discussion points

<Rachael> https://docs.google.com/presentation/d/1Hp460oEvI6dkCQJ3XsLDkvrGsnh9Ub8O/edit#slide=id.p7

<wendyreid> ... it's interesting that the drawings keep the policy regulation within accessibility/WCAG

<wendyreid> ... if you go to slides 11, 12, 13, all Andrew

<wendyreid> ... discussion of things of separate slides, 15 is the start of mine

<wendyreid> ... there's additional work Andrew had in his photoshop

<wendyreid> ... basically what they are trying to say, top left, in a wishful world, policy and regulation and conformance and a11y would all be the same

<wendyreid> ... greater a11y

<wendyreid> ... moving to the right, the more realistic endstates

<wendyreid> ... policy and regulation would only cover a part of what conformance is

<wendyreid> ... in the bottom right, where conformance might land

<wendyreid> ... conformance might have different levels

<wendyreid> ... A AA AAA, or bronze silver gold

<wendyreid> ... you may be conformant with an initial level, but additional policy requirements, which is not fully accessible

<wendyreid> mbgower: Slide 16 three states, the idea here is the user needs are not entirely covered by guidelines

<wendyreid> ... the goal is to make it as much as possible

<wendyreid> ... do the regulations need to have thresholds set by us

<wendyreid> ... can we express it differently from levels

<wendyreid> ... something that measures the relative state of an application

<wendyreid> ... a report in progress, if I'm 40% compliant once, then 55% compliance, neither achieve A right now

<wendyreid> ... there's no motivation for a vendor to put more into it

<wendyreid> ... if there's something that shows fluid development, to see progress

<wendyreid> ... you can also potnetially not have to say what the conformance is for a jurisdiction

<wendyreid> ... we may want to set a minimum

<wendyreid> ... we're defining what can be done to make sites more accessible

<wendyreid> ... regulators may decide what in their jurisdiction is applicable

<wendyreid> ... the more compliant you are, the more chance you have of achieving a11y in different jurisdictions

<wendyreid> ... the motivation exists for recording progress, and for orgs to put where they have higher expectations

<Ben_Tillyer> To add to mbgower. There was also a concern voiced during this discussion on Tuesday where if conformance threshold was too high, less jurisdictions would potentially adopt WCAG

<wendyreid> Ben_Tillyer: Concerns, if the conformance threshold was too high

<wendyreid> ... less jursidictions would potentially adopt

<wendyreid> jenniferS: Going back to my usual refrain, we need more information globally to better understand implementation constraints

<wendyreid> Ben_Tillyer: Could we have the final slide?

<Zakim> alastairc, you wanted to ask if policy regs often go outside of WCAG, e.g. to non-digital accessibility?

<wendyreid> alastairc: Tangent, is it worth recognizing that there is more to a11y than digital a11y

<wendyreid> ... physical world

<wendyreid> ... the green circle may be bigger

<wendyreid> Rachael: Would it help to qualify a11y as digital

<wendyreid> shadi: I agree, even if we just stay in digital

<wendyreid> ... the world of digital a11y is broader than WCAG

<wendyreid> ... this is a very WCAG-centric view

<wendyreid> ... the EAA for example requires customer support be trained for a11y

<wendyreid> shadi: You do not need to conform to be accessible

<wendyreid> ... if we look at WAI ARIA

<wendyreid> ... it does relate, but it goes beyond

<wendyreid> ... providing an a11y statement is another aspect

<wendyreid> ... written towards users, another type of regulation

<wendyreid> ... it's not filing with the government, it's reporting to the user

<wendyreid> ... helps build expectations

<wendyreid> ... what we're trying to do is make the digital world accessible

<wendyreid> ... what role does WCAG conformace have in that space?

<wendyreid> ... i.e. the maturity model

<wendyreid> ... does that have a role to create non-technical accessibility aspects

<wendyreid> ... not to get into complexity, I also disagree, I tinhk this is a particular perspective

<wendyreid> ... policy is a subset, it is but also has other things, like customer service

<wendyreid> ... like in EU, providing instructions in the local language

<wendyreid> ... policies and regulations are not a subset, but overlapping venn diagrams

<wendyreid> ... if I would draw it, I would draw a big circle of a11y, with lots of smaller a11y circles, like WCAG, policy, others

<wendyreid> ... ICT, the maturity model

<wendyreid> ... guidance we have at WAI, or others

<shadi> UAAG, ATAG, WAI-ARIA

<Karen> scribenick: Karen

John: I agree with Shadi as circle
... see them not as subset but as overlapping
... if we set bar too low, we are focused on minimum
... in US, ADA, there are no technical web standards, it comes down to functional tests
... what we might need to do
... if we have a score
... how does that score actually affect use by people with disabilities
... and from a courts perspective, how does that map to @'...how do we sort that out
... people need guidance making technical choices unless you pull expert witnesses to clear that out
... go back to requirements of WCAG 3

Rachel: people are welcome to fix the circles
... wondering if it makes sense to have a guide for enforcement agencies
... EOC
... and one geared towards law makers who want to assess the impact of conformance and why support full confirmance
... folks may be thinking about going to court
... arguing for WCAG to be implemented and adopted in whole

Jennie: Along similar path, I see citizens who don't see themselves represented
... not seeing enforcement and role of user
... standing up that supporting group
... how do people get involved in how to remedy processes
... especially those who may be in a vulnerable position
... rather than being a benefitter

Rachel: Is there a way you would add that into the diagram?

Jennie: the policy regulation circle is slightly different
... there is policy and regulation people
... and group that are the vetters, decision-makers, or governmental role
... and there is this user portion to the policy side
... to be able to be informed

<mbgower> scribe+

<mbgower> Karen: I have worked a lot with graphic interpretations. I wonder if you have thought of a mandala

Jennie: and be able to participate in enforcement; maybe there is a better word

<mbgower> ... If we have paper I could share. It's an easier way to show representations of constitutencies.

<mbgower> Rachael: Could you share?

<mbgower> scribe-

<Zakim> mbgower, you wanted to say that we had similar discussions about the policy 'circle' too

Karen: maybe consider a mandala model for the graphics

MichaelG: I cannot remember the original goal of different shapes
... we are trying to convey web content web accessibility needs and regulations
... and we need to balance between making WCAG so big that we cannot get regulation to follow and adopt
... we have to find ways to expand what we cover of user needs
... and pull policy regulation along in our wake for adoption
... it began as a Venn diagram
... there are regulations outside
... but what we cared about was in relationships to accessibility
... we're the WCAG team and that is what we are concerned with
... before we build a model of the universe
... we should figure out what we are trying to show
... this was meant as a thought exercize

Rachel: thank you for the reframing

<Zakim> jenniferS, you wanted to say that the illustrations are for conversation stimulation only, right? and should be replaced by verbiage?

Rachel: appreciate where you were going and the responses

JenniferS: the illustrations were for conversation stimulation only and augmented with verbage

Rachel: intent was for conversation

Shadi: I hope I can list some of Mikes' fruistrations
... I respectfully disagree with assertion about adoption of WCAG
... if not easy to be adopted, not a one liner; it's about how it is adopted
... it's important because if we don't map out what's important
... we are using WCAG as a definition of Accessibility and it's not

<Rachael> +1 to that the takeaway that wcag does not equal accessibility

Shadi: if we can map out what it is and not keep having discussions of policy v WCAG aspects
... interested in interface with standard and the policy
... and focus WCAG more on the technical aspects so that we can move forward more effectively

<Zakim> Lauriat, you wanted to ask a kind of foundational question of "vision" - does this mean the ideal state we want to work toward or documentation of the current state?

Rachel: final say on diagram

<Ben_Tillyer> Can we make sure there is a text alternative to these diagrams?

Shawn: I am still trying to decide what we are trying to work through
... former diagram looking at reality of filters
... and this looks like diagram of the current state
... understand what we are trying to work through and express; and what it is we are building toward

Rachel: great questions and final say
... from a chair perspective, it's to start to tease apart different perspectives
... point of exercise is there is this point or question of overlap
... will policy expand or not pick up all of WCAG is an interesting question that came out of it
... WCAG is not all of it
... what we expect from policy is going to be different
... as current text or audio description of image

<jaunita_george_> No matter how WCAG 3.0 looks in the end, we'll have to engage in a messaging campaign to get regulators/legislators/the business community to adopt it

<jaunita_george_> And understand how all of this guidance works together

Rachel: we have an large circle with digital accessibility and four other circles with overlaps with policy, user needs being informed
... and these other standards like ATAG, UAAG, maturity model, etc.
... it's a complex environment
... I would like to take the last half hour to do a brainstorm
... does anyone have difficulty with Google docs
... Like to have people go through a Google doc
... and write down what they think

<mbgower> Wishful: three overlapping circles 1) User needs (largest area); 2) WCAG Guidelines (almost covering all user needs); 3) policy/regulations (almost covering guidelines)

Rachel: What is in WCAG
... I'll say WCAG guidance, the non-conformance, the conformance, and regulatory ennvironment
... create headings for each of those
... take a few minutes to talk and brainstorms
... what things fall into it

<Rachael> https://docs.google.com/document/d/1ihe6BygeGYf8rc_u6supy6bm2IB7X4fBsMNkkn8A8h0/edit#heading=h.1ufxgmojqgu4

Rachel: add to it, make bullets or comments
... use this as a starting point for what falls into each of these category
... a heading one and three H2s
... WCAG Guidance, Conformance Model, Regulatory Environement
... if there is another category please add

<Rachael> https://www.w3.org/WAI/GL/task-forces/silver/wiki/Substantial_Conformance/Example_Scenarios

Rachel: Conformance group used slightly different divisions
... they used technical standards, guidance and policy regulators
... more interested in hearing what people think is compliance v conformance for example
... Feel free to enter something, or go in queue
... just talk about things

[pause while people are thinking]

Michael: you want everything we see in each of these areas?

Rachel: Add the things that you think fall into each of these areas
... it's an attempt to help us scope our conversations and avoid
... slipping into compliance when we should be in guidance

Michael: normative and non-normative?

Rachel: if helpful, go ahead and divide it
... we are in free-form brainstorming
... Let's discuss as a group
... If you are editing other people's content, add comments
... Shawn, you should not have to
... in general, avoid editing other people's comments
... we will talk about them as we get there
... Take about three more minutes
... Let's wrap up, finish what you are typing and let's talk through this

<Ben_Tillyer> +1

<shadi> +1

<Jennie> +1

<jaunita_george_> +1

Rachel: Please "+1 when you are done

<jon_avila> +1

<SuzanneTaylor> +1

<jenniferS> +1

<ShawnT> +1

Rachel: I will save this version so we have the content in its original form
... why don't we start under WCAG guidance and talk through what we think is here

Michael: that is all me
... I could not figure out what other people are saying; I did not get through the other parts
... we can go through this or skip

Rachel: from your perspective, all the normative and non-normative content
... next one has standards for delivering accessible outcomes

JenniferS: I did not fully understand the task, but I wrote what I think needs to be in WCAG guidance
... and added we need more people to get this done
... if you have been here for years, you know there are never enough hands

Rachel: for simplicity I acknowledge your point but will take it out since it's not content

JenniferS: I thought it was a brainstorm to understand what's in everyone's head

Jennie: I know that would be a marker of success for the work I do

Rachel: none of this content seems controversial
... does anyone disagree or think this content should go into another space

Michael: I like what Jennie wrote

<ShawnT> +1 to user roles

Michael: a good way to slice the information
... may be way to slice info a different way

Rachel: Thank you, Jennie

Jennie: this speaks backwards and forwards to compliance in MN
... all roles that touch digital assets of any kind, from writing email to writing @, apps and software
... everyone is required to comply
... want to make sure it's captured as being a requirement

<Zakim> ShawnT, you wanted to speak about roles

<jon_avila> EOWG did some work on roles and WCAG https://www.w3.org/WAI/EO/wiki/ARRM_Project_-_Accessibility_Roles_and_Responsibilities_Mapping

Shawn: When we start talking about user roles

<jeanne> EO mapping was used to help design the How-To Activities

Shawn: I thing about ARM project
... where does that fall into Silver
... it was in my head as to where that fits in Silver and WCAG
... ack Rachel

Rachel: Where I think this is intended to be is in the how-to documents
... the mapping goes there
... Jeanne confirms the mapping was intended to design the How-To Activities
... move on to conformance model
... optional statements of conformance not mandatory; John what do you mean by that?

John: Difference between making a model and a statement of conformance
... I think it's good we allow for people to not have to
... two things; help people make public statements and not requirement them to
... don't want situation where we write certain things into conformance; things not covered
... and hae situations is not in conformance; not have it in there
... rather leave it to regulators to decide
... really more to local control of what is or isn't covered

MichaelC: bullet about need for input from global community
... I added also from different global legal regimes
... we should check that

Ben: I wrote not mandatory on that first point
... I think having mandatory statements...from corporate, governance POV
... would definitely help some large companies

Rachel: Thank you
... next one says WCAG 3
... inclined to delete that unless someone feels strongly; may be due to bad directions
... next one is indications of how to go beyond basic conformance

Alastair: Maybe not be best place
... where people are looking to conform
... we need something around this is meeting those outcomes
... and the things you can do

Rachel: How a user without expertise can choose between product cites

SuzanneT: I added because I would love to see
... if a team officially has decisions to go beyond basic conformance, so get a badge for going beyond
... if someone has short-term memory issues, then maybe someone would use that site
... and perpetuate by social workers or organizations to "look for this stamp'

Rachel: I am adding visual indications of when sites meet a certain level of conformance

Suzanne: And divide by different types of accessibility
... different sites do things differently; give people places to go

Rachel: I think I put in reporting
... question of should we require or request a way of reporting
... I think it falls under conformance model if we do
... Need input from the global community
... Michael that was yours

<jon_avila> I agree with Susan that right now it's hard to know whether a site has done anything for accessiblity and that it will work for you as the user with a disability until you try to use it and get blocked.

MichaelC: yes

JenniferS: yes, that was me; also collaborate with global community
... and equity in conformance model needs to make sure certain disabilities should not be excluded
... try to start making a list of the folks to include
... I also added into the regulatory environment
... just trying to brain dump
... feel free to add others

Rachel: how to respect prerogative of regulators and respect the standards
... conformance model...falls there?
... any other comments
... We don't have time to go through the regulatory environemnt
... going to switch doc to comment rather than edit
... please go ahead and jump in to do that to comment over the next few days
... we will go through this in more detail in future
... Other comments before we close for this session?
... when we come back we will switch gears to talking about the conformance proposals
... set the stage for how we want to talk about them
... and we will close out
... we are back here in an hour
... enjoy lunch

[adjourned]

<jenniferS> scribe: jenniferS

<Rachael> https://docs.google.com/presentation/d/18W8M21acEeb5Z1ARdafTQc01_mzOvvjsS184_Qoma0c/edit#slide=id.g1544c217c09_2_213

on slide 4

slide 40 -

we wanted to start with revision criteria to evaluate if conformance model works or not

these are the different pieces to inform that taxonomy

40-44 go through the criteria to consider

first are based on requirements doc

design principles & requirements

design principles should target larger range of people with disabilities (PWD)

address emerging technologies

<mbgower> Drink our own champagne

follow our accessibility guidelines

intent we create should be equally accessible

include pwd in process, facilitate global participation

<jenniferS_> use cases…

<jenniferS_> situation 1: bugs and other issues

<mbgower> [slide 42]

<jenniferS_> slide 42

<jenniferS_> slide 43

<jenniferS_> anyone from conformance options subgroup to speak to those?

<jenniferS_> Shadi: these were not written for use cases for conformance

<jenniferS__> w3c will not be writing policies but we will provide guidance

<jenniferS__> [note my IRC dropped again for a bit]

<Fazio> Maturity Model can address Shadi's concern

<alastairc> You'd need some form of prioritisation to allow policy to use "reasonable efforts", the binary pass/fail doesn't support it.

<jenniferS__> is this small business issue something that we should look at? or is it a policy issue?

<jenniferS__> Rachael: I think we talked quite a bit about that in the last session. What should we title these, Shadi?

<jenniferS__> Will stub in Considerations for WCAG3 Conformance, for now, slides 42-43

<jeanne> Use Case for WCAG 3.0 Conformance Editor's Draft https://w3c.github.io/silver/use-cases/

<jenniferS__> Janina: we are mindful that if we could describe how to make something accessible and if you didn't do that

<jenniferS__> we should continue to define as we can how to make all kinds of content accessible

<jenniferS__> we may not know what to do about virtual environments

<jenniferS__> others we have pretty good ideas

<jenniferS__> is it reasonable to say in all situations that because something exists we know how to do it, and everybody has to do everything?

<jenniferS__> Shadi noted small business but shouldn't other compliance regulations and courts cover?

<jenniferS__> That's their specialty. We know how to write the guidelines,

<jenniferS__> they know how to do compliance

<jenniferS__> shift the paradigm of what conformance means

<jenniferS__> from everything we know to what regulators know

<Zakim> mbgower, you wanted to say there are a few other things that could be considered, such as content from templates, content from design systems, etc

<shadi> +1 to janina

<jenniferS__> mbgower: respond to Janina, I think that's what regulations are for

<jenniferS__> for example, Ontario, different adoption rates depending on entity size

<jenniferS__> it's more realistic for them to presume what to apply it to

<jenniferS__> that said when we think of our own stuff there are some things that are extremely valuable, motivation and reporting

<jenniferS__> for example my content is developed from a design system or template

<jenniferS__> so content of design system or template go through user testing to ensure accessible and such

<jenniferS__> what happens with stuff made from the system and templates?

<jenniferS__> do those lose their scoring if they don't go through all the resting again?

<jeanne> Jeanne notes thatt MikeG's example is in Situation 4

<jenniferS__> those are interesting questions to me

<jenniferS__> fazio: this is the essence of why started maturity model subgroup

<jenniferS__> hoped to tie into conformance, but discovered not the best route

<jenniferS__> to provide a policy framework for policy makers

<mbgower> I don't think situation 4 covers what I'm talking about, Jeanne. it might for a CMS system owned by someone else. Not if it's an in-house CMS or design system.

<jenniferS__> I don't think ag and guidelines should dictate policy

<jenniferS__> it should provide technical guidance and maturity model in accessible platform architectures (APA) etc drive policy

<jenniferS__> Jennie: in state of Minnesota, we're exploring WCAG 2.1 vs s.0 as state standard

<jenniferS__> and procurement work

<jeanne> +1 Jennie

<jenniferS__> when we sync with multiple states or orgs it gives us strength.

<jenniferS__> it is helpful when it is accepted as achunk

<jenniferS__> yet I understand the scoping requrest

<Judy> +1 Jennie, strength in numbers of coordinated purchases & coordinated conformance

<jenniferS__> I wonder if instead of considering small biz example, use idea of level of effort could be 3 pillars: knowledge time it take to do it and tools

<jenniferS__> Shadi: large orgs do not have an interest in different standards in diff countries, regions, etc.

<jenniferS__> if we divide this up as described, what is the right level of effort for who?

<jenniferS__> whatever the tech standard is there is an adoption for it.

<jenniferS__> I think Gregg van derH said something about harmonizing technical reqs.

<jenniferS__> for example the web accessibility directive in Europe completely removed live media because they thought it was too difficult for some orgs

<jenniferS__> this is the opposite of what we want to have

<jenniferS__> if we don't provide guidance and support on technical levels

<jenniferS__> help policy makers t put concepts in reasonable effort or accommodation

<jenniferS__> for example re video games

<jenniferS__> if game is about how quickly one hits the keyboard you don't have to change how the game works

<jenniferS__> some things are by nature less accessible

<jenniferS__> it's about trying to help policy makers adopt

<jenniferS__> Rachael: move on with the q

<Zakim> alastairc, you wanted to say that WCAG needs to provide some mechanism for policy makers to differentiate effort.

<jenniferS__> Alastair: building on Shadi,

<jenniferS__> org must put in reasonable effort concept

<jenniferS__> wcag2 doesn't make it easy to differentiate

<jenniferS__> for example live media, maps, we didn't make it easy for policy makers to draw any sort of levels. it didn't help with level A, AA, AAA…

<shadi> +1

<jenniferS__> we do need something like issue severity, or other differentiation of priority.

<jenniferS__> Janina: had interesting hallway conversations

we had to be pretty convincing it was achievable

don't think 508 is going away

there'sas much room for success

the world has changed for the better, not only for PWD but curb cuts

until this week I thought if you did it live you could be sloppier about spelling and timing

then I heard about the SAUR (synchronization requirements)

how synced must be to be comprehendable

if using captions or relying on descriptive text

then we heard a major broadcasting organization captions 100% of broadcast, 40% is live

we have to get smarter about how we give guidance

SLIDE 44

criteria to evaluate success

as coverage iincreases

slide 44

slide 46

<mbgower> nice set of questions on slide 44, Jeanne

this was the slide for evaluating results

Francis, John foliot & Jeanne worked on a year ago

looked at criteria from symposium and final report in 2012

adapted metrics they recommended

tested those proposals based on the metrics and plan

then Francis put in a tremendous amount of work building dummy websites with known errors for testing

so this is done and ready to evaluate different proposals

these 5 came out -- slide 46 second bullet inset list

sensitivity is important. if small change is made to a website, then small change should be reflected in scoring, large change reflected in large score change

adequacy: we want to be sensitive to rounding errors potential to impact scoring

complexity: don't want a complex proposal, i.e. the % proposal

<Rachael> Wiki: https://www.w3.org/WAI/GL/task-forces/silver/wiki/Conformance_Architecture_Testing

<Zakim> mbgower, you wanted to say I'm so grateful to Jeanne, Shawn and others who have done this due diligence to help us learn from and know about prior relevant work

mbgower: pause and ack all the work that Jeanne and Shawn and others have done

<Rachael> +1 to the Silver taskforce and task leads. Thank you

and patience to hear us rehash

<shadi> +1 to mbgower

<Jennie> +1

Rachael: slide 47

where we started the week

different conformance approaches

please read through these concepts over the next week if you haven't had a chance yet.

we will revisit this

many of these can be done at the test, outcome or entire level

slide 48

would like to discuss in more detail, hear pros/cons

not meant to be comprehensive

<Rachael> https://docs.google.com/document/d/1bvZofimn35x_Jk3u3htpvQjTPbS7-KI_nQTQm8pZ9G0/edit#

to keep track of what we talk about

first option: everything passes or it fails

pros/cons?

feel free to type in doc

<Zakim> jeanne, you wanted to talk about Silver research

Jeanne: number of stakeholder groups… consistently came up the problems with perfection or failure

making it difficult to motivate people to want to have good accessibility as it seemed unattainable

opened companies to legal risk due to single error, no matter how minor

<Zakim> mbgower, you wanted to say pro

mbgower: allows possibility to capture critical considerations, i.e. blinking text

actual requirement to reduce harm

reading through the pros & cons

any other comments?

mbgower: re supports automated eval over any other eval, please explain?

fazio: while I agree with all of Jeanne's statements and cons, I appreciate that it pushes to perfection

it's good for users if folks strive for perfection, but agree with the cons

moving on to percentages…

<matatk> scribe+ matatk

<matatk> Rachael: Pros and cons for percentages

<matatk> Rachael: in FPWD we counted everything that worked and created a percentage pass/fail

<matatk> Rachael: (chair hat off) Having people count the number of passes vs errors ended up being heigh effort, high complexity.

<matatk> qv?

<matatk> Fazio: There's a big possibility of leaving out disability groups entirely, e.g. cognitive. It's 80% accessible, e.g., but to whom?

<matatk> ... Unless we break it down (and include different groups).

<matatk> Rachael: Pros: seems to be potentially scalable; if there's a max good score, all scores are isomorphic with percentage anyways (if there's a score above which it's not possible to socre, then any other score is a percentage of that); allows for more granularity; allows for different levels of effort

<matatk> ... Can work for automated tests where a tool is doing the counting, but not for score as a whole; good for sensitivity generally.

<matatk> mbgower: Percentage gives you the ability to define a cutoff; you can also see how badly something fails and how well something passes.

<matatk> ... Allows to compare results (self, or other).

<Zakim> Lauriat, you wanted to add migration support

<matatk> Lauriat: Also better supports migrating from one iteration of WCAG to a newer incarnation - it's not that you completely failed the new one; it's that you passed this much of it.

<matatk> Rachael: Cons of percentages: will people stop just after the cutoff; what does 95% mean? (5% left out; 5% of site unusable); levels of success ascribed to percentages could be off-putting; too many instances to count (for manual tests); diffciult for some users to do w/o tech support.

<matatk> MichaelC: If your pass score is 70% then your failure rate is 30%

<matatk> Rachael: Complex to derive formula.

<matatk> ... [scribe is missing a couple of the last pros/cons but they're in the slides]

<matatk> mbgower: People stopping just after the cutoff can be a pro - there will be people who strive for more, and they'll be rewarded by recognition of their effort.

<matatk> ... VPATs at the moment punish people trying to be more accurate.

<matatk> Jennie: 2nd bullet could mean what's left out or is unusable, but for someone striving for 100% it may be hard to know what to do next, to improve.

<Zakim> mbgower, you wanted to say "95% accessible" needs to be reworded or reframed

<matatk> Jennie: How do I move from 95 to 100?

<alastairc> Similar to just looking at "3 WCAG AA SC fails", you have to go into the results to get the next steps.

<matatk> mbgower: I think we should try not to have things like '95% accessible' messaging effort needed. 100% doesn't mean you're 100% accesibile, but _compliant_

<matatk> ... I don't think this can ever by synonmmous with accessible.

+1 to highlighting that the standards cannot ensure 100% accessible.

<matatk> shadi: Standards give an expectation: if I request an ADA room, I don't expect full accessibility. It meets a level of expectation. Does 95% mean I can't use it 1/20 times?

<matatk> ... Percentages cascade a lot of info; may be hard to understand.

<matatk> JohnRochford: Wouldn't we be thrilled if millions of orgs met Bronze? Feels a little wishy-washy. Can we make Bronze less hard to acheive than other levels, to provide an incnetive to strive for more?

<matatk> ... So that of the 98% of sites in the world, that are inaccessible, more would be so.

<Jennie> * to John's point: 10,000 steps as a daily goal seems huge if you are relearning to walk

<matatk> Ben_Tillyer: I think how we get around the 95% accessible discoussion is what we're moving on to next - points. There's no limit to the number

<matatk> ... of points, so you can keep going for more. You will need so many to meet a given level like Bronze.

<alastairc> Is the points approach tied to the functional needs groups? The example is, but I assume points doesn't have to be.

<matatk> Ben_Tillyer: Percentages don't show you how many of the tests you ran (were N/A)

<matatk> ... Thereofre percentages aren't useful for comparing sites.

<matatk> Rachael: Points. Please add pros and cons to the slide.

<matatk> ... A cumulative number of points, assigned based on some criteria.

<matatk> Jennie: Back to JohnRochford's excellent point.

<matatk> ... 10k steps a day is a goal, but you may be just learning to walk.

<matatk> ... That same consideration JohnRochford raises has to be applied to all methods.

<matatk> ... Parking lot issue?

<Zakim> mbgower, you wanted to say and to say not applicable would mean there isn't a conformance need met. potential to game but what's the benefit to doing so?

<matatk> Rachael: Not going to parking lot, as it's already a criterion we've agreed to use, but good point.

<matatk> mbgower: if 70% of the site is irrelevant you have a higher percentage. Yes, you could game it. If it's so bare bones you get a high rating, but you can't do much with the site.

<matatk> ... If you make a simple, beautiful, good design, that gives the user what they need, this could be great, but it would score low in points because of a lack

<matatk> ... of bells and whistles. So it goes the other way with points.

<Zakim> alastairc, you wanted to ask a question on points

<jeanne> I think that the not applicable items would be excluded from the percentage. A good small site would get a high score, a bad small site would get a poor score

<matatk> alastairc: The e.g. for points was tied to proposal that had functional need groupings, scoring per functional needs. Is that necessarily part of points, or could they work differently?

<matatk> Rachael: Not necessarily part of the points. You're assigning a set value based on some criteria.

<matatk> Judy: What I'm not hearing is consideration for absolute barriers e.g. flashing images - not only can't use it, but it is dangerous to use. Not sure where this fits in to percentages or ponits. The category

<matatk> ... of absolute barriers needs to be clearly weighted.

<Zakim> Lauriat, you wanted to note that we cannot create a conformance model that prevents people from misrepresenting things, but we can make one that can help uncover that kind of thing

<matatk> Rachael: This concern could be appied to any of these systems.

<jeanne> Judy, we have discussing Issue Severity, and I think that is where we have been addressing this important issue

<matatk> Lauriat: It's impossible for us to create a system that stops people misrepresenting things.

<matatk> ... We need to figure out if the conformance model reflects the experience of pwd.

<matatk> ... E.g. if you have 99% but flashing, that's not good.

<matatk> ... If you have 80% and it _is_ good for pwd that's a better reflection.

<matatk> jenniferS: How do we know that the impacts are for some of these problems for users?

<matatk> ... Some could be very serious.

<matatk> shadi: Was there a discussion of cumulative problems?

<matatk> Rachael: The notion of cunulative and critical errors can apply regardless of scoring system.

<matatk> shadi: Here, you seem to get points for doing things. I thought in the other model we discussed, you got negative points for not doing things, or causing barriers.

<matatk> Rachael: We're counting the features.

<Zakim> Jennie, you wanted to ask Shadi if he means as the scorer or in terms of criticality

<matatk> Jennie: Would like to apply the concept of spoons, which shadi mentioned, to this. Concenrs around the complexity and cognitive load on the person doing the scoring.

<matatk> mbgower: Building on shadi's point: if there was a deductive element, and we get things in place to be counted by automated testing, every failure on that could be a deduction.

<matatk> ... Even a well-formed site may have a small bug on it, and not pass automated testing.

<matatk> ... I've been saying automated testing is a baseline, but also think you could use it as a way of reducing someone's score and that might address the 'bug' queestion.

<matatk> ... If you get 200 little things wrong from the tests, this could affect the score. Would be a shame to have them fully fail though. May give us more maleability.

<jeanne> +1 Ben

<matatk> Ben_Tillyer: If we did deductive scoring, people would be less willing to conduct more thorough tests, as they'd find more problems. I think if you were to add more to

<matatk> ... the score for doing more thorough tests, that would be a good thing.

<Jennie> Excellent point!

<matatk> ... With deductive, the onus for improving the site's accessibility may fall more on the end user.

<mbgower> i.e., equal yardstick for the baseline measuring

<matatk> JohnRochford: If we start with 100 points and deduct, or 0 points and add, would that make a difference?

<matatk> Rachael: points - pro: no maximum score; could express IRL accessibility more.

<matatk> ... Cons: could punish both simple and complex sites; may introduce bias if based in a certain way on functional needs; additional cognitive load [and others from the doc!]

<Rachael> Adjectival

<matatk> Rachael: This is where you have categories for results (bad/good/ok). Often used in US govt as a way of grading people.

<matatk> Fazio: We've learnt that positive reinforcement is the most effective way to deal with this.

<matatk> ... If there's a way that we can structure that rewards thorough testing that would be a good way to go.

<Ben_Tillyer> +1 to Fazio

<matatk> Fazio: Disability Equality Index (DEI) is a good example of this.

<alastairc> I don't buy that points scoring would impact how much people test. If that were the case it would happen now. The same dynamic would apply where if a site says their accessibility is great but people have problems, they still have a problem.

<matatk> Rachael: Pros: could help by defining bands of results; already used; could be quick to arrive at a rough result

<shadi> +1 to hard to spell

<matatk> Rachael: cons: hard to spell!; categorical; may be less reliable than other test measures [and others in the doc!]

<matatk> Jennie: Seens ome adaptive thermometers that have words on one side and numbers on the other. Is there a reaosn why we couldn't associate a number with a word?

<matatk> Rachael: no

<shadi> +1 to Jennie (and percentages too)

<matatk> jeanne: +1 to Jennie as this is what we did in an early proposal. We associated a numbrr with each of the bands.

<matatk> ... We were using it primarily for qualitative tests, so we could mix the two in the same scoring system.

<Jennie> *Symbols could also then be associated as well, with personalization to use your own symbol set

<matatk> ... Adjectival works well for an individual test.

<matatk> Fazio: I know we talked about this years ago and had issues. When you move to adjectival you're deciding what's good enough. With numbers it is more clear cut. This adds subjectivity and we shouldn't be making that sort of decision.

<matatk> mbgower: This makes more sense when talking about qualitative measures.

<matatk> ... Do I understand that it would not be used exclusively?

<matatk> ... Orgs are going to be coming up with metrics. Understand Fazio's concern but however we work, once we get to qualitative stuff

<Zakim> alastairc, you wanted to say/ask whether it works best as an intermediate later? and to say/ask whether it works best as an intermediate layer?

<matatk> ... and further from prescriptive qualitative tests, it is going to be more subjective.

<Zakim> jeanne, you wanted to say that we recieved some positive feedback from US regulators. When the bands are defined it becomes more clear what the qualitative decision should be.

<matatk> alastairc: We need to think about what level it's being used. Not test (pass/fail); could be at method level for qualitative things; could be intermediate - above guideline, e.g. language: this goes inbetween good and terrible.

<matatk> ... You could have that layer across all of the guidelines, to give a simpler way to provduce an overall score. I think this is

<matatk> ... what jeanne was saying and it works well form a regulatory perspective.

<matatk> jeanne: As it becomes more subjective it gets more divergent. We pursued this because research shows you reduce subjectivity when you have clearly defined bands.

<matatk> ... I.e. a rubrik

<matatk> ... 'These are the qualities for pass/fail'

<matatk> jeanne: This increases reliabilty.

Matt: apologize in advance, concerned about mixing results,

sure lots of people are aware, you can't mix contegorical and discrete data and come up with any meaningful result

<jeanne> +1 to Matt about validity reminder of mixing scoring types

<matatk> mbgower: This could be really useful for capturing user test responses.

<Zakim> mbgower, you wanted to say I can picture it being very useful for user testing results/responses

<matatk> matatk: [note my commentn was around not falling into the trap of mixing categorical and discrete data - again sorry if already considered, which I'm sure you did!]

<matatk> Fazio: Neuropsychological evaluations provide a great example of how to do this type of analysis.

<matatk> Rachael: Leaving the doc open for contribution!

just a note that the DSM is not a perfect document and is well behind latest neuropsychological research

<Jennie> Congrats to chairs and staff that put this together!

<matatk> Rachael: Next steps. Thanks all for contributions. We have kept the parking lot. I will summarize and send out an email about what we've done.

<matatk> ... When we get back to our regular schedule we'll do a retrospective on the subgroup process.

<matatk> ... If we continue with them we'll need to spin them up.

<matatk> ... We'll continue this conformance discussion with the whole group.

<matatk> ... We have a gooal of publishing a WD fairly soon.

<matatk> ... Made good progress today; need to explore test types etc. soon first.

<matatk> ... We will then have a meeting with regulators form different areas.

<matatk> ... We'll talk with them about conformance and hear their feedback directly.

<matatk> ... Really appreciate everyone's time, energy, insights and thoughts. Safe travels!

+1

<matatk> shadi: We've been discussing what a wonderful job all the leaders have been doing.

<matatk> *group agrees*

<Jennie> +1

<Lauriat> +1 Rachael, THANK YOU!

<jeanne> +1 for outstanding job chairing

<Judy> +1

<alastairc> +1, and Rachael has done a huge amount of work

<jaunita_george_> +1 Thank you!!