Meeting minutes
Evaluating Procedures Proposal
https://
jg: <reads from proposal>
jf: what is goal for this review?
jg: send the proposals to the WG
don´t expect consensus of the subgroup, we have two areas of consensus
jf: will we review first?
jg: can do some cleanup before submitting
rbm: I see next step as solicit review from the parent groups of both proposals
we can refine them if it´s productive
jf: I see lots of questions open
jg: because we have two areas of consensus, we expect that
taking to the larger group will give us more points of view
don´t think we will be able to agree here on a single proposal
per RBM we can start to prepare the editors´ notes of outstanding issues
Points for Protocol Proposal
https://
<jennifer_> I thought I asked a question about the two approaches, and the response was that the two approaches would work in tandem.
jf: my understanding of conformance is that you perform ACT-style tests
failing tests lowers the score and potentially conformance level
losing points is demoralizing
think protocols can be used to (re)gain some points
define what success looks like given unknown parameters
promising to adopt best practices docs gives you points
<Zakim> jeanne, you wanted to ask that a concern be listed that protocols can be gamed to pass a minimum on a site that isn't accessible.
protocol gives content creator info to make well informed decision
jeanne: should note concern about raising score where it @@
<Zakim> Rachael, you wanted to clarify what would go forward with this proposal
jennifer: would otherwise fail a minimum bar of objective tests. It makes it more vulnerable to gaming
rbm: we´d add all concerns to either proposal
Discussion of proposals
jf: my concern with evaluating procedures proposal allows self-documented protocols
could lead to toothless protocols
orgs do this all the time but it doesn´t necessarily improve accessibility
jg: the evaluating procedures proposal includes requirements for self-documented protocols
and states that they must be public so they can be inspected
jf: who vets them?
jg: protocols relate to specific outcomes and are measured in context of that
jf: don´t see how we evaluate stuff that can´t be measured
it´s still subjective
jeanne: how does Points for Protocols provide the rigour? I see the same question applying to both
<Zakim> jeanne, you wanted to ask how Points for Protocols addresses this?
jf: protocols are vetted by AG
hence subject matter experts
jeanne: that´s not clear in the proposal right now, think it needs editing
jf: AG doesn´t have to author all protocols
but it reviews them before they can be used to gain conformance points
jeanne: that needs to be written up
<jaunita_george> https://
rbm: might be useful to compare the proposals in table form to find the similarities and differences
<jennifer_> +1
<jeanne> New document
jennifer: I think we should come together with a harmonized proposal
<Rachael> Just a note that multiple proposals at the exploratory phase is built into our process
need to be nice to each other, hear one another, approach with best of intentions
jf: @@ haven´t decided what we´re doing with conformance
public declaration of adopting protocol
accomplishes @@
the public promise means you have to follow through on it
in a legal situation
ack \
jg: accessibility statements aren´t generally enforceable
jf: one of the questions is, what will regulators accept
<jennifer_> * Jennifer needs access
<Rachael> Chair note: We are working our way towards conformance conversations but we are still not there yet so pre-discussing has not proven to be a productive use of time
jg: do we want to develop a single proposal, or submit multiple?
jf: are we even working on the same thing?
maybe we´re working on different things that both use word protocol
<jennifer_> * Jennifer: Thanks, Jeanne. I refreshed before, but now I do have access.
for me, the question is what @@
mc: I think we won´t form consensus on a single proposal, thing broader input from the WG is needed
raising questions and feature comparisons is useful
jennifer: think we should be able to come to agreement
rbm: discussion of conformance keeps side-tracking us
there are unknowns that we need to work around, until we´re able to get to them
<JF> +1 to Jen
jennifer: all work should halt until conformance sorted out
rbm: we tried addressing conformance first, but didn´t have enough base information to have non-circular conversation
the chairs have to find a way to work through things
having multiple options is ok
we can compare, discuss, merge, etc.
people may strongly favour a proposal, but we have to develop an overall consensus
chairs are handling this with a strict schedule, which we have to use to be able to move forward
jennifer: can we put a bracket around conformance-related aspects of proposal
rbm: thinks the points for protocols proposal has done that
it will be part of what we forward to group
we need to make progress, we´re recycling discussion
jennifer: think a group should be able to work together
and evaluating procedures group has developed something different
jf: we have two unique ideas, both have some value, neither are fully baked
should ask the WG to evaluate them independent of each other
with both using the word ¨protocol¨ that´s harder
see conformance being addressed in the evaluating procedures proposal as well
<Zakim> JF, you wanted to note that an end state does not have to be one or the other - it may be both
mc: think we were ¨bracketing¨ conformance but hadn´t actually put brackets in
think we need to have placeholder conformance info for the proposal and as fodder for later discussion
it´s ok to rename if there is concern they are being presented as exclusive of each other
jg: what about renaming?
jf: ¨development protocols¨ and ¨evaluation protocols¨
jg: development would be points for protocols, evaluation would be evaluating procedures
<jaunita_george> Proposed Resolution: Bring both proposals to AGWG as Development Protocols (Points for Protocol) and Evaluation Protocols (Evaluating Procedures) and work to get a table together outlining the differences/similarities ahead of the AGWG meeting we'll be discussing it.
<JF> +1
+1
<jaunita_george> +1
<Rachael> 0
<jennifer_> 0
<jeanne> +1 for two propo9sals, but I don't like the new names. I don't think they accurately describe the proopsals
<jennifer_> Agree with Jeanne — I don't understand them at all.
<Rachael> What about procedures and protocols?
RESOLUTION: Bring both proposals to AGWG as Points for Protocol and Evaluating Procedures and work to get a table together outlining the differences/similarities ahead of the AGWG meeting we'll be discussing it.
jg: we´ll prepare to present these to the AG WG
meanwhile let´s fill out the table of similarities and differences
jg: let´s meet next week (12:00 pm Eastern) to review the table