Get Accessibility Properties

[spectranaut]

See WICG/aom#203

Added/update sections:

• Added route under "Element" section
• Added "Accessibility" Section
• Add references

Things to maybe revisit:

• Discussion on errors in explainer - should we/can we have more specific errors?
• Discussion on IDs in the explainer - should the accessibility ID be more defined?

---

This change is 

---

Preview | Diff

[spectranaut]

First pass at fixes to your comments, @cookiecrook, but I still need to work on definitions and where they belong!

[jcsteh]

Thanks for all of your work on this.

[spectranaut]

I can't actually add people as reviewers, but, this is ready for re-review after all the feedback from the meetings the last few days, and the review comments from James and Jamie.

@jcsteh @cookiecrook @jugglinmike @lolaodelola , if you could all take a look :)

[jcsteh]

Thanks. Looks good to me.

[lolaodelola]

This looks good to me

[jugglinmike]

A couple typos: "accessibility", and we can nix the trailing forward-slash.

Taking a step back, though, I wonder if we should design this resource more like the element commands. That is, given that we have:

/session/{session ID}/element/{element ID}/rect

A resource like:

/session/{session ID}/accessibility/node/{accessibility node ID}/properties

Looks a little more familiar (and hopefully more predictable) than

/session/{session ID}/accessibility/properties/{accessibility node ID}

I'd personally be in favor of such an alteration purely on the basis of parity, but it would help if there was a credible expectation for further accessible-node-based commands in the future. Do you know any? Maybe /element to get the backing DOM element (where available)?

[spectranaut]

Thanks for the ideas here, I wonder what the other reviews would think, and we should discuss this at the next accessibility interop meeting.

[spectranaut]

We agreed to make this switch!

[jugglinmike]

An unadorned "relevant" seems a little dubious for conformance-seekers. I imagine that the properties are implementation-defined (though I can't find "computed accessibility properties" in CORE-AAM 1.2 or its issue tracker); if that's part of the definition, is "relevant" necessary here?

[spectranaut]

Right, there is a PR for "computed accessibility properties": w3c/aria#2800

To see the spec text: https://deploy-preview-2800--wai-aria.netlify.app/core-aam/#computed_accessibility_properties

The spec text does explain exactly which properties should appear for a given accessibility node. So maybe you are right, the relevant is superfluous, the "computer accessibility properties" is a defined set, I'll remove it :)

[OrKoN]

this link does not seem to work for me? is the content for this in a PR?

[spectranaut]

So sorry -- I should have made that more clear. This is also currently being specified and there is a PR: w3c/aria#2800

To see the spec text: https://deploy-preview-2800--wai-aria.netlify.app/core-aam/#computed_accessibility_properties

We need to get agreement between all the browsers on this, and we are working on that actively, it seems like it's close to landing.

[OrKoN]

Do you know how well does this corresponds to https://chromedevtools.github.io/devtools-protocol/tot/Accessibility/#type-AXNode?

[spectranaut]

Yeah, actually, this will be backed by the devtools node -- the webdriver command goes through the devtools protocol to get this information. The difference is that to start it will be a subset of the properties -- just the ones all three browsers agree to expose, and the agreement is being worked out in the other PR (w3c/aria#2800).

[OrKoN]

are you also working on the implementation in Chromium?

[spectranaut]

Yes, I sent it to you, but just in case here is is again: https://chromium-review.googlesource.com/c/chromium/src/+/7922212

[spectranaut]

Thanks @jugglinmike and @OrKoN ! I've some fixes, answers and questions.

[spectranaut]

Right, there is a PR for "computed accessibility properties": w3c/aria#2800

To see the spec text: https://deploy-preview-2800--wai-aria.netlify.app/core-aam/#computed_accessibility_properties

The spec text does explain exactly which properties should appear for a given accessibility node. So maybe you are right, the relevant is superfluous, the "computer accessibility properties" is a defined set, I'll remove it :)

[spectranaut]

Thanks for the ideas here, I wonder what the other reviews would think, and we should discuss this at the next accessibility interop meeting.

[spectranaut]

So sorry -- I should have made that more clear. This is also currently being specified and there is a PR: w3c/aria#2800

To see the spec text: https://deploy-preview-2800--wai-aria.netlify.app/core-aam/#computed_accessibility_properties

We need to get agreement between all the browsers on this, and we are working on that actively, it seems like it's close to landing.

[spectranaut]

Yeah, actually, this will be backed by the devtools node -- the webdriver command goes through the devtools protocol to get this information. The difference is that to start it will be a subset of the properties -- just the ones all three browsers agree to expose, and the agreement is being worked out in the other PR (w3c/aria#2800).

[OrKoN]

would these endpoints allow getting the entire a11y tree? what is the expected workflow? getting accessibilityproperties for the root element and then using accessibility/properties/<accessibility id> to traverse?

[OrKoN]

it also seems like it would require many roundtrips to get properties for all elements.

[spectranaut]

Yes we will have some "subtree" tests which walk the accessibility tree by accessibility ID, for example: https://github.com/web-platform-tests/wpt/blob/master/wai-aria/subtree/tablist.tentative.html

(this test runs on firefox because of the marionette endpoint: https://wpt.fyi/results/wai-aria/subtree?label=experimental&label=master&aligned)

In general, though, the subtrees are not testable, because they are pretty different between browsers, and we don't think we will ever have full alignment. The main issue is how much "generic" nodes there are (things that represent something meaningless in the a11y tree, like a div for styling purposes). We expect only to test specific scenarios. We discussed having an endpoint for the whole subtree, but we think it's more comment to just test a node or a few children/parents, thus the current design.

[OrKoN]

Thanks, that makes sense but I wonder once this is published if other WebDriver clients might start building out features based on the subtree iteration mechanisms available (thus starting to rely on the non-specified behavior) and require many roundtrips to implement the subtree retrieval.

[cookiecrook]

I expect that will be the case, and don't see harm in testing the subtree of smaller features, like a known list or table with a known style sheet... However, it won't be a reliable cross-UA WPT test yet to include something like dumpAccessiblityTree() at the root node for a complex web site. Perhaps someday if and when we are able to standardize the rest.

[OrKoN]

would it make sense to add a note about this to the spec text here?

[OrKoN]

does it mean that the computed properties are keys on the same level as accessibilityId, parent, children? is there a chance to have conflicts in the future?

[spectranaut]

Good question. From my perspective, working on ARIA and being an editor of Core-AAM, I'd guess we won't end up defining it there... but it's possible it would be a fall out of testing these internal accessibility trees and trying to get more alignment on them.

These computed accessibility trees started out as something like an "implementation detail", the original specified accessibility trees are in terms of the browser-exposed accessibility API (these are the things the screen reader actually uses to interact with the browser). This "computed accessibility tree", however, is useful to test because it is the platform independent and the same thing you see in the devtools. Getting alignment on it for role and name has solved a lot of interop issues, so we want to surface more properties. Sorry I don't know how much you know about the accessibility part of the browser, just thought I'd provide more information if it's helpful :)

Ultimately, thought, I think it's fine if we bundle the computed properties in a key like "properties." I bring it up in the next WPT accessibility interop meeting.

[OrKoN]

Thanks, I think it would be useful to put the computed properties into a separate key.

[cookiecrook]

I would want these are same level as parentNode/childNodes/id, etc. I don't see a lot of benefit in putting an arbitrary collection of them into a sub-level properties container. Many of the to-be-specified don't align with attributes per se... location and size/bounds for example, would be computed using CSS and other features, but should that any different in the final object representation? What about the other relationships like aria-controls (or label-from)? Why should those be any different from the parent and children relationships?

[OrKoN]

The reason I asked this question is to avoid potential conflicts if some properties defined by the computed accessibility properties end up clashing with properties defined here. Alternatively, we could say that the properties coming from computed accessibility properties override conflicting keys if any (or the other way around). I think called the key properties would be fine because it the algorithm is data it provides is called computed accessibility properties. If you think conflicts are unlikely, I think the current version looks good to me.

I am also wondering if perhaps the accessibility node ID term might move to the core-aam spec so that WebDriver spec uses it instead of defining it. Then the Core-AAM spec can define how to produce all of the attributes including parentNode, childNodes and all keys on the same level would be part of the same spec. It would also make it easier to re-use the definitions in WebDriver BiDi spec later on.

This is also somewhat related to the other question I had about how child nodes should be computed and whether the a11y nodes should span over iframes or if the client needs to use SwitchToFrame to inspect a11y tree that belongs to iframes.

[spectranaut]

We talked about this in the interop meeting, I'm going to try to define it along side the computed accessibility properties in Core-AAM, for re-using in bidi -- but I won't be able to move this forward for a few weeks!

[OrKoN]

as far as I see the a11y properties do not link back to DOM element IDs? it seems like it might limit usefulness of the a11y properties?

[spectranaut]

They don't link back, it's true. Can you explain why it might limit the usefulness? Oh maybe you mean, if you have an accessibility node, and you want to click it, you don't know where to send that click?

That is a problem, I'm not sure right now how to get around, I'll bring it up.

[OrKoN]

yes, this would be a use case we have in Puppeteer or in Chrome DevTools (a11y tree view), for example, but also if the a11y tree is traversed using the methods in PR, there seems to be no mechanism to do something with the backing DOM node or even know what DOM contributed this a11y node (if any). I understand that the proposed extension is probably useful for the WPT use cases but I also wonder how it would work for other WebDriver clients.

[cookiecrook]

There should be an optional DOM Node ID reference from the Accessibility Node, if relevant... The reason it's optional is that not all Accessibility Nodes have a relevant DOM Node, and vice versa, not all DOM Elements have a backing Accessibility Node...

An example of the latter is any hidden DOM node; not mapped in the AX Tree. An example of the former (AX Nodes without a DOM Element) is any CSS generated content, or sometimes (depending on implementation details, dynamic nodes like table columns or other in-betweener AX nodes)

[OrKoN]

I do not see an optional DOM Node ID reference in the spec. Could you please point me to the definition?

[cookiecrook]

That could be defined here, but would more likely come from the TBD property bag definition in Core-AAM. @spectranaut?

[spectranaut]

In the case of webdriver, it would be the webdriver element ID. Maybe we will make some kind of abstraction in order to define this in Core-AAM -- because other testing frameworks (like marionette driver) will have a different kind of ID references to get back to the original DOME element.

[OrKoN]

Is there something to link to that defines what child nodes are? should this extend into iframes or is it expected to work with https://www.w3.org/TR/webdriver2/#switch-to-frame?

[cookiecrook]

It's a good question. The cross-frame and cross-process hop makes it tricky, and I'd defer to a security analysis of whether its wise to allow that here. @gsnedders, @twilco, @minorninth, @jcsteh, do you have any thoughts?

[spectranaut]

Thanks again @OrKoN ! I'm out for a week, but wanted to reply to a few things. There is an WPT accessibility interop meeting July 1, I'll bring up some points there.

[spectranaut]

Yes, I sent it to you, but just in case here is is again: https://chromium-review.googlesource.com/c/chromium/src/+/7922212

[spectranaut]

Yes we will have some "subtree" tests which walk the accessibility tree by accessibility ID, for example: https://github.com/web-platform-tests/wpt/blob/master/wai-aria/subtree/tablist.tentative.html

(this test runs on firefox because of the marionette endpoint: https://wpt.fyi/results/wai-aria/subtree?label=experimental&label=master&aligned)

In general, though, the subtrees are not testable, because they are pretty different between browsers, and we don't think we will ever have full alignment. The main issue is how much "generic" nodes there are (things that represent something meaningless in the a11y tree, like a div for styling purposes). We expect only to test specific scenarios. We discussed having an endpoint for the whole subtree, but we think it's more comment to just test a node or a few children/parents, thus the current design.

[spectranaut]

Good question. From my perspective, working on ARIA and being an editor of Core-AAM, I'd guess we won't end up defining it there... but it's possible it would be a fall out of testing these internal accessibility trees and trying to get more alignment on them.

These computed accessibility trees started out as something like an "implementation detail", the original specified accessibility trees are in terms of the browser-exposed accessibility API (these are the things the screen reader actually uses to interact with the browser). This "computed accessibility tree", however, is useful to test because it is the platform independent and the same thing you see in the devtools. Getting alignment on it for role and name has solved a lot of interop issues, so we want to surface more properties. Sorry I don't know how much you know about the accessibility part of the browser, just thought I'd provide more information if it's helpful :)

Ultimately, thought, I think it's fine if we bundle the computed properties in a key like "properties." I bring it up in the next WPT accessibility interop meeting.

[spectranaut]

They don't link back, it's true. Can you explain why it might limit the usefulness? Oh maybe you mean, if you have an accessibility node, and you want to click it, you don't know where to send that click?

That is a problem, I'm not sure right now how to get around, I'll bring it up.