Peter Krautzberger · on the web

Three interesting results from the AIM workshop

When I look back at looking ahead to 2018, I'm going to fall short of my goals for this year. Life was on full throttle this year; no regrets. But I want to at least acknowledge what an excellent adventure the AIM Workshop in May has been. So let's take a look at three interesting results.

The first rule of workshops

It's all in the people.

Of course, the very best part of this workshop was the people who attended it. It's amazing to get people from NVDA, JAWS and ChromeVox into a room for a few days. It's even better when you have people from MathJax, MathLive, Desmos in the same room. It gets even better when you have publishing experts from Wiley and Pearson on board. It's incredibly much better to have the vast expertise of people such as T.V. Raman and Joanie Diggs there. But for me, the most thrilling was the educators in the room. They are the key and without them we are all lost. And I'm the first to admit the workshop didn't serve them well enough. Even more importantly, at future workshops we need to get students in the room as well. Because what the hell are we doing without them.

In extension, this is a compliment to AIM's workshop design. Providing funding not only for a workshop but for everyone's travel and accommodation was excellent but also crucial. We would never have been able to get all these people in a room. This is the right way to hold workshops, especially when inclusiveness is a huge issue.

Towards an optional Braille stream

There's a particular limitation of today's accessibility landscape: we cannot specify separate textual alternatives for voice and Braille.

Generally, not having separate streams for voice and Braille does not seem like a huge problem. As long as all accessibility needs are covered by a fixed set of standard elements that are designed for both aural and tacticle interfaces, then assistive technologies can reliably implement a split in the stream, i.e., present separate voice and Braille streams from that. For example, if you have a dedicated button element, it can represented as a btn contraction on a Braille display and voiced as Button.

As usual, not all things can be covered by standards. Say your button is is used as a control in a game then you may want to augment the button's accessible name to include the action. So if the button opens a selection of planets to travel your to in your game, then you may want to have this voiced as planet. You can do that of course and then you might get a voicing of planet; button and something akin to pln btn on a Braille display.

Unfortunately, you might find yourself in a situation where you need to prevent the addition of button in the voicing because it may be problematic for your aural users, e.g., users with learning disabilities may find it to be distracting noise. But now how do you identify the button on a Braille display?

For equation layout, the situation is much like that final situation. In many countries, specialized Braille formats such as Nemeth, UEB, or Marburg have been developed to represent equation layout in Braille. These are well established and there are not too difficult to create. But they differ considerably from what you would might want to render aurally (and visually). In fact, since most precede the web, they try to capture (simplified) visual layout, including all the ambiguities we face there.

For me, the greatest positive experience of the workshop was to see the group assess the problem, come to an agreement that it needs a solution, add it to the ARIA tracker, build demos and even see NVDA whip up a first implementation that we could explore by the end of the week.

This was huge.

And yet, it is the easy part. Now the long road towards a proper standard with widespread implementations lies ahead.

Progress on deep aria labels

I brought my favorite problem to the workshop - deep aria-labels and I was not disappointed.

Assistive technologies for equation layout (in particular for MathML) have to apply heuristics (read: guess) the semantics of an expression so that they can generate meaningful non-visual representations. This is a problem because heuristics that are hard coded into an external tool such as screenreaders cannot be altered by standard means, leaving authors without adequate means to ensure the quality of their content. (If a screenreader voices every superscripted 2 as squared and you have no way of changing that, then you're screwed.)

More importantly, since equation layout is, ultimately, only visual, a perfectly correct representation in HTML is as spans, i.e., there are no semantics. Finally, ARIA (naturally) does not have a dictionary of equation layout terminology (let alone mathematical or scientific terminology) to use - a) because all past dictionary based approaches have failed and b) such a dictionary would have to be extensible (read: infinite) which ARIA, so far, does not really want to be (role-description notwithstanding).

So the pragmatic answer is: you'll just have to do it yourself and use deep aria-labels: you override every single accessible name computation by slapping a fixed label to things. Because, ultimately, this is how we read equational content - with words.

The trouble is that it's easy to add a single aria-label at the root but it is hard to provide an explorable structure that provides a decent user experience. You'll want to provide labels at the leaf level but the state of ARIA prevents those from adequately building up an explorable tree. (And we're not even talking about refinements such as providing summaries and structural and positional information during exploration.)

At the workshop, David Tseng from Google's ChromeVox team, Volker Sorge from Speech Rule Engine and Davide Cervone from MathJax sat down nd build a first demo that tries two things

This is, simply put, a fantastic step forward.

The approach builds on existing parts of ARIA and identifies reasonable, incremental improvements to it. It raises important questions on general exploration, e.g., how is there a generic aria-table walker in every screenreader but not some basic aria-tree walker (such as breadth or depth first)?. And yet it pragmatically builds an unobtrusive solution anyway that works at 60FPS. It works with any markup, in particular any approach using CSS or SVG markup. And to top it off, it leverages existing open-source tool to enrich pre-existing content. And while it shows just how far ahead MathJax and Speech Rule Engine are, this approach is transparent and easily used by any other equation layout library.

In terms of UX, this is also a critical step forward. The approach is should be able to provide a seamless an interaction for visual and non-visual users alike, in synchronization. Effectively, it pushes MathJax's Accessibility Extensions from client- to server-side, requiring minimal JavaScript (just a keyboard event listener) to expose the content, without live region hacks, and with a solid non-JS fallback. It provides a clear path for making even that bit of JS obsolete through natural improvements to ARIA. It opens a path to finally get rid of the horrible hackery such as JAWS did back in the day, manipulating IE's DOM to manipulate MathJax, or Texthelp is doing today by injecting JS on the client (yuck!, and also badly failing when content-security is in place).

Even better, if you combine it with the previous part (exposing specialized Braille which SRE can soon produce), then this would immediately become the by far best, universal rendering of equation layout on the web: robust, high-quality, customizable, precise. And it is a solution that will only get better as standards evolve while leaving the full control with the author (with or without aid of heuristics at authoring time).

I'll dig into this more some other time but admittedly, I'm pretty excited.

Coda

You can find the organizers' report at aimath.org but you can take one thing away: It's looking very good for accessible equation layout on the web these days. And it will only get better.

If we can continue these workshops, things will move faster for everyone. And maybe, just maybe, we can even finally move on to actual mathematics (and other STEM content) on the web.