зеркало из https://github.com/mozilla/pjs.git
Update to MSAA support documentation. NPOB.
This commit is contained in:
aaronl%netscape.com
Родитель
5dc7316f87
Коммит
a198355723
|
|
@ -1,190 +1,586 @@
|
|||
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
|
||||
<html>
|
||||
<head>
|
||||
<title>
|
||||
How the Accessible Module works
|
||||
</title>
|
||||
<title>How the Accessible Module works</title>
|
||||
</head>
|
||||
<body>
|
||||
<h1>How the Accessible module (accessibility.dll) works</h1>
|
||||
<h1>Explanation of Mozilla's MSAA implementation<br>
|
||||
and Cheat Sheet for Engineers new to MSAA</h1>
|
||||
<ul>
|
||||
<p>
|
||||
See also:
|
||||
<a href="http://www.mozilla.org/projects/ui/accessibility/vendors-win.html">Gecko Info for Windows Accessibility Vendors</a>, a primer for vendors of 3rd party accessibility software, on how to use our MSAA and other relevant API's.
|
||||
</p>
|
||||
<p>
|
||||
The <a href="http://lxr.mozilla.org/seamonkey/source/accessible/">Accessible module</a> is where we implement support for the <a href="http://www.microsoft.com/enable/msaa/">Microsoft Active Accessibility (MSAA) API</a>
|
||||
(<a href="http://bugzilla.mozilla.org/show_bug.cgi?id=12952">bug 12952</a>).
|
||||
Support for Sun's <a href="http://www.sun.com/access/gnome/">Gnome Accessibility API</a> is part of our future plans as well.
|
||||
</p>
|
||||
<p> See also: <a
|
||||
href="http://www.mozilla.org/projects/ui/accessibility/vendors-win.html">Gecko
|
||||
Info for Windows Accessibility Vendors</a>, a primer for vendors of 3rd
|
||||
party accessibility software, on how to use our MSAA and other relevant
|
||||
API's. This has the details of what we do and do not support.<br>
|
||||
</p>
|
||||
<p>Mozilla supports the <a
|
||||
href="http://msdn.microsoft.com/library/default.asp?url=/library/en-us/msaa/msaastart_9w2t.asp?frame=true">Microsoft
|
||||
Active Accessibility (MSAA) API</a> , used on Windows operating
|
||||
sytstems. to support assistive technologies for users with disabilities.
|
||||
The <a href="http://lxr.mozilla.org/seamonkey/source/accessible/">Accessible
|
||||
module</a> is where we implement the support for MSAA (<a
|
||||
href="http://bugzilla.mozilla.org/show_bug.cgi?id=12952">bug 12952</a>)
|
||||
as well as for Sun's <a
|
||||
href="http://wwws.sun.com/software/star/gnome/accessibility/architecture.html">ATK</a>
|
||||
API for Linux and UNIX. For documentatin specific to the Mozilla ATK
|
||||
effort, supported by Sun Microsystems, see the <a
|
||||
href="http://www.mozilla.org/projects/ui/accessibility/unix/index.html">Mozilla
|
||||
accessibility on Unix</a> page.<br>
|
||||
</p>
|
||||
</ul>
|
||||
<h2>What is MSAA?</h2>
|
||||
<ul>
|
||||
<p>
|
||||
A 3rd part accessibility aid, such as a screen reader, wants to track what's happening inside Mozilla. It needs to know about
|
||||
focus changes and other events, and it needs to know whtat data nodes there are in the layout tree.
|
||||
Using this information,
|
||||
the screen reader will speak out loud important changes to the document or UI, and allow the user to track
|
||||
where they navigate. Some screen readers also magnify text and images in the currently focused area, and others
|
||||
show information on a <a href="http://www.audiodata.de/e/produkte/pc/lapbraille/">refreshable braille display</a>.
|
||||
</p>
|
||||
<p>
|
||||
In Windows, accesibility aids acquires the necessary information to do this using hacks and MSAA. MSAA is supposed
|
||||
to be the "right way" for accessibility aids to get information, but sometimes the hacks are more effective.
|
||||
For example, screen readers look for screen draws of a vertical blinking line, to determine the location of the caret.
|
||||
Without doing this, screen readers would not be able to let the user know where there caret has moved to in most programs,
|
||||
because so many applications do not use the system caret (ours is an example). This is so commonly done, that
|
||||
no one even bothers to support the MSAA caret, because the hack works.
|
||||
</p>
|
||||
<p>
|
||||
MSAA provides information in two different ways:
|
||||
<ol>
|
||||
<li>a COM interface (IAccessible) that allows applications to expose the tree of data nodes that make up
|
||||
each window in the user interface currently being interacted with and</li>
|
||||
<li> a set of system messages
|
||||
that confer accessibility-related events such as focus changes, changes to document content and alerts.</li>
|
||||
</ol>
|
||||
</p>
|
||||
<p>
|
||||
To really learn about MSAA, you need to download
|
||||
the entire <a href="http://www.microsoft.com/enable/msaa/download.htm">MSAA SDK</a>.
|
||||
Without downloading the SDK, you won't get the complete documentation.
|
||||
The SDK also contains some very useful tools, such as the Accessible Event Watcher, which shows what accessible
|
||||
events are being generated by a given piece of software. The Accessible Explorer and Inspect Object tools
|
||||
show the tree of data nodes the Accessible object is exposing through COM.
|
||||
</p>
|
||||
<p>3rd party assistive technology, such as screen readers, screen
|
||||
magnifiers amd voice input software, want to track what's happening
|
||||
inside Mozilla. They needs to know about focus changes and other events,
|
||||
and it needs to know what objects are contained in the current document
|
||||
or dialog box. Using this information, a screen reader will speak out
|
||||
loud important changes to the document or UI, and allow the user to
|
||||
track where they navigate. The screen reader user can navigate the web
|
||||
page using screen reader commands or browser commands, and the two
|
||||
pieces of software must remain in sync. Some screen readers can even
|
||||
show information on a <a
|
||||
href="http://www.audiodata.de/e/produkte/pc/lapbraille/">refreshable
|
||||
braille display</a>. Screen magnifers will zoom to the focus, keeping it
|
||||
on the screen at all times, or even allow the user to enter a special
|
||||
low vision document reading mode. Finally, voice dictation
|
||||
software needs to know what's in the current document or UI in order to
|
||||
implement "say what you see" style features.<br>
|
||||
</p>
|
||||
<p> On Microsoft Windows, these kinds of assistive technology acquire
|
||||
this necessary information via a comination of hacks, MSAA and
|
||||
proprietary DOMs. MSAA is supposed to be the "right way" for
|
||||
accessibility aids to get information, but sometimes the hacks are more
|
||||
effective. For example, screen readers look for screen draws of a
|
||||
vertical blinking line, to determine the location of the caret. Without
|
||||
doing this, screen readers would not be able to let the user know where
|
||||
there caret has moved to in most programs, because so many applications
|
||||
do not use the system caret (ours is an example). This is so commonly
|
||||
done, that no one even bothers to support the MSAA caret, because the
|
||||
hack works. </p>
|
||||
<p> MSAA provides information in two different ways: </p>
|
||||
<ol>
|
||||
<li>a COM interface (IAccessible) that allows applications to
|
||||
expose the tree of data nodes that make up each window in the user
|
||||
interface currently being interacted with and</li>
|
||||
<li> a set of system messages that confer accessibility-related
|
||||
events such as focus changes, changes to document content and alerts.</li>
|
||||
</ol>
|
||||
<p></p>
|
||||
<p> To really learn about MSAA, you need to download the entire <a
|
||||
href="http://msdn.microsoft.com/library/default.asp?URL=/downloads/list/accessibility.asp">MSAA
|
||||
SDK</a>. Without downloading the SDK, you won't get the very useful
|
||||
tools, which help a great deal in the learning process. The Accessible
|
||||
Event Watcher shows what accessible events are being generated by a
|
||||
given piece of software. The Accessible Explorer and Inspect Object
|
||||
tools show the tree of data nodes the Accessible object is exposing
|
||||
through COM, and what the screen boundaries of each object are.<br>
|
||||
</p>
|
||||
</ul>
|
||||
<h2>IAccessible Interface</h2>
|
||||
<h2>How to hook in / How the IAccessible's are Created</h2>
|
||||
<ul>
|
||||
<p>
|
||||
The IAccessible interface is used in a tree of IAccessible's, each one representing a data node, similar to a DOM.
|
||||
</p>
|
||||
<p>
|
||||
Here are the methods supported in IAccessible:
|
||||
<p> The first thing that happens when an accessibility aid wants to
|
||||
watch our application is calls the Windows API function
|
||||
AccessibleObjectFromWindow(). This in turns sends the window in question
|
||||
a <a href="http://lxr.mozilla.org/seamonkey/search?string=WM_GETOBJECT">WM_GETOBJECT</a>
|
||||
message requesting an IAccessible for the window. In our case, this
|
||||
event is received in mozilla/widget/src/windows/nsWindow.cpp. We send
|
||||
back an IAccessible interface that represents that root window. The
|
||||
accessibility aid will use that first IAccessible to reach rest of the
|
||||
IAccessible hierarchy, by asking for it's children IAccessibles, asking
|
||||
the children for the grandchildren IAccessibles, and so on. Until this
|
||||
WM_GETOBJECT message is processed, the accessibility.dll is not loaded,
|
||||
so there is almost zero overhead for accessibility in Mozilla.<br>
|
||||
</p>
|
||||
<p> To create the root IAccessible for a window the first time
|
||||
it gets the <a
|
||||
href="http://lxr.mozilla.org/seamonkey/search?string=WM_GETOBJECT">WM_GETOBJECT</a>
|
||||
message in, nsWindow.cpp first generates an internal event called <a
|
||||
href="http://lxr.mozilla.org/seamonkey/search?string=NS_GETACCESSIBLE">NS_GETACCESSIBLE</a>,
|
||||
which is handled in nsFrame.cpp via the creation of an <a
|
||||
href="http://lxr.mozilla.org/seamonkey/find?string=msaa/nsDocAccessibleWrap">nsDocAccessibleWrap</a>
|
||||
for an inner window or <a
|
||||
href="http://lxr.mozilla.org/seamonkey/find?string=msaa/nsRootAccessibleWrap">nsRootAccessibleWrap</a>
|
||||
for a top level window. These classes implement both nsIAccessible, our
|
||||
cross platform API, and IAccessible, which is specific to
|
||||
Windows/MSAA/COM. The nsRootAccessibleWrap class is then told to listen
|
||||
for DOM, page load and scroll events. implementation It uses
|
||||
these events to fire MSAA-specific events such as EVENT_OBJECT_FOCUS or
|
||||
EVENT_OBJECT_STATECHANGE.. We'll explain more about events in a moment.<br>
|
||||
</p>
|
||||
</ul>
|
||||
<h2>The Accessible Tree and the DOM Tree<br>
|
||||
</h2>
|
||||
<ul>
|
||||
<li>get_accParent: Get the parent of an IAccessible.</li>
|
||||
<li>get_accChildCount: Get the number of children of an IAccesible.</li>
|
||||
<li>get_accChild: Get the child of an Iaccessible.</li>
|
||||
<li>get_accName: Get the "name" of the IAccessible, for example the name of a button, checkbox or menuitem.</li>
|
||||
<li>get_accValue: Get the "value" of the IAccessible, for example a number in a slider, a URL for a link, the text a user entered in a field.</li>
|
||||
<li>get_accDescription: Get a long description of the current IAccessible. This is not really too useful.</li>
|
||||
<li>get_accRole: Get an enumerated value representing what this IAccessible is used for, for example.</li>
|
||||
is it a link, static text, editable text, a checkbox, or a table cell, etc.</li>
|
||||
<li>get_accState: a 32 bit field representing possible on/off states, such as focused, fousable, selected, selectable, visible, protected (for passwords),
|
||||
checked, etc. </li>
|
||||
<p>After the root or doc accessible for a window has been created and
|
||||
handed back to the MSAA client, it is used to traverse the rest of the
|
||||
IAccessible tree using accNavigation, get_accChild and get_accParent.
|
||||
Any IAccessible will support those methods. We also support
|
||||
IEnumVARIANT::Next() which allows for fast marshaling of all of an
|
||||
objects children to a client via COM. In other words, the assistive
|
||||
technology can say "guve me all 20 children of this object into this
|
||||
array". That's much faster than 20 separate calls, one for each child.<br>
|
||||
</p>
|
||||
<p>In Mozilla, the client has another choice for tree navigation --
|
||||
it can use the DOM via Mozilla's <a
|
||||
href="http://lxr.mozilla.org/seamonkey/source/accessible/public/msaa/ISimpleDOMNode.idl">ISimpleDOMNode</a>interface.
|
||||
An IAccessible can be used to QueryInterface to an ISimpleDOMNode, and
|
||||
vice versa for a round trip. However, one might QI ISimpleDOMNode to
|
||||
IAccessible only to find it is null, which means that particular node in
|
||||
question is not exposed in the IAccessible tree.<br>
|
||||
</p>
|
||||
</ul>
|
||||
<h3>MSAA tree vs. DOM tree - what's the relationship?</h3>
|
||||
<ul>
|
||||
<p> <img
|
||||
src="http://www.mozilla.org/projects/ui/accessibility/images/tree-relativity.gif"
|
||||
alt="Diagram showing MSAA tree is a subset of the DOM tree"
|
||||
title="Diagram showing MSAA tree is a subset of the DOM tree"> </p>
|
||||
The MSAA tree and the DOM tree are parallel structures, although the
|
||||
MSAA tree is a subset of the DOM tree. <code>QueryInterface()</code> can
|
||||
be used to switch between the interfaces used in the two trees
|
||||
(IAccessible and ISimpleDOMNode). If there is no MSAA node for a DOM
|
||||
node, <code>QueryInterface()</code> will return null.
|
||||
</ul>
|
||||
<h2>IAccessible Methods - Cheatsheet for Implementors</h2>
|
||||
<ul>
|
||||
<p> The IAccessible interface is used in a tree of IAccessible's, each
|
||||
one representing a data node, similar to a DOM. </p>
|
||||
<p> Here are the methods supported in IAccessible - a minimal
|
||||
implementation would contain those marked "<span
|
||||
style="font-weight: bold;">[important]</span>" :<br>
|
||||
</p>
|
||||
<ul>
|
||||
<li>get_accParent: Get the parent of an IAccessible. <span
|
||||
style="font-weight: bold;">[important]</span><br>
|
||||
</li>
|
||||
<li>get_accChildCount: Get the number of children of an IAccesible. <span
|
||||
style="font-weight: bold;">[important]</span></li>
|
||||
<li>get_accChild: Get the child of an IAccessible. <span
|
||||
style="font-weight: bold;">[important]</span></li>
|
||||
<li>get_accName: Get the "name" of the IAccessible, for example the
|
||||
name of a button, checkbox or menuitem. <span style="font-weight: bold;">[important]</span></li>
|
||||
<li>get_accValue: Get the "value" of the IAccessible, for example a
|
||||
number in a slider, a URL for a link, the text a user entered in a
|
||||
field. <span style="font-weight: bold;">[important]</span></li>
|
||||
<li>get_accDescription: Get a long description of the current
|
||||
IAccessible. This is not really too useful.</li>
|
||||
<li>get_accRole: Get an enumerated value representing what this
|
||||
IAccessible is used for, for example. <br>
|
||||
</li>
|
||||
is it a link, static text, editable text, a checkbox, or a table
|
||||
cell, etc. <span style="font-weight: bold;">[important]</span><span
|
||||
style="font-weight: bold;"></span><li>get_accState: a 32 bit field
|
||||
representing possible on/off states, such as focused, fousable,
|
||||
selected, selectable, visible, protected (for passwords), checked, etc. <span
|
||||
style="font-weight: bold;">[important]</span> </li>
|
||||
<li>get_accHelp: Get context sensitive help for the IAccessible.</li>
|
||||
<li>get_accHelpTopic: We don't use this, it's only if the Windows help system is used.</li>
|
||||
<li>get_accKeyboardShortcut: What is the keyboard shortcut for this IAccessible.</li>
|
||||
<li>get_accFocus: Which child is focused?</li>
|
||||
<li>get_accHelpTopic: We don't use this, it's only if the Windows
|
||||
help system is used.</li>
|
||||
<li>get_accKeyboardShortcut: What is the keyboard shortcut for this
|
||||
IAccessible (underlined alt+combo mnemonic)<br>
|
||||
</li>
|
||||
<li>get_accFocus: Which child is focused? <span
|
||||
style="font-weight: bold;">[important]</span></li>
|
||||
<li>get_accSelection: Which children of this item are selected?</li>
|
||||
<li>get_accDefaultAction: Get a description or name of the default action for this component, such as "jump" for links.</li>
|
||||
|
||||
<li>accSelect: Select the item associated with this IAccessible.</li>
|
||||
<li>accLocation: Get the x,y coordinates, and the height and width of this IAccessible node.</li>
|
||||
<li>accNavigate: Navigate up, down, left or right from this IAccessible.</li>
|
||||
<li>accHitTest: Find out what IAccessible exists and a specific coordinate.</li>
|
||||
<li>accDoDefaultAction: Perform the action described by get_accDefaultAction.</li>
|
||||
<li>get_accDefaultAction: Get a description or name of the default
|
||||
action for this component, such as "jump" for links.</li>
|
||||
<li>accSelect: Select the item associated with this IAccessible. <span
|
||||
style="font-weight: bold;">[important]</span></li>
|
||||
<li>accLocation: Get the x,y coordinates, and the height and width
|
||||
of this IAccessible node. <span style="font-weight: bold;">[important]</span></li>
|
||||
<li>accNavigate: Navigate up, down, left or right from this
|
||||
IAccessible. <span style="font-weight: bold;">[important]</span></li>
|
||||
<li>accHitTest: Find out what IAccessible exists and a specific
|
||||
coordinate.</li>
|
||||
<li>accDoDefaultAction: Perform the action described by
|
||||
get_accDefaultAction.</li>
|
||||
<li>put_accName: Change the name.</li>
|
||||
<li>put_accValue: Change the value.</li>
|
||||
</ul>
|
||||
</ul>
|
||||
</p>
|
||||
</ul>
|
||||
<h2>The Basics</h2>
|
||||
<h2>The Implementations Behind IAccessible</h2>
|
||||
<ul>
|
||||
<p>
|
||||
Rather than directly implement IAccessible with an Accessible class, we have chosen to proxy to our own cross-platform interface,
|
||||
called nsIAccessible, which is more robust. It has the capability of supporting other new accessibility API's such
|
||||
as Sun's Gnome Accessiblity API. The nsIAccessible interface is implemented by a variety of classes for each of the
|
||||
various objects in HTML. Each class is tailored to the specific abilities and properties of the HTML objects it applies to.
|
||||
</p>
|
||||
<p>
|
||||
The first thing that happens when an accessibility aid wants to watch our application is calls the Windows API function
|
||||
AccessibleObjectFromWindow(). This in turns sends the window in question
|
||||
a <a href="http://lxr.mozilla.org/seamonkey/search?string=WM_GETOBJECT">WM_GETOBJECT</a> message requesting an IAccessible for the window.
|
||||
In our case, this event is received in mozilla/widget/src/windows/nsWindow.cpp.
|
||||
We send back an IAccessible interface that represents that root window. The accessibility aid will use
|
||||
that first IAccessible to reach rest of the IAccessible hierarchy, by asking for it's children IAccessibles, asking the children for the
|
||||
grandchildren IAccessibles, and so on. Until this WM_GETOBJECT message is processed, the accessibility.dll is not loaded,
|
||||
so there is almost zero overhead for accessibility in Mozilla.
|
||||
</p>
|
||||
</ul>
|
||||
|
||||
<h2>How the IAccessible's are Created</h2>
|
||||
<ul>
|
||||
<p>
|
||||
To create the root IAccessible for a window the first time it gets the <a href="http://lxr.mozilla.org/seamonkey/search?string=WM_GETOBJECT">WM_GETOBJECT</a> message in,
|
||||
nsWindow.cpp first generates an internal event
|
||||
called <a href="http://lxr.mozilla.org/seamonkey/search?string=NS_GETACCESSIBLE">NS_GETACCESSIBLE</a>,
|
||||
which is handled in nsFrame.cpp via the creation of an nsRootAccessible implementation of the nsIAccessible interface.
|
||||
The first IAccessible is then created by instantiating a RootAccessible class. This RootAccessible is also cached by
|
||||
the nsWindow it's for, so that any additional WM_GETOBJECT messages use the same RootAccessible.
|
||||
The RootAccessible class used to implement IAccessible here is slightly different from the normal Accessible class
|
||||
that's used, in that it keeps track of event data.
|
||||
RootAccessible and Accessible are both implemented
|
||||
in <a href="http://lxr.mozilla.org/seamonkey/source/widget/source/windows/Accessible.cpp">
|
||||
mozilla/widget/src/windows/Accessible.cpp</a>).
|
||||
</P>
|
||||
</ul>
|
||||
<h2>The Real Power Behind IAccessible's</h2>
|
||||
<ul>
|
||||
<p>The implementations of IAccessible (Accessible and RootAccessible), don't know anything about Mozilla
|
||||
objects. They merely proxy to our cross platform accessibility classes, which all have an nsIAccessible interface.
|
||||
</h2>
|
||||
<p>
|
||||
The base implementation for nsIAccessible is called nsAccessible. It has default implementations for all the
|
||||
nsIAccessible methods. It also knows how
|
||||
to walk Mozilla's content DOM and frame tree, exposing only the objects that are needed for accessibility.
|
||||
Essentially, nsAccessible knows what it needs to expose by asking each DOM node's primary frame for
|
||||
an nsIAccessible, using the GetAccessible() method. If it gets one, it's considered an accessible object.
|
||||
A frame that wishes to return
|
||||
an nsIAccessible GetAccessible() is called, creates one of the correct type on the fly using
|
||||
nsIAccessibilityService methods built for that purpose.
|
||||
</p>
|
||||
<p>
|
||||
The specific implementations
|
||||
of nsIAccessible for each widget or content type inherit from nsAccessible.
|
||||
Each implementation then overrides those methods
|
||||
it wishes to implement, and does nothing for those methods it wants the default behavior for.
|
||||
For example, the default behavior for nsIAccessible::getAccFirstChild is to
|
||||
instantial a nsDOMTreeWalker, and ask it for the first child. However, nsImageAccessible overrides getAccFirstChild,
|
||||
returning the first area of an image map if there is one, otherwise nsnull.
|
||||
</p>
|
||||
|
||||
<p>The methods in ns*AccessibleWrap classes don't implement much
|
||||
accessibility support. These platform-specific classes inherit from
|
||||
cross-platform classes, where the real implementations exist. For
|
||||
example, nsAccessibleWrap inherits from nsAccessible. Every accessible
|
||||
object in the MSAA tree has an nsAccessible class, which exposes
|
||||
accessibility information through nsIAccessible, in a cross-platform
|
||||
manner. This base implementation for nsIAccessible knows how to walk
|
||||
Mozilla's content DOM and frame tree, exposing only the objects that are
|
||||
needed for accessibility.<br>
|
||||
</p>
|
||||
<p>Essentially, nsAccessible knows what it needs to expose by asking
|
||||
each DOM node's primary frame for an nsIAccessible, using the
|
||||
GetAccessible() method. If it gets one, it's considered an accessible
|
||||
object. A frame that wishes to return an nsIAccessible GetAccessible()
|
||||
is called, creates one of the correct type on the fly using
|
||||
nsIAccessibilityService methods built for that purpose.</p>
|
||||
<p>The nsIAccessible interface is implemented by a variety of
|
||||
cross-platform classes for each of the various objects in HTML. Each
|
||||
class is tailored to the specific abilities and properties of the HTML
|
||||
objects it applies to, and can support both MSAA, ATK and hopefully any
|
||||
future accessibility API's we need to support.<br>
|
||||
</p>
|
||||
<p>The specific implementations of nsIAccessible for each widget or
|
||||
content type override methods from nsAccessible. For example
|
||||
nsHTMLButtonAccessible overrides get_accRole to expose ROLE_BUTTON for
|
||||
that getter. <br>
|
||||
</p>
|
||||
<p>A more complicated set of nsIAccessible methods which can be
|
||||
overriden are GetAccFirstChild/GetAccLastChild/GetAccChildCount, which
|
||||
allows for objects to define their own decendent subtrees. The default
|
||||
behavior for nsIAccessible::getAccFirstChild is to instantiate a
|
||||
nsDOMTreeWalker, and ask it for the first child. However,
|
||||
nsImageAccessible overrides getAccFirstChild, returning the first area
|
||||
of an image map if there is one, otherwise nsnull. This is necessary
|
||||
because the image map areas can be in a completely different area of the
|
||||
DOM from the image they apply to.<br>
|
||||
</p>
|
||||
</ul>
|
||||
<h2>MSAA Events</h2>
|
||||
<ul>
|
||||
<p>
|
||||
When an accessibility-related event occurs within an application such as Mozilla, it must use NotifyWinEvent from
|
||||
the Win32 API. NotifyWinEvent is passed arguments for the window the event occured in, and the number of the child
|
||||
within that window. Accessibility aids use the Win32 call SetWinEventHook() to register as a listener for these events.
|
||||
</p>
|
||||
<p>
|
||||
The accessibility aid is choose which events it is interested in learning more about by using the Win32 API call
|
||||
AccessibleObjectFromEvent, requesting the IAccessible to the node corresponding to the
|
||||
child number that had been indicated from NotifyWinEvent(). This ends up asking our RootAccessible for the child
|
||||
IAccessible that matches the window handle and child id we indicated through NotofyWinEvent().
|
||||
</p>
|
||||
<p>
|
||||
In Mozilla, this creates a problem. We cannot
|
||||
keep track of a child number for every important accessible node in a document. We deal with this by generating fake
|
||||
child IDs for the most recent accessibile events that we have generated, in a circular array.
|
||||
</p>
|
||||
<p>
|
||||
Since there is a RootAccessible for each top level window that might generate MSAA events, that's where we do the bookkeeping
|
||||
for these events and their nsIAccessible's. Whenever NotifyWinEvent() is called, a new fake ID is generated (We use
|
||||
negative numbers for the fake IDs). When the callback comes to request the IAccessible for that child number,
|
||||
we check the circular array for that ID, and voila, we have the corresponding nsIAccessible to proxy.
|
||||
</p>
|
||||
<p>
|
||||
Every RootAccessible has an nsRootAccessible which is an nsIAccessibleEventReceiver. The RootAccessible
|
||||
uses this to register itself as an nsIAccessibleEventListener. In the end, nsRootAccessible registers itself as a listener
|
||||
of Mozilla's internal and DOM events. It's HandleEvent routine translates these events into MSAA events, and passes them along to
|
||||
with an nsIAccessible to the original RootAccessible::HandleEvent
|
||||
which turns that nsIAccessible into a NotifyWinEvent call, complete with a fake child ID.
|
||||
</p>
|
||||
<p>
|
||||
Most MSAA events aren't utilized by accessibility aids. Therefore we implement only the handful that matter.
|
||||
The most important event is the focus event, followed by name, state and value change events.
|
||||
</p>
|
||||
<p> When an accessibility-related event occurs within an application
|
||||
such as Mozilla, it must use NotifyWinEvent from the Win32 API.
|
||||
NotifyWinEvent is passed arguments for the window the event occured in,
|
||||
and the number of the child within that window. Accessibility aids use
|
||||
the Win32 call SetWinEventHook() to register as a listener for these
|
||||
events. </p>
|
||||
<p> The accessibility aid is choose which events it is interested in
|
||||
learning more about by using the Win32 API call
|
||||
AccessibleObjectFromEvent, requesting the IAccessible to the node
|
||||
corresponding to the child number that had been indicated from
|
||||
NotifyWinEvent(). This ends up asking our nsDocAccessibleWrap or
|
||||
nsRootAccessibleWrap for the child IAccessible that matches the window
|
||||
handle and child id we indicated through NotofyWinEvent(). </p>
|
||||
<p>In Mozilla, we use the DOM node pointer in the accessible object
|
||||
as its child ID, which is also used as a hashkey into our cache. During
|
||||
the callback, we look up the original accessible node in the
|
||||
nsDocAccessible's cache and return it.<br>
|
||||
</p>
|
||||
</ul>
|
||||
<ul>
|
||||
<p>Most MSAA events aren't utilized by accessibility aids. Therefore
|
||||
we implement only the handful that matter. See <a
|
||||
href="http://www.mozilla.org/projects/ui/accessibility/vendors-win.html">Gecko
|
||||
Info for Windows Accessibility Vendors</a> for the list of events we
|
||||
implement. By far the most important one is EVENT_OBJECT_FOCUS.<br>
|
||||
</p>
|
||||
</ul>
|
||||
<h2>MSAA Events - Cheatsheet for Implementors<br>
|
||||
</h2>
|
||||
<ul>
|
||||
<p>For information on what each event does, see the <a
|
||||
href="file:///c:/moz/mozilla/accessible/Hello">MSDN Event Constants
|
||||
page</a>.</p>
|
||||
<p>Check with your assistive technology partners to find out what
|
||||
events you need to support. There's a very good chance they won't ask
|
||||
for more than the events marked <span style="font-weight: bold;">[important]</span>:<br>
|
||||
</p>
|
||||
</ul>
|
||||
<table
|
||||
style="text-align: left; margin-left: auto; width: 75%; margin-right: auto;"
|
||||
border="0" cellspacing="2" cellpadding="2">
|
||||
<tbody>
|
||||
<tr>
|
||||
<td style="vertical-align: top;">EVENT_SYSTEM_SOUND<br>
|
||||
EVENT_SYSTEM_ALERT<br>
|
||||
EVENT_SYSTEM_FOREGROUND<br>
|
||||
EVENT_SYSTEM_MENUSTART<br>
|
||||
EVENT_SYSTEM_MENUEND<br>
|
||||
EVENT_SYSTEM_MENUPOPUPSTART <span style="font-weight: bold;">[important]</span><br>
|
||||
EVENT_SYSTEM_MENUPOPUPEND <span style="font-weight: bold;">[important]</span><br>
|
||||
EVENT_SYSTEM_CAPTURESTART<br>
|
||||
EVENT_SYSTEM_CAPTUREEND<br>
|
||||
EVENT_SYSTEM_MOVESIZESTART<br>
|
||||
EVENT_SYSTEM_MOVESIZEEND<br>
|
||||
EVENT_SYSTEM_CONTEXTHELPSTART<br>
|
||||
EVENT_SYSTEM_CONTEXTHELPEND<br>
|
||||
EVENT_SYSTEM_DRAGDROPSTART<br>
|
||||
EVENT_SYSTEM_DRAGDROPEND<br>
|
||||
EVENT_SYSTEM_DIALOGSTART<br>
|
||||
EVENT_SYSTEM_DIALOGEND<br>
|
||||
EVENT_SYSTEM_SCROLLINGSTART<br>
|
||||
EVENT_SYSTEM_SCROLLINGEND <span style="font-weight: bold;">[possibly
|
||||
important, talk to AT vendor]</span><br>
|
||||
EVENT_SYSTEM_SWITCHSTART<br>
|
||||
EVENT_SYSTEM_SWITCHEND<br>
|
||||
<br>
|
||||
</td>
|
||||
<td style="vertical-align: top;">EVENT_SYSTEM_MINIMIZESTART<br>
|
||||
EVENT_SYSTEM_MINIMIZEEND<br>
|
||||
EVENT_OBJECT_CREATE <span style="font-weight: bold;">[don't implement,
|
||||
watching system generated versions of this event causes </span><span
|
||||
style="font-weight: bold;">assistive technology </span><span
|
||||
style="font-weight: bold;">crashes]</span><br>
|
||||
EVENT_OBJECT_DESTROY <span style="font-weight: bold;">[don't
|
||||
implement, watching system generated versions of this event causes
|
||||
assistive technology crashes]</span><br>
|
||||
EVENT_OBJECT_SHOW<br>
|
||||
EVENT_OBJECT_HIDE<br>
|
||||
EVENT_OBJECT_REORDER <span style="font-weight: bold;">[important for
|
||||
mutating docs in future, but not yet]</span><br>
|
||||
EVENT_OBJECT_FOCUS <span style="font-weight: bold;">[important]</span><br>
|
||||
EVENT_OBJECT_SELECTION<br>
|
||||
EVENT_OBJECT_SELECTIONADD<br>
|
||||
EVENT_OBJECT_SELECTIONREMOVE<br>
|
||||
EVENT_OBJECT_SELECTIONWITHIN<br>
|
||||
EVENT_OBJECT_STATECHANGE <span style="font-weight: bold;">[important
|
||||
for checkboxes and radio buttons]</span><br>
|
||||
EVENT_OBJECT_LOCATIONCHANGE<br>
|
||||
EVENT_OBJECT_NAMECHANGE<br>
|
||||
EVENT_OBJECT_DESCRIPTIONCHANGE<br>
|
||||
EVENT_OBJECT_VALUECHANGE<br>
|
||||
EVENT_OBJECT_PARENTCHANGE<br>
|
||||
EVENT_OBJECT_HELPCHANGE<br>
|
||||
EVENT_OBJECT_DEFACTIONCHANGE<br>
|
||||
EVENT_OBJECT_ACCELERATORCHANGE<br>
|
||||
</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
<h2>Dealing with the Quirks of MSAA</h2>
|
||||
<p style="margin-left: 40px;">MSAA has a well deseved reputation for
|
||||
quirkiness. It is not "plug and play", and will take a lot of
|
||||
testing/refinement before your solution works with any product. Here
|
||||
are some of it's quirks and some solutions/workarounds:<br>
|
||||
</p>
|
||||
<ul>
|
||||
<li>Crash prone - more than one process refcounting same object</li>
|
||||
</ul>
|
||||
<div style="margin-left: 40px;">Problem: When your application closes,
|
||||
different signals are broadcast. For example, the application window
|
||||
closes and the window is blurred. It is impossible to know if and when
|
||||
the 3rd party assistive technology will use one of these signals to
|
||||
release the objects of yours that are being refcounted. This can lead
|
||||
to crashes where some of your dll's are unloaded but not others, and a
|
||||
destructor is called in an unloaded DLL.<br>
|
||||
<br>
|
||||
Solution: Create a "shutdown" method for each internal accessible
|
||||
object, to remove any references to other internal objects before any
|
||||
of your dll's are unloaded. In order to do this effectively, you will
|
||||
have to keep track of every accessible object that you create. The
|
||||
easiest way to do that is to keep a pointer to an accessible in each
|
||||
internal UI object. If that pointer is non-null, then there is an
|
||||
accessible object for it. Whenever a UI object is destroyed, shutdown
|
||||
it's accessible object as well. In Gecko/Mozilla we are not allowed to
|
||||
keep this extra pointer for each accessible object, so we use a hash
|
||||
table cache, which is kept in sync with the tree of UI objects. Also,
|
||||
don't implement EVENT_OBJECT_CREATE or EVENT_OBJECT_DESTROY. Vendors
|
||||
have found that watching these events causes crashes.<br>
|
||||
</div>
|
||||
<ul>
|
||||
<li>Hacky caret tracking interferes with MSAA solution<br>
|
||||
</li>
|
||||
</ul>
|
||||
<div style="margin-left: 40px;">Problem: The screen reader does not use
|
||||
the MSAA caret. It follows screen draws, and looks for a vertical
|
||||
blinking line. Unfortunately, some screen readers can get confused by
|
||||
the vertical lines on other objects, such as list boxes, even though
|
||||
those lines are not blinking.<br>
|
||||
<br>
|
||||
Solution: Make sure there is a configuration file for each screen
|
||||
reader specific to your application. Read the manual and find the
|
||||
keystroke or commands for training the caret, and save this information
|
||||
in the configuration file.<br>
|
||||
</div>
|
||||
<ul>
|
||||
<li> Event window confusion<br>
|
||||
<br>
|
||||
Problem: The screen reader or other assistive technology does not track
|
||||
the focus or other events correctly.<br>
|
||||
<br>
|
||||
Solution: This may be because you are reporting that the events are
|
||||
occuring in a different window from the system window focus that the
|
||||
screen reader gets from the Win32 call GetGUIThreadInfo() in the
|
||||
hwndFocus field. This might be because although you are visibly showing
|
||||
window focus on the correct window, you have not told the operating
|
||||
system to change focus to this window. You must keep the two in sync.<br>
|
||||
<br>
|
||||
</li>
|
||||
<li>Confusion with system-generated events<br>
|
||||
<br>
|
||||
Problem: When you test with Accessible Event Watcher in the MSAA SDK,
|
||||
you will see many events that you yourself did not generate. Microsoft
|
||||
Windows generates some events, like focus events, for you. This is
|
||||
extremely annoying. The assistive technology has no way to tell whether
|
||||
the event came from your application or from Windows. For example, when
|
||||
you set window focus, Windows will generate an EVENT_OBJECT_FOCUS
|
||||
event. If you happen to set window focus after you fired your own
|
||||
EVENT_OBJECT_FOCUS event, your event will be ignored, because assitive
|
||||
techology software tends to pay attention only to the last focus event.<br>
|
||||
<br>
|
||||
Solution: When an object is about to get focused in a different window,
|
||||
make sure you focus a window before you fire your own focus events for
|
||||
objects inside it. Test using Accessible Event Watcher in the MSAA SDK,
|
||||
and use the settings to watch subsets of accessibility events. Count on
|
||||
the assistive technology to make sense out the jumble.<br>
|
||||
<br>
|
||||
</li>
|
||||
<li>No unique child id for object in window.<br>
|
||||
<br>
|
||||
Problem: MSAA expects events to be reported using
|
||||
NotifyWinEvent(eventNum, hWnd, worldID, childID), and there may not be
|
||||
an obvious way to get a window handle and a 32 bit childID for an
|
||||
object. You may be using windowless controls, or have an entire
|
||||
document with lots of objects in a given window. The child ID must be
|
||||
unique, because this is what the assistive technology will use to
|
||||
retrieve the accessible via AccessibleObjectFromEvent() which ends up
|
||||
using get_accChild on the accessible for the given window.<br>
|
||||
<br>
|
||||
Solution: In Gecko/Mozilla, we did not want to store an extra 32 bit
|
||||
unique ID value on every object. Instead, we hand back a 32 bit value
|
||||
derived from the UI object's pointer, which is unique. We ensure that
|
||||
the value we hand back is always negative. When the get_accChild call
|
||||
comes back, we check our hash table cache for that window to see if
|
||||
there's an accessible object still associated with that unique value.
|
||||
This means the client must use AccessibleObjectFromEvent immediately,
|
||||
because there is a danger that the object will go away, and another
|
||||
differnt object will be created with the same pointer value.<br>
|
||||
<br>
|
||||
</li>
|
||||
<li>Not all features utilized by 3rd party vendors<br>
|
||||
<br>
|
||||
Problem: The assistive technology does not use 50% of what's available
|
||||
in MSAA, e.g. MSAA caret, a lot of events, roles, states and methods.<br>
|
||||
<br>
|
||||
Solution: Use this document to see what is generally considered
|
||||
important by assistive technology manufacturers. Contact the vendors
|
||||
early and often as you plan and implement your architecture, to see
|
||||
what's important to them. Ignore every OBJID except for OBJID_CLIENT in
|
||||
the WM_GETOBJECT message. Don't try to build a palace that no one will
|
||||
live in. Implement only what's needed.<br>
|
||||
<br>
|
||||
</li>
|
||||
<li>Missing functionality in MSAA<br>
|
||||
<br>
|
||||
Problem and Solutions: Assistive technology vendors need some things
|
||||
which MSAA does not provide, such as:<br>
|
||||
</li>
|
||||
<ul>
|
||||
<li>No way of signifying that a document has finished
|
||||
loading. Fire EVENT_OBJECT_STATECHANGE for a window/client/pane
|
||||
object when it starts to load a new document. Use STATE_BUSY to
|
||||
indicate that a new document is being loaded. When the loading has
|
||||
finished, fire another EVENT_OBJECT_STATECHANGE event and clear the
|
||||
STATE_BUSY flag. </li>
|
||||
</ul>
|
||||
<ul>
|
||||
<li>No method to get clipped/unclipped bounds of a piece of text
|
||||
within a text object. This is needed by screen magnifiers. No scrollTo
|
||||
method, also needed by screen magnifiers. Implement a custom interface
|
||||
for text objects, and support it through QueryInterface or QueryService
|
||||
if it's being implemented on a different object than IAccessible is.
|
||||
Support a scrollTo method which takes a text index, and a
|
||||
getClippedBounds and getUnclippedBounds method which takes a start and
|
||||
end index. Publish your custom interface.</li>
|
||||
<li>No way for assistive technology to know when scrolling has
|
||||
stopped. Fire the EVENT_SYSTEM_SCROLLINGEND event to indicate when
|
||||
scrolling has ended (try not to fire too many of these, wait until
|
||||
scrolling has truly stopped). There is no need to support
|
||||
EVENT_SYSTEM_SCROLLINGSTART, it is not used by assistive technology.<br>
|
||||
</li>
|
||||
<li>No support for document formatting or "DOM" as requested by
|
||||
some vendors: support a custom interface that gives them the formatting
|
||||
information they are requesting.<br>
|
||||
<br>
|
||||
</li>
|
||||
</ul>
|
||||
<li>Dueling text equivalents<br>
|
||||
<br>
|
||||
Problem: There are three kinds of text equivalents, and it is difficult
|
||||
to know when to use each. Acrobat uses accessible value for text labels
|
||||
where as most programs use accessible name. There are different roles
|
||||
for text objects, ROLE_STATICTEXT and ROLE_TEXT (editable text), which
|
||||
seems to be used for non-editable text in many places.<br>
|
||||
<br>
|
||||
Solution: Be as consistent with Internet Explorer as possible. Use
|
||||
accessible name for most text equivalents, and accessible value for
|
||||
URL's. Don't use accessible description unless you really do have a
|
||||
long description for the object you need to expose -- most assistive
|
||||
technology makes little use of it. Use ROLE_STATICTEXT for labels
|
||||
specific to dialog controls, and always use ROLE_TEXT for document text
|
||||
even if the text is not editable (in that case use ROLE_TEXT with
|
||||
STATE_READONLY).<br>
|
||||
<br>
|
||||
</li>
|
||||
<li>Issues with Links<br>
|
||||
<br>
|
||||
Problem: The assistive technology has inflexible heuristics when it
|
||||
comes to reading links. First, it won't read the object unless the
|
||||
states are correctly set. Second, it won't read the object if it cannot
|
||||
parse the whitespace according to its own rules.<br>
|
||||
<br>
|
||||
Solution: Make sure the ROLE_LINK object and its child ROLE_TEXT
|
||||
objects all have STATE_LINKED set. For multi-line links with a line
|
||||
break in the middle, make sure there is no whitespace at the beginning
|
||||
or end of any of the accessible names, and make sure there is a \r\n
|
||||
where the line breaks occur in the accessible name for the ROLE_LINK.
|
||||
For an example of how to do this properly, see Internet Explorer or
|
||||
Gecko. Again, if it's not done exactly this way, some links will not be
|
||||
read.<br>
|
||||
<br>
|
||||
</li>
|
||||
<li>MSAA Implementation is Not Performant<br>
|
||||
<br>
|
||||
Problem: The assistive technology may interact slowly with your
|
||||
application.<br>
|
||||
<br>
|
||||
Solution: Try not to calculate the same things more than once or create
|
||||
the same objects more than once. For example, create and cache an
|
||||
object's children when you look for them in get_accChildCount(), so
|
||||
that you can just hand them back when asked for using get_accChild() or
|
||||
accNavigate(). Support IEnumVARIANT so that the MSAA client can ask for
|
||||
a number of children in one call. In custom interfaces, create methods
|
||||
that hand back a lot of data in one call, rather than requiring a large
|
||||
number of calls. The fewer calls, the better, because a lot of slowness
|
||||
is caused by COM Marshaling.<br>
|
||||
<br>
|
||||
</li>
|
||||
<li>Differing client implementations<br>
|
||||
<br>
|
||||
Problem: Every assistive technology uses MSAA differently.<br>
|
||||
<br>
|
||||
Solution: We don't know of any outright conflicts in the differing uses
|
||||
of MSAA (yet). However, be on guard. If a vendors asks you to do
|
||||
something different from the spec, it's better to check with the other
|
||||
vendors before moving forward. Check to see what applications from
|
||||
Microsoft do in a similar situation.<br>
|
||||
<br>
|
||||
</li>
|
||||
<li>Vendor quirks:<br>
|
||||
<br>
|
||||
Problem: Because assistive technology tends to utilize MSAA as an
|
||||
additional solution resting on top of old hacks, rather than a
|
||||
completely clean and separate way to deal with an application, and
|
||||
because of the quirky nature of MSAA and of the inflexible heuristics
|
||||
that screen readers use, we do not have a "plug and play solution". In
|
||||
addition, assistive technology vendors are tiny companies, often
|
||||
scrambling to keep up with changes in the applications they already
|
||||
support, or new products other than yours which they need to support.
|
||||
It's very difficult to get vendors to spend time testing an MSAA
|
||||
implementation, send feedback or help find out why things aren't
|
||||
working. Time and version commitments often fall through. There is
|
||||
always a belated new version due around corner, after which you will be
|
||||
promised to be the next prioritiy.<br>
|
||||
<br>
|
||||
Solution: Try to reach out as a friend of the assistive technology
|
||||
company. Be as easy to work with as you possibly can, including being
|
||||
extremely responsibe to bug reports with new test builds, as well as
|
||||
being very communicative about what you have fixed. Do as much work as
|
||||
you possibly can without their help. See if your organization can offer
|
||||
something they can't get for themselves. Be patient, set your
|
||||
expectations to a reasonable level. Realize that it's still business
|
||||
for these companies, and that they need to sell more copies of their
|
||||
software to make up for the time they spend on compatibility with your
|
||||
application. Remember that no matter how small they are, you need them
|
||||
more than they need you, unless your application's accessibility is
|
||||
being demanded by end-users.</li>
|
||||
</ul>
|
||||
<h2>Feedback</h2>
|
||||
<div style="margin-left: 40px;">How can this document be improved? Was
|
||||
it useful? Questions? Contact <a href="mailto:aaronl@netscape.com">aaronl@netscape.com</a><br>
|
||||
</div>
|
||||
</body>
|
||||
</html>
|
||||
Загрузка…
Ссылка в новой задаче