How to: Implement accessibility for custom widgets in Qt (for NVDA screen-reader)

As promised I’m going to share with you some things that I’ve learned this summer while developing accessibility for MuseScore. This post will be about implementing accessibility for your custom widgets in Qt. Please keep in mind that these are only things that I’ve learned myself and might not be the official way to do them. Also, if you need to set just a string to a widget just call QWidget::setAccessibleName and QWidget::setAccessibleDescription and Qt will do the rest. This how to is if you need a more complex approach and the informations here are not exhaustive. I will try to tell you exactly what you need to know to get thigs running, but if need stuff more complex than what is here, you need to struggle yourself. ūüôā

Link to an example project[0].

First things first. Don’t forget to put the accessible plugin in the folder where your binary is when creating the installer.

Also, it is important to understant from the begining that every screen-reader has its own “personality”, in the sense that what works for one screen-reader might not work as well, or at all for other, unfortunatelly. Qt provides the same informations for all the screen-reader, but it’s the screen-reader who choses what to do with those informations, what and how to read and what to ignore.

There are 2 main steps when you are trying to do this:

Step 1: Implement QAccessibleInterface for your widget. 

The good thing is that you don’t have to implement all the methods of QAccessibleInterface. For widgets it’s easier to subclass QAccessibleWidget which does part of the job for you. Still you will have to implement these¬†methods:


int childCount() const ;
QAccessibleInterface* child(int index) const;
QAccessibleInterface* parent() const;
QRect rect() const;
QAccessible::Role role() const;
QString text(QAccessible::Text t) const;

1.1¬†Let’s take them one at the time, but let’s start with¬†QString text(QAccessible::Text t) const;.

The QAccessible::Text enum defines these types of text:

QAccessible::Name 0 The name of the object. This can be used both as an identifier or a short description by accessible clients.
QAccessible::Description 1 A short text describing the object.
QAccessible::Value 2 The value of the object.
QAccessible::Help 3 A longer text giving information about how to use the object.
QAccessible::Accelerator 4 The keyboard shortcut that executes the object’s default action.
QAccessible::UserText 0x0000ffff The first value to be used for user defined text.

Depending on the “personality” of the screen-reader, it will tell you some of them. For NVDA, I think that it’s ok to return something for¬†Name, Value¬†and¬†Description. (of course if you have that much info, you don’t need to put information there, just not to leave those fields empty).

Taking for example a QSlider, the QAccessible::Name¬†would be what you set for¬†QSlider::accessibleName, the QAccessible::Description would be what it’s set for QSlider::accessibleDescription¬†and the QAccessible::Value¬†is the value of the slider.

Important:¬†It’s important to understand that Qt intermidiates all the interactions between widgets and assistive technologies using the QAccessibleInterface. The accessibleName and accessibleValue are \not called directly by the screen-reader. For the QSlider the text method, probably looks like this in Qt:

QString AccessibleSlider::text(QAccessible::Text t) const
{
   QSlider* s = static_cast<QSlider*>(widget());
      switch (t) {
         case QAccessible::Name:
            return s->accessibleName();
         case QAccessible::Value:
            return QString::number(s->value());
         case QAccessible::Description:
            return s->accessibleDescription();
         default:
            return QString();
         }
         return QString();
   }

So you will want to override this behaviour¬†and make it return what you want. For example, in MuseScore, in order to add accessibility support for the score, I subclassed the QAccessibleWidget for ScoreView. For clarifications, in MuseScore, the score as internal representation is done by the Score class and the widget that you see on screen is a ScoreView. So the ScoreView will return the name of its Score for QAccessible::Name and for the QAccessible::Value, I’m returning a string that is constructed for the selected score element in the moment of the selection.

1.2 int childCount()

QAccessibleInterface* child(int index) const;

QAccessibleInterface* parent() const;

As you know, the Qt widget interface is actually a tree of QWidgets and QObjects. For accessibility, you can take into account the actual tree, or you can do it anyway you want. For this pourpose you have these methods that at the first sight are somewhat duplications of those from QWidgets. You can skip branches, you can skip levels, you can add branches, etc. (I didn’t do this kind of things, so I can’t give more details here).

IMPORTANT: What I can tell you is that child(0) is not the current widget itself. This behaviour is deprecated in Qt 5.

1.3 QAccessible::Role role() const

Here, you must return a value of the QAccessible::Role. You should find the value that is closest to what your custom widget represents.¬†Because of the screen-reader’s “personality”, you might receive different outputs for different values of QAccessible::Role, even though the text() method returns the same thing. (this can happen even for different values of QAccessible::Role, with the same screen-reader).

1.4 QRect rect() const

I’m not 100% sure here, but common sense tells me that this value reflects the area that is ocupied by the widget. I returned the value given by the QWidget::rect().

1.5* QWindow* window() const

This is not one of the methods that you *have* to implement, but I highly recommend it. By default it returns 0 and if going from parent to parent, there is no QAccessibleInterface that returns a valid value, the information is not sent to the screen-reader.

Step 2: Factory

Even though you have implemented the QAccessibleInterface. Your work is not done yet. You need to tell Qt about it and how and when it should be instantiated. You will not instantiate it yourself. You nead to create a factory method and install this factory.

It needs to have that exact signature for it and it would look something like this:

QAccessibleInterface* ClassNameFactory(const QString &classname, QObject *object)
{
   qDebug("Creating interface for ClassName object");
   QAccessibleInterface *iface = 0;
   if (classname == QLatin1String("ClassName") && object && object->isWidgetType()){
      iface = static_cast<QAccessibleInterface*>(new ClassName(object));
}

return iface;
}
And in order to install it you will need to call QAccessible::installFactory(ClassNameFactory), in main, before running exec(). What will this do is to create a function pointer to your factory method and when the screen-reader asks for info about a specific widget, Qt calls first all your factory methods for it and only if none of them returns a new QAccessibleFactory Qt will instantiate the default one. I recommend making this a static method in the ClassName, because it would keep all your code related to this in one place.

3. Updates

The screen-reader will tell by default the informations related to a widget when it gets focus, or when you hover over it with the mouse cursor, but you might want to receive updates when things change. For example in MuseScore, I made it so that the user receives the informations about the selected element after every change it makes and the focus doesn’t leave the ScoreView.

In order to do this you will need to call QAccessible::updateAccessibility(QAccessibleEvent ev).

IMPORTANT:¬†Calling this method will have no effect if you haven’t done Steps 1 and 2¬†are not done properly.

As for the events, most of them are created like this QAccessibleEvent ev(targetObject, QAccessible::NameChanged). There is an enum in QAccessible for event types. However, there are events like QAccessibleValueChange event that need to be instantiated with their specific class, if you try something like QAccessibleEvent ev(targetObject, QAccessible::ValueChanged) the compiler won’t say anything, but you will get a runtime error, so when you decide what events fits best your pourpose, check if there is a specific class for it.

4. Tips and tricks for NVDA

(Some of these might be “placebos”, but they worked for me, so if you run out of ideas, you might want to try them ūüôā again they are not from the official Qt documentation and might not work for other screen-readers)

¬†If you call QAccessible::updateAccessibility and the widget doesn’t have focus in that moment, NVDA will ignore those informations (this is not a placebo!)

–¬†try to call QAccessible::updateAccessibility as late as possible. If there is another call after yours, (made by you, or internally by Qt) NVDA will only take the last one, so try calling it as close as possible to the point where you give control to the event loop.

– There is a method QAccessible::isActive that tells you if at any point there was an assistive technology connected to your GUI, but keep in mind that it’s not a signal and it doesn’t return false once you close the screen-reader.

РThere might be a bug on Windows, so that if you open a GUI directly maximized you need to minimize and restore the window, before the updates kick in.

– Try adding your factory method as earlier as possible, don’t leave it right before the exec() (might be a placebo)

– Implement QWindow QAccessibleInterface::window() const yourself

РUse Q_DECL_OVERRIDE after every method declaration, if you not override the default behaviour, you might not get a compiler error, but you will spend hours trying to debug this.

I highly recommend that you also read the Qt official documentation, I know that there are things still in the development stage and there might be bugs. Once my code gets merged I will also post a link to the implementation I did for MuseScore. However I will post a link to a small test project I did[0] and will leave a link[1] to a tutorial that helped me, but has some deprecated code.

Hope this helps!

[0] https://www.dropbox.com/s/zu2pm5w4d30i6rv/Implementing%20accessibility.zip

[1] http://www.ranorex.com/blog/enabling-automation-for-custom-qt-widgets-by-adding-accessibility

Update:¬†As promised here are the¬†links from the official repository with MuseScore’s implementation

https://github.com/musescore/MuseScore/blob/master/mscore/scoreaccessibility.h

https://github.com/musescore/MuseScore/blob/master/mscore/scoreaccessibility.cpp

 

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s