MuseScore 2.0 Beta with accessibility!

I’m very happy to say that MuseScore announced the release for Beta version of MuseScore 2.0 on 25 august 2014 and that my accessibility project will be among the new features! 🙂


How to: Implement accessibility for custom widgets in Qt (for NVDA screen-reader)

As promised I’m going to share with you some things that I’ve learned this summer while developing accessibility for MuseScore. This post will be about implementing accessibility for your custom widgets in Qt. Please keep in mind that these are only things that I’ve learned myself and might not be the official way to do them. Also, if you need to set just a string to a widget just call QWidget::setAccessibleName and QWidget::setAccessibleDescription and Qt will do the rest. This how to is if you need a more complex approach and the informations here are not exhaustive. I will try to tell you exactly what you need to know to get thigs running, but if need stuff more complex than what is here, you need to struggle yourself. 🙂

Link to an example project[0].

First things first. Don’t forget to put the accessible plugin in the folder where your binary is when creating the installer.

Also, it is important to understant from the begining that every screen-reader has its own “personality”, in the sense that what works for one screen-reader might not work as well, or at all for other, unfortunatelly. Qt provides the same informations for all the screen-reader, but it’s the screen-reader who choses what to do with those informations, what and how to read and what to ignore.

There are 2 main steps when you are trying to do this:

Step 1: Implement QAccessibleInterface for your widget. 

The good thing is that you don’t have to implement all the methods of QAccessibleInterface. For widgets it’s easier to subclass QAccessibleWidget which does part of the job for you. Still you will have to implement these methods:

int childCount() const ;
QAccessibleInterface* child(int index) const;
QAccessibleInterface* parent() const;
QRect rect() const;
QAccessible::Role role() const;
QString text(QAccessible::Text t) const;

1.1 Let’s take them one at the time, but let’s start with QString text(QAccessible::Text t) const;.

The QAccessible::Text enum defines these types of text:

QAccessible::Name 0 The name of the object. This can be used both as an identifier or a short description by accessible clients.
QAccessible::Description 1 A short text describing the object.
QAccessible::Value 2 The value of the object.
QAccessible::Help 3 A longer text giving information about how to use the object.
QAccessible::Accelerator 4 The keyboard shortcut that executes the object’s default action.
QAccessible::UserText 0x0000ffff The first value to be used for user defined text.

Depending on the “personality” of the screen-reader, it will tell you some of them. For NVDA, I think that it’s ok to return something for Name, Value and Description. (of course if you have that much info, you don’t need to put information there, just not to leave those fields empty).

Taking for example a QSlider, the QAccessible::Name would be what you set for QSlider::accessibleName, the QAccessible::Description would be what it’s set for QSlider::accessibleDescription and the QAccessible::Value is the value of the slider.

Important: It’s important to understand that Qt intermidiates all the interactions between widgets and assistive technologies using the QAccessibleInterface. The accessibleName and accessibleValue are \not called directly by the screen-reader. For the QSlider the text method, probably looks like this in Qt:

QString AccessibleSlider::text(QAccessible::Text t) const
   QSlider* s = static_cast<QSlider*>(widget());
      switch (t) {
         case QAccessible::Name:
            return s->accessibleName();
         case QAccessible::Value:
            return QString::number(s->value());
         case QAccessible::Description:
            return s->accessibleDescription();
            return QString();
         return QString();

So you will want to override this behaviour and make it return what you want. For example, in MuseScore, in order to add accessibility support for the score, I subclassed the QAccessibleWidget for ScoreView. For clarifications, in MuseScore, the score as internal representation is done by the Score class and the widget that you see on screen is a ScoreView. So the ScoreView will return the name of its Score for QAccessible::Name and for the QAccessible::Value, I’m returning a string that is constructed for the selected score element in the moment of the selection.

1.2 int childCount()

QAccessibleInterface* child(int index) const;

QAccessibleInterface* parent() const;

As you know, the Qt widget interface is actually a tree of QWidgets and QObjects. For accessibility, you can take into account the actual tree, or you can do it anyway you want. For this pourpose you have these methods that at the first sight are somewhat duplications of those from QWidgets. You can skip branches, you can skip levels, you can add branches, etc. (I didn’t do this kind of things, so I can’t give more details here).

IMPORTANT: What I can tell you is that child(0) is not the current widget itself. This behaviour is deprecated in Qt 5.

1.3 QAccessible::Role role() const

Here, you must return a value of the QAccessible::Role. You should find the value that is closest to what your custom widget represents. Because of the screen-reader’s “personality”, you might receive different outputs for different values of QAccessible::Role, even though the text() method returns the same thing. (this can happen even for different values of QAccessible::Role, with the same screen-reader).

1.4 QRect rect() const

I’m not 100% sure here, but common sense tells me that this value reflects the area that is ocupied by the widget. I returned the value given by the QWidget::rect().

1.5* QWindow* window() const

This is not one of the methods that you *have* to implement, but I highly recommend it. By default it returns 0 and if going from parent to parent, there is no QAccessibleInterface that returns a valid value, the information is not sent to the screen-reader.

Step 2: Factory

Even though you have implemented the QAccessibleInterface. Your work is not done yet. You need to tell Qt about it and how and when it should be instantiated. You will not instantiate it yourself. You nead to create a factory method and install this factory.

It needs to have that exact signature for it and it would look something like this:

QAccessibleInterface* ClassNameFactory(const QString &classname, QObject *object)
   qDebug("Creating interface for ClassName object");
   QAccessibleInterface *iface = 0;
   if (classname == QLatin1String("ClassName") && object && object->isWidgetType()){
      iface = static_cast<QAccessibleInterface*>(new ClassName(object));

return iface;
And in order to install it you will need to call QAccessible::installFactory(ClassNameFactory), in main, before running exec(). What will this do is to create a function pointer to your factory method and when the screen-reader asks for info about a specific widget, Qt calls first all your factory methods for it and only if none of them returns a new QAccessibleFactory Qt will instantiate the default one. I recommend making this a static method in the ClassName, because it would keep all your code related to this in one place.

3. Updates

The screen-reader will tell by default the informations related to a widget when it gets focus, or when you hover over it with the mouse cursor, but you might want to receive updates when things change. For example in MuseScore, I made it so that the user receives the informations about the selected element after every change it makes and the focus doesn’t leave the ScoreView.

In order to do this you will need to call QAccessible::updateAccessibility(QAccessibleEvent ev).

IMPORTANT: Calling this method will have no effect if you haven’t done Steps 1 and are not done properly.

As for the events, most of them are created like this QAccessibleEvent ev(targetObject, QAccessible::NameChanged). There is an enum in QAccessible for event types. However, there are events like QAccessibleValueChange event that need to be instantiated with their specific class, if you try something like QAccessibleEvent ev(targetObject, QAccessible::ValueChanged) the compiler won’t say anything, but you will get a runtime error, so when you decide what events fits best your pourpose, check if there is a specific class for it.

4. Tips and tricks for NVDA

(Some of these might be “placebos”, but they worked for me, so if you run out of ideas, you might want to try them 🙂 again they are not from the official Qt documentation and might not work for other screen-readers)

 If you call QAccessible::updateAccessibility and the widget doesn’t have focus in that moment, NVDA will ignore those informations (this is not a placebo!)

– try to call QAccessible::updateAccessibility as late as possible. If there is another call after yours, (made by you, or internally by Qt) NVDA will only take the last one, so try calling it as close as possible to the point where you give control to the event loop.

– There is a method QAccessible::isActive that tells you if at any point there was an assistive technology connected to your GUI, but keep in mind that it’s not a signal and it doesn’t return false once you close the screen-reader.

– There might be a bug on Windows, so that if you open a GUI directly maximized you need to minimize and restore the window, before the updates kick in.

– Try adding your factory method as earlier as possible, don’t leave it right before the exec() (might be a placebo)

– Implement QWindow QAccessibleInterface::window() const yourself

– Use Q_DECL_OVERRIDE after every method declaration, if you not override the default behaviour, you might not get a compiler error, but you will spend hours trying to debug this.

I highly recommend that you also read the Qt official documentation, I know that there are things still in the development stage and there might be bugs. Once my code gets merged I will also post a link to the implementation I did for MuseScore. However I will post a link to a small test project I did[0] and will leave a link[1] to a tutorial that helped me, but has some deprecated code.

Hope this helps!



Update: As promised here are the links from the official repository with MuseScore’s implementation


weeks[12].toString(); // End of GSoC 2014!

Hello everyone!

This post is the last weekly blog post of Google Summer of Code 2014.

1. Acknowledgments

First of all, I would like to thank my mentor, Marc Sabatella, for all his help and patience! He was a true mentor during this whole summer, guiding me and responding to my every question (and I tend to put a *lot* of questions! 🙂 ).

I also want to thank the other members of the developing team and MuseScore open source community who helped me and gave me feedback  : Nicolas Froment (lasconic), Joachim Schmitz, Werner Schweer, Maurizio M. Gavioli. 

There were also people outside the MuseScore developing team who were really helpfull and I would like to thank them too: Frederik Gladhorn (accessibility developer for Qt) and Jaffar Ahmad Sidek.

I also want to congratulate all the other GSoC students who worked for MuseScore this summer: Ruchit Agrawal (shredpub), John Pirie (jpirie), Bartłomiej Lewandowski, Maxim Grishin (igevorse).  They all did amazing jobs!

(I hope I didn’t forget anyone here 🙂 ).

2. Review

Lets see what I did this summer:

– Provided accessibility for menus

– Prodivded accessibility for the main window of MuseScore:

     – Restructured actions and shortcuts so they only get triggered for their appropriate widgets.

     – Added accessibility for the toolbar (tab order + screen-reader info)

     – Added accessibility for inspector   (tab order + screen-reader info)

– Provided accessibility for the main dialogs

– Score accessibility (the most important part 🙂 ) – a way for blind users to read the score using the NVDA Screen-Reader

     – Ability to traverse the main elements of the score

     – Screen-reader feedback regarding the selected element and all the annotations/articulations/lyrics/etc (all the attached elements to it).

3. Why this project?

Well, I have to say that beeing a programmer and being passionate about technology doesn’t mean that I get excited about every new app. I’m not the kind of person that uses apps for time schedueling , doesn’t play games and who generally uses the computer/phone just for programming, movies and music (composing and listening). You can say that I’m an atypical geek in this way. 🙂 With this in mind, it’s easy to understand why it would be hard for me to find a project that I can be passionate about. I found MuseScore. This looked like a good organization to find a project; it went very well with my passion for composing music. The first project on their idea list was this one and after talking a bit with Marc, I knew that I simply had to work on it. With this project I could really feel that I was going to make a difference. I have to say that during the time I worked on accessibility I found out that the programs that offer support for it are expensive and the assistive technologies alike. So a free software for writing music, with free support for accessibility (which I would’ve developed) and with the goal to make it compatible first with free screen-readers. For me the alternative for an app that remainds me about a meeting is a sticky post, but the alternative of this project is far more expensive.

4. What is next?

I’ve already talked to Marc Sabatella and I’m commited to continue to develop accessibility for MuseScore. This project was just the begining, it allows the user to find his way through the window and read the score. There are plenty that can be done: score editing, braille displays, braille exports/imports, support for other accessible forms of music notation.

Aside from accessibility development, I will also do translation and… who knows? Ideas for features are certainly not something that is missing from MuseScore. 🙂

5. What I’ve done this week?

This week, I polished and integrated everything that I’ve worked on and fixed all the important bugs.


Although I’m not going to be posting every week here, you will certainly view new posts regarding the work I will be doing.

See you soon!



weeks[11].toString(); // Polishing and bug hunting

With the end of Google Summer of Code approaching and the score accessibility system in place, I’ve started fixing glitches and bugs and making everything work properly so that they can be integrated in the main branch.


1. Next element comand

I finnished this task some time ago, but I didn’t made a PR for it back then, some now went over it and fixed the inconsitencies that were generated by the changes in the main branch, I fixed some codying style issues and I’ve made the PR.


2. Score accessibility

 Here things falled in place very nicely. The design that I had changed a lot and with Marc Sabatella’s and Nicolas Froment’s (lasconic) help I finally got it to a stage where it is very easy to maintain.

First of all there was a major change in the workflow of the system. The score doesn’t announce anymore that the selection has changed. Instead, at the end of every action (in the MuseScore::endCmd method), ScoreAccessibility::updateAccessibilityInfo is called the ScoreAccessibility class knows to look in the current selection and extract any information it needs.

Also there are two new methods. Along with Element::accessibleInfo there is Element::screenreaderInfo and Element::accessibilityExtraInfo. The first one returns the string that is placed in the status bar. The second and last one are speacially created for the screen-reader. Element::screenreaderInfo returns by default Element::accessibbleInfo, but can be overriden in case the screen-reader doesn’t know any symbol from that string (see bug fix section*) and Element::accessibleExtraInfo returns a string that is appended by the ScoreAccessibility class to the Element::screenreaderInfo and bar/beat info and this more detailed string is given to the screen-reader.

3. Bug fix

– Tempo text. Here the status bar was showing private information about how the symbols are stored and now it shows and exact text represantation of that is in the score

– *Note pitch, Ambitus pitch. This is one example why I created a virtual method that returns by default Element::accessibleInfo. For notes the pitch was for example ‘C#3’, but the screen-reader was telling only ‘C3’. In this case I overriden the screenreaderInfo method so it creates the exact same string as Note::accessibleInfo, just instead of ‘C#3’ it returns ‘C sharp 3’ and instead of ‘Cb3’ it says ‘C flat 3’.

– Concert pitch. The Note::accessibleInfo and Ambitus::accessibleInfo didn’t take into account the concert pitch

– There is one curios bug that for which I did a temporary fix. If MuseScore starts maximized, the screen-reader doesn’t receive, or ignores all QAccessible::updateAccessibility calls until the main windows is minimez and restored. For this I made so that if any screen-reader is detected, MuseScore is not started maximized.

4. Extra info

This info is returned by Element::extraInfo as I said before. For this part neither me, or Marc were very sure how things would be better, so Marc let me decide what should go where so things work nicely with next element and we can change them later acording to the user feedback.

Next week, GSoC ends so this week will be all about fixing bugs, integration, resolving conflits, PRs and making sure everything works smoothly. 🙂

See you next week! 

weeks[10].toString(); //Screen reader works!

Hello everyone! As the title says, I finally constructed the accessibility info for all the elements and connected the screen reader. 🙂

1. Changes in design

There were two main changes in the design that I presented last week. First of all, I removed the ElementAccessibilityInfo class and implemented a virtual method in Element, named accessibleInfo that returns the necessary information as a QString. Also, I removed the signal from Element class an replaced it with a call to ScoreAccessibility::currentInfoChanged in the MuseScore::endCmd function. Then endCmd methos is called at the end of every action performed in the score so after every action the user knows what has changed.

2. Screen Reader

2.1 What I’ve done

– Like I’ve said, I’ve implemented the accessibleInfo method for all classes that represent score elements

– I’ve implemented the QAccessibleInterface for ScoreView.

2.2 Problems

There are a few glitches. For some reason that I don’t know yet, the screen-reader skips some elements for specific commands.

Some examples with the rate of success in feedback:

– next-chord (Right arrow, LeftArrow) ~50%

– altUp/altDown  (Alt+Up, Alt+Down) ~90%

– pitchUp/pitchDown (Up arrow, DownArrow) ~100%

2.3 What else needs to be done

– For now, the mechanism works for only one score view, in the sense that if you have two scoreviews open and hover with the mouse from one to the other, you will not receive info about one of them.

I’m very excited to have finally made this work, this was a very important part of my project. This, along with the next element command that I also implemented, will allow blind users to read a score. 🙂

weeks[9].toString(); // More Accessibility Info

Ok, let’s see what I did this week. 🙂

Beside of what I presented in the post from yesterday I also made the status bar work properly for the next type of elements:

– Time signature

– Markers

– Jumps

– Measures

– Accidentals

– Articulations

– Dynamics

– Rest

– Repeat Measures

I also implemented a system that tells the ScoreAccessibility class that the the current information has changed. For those of you who don’t want to read the long version in the previous post, here is the short one. 🙂

Bassically is an Observer pattern implemented with signals and slots. So the ScoreAccessibilityInfo has a slot that updates the status bar and the ElementAccessibilityInfo and Element each have a signal for telling the the information has changed. The signal from Element is connected to the signal from ElementAccessibilityInfo which in the end is connected to the slot from ScoreAccessibilityInfo for the object the ocupies the status bar. The reason why I like very much the signal/slot system in combination with the Observer pattern is that you don’t have to check if there is something else connected to the other end. You just emit the signal and if there is a ElementAccessibilityInfo instantiated, it will propagate the signal and if it’s also connected to the slot, it updates the status bar, but like I said there is no checking required. I think that this is a very nice way of applying the “Tell, don’t ask” principle.

Also, I got in touch with one of the main developers for Accessibility from Qt, Frederik Gladhorn, and he was able to help me with most of the problems that I had. Now the code works in my testing project so I can start moving it in MuseScore. I wasn’t yet able to give “fake” focus to a QObject so that the screen-reader can view it, but once I do that I will unpload the project so that anyone can use it. There are two things that I would like to mention here, that are not specifically written in the Qt documentation:

– Most screen-readers ignore any update that is not about the widget that has focus

– Child(0) is not the object itself, it’s the first(if it exists) child widget

This week I’ll probably have working the screen-reader and the basic information collected about all the Element objects so I can start collecting extra info. 

As I discussed with Marc, the status bar will only have the basic information, but the screen-reader will actually tell more, like for example for a note: what annotations, articulations it has, if it has explicit accidentals, etc. We will provide all this info, but the user doesn’t need to listen to it every time, since moving to the next element makes the screen-reader tell the info of that one.

Thank you for reading!

See you next week! 🙂

Design of the score accessibility system in MuseScore

In this post I’m going to present in detail my idea for the system that is going to provide feedback for the score. Almost all of the following design is already implemented in the score_accessibility branch of my repository.

For this I’ve created three classes:

1. ScoreAccessibility

2. ElementAccessibilityInfo

3. AccessibleScoreView (not implemented yet)

A. Class description

1. ScoreAcessibility

Since there is only one status bar that needs to be updated and only one screen-reader at a time it makes sense that this class will have at most one instance. I chose to use the Singleton pattern instead of static functions because it makes more sense to me to have an object that represents a curent state than static members. Other advantages of using this pattern are the fact that the object can be accessed from any part of the code by calling ScoreAccessibility::instance and that object can be used as a parameter in functions later on if that is the case.

This class agregates the ElemenetAccessibilityInfo class and the AccessibleScoreView class.

This class exposes two methods and one slot:

– updateAccessibility(ElementAccessibilityInfo* e);

this method updates the status bar using the informations received from parameter “e” and when the the AccessibleScoreView will be implemented it will notify the screen-reader that the accessibility informations have changed.

– clearAccessibilityInfo()

for the moment this method only cleans the status bar.

– slot currentInfoChanged()

this slot updates the information of the current ElementAccessibilityInfo (the one that currently ocupies the status bar)

2. ElementAccessibilityInfo

Here there were two options:

i ) add a virtual method to Element class that will return a string with the accessible information

ii) create a new class that takes care of constructing the accessibility information for an Element

I chose the second one for the following reasons (advantages):

R1) (Single responsability principle) I didn’t want to overcrowd the Element hierarchy with methods that compute and update its accessibility information. By going with the second solution, I’m defering this responsability to the ElementAccessbilityInfo class and the Element class gets (idealy) only:

    – one member accInfo (a pointer to it’s ElementAccessibilityInfo object)

    – a virtual method ElementAccessibilityInfo* accessibilityInfo() that instantiates and returns the object

    – a signal that is emited when the internal information changes and the status bar might need to be updated

R2) Reduce the duplicated code. The initial idea was to not have duplicated code at all, the informations would be added using inheritance.

R3) The visual/conceptual way of looking at score elements doesn’t match always 1:1 to how the inheritance hierarchy is implemented for Elemenets. By creating a new class I’m able to create a new hierarchy.

(this example might debatable and I won’t inherit myself also NoteAccessibilityInfo from RestAccessibilityInfo, but it goes to show that, musically, things are a bit different from how they are programmed)

For example: a note can be viewed musically as a rest with a pitch – both have a duration, a measure, a beat, the rest has a reduced set of annotations that can be added to it, where as the note can have annotations, accidentals, articulations, grace notes, etc. If you look at things this way, the note can inherit from the rest class. But every note has a duration only musically, programatically, notes are grouped in chords and the chord has the duration. It makes sense to implement things the way they are now, but it doesn’t make sense to provide info about a note to the user without giving its duration.

R4) Saving memory (not implemented yet completly). It makes sense to me (at least now, maybe things will change as I implement further) that at every point, only one accessibility object/info needs to be instantiated, the one that curently occupies the status bar. At this point, the object that contains all the information is not created until the respective element is not selecte. ( but it is not destroyed once created, at the moment )

R5) At one point I might want to implement feedback for commands (and maybe other things) not just for elements. Then idealy I need a base class and I don’t see any way (conceptually speaking) to connect elements with commands, but I can see a base class AccessibilityInfo that is inherited by ElementAccessibilityInfo and CommandAccessibilityInfo

R6) Self mentaining code – As long as the getters in the interfaces of Element hierarchy classes remain the same, there is no reason to change any code in the ElemenetAccessibilityInfo hierarchy, even if the internal structure changes.


D1) R3) unfortunatelly goes both ways, so I can’t always do things as efficient as I want and without duplicating code, or finding a compromise solution.

D2) I might not always have getters for the information that I need

The ElementAccessibilityInfo class has only two public functions and a signal:

QList<QString> info() that returns a list with all the informations as QStrings

updateInfo() which updates the strings in the list returned by info

– (signal) infoChanged() this signal annouces that the strings have changed and you might want to call info() again.

ElementAccessibilityInfo can be see from the point of view of design patterns as a Template, because info calls updateInfo, which is virtual, since every derived class needs to update in its own way.


This class will inherit QAccessibleWidget (implementing in this way QAccessibleInterface) so that the screen-reader will see the informations that I will provide. By talking to Frederik Gladhorn, one of the main developers for accessibility from Qt, I understood that the idea with the status bar was not enough, because I can’t make the screen-reader tell those informations without giving focus to the status bar. Most screen-readers ignore any changes that happen to widgets that don’t have focus. Since the ScoreView is the widget that has focus when selecting different score elements, I chose to implement this class.

B. Workflow

1) the client (in my case the selectSingle function) calls the Element::accessibilityInfo method

2) the Element::accessibilityInfo method checks if Element has an instance of ElementAccessibilityInfo, or subclass, if not it creates one and it returns it.

3) The ElementAccessibilityInfo class constructs the necesary info in the constructor and connects its infoChanged signal with the parent’s accessibilityInfoChanged signal

4) the client gets the instance of ScoreAccessibility, by calling ScoreAccessibility::instance()

5) the client calls using instance from 4) the ScoreAccessibility::updateAccessibility method using the ElementAccessibilityInfo object that he obtained at 2) as parameter

6) ScoreAccessibility::updateAccessibility looks if the current given parameter is the object that it already had, if not, it dissconects its infoChanged slot from the signal of the former object, it connects the signal to the the new object, and it replaces the former object with the parameter. After that it calls ElementAccesibilityInfo::info and updates the status bar and the screen-reader.

7) Everytime the ElementAccessibilityInfo::infoChanged is emited, the ScoreAccessibility class calls updateAccessbility giving as a parameter the object that it already has.

Other advantage:

– Loose-coupling – as can be seen in the workflow the ScoreAccessibility class has no idea about the Element or ElementAccessibility hierarchy, or about what are those informations that it receives. This is a big plus if as I said earlier I want to implement feedback for commands. The only change that I would need to do is modify everywhere is ElementAccessibilityInfo to AccessibilityInfo (2 places 🙂 )

At first it seems like if I would’ve implemented everything directly in the Element I would’ve not needed the signals, but I would still need a system to tell the ScoreAccessibility that the information has changed and it needs to update itself. At most, the accessibilityInfoChanged signal would be directly connected to the currentInfoChanged slot. This way there is another signal involved, but it is connected and disconected in the constructor, respectively in the destructor of the ElementAccessibilityInfo, so it was taken care of and I see no reason in the future for someone to manually connect these 2 signals in any other way.

One of the reasons I wrote this post is so Marc Sabatella can see my full thinking process behind the code, otherwise it would have probably been included in the tommorow’s weekly update. I’m going to discuss these ideas with him and I will let you know if anything changes. Mostly we are going to talk about the disadvantages that I haven’t seen, because naturaly if I had seen them, I wouldn’t have chosen these solutions. 😀 Of course, if anybody else has a comment, or advice I’m happy to hear them all. 🙂