weeks[8].toString(); // Screen Reader and Accessibility Info

Hello everyone! 🙂

This week I continued my reasearch on how Qt intereacts with the screen-reader and the implementation of classes that update the status bar with the necesary informations.

1. Screen-reader

Unfortunately, here I’ve hit a dead end. I’ve succesfully implemented the QAccessibleInterface in order to provide custom accessibility info, by subclassing the QAccessibleWidget class and the QAccessibleInterface class, but I couldn’t make the screen-reader to update the informations using the QAccessible::updateAccessibility method.  I send an email on the Qt Accessibility mailing list, and asked on two forums, but no luck… In the end I’ve sent a bug report to Qt about this. For the moment I’m goind to put this aside until I here back from Qt Accessibility developers and I’m focusing on gathering all the necesary informations about the score elements so when I find out what needs to be done, everything will be in place. You can find the forum posts here[0][1] and the bug report here[2]. Also in the bug report I’ve sent my testing project so you can view there how to implement the QAccessibleInterface and if by any chance you manage to update the screen-reader feedback, please let me know. 🙂 I’ve set some debugging messages so you can see what informations Qt is querying, but it seems that either it doesn’t send them to the screen-reader, or the screen-reader ignores them.

 

2. Accessibility info

2.1 What I’ve done

I’ve implemented the apropriate subclasses of the ElementAccessibilityInfo that I’ve talked about for the next elements and they provide the respective info (for now):

– Note:

Note type, Pitch, Duration, Bar, Beat

– Cleff:

Cleff type, Bar

– Key signature (for classic key signatures)

Key signature, Key Signature type, Bar

– Barline

Barline, Barline type

Also, the information updates for consecutive selections.

2.2 Problems

No problems so far. I didn’t even had to break the encapsulation, by making the ElementAccessibilityInfo a friend class of Element as I was worried.

2.3 What else needs to be done

– Write the implementation for other element types. For now, the default behavior is to write in the status bar the element type.

– Update the status info when changing the element and the selection remains on it. For this I think I will use signals.

– Save memory space. Another advantage that I didn’t see right away is that I can keep only one ElementAccessibilityInfo object “alive”. The one that is currently in the status bar, the other ones can be deleted and reinstantiated when it is needed.

I’m very dissapointed that I’ve spent almost the whole week trying to make the screen-reader work with no luck, but hopefully the developers will be able to help me. Also, I think that by end of next week I will finish gathering the info about all the element types (at least the basic info).

Thank you! See you next week! 🙂

Advertisements

weeks[7].toString(); // Screen Reader feedback for score

First of all I want to appologise for being late with this weekly blog post. I just got back from camping last night and I didn’t have my laptop with me.

This week did a lot of reasearch on how to provide accessibility using the  screen reader for the score and I’ve started implementing the design for the it.

1. Reasearch

1.1 What I’ve done

– I’ve read the documentantion for the following classes: QAccessible, QAccessibleObject, QAccessibleInteface

– I’ve read some of the code from the Qt project regarding accessibility, especially for the QAccessible::updateAccessibility function

– I’ve created small programs in order to figure out what can be done and how the API works

1.2 Problems

– Unfortunatelly there is one major flaw in the idea with the status bar. The initial idea was to populate the status bar with a QLabel (or more QLabels) and make the screen reader tell the informations that are presented there. From what I can tell both by running tests and reading Qt’s code the updateAccessibility function only works when the widget for which is called has focus. Also, it is called internally by Qt when the QWidget::setFocus method is called. This means that in order for the screen reader to give the user all the informations, the status bar needs to keep the focus in that period of time. This is not an option, because the user could not would have to wait until all the information is provided and the score gets the focus again.

In order to solve this problem, I’m thinking about using the scoreview, or scoretab to place the information in their accessibleName/accessibleDescription. I need to do more tests in order to be sure.

– One thing that I don’t yet understand is how the QAccessibleObject works since a QObject that is not a QWidget cannot gain focus.

2. Design

2.1 What I’ve done

– I’ve created the ScoreAccessibility class that will manage and update all the informations using the method updateAccessibilityInfo. In the implementation of this class I used the Singleton pattern. In this way there is only one global instance for the aplication and I can access it from any object using the static method ScoreAccessibility::instance.

– As I said the ScoreAccessibility class will only manage the informations. These informations will be provided using another class, ElementAccessibilityInfo. The ElementAccessibilityInfo class will be inherited in order to get the appropriate informations for all the element types. ElementAccessibilityInfo keeps a private list with QStrings(or QLabels, or another custom class, I haven’t decided yet) and it will be populated by subclasses with the apropriate informations and this list is what the ScoreAccessibilityInfo will use.

I could’ve just added the QList to the Element class, but I think that these solution is better from a design point of view, by deferring responsability through agregation and by keeping the entropy.

2.2 Problems

– I might need to break the encapsulation by making the ElementAccessibilityInfo a friend class of Element if at some point I need to access protected, or private members

 

I’ve made this design work for the Note element and for now it works well, but I have to implement more in order to see if it holds for every element type, so things might change along the road. Also there is still that major problem of finding a place to put the informations so that the screen-reader can access them.

Because I was away for a few days I will work more this week in order to recover the lost time. 🙂

That’s all for this week!

Thank you!

weeks[6].toString(); // Navigation and Screen Reader feedback

Ok, lets see what I’ve done this week. 🙂

1. Navigation
First of all I finished every aspect of the next/prev element command, so the only thing that remains to be done for navigation is the next annotation/articulation command, but as discussed in the previous post I will implement this after the screen reader feedback.

1.1 What I’ve done

– Fixed all the problems with the barlines. I know I said last week that I only have to test the navigation for barlines, but when I did, I discovered that it didn’t cover all the use cases, so I had to find a more general implementation. Now everything works as expected

– Refactored the functions. There was a lot of duplicated code in two huge functions, so I moved some of it in the segment object, creating three smaller functions and now the code is much more easier to read and understand.

– Made the canvas adjust its position during the navigation

– Fixed the coding style

1.2 What else needs to be done

– As I said, the only thing that needs to be implemented is the next annotation/articulation command.

 

2. Screen reader feedback

As I mentioned in the proposal a very important part of my project is making the screen-reader tell the score to visual impaired musicians. After talking to Marc I think I found a good solution in order to do this. First of all I will implement an accessibility interface for the status bar using the accessibility API provided by Qt like this example [0] and after that I will make the select command populate the status bar with all the informations that the user needs.

For example:

“Select note 1 of 3 Pitch: C4 Duration: 8th note Bar: 4 Beat: 1”

I’ve started doing the reasearch for how to implement this, I’m playing with the above example and reading the documentations provided by Qt for the accessibility API for QAccessibleInterface, QAccessibleObject and QAccessibleWidget, but I don’t have yet anything functional. Hopefully, if everything works as planned I should have the status bar working by the end of the week.

3.  Bug fix

– I also fixed this bug: http://musescore.org/en/node/24243

-See Pull Request here:  https://github.com/musescore/MuseScore/pull/1027

See you next week! 🙂

[0] http://www.ranorex.com/blog/enabling-automation-for-custom-qt-widgets-by-adding-accessibility

weeks[5].toString(); // Navigation

First of all, the good news: I’ve passed the midterm evaluation, so the project goes on! 🙂

I want to thank Marc Sabatella for all his help and patience so far! He is a great mentor and I’m very happy to work with him! 🙂

This week I’ve continued working at the navigation commands.

1. Navigation

1.1 What I’ve done

– Finished the home command

– Finished the end command

– Finished the next element command

– Finished the previous element command

1.2 Problems

– There were some problems with the barlines, because they weren’t traversed just for the first staff, since the assigned voice was voice 0. To fix this problem I’ve created a variable that keeps the track of the previous element selected. In this way I can “look behind” and to find on what staff I am. I just finished writing the code for this, so I still need to do a bit of testing.

– I made a wrong assumption that in a ChordRest segment, on a staff can be either chords, or a rest. I fixed this problem, but I didn’t like the fact that I was jumping from one voice to another with my functions, so I used Marc Sabatella’s functions upAlt and downAlt.

– I found a solution to the segment problem that I’ve stated in the previous post. To find the segment for any element, for spanners I’m using the starting/ending segment – depending on the direction of navigation – and for other elements I’m going from parent to parent until I find the element that inherits the Segment class.

1.3 What else needs to be done

– I still need to implement a way to traverses articulations and annotations, but I think I will do this after I implement the screen-reader feedback. There are to ways to do this: either implement a diferent command, or add them to this command. For now, if an element from those to groups is selected the navigation command finds the first non-attached element and selects it. For note-attached elements, it goes to the note, for segment attached elements, it goes to the first element from that segment, etc. Like Marc said, it would be best to implement both variants and pick one after that, but I’m not yet sure how to do them, so as I said, I’m moving to the screen-reader feedback for now. This will also be a factor in deciding how to implement the traversing.

 

That’s all for this week! See you next Sunday!

weeks[4].toString(); // More Dialogs and Navigation

It’s time for a new weekly blogpost! 🙂

This week I’ve continued working on dialogs and I started implementing new navigation commands.

1. Dialogs

1.1 What I’ve done

As usual I’ve made sure that the focus policies and the tabbing order are correct and also that the appropriate screen-reader is provided for the next dialogs/wizards:

– Inspector (all UIs)

– Instrurment Wizard

– New Wizard

– Time Signature wizard

– Create new score dialog

1.2 Problems

–  I fixed two minor bugs in the shorcut capture dialog: Clear button was a QToolButton instead of a QPushButton, so when pressing Enter, the Add button was pressed, also pressing Clear button did not disable the Add and Replace buttons.

– I’ve found a Qt bug regarding the QTreeWidget object. When pressing Enter the QTreeWidget object handles the key press event, but doesn’t stop it, so the event goes to the next object. This is why, if you select an instrument in the Create new score dialog and press Enter, the instrument is added, but also the Next button is pressed. The same thing happens in the Prefferences dialog with shotcuts, if you press Enter the shorcut capture dialog is opened, but when it’s closed, the Ok button is pressed as well and the Prefferences dialog closes too.

– I also have some problems with the screen-reader feedback for the added instruments

1.3 What else needs to be done

– Fix the last two problems mentioned

– Since the key signature is added using the palettes, that part will work once I start working at them

– Other dialogs

– Now that the focus policy for every individual element of the Inspector has been set, I’m waiting for the branch with the action restructuring to be merged into the main repository, so that I can rebase this branch on top of those changes and add the Inspector to the tabbing order

 

2. Navigation

The goal here is to create some commands that will enable blind users to traverse all the score elements. This week I familiarized with how the score is stored.

2.1 What I’ve done

I’ve almost finished writing a command that will enable blind users to “read” the score. This command traverses staff by staff all the main (non attached) elements: clefs, key signatures, notes (from top to buttom in a chord), rests, barlines, breaths. This command will work great in combination with the screen-reader feedback that I will implement later, since the screen-reader will have to tell not only the main elements, but also the attached elements. For example, if I have a C note, with a # accidental, I don’t want to have to traverse the C note, and after that the accidental, to find out the pitch. I want to traverse the note and the screen-reader should tell me C#. Of course, I will implement another command to traverse this element’s attached elements, since users don’t need just to know that they are there, they have to be able to select them and edit them, without using the mouse.

2.2 Problems

– The implemented command works well when the selected element is one of those enumerated above, the problem appears when the selected element is not of them. To solve this I need to find a way to get to the parent segment of every element, prefferably with as few ifs as possible.

2.3 What else needs to be done

– Now, the command, only traverses one staff and I need to implement something to make it go to the next staff as well

– Implement the command for traversing the attached elements

 

That’s all for this week! 🙂

 

 

weeks[3].toString(); //Actions, shortcuts and dialogs

I would like to start this post by saying that I’ve finished my finals. Starting from now, I’m going to dedicate my full time to this project. 🙂

This week I’ve worked on the following issues:

1. Actions and shorcuts

1.1 What I’ve done

I’ve implemented the solution that I’ve talked about in the previous post.

Here is the discussion from the mailing list[0]. You will find there more details then in my previous blog post.

Remainder:

There were three steps in this solution

I.  Move every QAction that affects the score in the ScoreTab object and change their shorcutcontext from WindowShortcut to WidgetWithChildrenShortcut, leaving in the MuseScore object (Main window) just those that open subwindows, dialogs,etc.
II.  Set the focus policy for all the other subwindows of the main window so that the they don’t receive focus by clicking on their elements (except for the case when text editing is necesary)
III. (optional) Restrict the user from assigning keys like Return, Tab, Arrow keys as shortcuts.

After talking to Marc, I’ve only created a restriction for the Tab key, because there was not way for a blind user to move around the shortcut capture dialog. Before, if the Tab key was pressed it was seen as a shorcut assignement and it was not ok, since it is crucial for blind users to be able to configure their preffered shorctus.

1.2 Problems

The only problem here was knowing which action should go where. There wasn’t an exact criteria for this.

You can find a google doc with the shorcut asiggnmets here[1]:

 

And with this solution I’ve created my first pull request for the accessible-toolbar. 🙂

 

2. Dialogs

2.1 What I’ve done

I’ve finished adding the screen-reader feedback for all the elements of the prefferences dialog and also for the shorcut capture dialog.

2.2 Problems

I didn’t have big problems, but there are some glitches.

– One of the things is that the screen-reader doesn’t tell the new values of the combo boxes when they are changed

– The second one is a two way street. On one hand I don’t like that in dialogs like Preferences the screen-reader reads the QLabels, but on the other hand it is importand that it reads them in small dialog windows, like shortcut capture dialog, or warning/error dialogs

2.3 What else needs to be done

Here the next will be to start working on the Inspector.

[0] http://dev-list.musescore.org/Keyboard-usability-and-accessibility-tt7578844.html

[1] https://docs.google.com/spreadsheets/d/1RbOLweFvoXxSYrJ-BMFmUiL1_b0lgzJvlDjPbbRcCew/edit?usp=sharing

weeks[2].toString(); //Keyboard control (Actions and shortcuts)

It’s time for a new weekly blog post. 🙂

This week was a bit unusual, so I can’t structure this blog post in the way that I did before. I’ve mostly worked on tabbing through the main window. I’ve managed to remove from the tabbing order every subwindow of the main window that doesn’t have yet support for accessibility. These are: MuseScore Connect, Palette, Piano and Inspector. Everything looked good at this point, I was preparing to make my first Pull Request and just when I was thinking about it, I started talking to Marc Sabatella, telling him what I’ve did, what works and what are my concernes. At the end of this conversation, we both understood that the Return key problem was just a piece of the bigger picture. There is the same problem with the arrow keys and some other shortcuts used for editing and selecting text (like Ctrl+A, Shift+RightArrow, etc).

It was clear that I could not treat every shortcut that creates conflicts like a KeyPressEvent for two main reasons:
– first of all this would meant that a lot of shortcuts would have been hardcoded. Shortcuts are now configurable through the preferences dialog, using the shortcuts tab, or using directly the shortcuts.xml file
– secondly, there was no way of knowing what shortcuts would every user add/change. I’ve removed that Enter key from being a shortcut, but that doesn’t stop any user to add it again .

Remainder: The problem with the Return key was that the shortcut was global and other objects did not receive an event when it was pressed, because Qt was sending the event to the QAction object that had the key assigned to instead of seding it to the widget that had focus at that moment.
The bigger picture: All shortcuts are global and any key or key sequence that is assigned as shortcut will be sent to the QAction object, instead of the widget that has focus.

I needed to find a new solution, that was general enough to cover if not all, at least most of the usecases. This is the solution that I was able to come up with:
1. Move every QAction that affects the score in the ScoreTab object and change their shorcutcontext from WindowShortcut to WidgetWithChildrenShortcut, leaving in the MuseScore object (Main window) just those that open subwindows, dialogs,etc.
2. Set the focus policy for all the other subwindows of the main window so that the they don’t receive focus by clicking on their elements (except for the case when text editing is necesary)
3. (optional) Restrict the user from assigning keys like Return, Tab, Arrow keys as shortcuts.

Details:
For 1. I will create a global enum that will specify for what window/object is the QAction intended. In this way, if in the future other actions will be added that are supposed to go in other objects, the only thing that will be needed is to add a new value for that enum. Since every action is instantiated in the action.cpp file using a Shortcut object, I will put a variable of that enum type as field in the Shortcut object.

For 2. : Since all actions are in the main window at the moment and their shortcut context is WindowShortcut, the focus *appears* to be always in the scoretab. In order to preserve this behavior where this is desired and avoid the bad examples I’ve given above I need to set the focus policy for all the other objects so that they don’t give away the focus to other elements. It would be nice to set a property for scoretab so that it doesn’t give away the focus, but this is not possible. The only usecase I see when the focus should leave the scoretab is when editing text in another subwindow.

For 3: This step is debatable. With the default shortcuts assigned in the way they are now, everything will work as desired, but this doesn’t mean that a user cannot reassign keys like: Return, Tab, Arrow keys as shortcuts for other actions and create problems. This is a question if we consider that is MuseScore’s responsability to restrict the user from poorly assigning shortcuts, or it is the user’s responsability. In the first case, the above mentioned keys will be handled like KeyPressEvents in the scoretab object and their behavior for the scoretab will be hardcoded.

I’ve sent this proposal to the MuseScore mailing list hoping that it will be aproved by the community, since it is a big change. I also asked the following questions:

Q1: What do you think about step 3?
Q2: Do you know other usecases in which  the scoretab should give away the focus, except when editing text?

What I forgot to mention on the mailing list is that this change will only affect the way in which the actions are triggered. The chain of functions and verifications that an action travels through for something to actually happen will remain exactly the same. They will be passed as before to the cmd function from the main window using a signal.

The good part is that the time I’ve spent on what I worked until now was not wasted. I implemented enough of the solution so that I can test it and I’ve done this by applying few changes to the code that I’ve already wrote. For step 2 of the solution I’ve used the knowledge that I’ve gain by trying to remove the subwindows from the tabbing order.

Now, I have to make a list of the actions that will remain in the main window. 

As Marc said, things might get even more complicated when I get to the Palettes, but hopefully this solution will be good enough and it will not need (many 🙂 ) changes.

I know I promissed a howto post this week, but as it turns out, it is a good thing that I didn’t publish it, since it would have lacked many usefull informations.

Some good news: Thursday I will have my last exam, so after that I’m going to devote all my time to this project.

Thank you for taking the time to read this! 🙂