Switch access and scanning: control any app using Mouse4all

2019 Jul 31

Imagine having to control everything you want using only a light switch. That is the situation of many persons with severe physical disabilities that cannot type on a keyboard, move a mouse or speak to a voice assistant like Alexa. They can just press one switch or button with their foot, their hand, their chin or their head.

In this article, we are going to talk about how persons with disabilities can use Mouse4all together with a switch to access any app on their smartphone or tablet. This article is based on the excellent guide “Switch access to technology” written by David Colven and Simon Judge of the ACE Centre (you need to register, but it is free to download).

How Do We Multiply the Power of the Switch

Usually, you can only control one thing with a switch, that is, one light. However, using a technique known as “scanning” you can control many more things. If a list of options is presented to you sequentially, you can press the switch when the choice you want is presented to you.

This way you could control multiple lights. Using the analogy in the guide, it is a bit like a sushi bar. You cannot reach all the dishes on the bar. You have to to wait until the one you want is in front of you.

Switching on and off lights is nice, but using a mobile app to send a WhatsApp message, play a game or watch a video is something much more complex.

How can we do it with a switch? Instead of scanning objects or lights, we scan actions, called “items”, that need to be performed on the smartphone: move the mouse pointer up, drag an object, turn off the screen, take me to the home screen, etc.

If we can provide a rich set of actions, the user can perform anything on his device. However, if the list (i.e. the sushi bar) is too long, it will take a long time until she can select her chosen item. That is why actions or options are organised in selection sets, grids, pages or menus (“menu” is the word that we use at Mouse4all).

The user can select among a small number of selection sets and then once inside that set or page, she can choose the detailed action she wants to perform. For example, the user can select the “Move” page and then once inside, select exactly the direction of the movement (“Move up”, “Move down”, “Move left” or “Move right”).

The options of a grid can be presented ordered in a single line or row (that is the case of Mouse4all) or on multiple lines. The benefit of a single row is that the grid does not cover as much space of the display, already quite small in a smartphone screen.

Moving between selection sets is like moving on a web page, you select a link “item” and then the content of the grid changes to the chosen page. There is also an option to move back to a previous page or to the home page.

To let the user know which item is the currently active item of the grid (which dish is in front of her at the sushi bar), the item is highlighted to make is stand out from the rest. In the case of Mouse4all, the current item is marked with a thick border in a different colour.

The visual representation of the grid and the items is a very important aspect of any scanning solution. It helps the user to learn and move faster. It must be configurable (size, colours, images) to the needs and preferences of each user. Mouse4all includes a simple but large set of configuration options for this.

Functions of the Scanning Process

In the next paragraphs, we focus on the technical details of the scanning process. We follow the order and terminology used by ACE Centre’s guide.

Scanning sequencing methods

How does the highlight in a grid move from one item to the next one?

• Simple scan: The highlighted item moves sequentially from one item to the next. The order of items is predefined. This is the implementation of Mouse4all.
• Group scan: Instead of highlighting a single item, a group of items is highlighted (typically a row or a column of items). When a group is selected, simple scanning item by item is performed among the items of the group.

Highlighter movement control

What event triggers the “jump” from one highlighted item to the next?

• Autoscan: The highlight moves automatically and cyclically every x seconds. When the highlight reaches the desired item, the user presses the switch to select it. Mouse4all works this way when configured for one single switch.
• Step scan: This requires typically two switches. When the user presses one switch, the highlight moves to the next item. She needs to keep on pressing this switch until the highlight reaches the desired item. At this point, she can press the second switch to select the item. Mouse4all works this way when configured for two switches.

There are other more complex scan mechanisms that you can review in ACE Centre’s guide.

Manipulating the highlighter movement

How does the highlight behave after selecting an item:

• Starting: The highlight moves back to the first item of the grid.
• Restarting: The highlight starts moving from the last selected item.

Mouse4all supports these two options. Additionally, Mouse4all defines one setting (“Slower initial speed”) to highlight the first item for a longer period than the rest of the items. This feature lets the user some additional time to get used to a new grid that has just been displayed.

Item selection

How does the user select the desired item once it is highlighted?

• Switch actions: the item is selected when the user “presses” or “releases” the switch. For some continuous actions (move, scroll) the action associated to the item can keep executing from the moment the switch is pressed until it is released (e.g. the web page keeps scrolling down while the user keeps the switch pressed). Mouse4all allows configuring any of these options based on the needs of the user.
• Dwell selection: the item is selected when the user does nothing for a preconfigured number of seconds. If this timeout expires without any switch being pressed, the system interprets that the user wants to select the currently highlighted item. This mechanism is typically used together with other input mechanisms (e.g. ball mouse to control mouse buttons).

Timing and input

• Scan Time: The time it takes to the highlight to move from one item to the next. Typically when a user starts using a scanning solution like Mouse4all, she requires longer scan times. As she gets used to scanning, scan time can be shortened. Mouse4all implements a setting for this (“Scan speed of the virtual menu”).
• Initial scan time: When a new grid or table is presented to the user, it may be convenient to let an extra delay to the user before the highlight moves to the first item. Mouse4all implements a setting for this (“Acceptance delay for menu”).
• Dwell time: Used only for dwell selection.

Repeat functions

Some items (in fact the actions associated to those items) can be repeated without having to reselect the item. Note that this is only suitable for specific actions (e.g. move, scroll)

• Repeat delay: How long does the switch state need to be held before a command is repeated.
• Repeat time: How often is the action repeated once repetition has been triggered. Mouse4all provides settings to control the repetition of scroll and swipe actions.
• Auto-repeat: Once an action is selected, it will keep repeating until the switch is pressed again. Mouse4all supports this feature for move, drag, scroll and swipe actions. For example, if the user selects “Move left” action, the mouse pointer will start moving left. The move will go on until the user presses a switch again. If the pointer reaches the border of the screen, it can either bounce back and start moving right or start moving left form the opposite border of the screen.

Input filtering

This functionality removes unwanted switch actions. These actions can be triggered by involuntary movements of the user or by the mechanical and electrical properties of the switch.

There are multiple properties that Mouse4all can filter in the connection box (debounce time and acceptance time) and in the Mouse4all app (post-acceptance time).

Read the ACECente guide for a detailed description of these features.

Conclusion

Switches are a very powerful tool that enables persons with physical disabilities to use computers, smartphones and tablets. With the right scanning solution, a switch can control any application.

We have used ACE Centre’s “Switch access to technology” guide to explain how Mouse4all works on Android devices and how it can be personalised to the needs of every single user.

Thanks to Mouse4all, persons with cerebral palsy, SCI (Spinal Cord Injury) or ALS (Amyotrophic Lateral Sclerosis) are using WhatsApp, Facebook, YouTube, Google Home and many other apps.

We encourage you to try out our app Mouse4all Switch and experience it by yourself.