Beyond the mouse – how people actually interact with your website or app

14 November 2022

It’s easy to think everyone interacts with their devices in the same way we do. But there are many different ways.

And there are some simple, practical steps we can take to ensure everyone gets a great experience when using our websites and apps.

Everyone has access needs

A reminder: 100% of people have access needs some or all of the time.

Of those people, in the UK, more than 20% are disabled.

The other 80% have a range of permanent or temporary access needs resulting from things like low vision, degenerative illness and learning difficulties to injured limbs, migraine headaches, or simply needing to use a device in bright sunlight.

How easy is it to use a mouse?

It is very easy? Do you normally use a mouse? Are you used to using it and find it intuitive?

Well, those of us who are skilled mouse users can easily take it for granted.

But, using a ‘normal’ mouse requires:

At least one usable hand
Fine motor control
Hand-eye coordination
Ability to see both cursor and target on a screen (this is perhaps the most important of all!)
Ability to perceive movement
Smooth, flat surface
Charged batteries (has your wireless mouse ever fun out of power? Were you able to continue working efficiently?).

Here are a few common reasons some people can’t easily use a mouse:

Repetitive strain injury (this is a very common reason why people with good vision and normally good motor control might need keyboard access)
Low vision (the cursor and target is a critical component of mouse interaction. If you can’t see either, a mouse isn’t much use to you)
Hand tremors (can make fine control difficult or impossible)
Missing or limited use of limbs
Paralysis
Poor hand-eye-coordination
Learning difficulties
Uneven surface
Empty batteries
Fluff in the rollerball (if you used a mouse in the 90’s you’ll remember this being a problem, but some people still prefer or need the tactile feedback of a rollerball mouse).

Different pointing devices

The mouse, however is just one type of pointing device.

For those users who cannot use a ‘normal’ mouse because of difficulties with their movement, there are a range of alternatives.

And you don’t have to have a motor impairment to use any of these. Maybe you already do use one, or will be inspired to try.

Ergonomic mouse (usually a more upright shape than a normal mouse, so the hand position and posture is more natural)
Trackball/rollerball (where the ball is on top and is rolled directly by the user; the device itself doesn’t move across the desk)
Touchpad/trackpad (many of us will have used one on devices such as laptops; it requires very fine motor control of an individual finger, but for some people that might be all they can effectively control)
Pen mouse (held just like a pen)
Graphic Tablet (higher fidelity than a pen mouse, but the same type of movements)
Joystick (why not use a joystick to control the cursor? You can even get retro-Atari-style USB joysticks)
Foot mouse (usually a pair of pads on the floor – one a slide pad for moving your foot across to control the cursor, the other for button presses)
Keyboard cursor control (use the arrow keys to move the cursor; you can probably do this on your own device)
Head movement or gesture tracking (uses cameras to track body movements to control the device; often users will have a range of gestures that go beyond just controlling cursor position)
Eye tracking (like gesture tracking, but often more precise and suitable for people with very limited movement)
Touch screen (many of us have used one, most commonly on our smartphones; but have you tried it with a screen reader?).

Like the mouse, these are all pointing devices. They all have different levels of fidelity and some users will benefit from larger target areas, zoomable responsive layouts (that don’t break or require excessive scrolling), and more time.

Different input methods

Pointing devices aren’t the only input method people use to interact with your software:

Keyboard (when it works, even for people who can use a mouse, the keyboard can be faster)
Ergonomic keyboard (different key layout, usually to help with repetitive strain injuries)
Keyboard shortcuts/commands (so keyboard users can perform common tasks like entering edit mode or saving changes even faster)
Voice (you don’t even have to touch the device! And for those who can’t touch their device it can be life changing technology. It’s another very impressive way of interacting that you can try today on your own smartphone)
Braille keyboard (something as low-fi as adding braille to the keys on a standard keyboard can be the difference between easy and difficult – or impossible – access for someone)
Screen reader touch-to-find (on touch devices, with a screen reader enabled it’s possible to explore the screen just by moving a finger around and listening to what’s there. The importance of good HTML markup can’t be underestimated here)
Switch control – button (switch control works by the device highlighting parts of the screen one-part-at-a-time and the user clicking a button when what they need is highlighted. In this case, a physical button – or multiple buttons for different commands – is used)
Switch control – tap on screen (works in the same way as a physical button; you can set this up on your smartphone, and I recommend you try it)
Switch control – sip and puff (literally like a straw, usually used by people with very limited movement).

Professor Stephen Hawking used an infra-red sensor mounted on his glasses to detect voluntary twitches of his cheek muscle. With this he was able to write at a rate of approximately one word per minute. Which seems slow to some of us. But even so, it allowed him to do some amazing work. Many of our users can benefit in similar ways.

Ensuring interactions like these are at least possible is something we can do that will make a real difference to the quality of our products, and the quality of our users’ experiences.

Different ways of perceiving the content

We’ve seen there are many ways people interact with our software.

But what about the different ways there are of perceiving the content?

Large screens (such as desktop computer screen. Maybe you have a good quality screen with excellent colour and luminosity; maybe you have multiple monitors; maybe you’re reading this on a standard definition TV from across the room)
Small screens (smartphone screens are small. So is the really old monitor your semi-retired accountant neighbour is still using to ‘log on’ to the Internet)
No screen (not everyone has one, because they couldn’t see it anyway; often a blind user who uses a screen reader will have the screen itself turned black for their own privacy)
Low resolution screen (because not everyone can afford, wants or needs an upgrade)
Zoomed screen (with reduced vision often comes the need to zoom. Responsive layouts done properly are very valuable here)
Customised font, font size, line spacing, character spacing (they may make your app look very different to how it was intended, but if it makes it easier – or even possible – for some people to use it, let’s make sure it’s supported!)
Customised colour scheme (we can design our use of colour in a way that includes as many people as possible; but some people need low contrast or different colours all together, so make sure users can still apply their own preferred colour scheme if they need it)
Dark mode (it’s not just for developers who want to be cool! For many users – but not everyone – it’s a lot easier on the eye. Does your app support it?)
Screen reader/text-to-speech (where your device can read out what’s on screen; it’s used by people who are blind, have reduced vision, have learning difficulties or have reading or comprehension difficulties such as dyslexia)
Refreshable braille display (they work just like a screen reader, but instead of audible output, they ‘write’ lines of braille)
Captions on videos – because not everyone can hear
Audio description on videos – because not everyone can see
Sign language – because not everyone can hear, and often people who are Deaf use sign language as their primary language; meaning, for them signing is perhaps easier to understand than written captions.

Limited time, simple tactics

But, I hear you say, we have limited time to develop and test our software! It’s too much to deal with!

And you would be correct.

Because, when creating and maintaining software, there are limits to both time and money that we have to take into account.

We can, of course, keep it simple. So here are three basic tactics to help you and your team improve the way your work:

1. Follow basic coding standards

It’s really easy to forget about the basics when we’re focused on more complex things.

The single most important thing you can do as developers and quality engineers is make sure you always use standards-compliant HTML

For accessibility, the single most important thing you can do as developers and quality engineers is make sure you always use standards-compliant HTML.

Yes, you can use just about any HTML tag you want and get something to work, because browsers are forgiving. But users aren’t so forgiving.

There is a correct HTML tag to use for almost every UI element you can think of.

HTML easy to learn, easy to write, easy to test for and makes an enormous difference for a significant number of your users.

Adding WAI-ARIA to your HTML makes things even better. Especially for those people using screen readers or who perceive your app in other different ways.

It can even help when it comes to improving old code when you need to do it with minimal change risk.

2) Test with a keyboard

Always test with a keyboard. It’ll quickly and easily reveal common issues you’ll miss if you only ever test your work with a mouse:

Keyboard traps
Unpredictable tab order
Un-focusable elements
Missing focus styles
Focus not moving to the current context (like when you open a modal dialogue)
Un-openable or unclickable elements
Custom components that can’t be used
A surprising change of context when something gets focus.

A lot of these issues result from poor use of HTML or misuse of attributes such as tab-index.

Almost everything is easily fixed.

3) Use automated tests to help maintain quality

Some the common accessibility issues can be found using browser testing plugins such as WAVE or Lighthouse:

Standards-compliant HTML
Duplicate IDs
The presence of expected WAI-ARIA attributes
Colour contrasts
Headings order.

You can set up automated tests to check for these things as part of your normal test routines.

You should also creating regression tests to make sure certain tags, or attributes such as WAI-ARIA, aren’t accidentally removed or broken by future changes.

You can’t automate everything, but you should automate what you can.

If you want to really make sure what you build is usable, try testing with a screen reader.

User testing with people who actually use these devices and technologies is something you should also be doing as part of the overall process, but you can’t realistically do a user test for every change during development.

So get to know the screen reader on your own devices – there’ll be one installed and others are available to download.

Get good at moving around the screen and jumping to different landmarks, headings or links.

You’ll quickly learn how to easily spot things like:

Broken keyboard support
Missing landmarks
Missing or badly-ordered headings
Form inputs without proper labels
Poor link text.

While some of these things can be automated, actually using the assistive technology at your disposal will help you create features with much improved user experiences.

That’s it

Just a couple or reminders to finish:

100% of people have access needs some or all of the time.

A mouse is just one of the many ways people access your software.

So, to make sure everyone is able to use your app or website, write good HTML, test with a keyboard, automate what you can, and try using some assistive technology yourself.