Skip to content

Charlie Freestone Blog Post - Some Best Practices for Accessible Automation Testing #317

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 6 commits into
base: gh-pages
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions _data/authors.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ active-authors:
- ceberhardt
- cferguson
- cfisher
- cfreestone
- chayes
- ckoris
- colive
Expand Down Expand Up @@ -1388,6 +1389,11 @@ authors:
author-summary: "I am a Graduate Software Developer at Scott Logic. "
twitter-handle: null
picture: profile-pic-square.jpg
cfreestone:
name: "Charlie Freestone"
author-summary: "I am a Tester at Scott Logic. "
twitter-handle: null
picture: profile-pic.jpg
jleftley:
name: "James Leftley"
author-summary: "I am a Lead Developer at Scott Logic."
Expand Down
58 changes: 58 additions & 0 deletions _posts/2025-05-14-some-best-practices-for-accessible-automation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
---
author: cfreestone
summary: While on my most recent project I had the unique experience of working closely with many testers and test minded individuals. This allowed me to learn some much-needed lessons about how to best implement automation testing with accessibility in mind, a sometimes-overlooked area of test automation.
categories:
- Testing
---
# Some Best Practices for Accessible Automation Testing
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove this one, as your title is provided for you:

image

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still need this correction.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry I have been busy with client work I shall make this adjustment.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Other way around, need a title in the head section, not in the body of the post:

author: cfreestone
title: Some Best Practices for Accessible Automation Testing
summary: ...

While on my most recent project I had the unique experience of working closely with many testers and test minded individuals. This allowed me to learn some much-needed lessons about how to best implement automation testing with accessibility in mind, a sometimes-overlooked area of test automation.

## Introduction
What I hope to share with you is how simple it can be to both think about and implement the approach to make sure your tests are accessible. This can be because you have strict reasons on the project to ensure compliance with certain metrics, because this can lead to improved readability of your code and its longevity, or because you want to read some wonderful thoughts "from certain testers perspective".

## What to expect()
Now it’s good to make note that the lessons learnt here are all from the work I have carried out while implementing Playwright tests. This will mean you will need some basic understanding of TypeScript and Playwright when I discuss some examples. So without further ado let's discuss the `expect()` function.

This neat little function is the bread-and-butter basics of automating in playwright, it's what we want to see when we place our test into a certain configuration. This doesn’t mean it has to do handstands or backflips, but just like any gymnastics routine we do wait on it with bated breath, hoping for results we will cheer for. This anticipation makes it easy for us to write something which makes us wait for that big finale in our tests, even if the tests took the literal definition of breaking a leg.

And therein lies the issue; during a test we usually assert for confirmation to ensure we are where we expect. That means we can `expect(response.status()).toBe(200)` which is a wonderful successful request that has found the requested resource of a page. Instead we get a `201 response`, which is just as equally a successful request but it led to the creation of a resource. Test results can’t always be perfect and that means that this stopping point during the test, before we check the larger critical parameters later on, will skip the test. Instead, we have to accept that not every response will be exciting and just `expect(response.status()).toBe(OK)` which allows us to still capture data from a test even if the status code is different.

## Don’t just getBy, Role with it
After correctly measuring our expectations, we want to roll into selecting all the necessary parameters we can get by with. Therefore, it comes as no surprise we use the `.getByRole function`, but in order for us to make sense of it then it's all about those pesky selectors.

This is because there is an interface that runs behind the scenes of the UI and Playwright can pick selectors which will not be seen by someone who needs to use the keyboard to navigate around the system. For these users, the elements they have access to are either tabbed to or navigated to via arrow keys. If using a screen reader these selectors need to have a consistent and readable naming convention. Otherwise, any software that reads it aloud would read out jargon which will only confuse the User.

Things like filters and drop-downs can be difficult to locate but they can be found. A good tip for finding this is by entering the webpage and open up DevTools, then clicking near the item you wish to find the selector of, but not on it. After that press tab and open up the console within DevTools. If you then type the command: `document.activeElement` you will find the info you need.

By the same accessible reasoning it is good practice to not use `.getbyTestID`.This is due to the fact that names of those same ID’s will not be useful to someone with an accessibility issue, as it will sound like Jargon when navigated to.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe you may have undercooked the importance of selecting by role (or label or title), not just for the tester but for the developer also. It's impossible to select by role if the developer hasn't used semantic html or has made everything a <div>.

Making a rule that the tester will select by role or label or title, or text where those are not possible, and never use the dreaded byTestID, means the developer is forced to implement good practices, and the tester can verify that those are adhered to by testing the page in a way that mirrors how a (screen reader) user will navigate and understand the page.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok what would you suggest this would need changes too. I'd say that this still talks about the point, what detail needs to be added here?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, you state this:

If using a screen reader these selectors need to have a consistent and readable naming convention.

But then you don't elaborate on that naming convention, nor on how using getByRole has anything to do with this naming convention.

Remember also that use of getByRole is entirely dependent on the developer using roles correctly when implementing the page, so it's all well and good to say "test by role" but you can only do that if the developers are on board with that.

Similarly, finding an element using devtools will not necessarily give you the information you need, if the developer has used test IDs on divs instead of roles or labels, or generally non-semantic html.

What I am suggesting is that you are missing the important interplay between developer and test engineer. Have a think about the conversation you would have with the developer, if you tabbed to an element, used devtools to identify it, and the result was a <div> with no role and no accessible name.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok I can add some more detail surrounding this. I don't want to make the conversation about how to have the conversation with the developer about what needs changing. I am purely saying and discussing why this way is a good implementation and how its something which links up with the skills I learnt about accessibility testing.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the record, I think you are looking at this the wrong way around. You are advising that testers not use getByTestId, but the tester may not have any choice. This is when a testing strategy agreed with the developers can be worth its weight in gold, because you can all agree up front how the UI will be implemented.


Sometimes though this may not of been implemented on the project. This requires roles to be assigned correctly on the project you are on, things like test ID's or divs could of been used instead which makes this impossible to implement. When this is the case its always best to have that conversation with the team so that you can explain how useful and helpful having roles can be.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

may not have been implemented

could have been used instead

When this is the case it is best to

how useful and helpful HTML roles are.


## Some notable names worth tagging
And after all of these suggestions, we find ourselves back at the top of a finished test, ready to move on and forget about all that good work we have done. If only there was a way to easily locate your previous work when you have some busy moments on the project and need to hand over what you’ve created. Well, to name a few reasons, let’s start with the Name of your test and follow with some tagging advice.

When creating names of tests they tend to look like this:

```test('should return a 400 when User is Valid')```

which uses single quotation marks `'` as the name is a string. However, it can be difficult when you wish to pass in a variable to that same string, as variables are noted by single quotation marks also.

So, when creating Test Names its best practice to instead use the back tick ``` ` ``` as this will then still allow you to use a variable in the test name as seen in this example:

```test(`should return a 400 when User Infomation is '${variable}'`)```

For those who don’t know where it is, the back tick can be located underneath the ESC key on the majority of keyboards.

Going forward with the creation of our tests we want to ensure that they can be associated with the correct story or ticket from which they were created. When paired with the previous section for “Test Names”,. The tag function can be used in the test name and can be done like this:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for "Test Names", the tag function

Comma is correct, not full-stop.


```test(`a test example to show a test name for a test under ticket THW-000`, {tag: 'THW-000'}, async()```

Here you can see that the tag simply needs to be placed at the end of the test name and is separated with a comma. For good practice this should be the ticket number that the work is being generated from.

If you want to go the extra mile its even better to use annotations for ticket numbers. That way tags can be used excusivly to filter what tests are ran, filtering test results. This is what is documented in the Playwright documenteation.
Copy link
Member

@chriswilty chriswilty Jun 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's even better (need apostrophe)

exclusively (typo)

what tests are run (typo)

Playwright documentation (typo)


## Conclusion
Well, there have been some odd analogies and puns along the way but hopefully this peek into a tester’s mind has helped you understand the importance of, and some ways to implement some good accessible skills in your automation tests while also keeping it simple to implement. Maybe there are even more simple techniques out there so go out and find them, or better yet bring them up with others and let's get everything even more accessible!



4 changes: 4 additions & 0 deletions cfreestone/atom.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
---
author: cfreestone
layout: atom_feed
---
4 changes: 4 additions & 0 deletions cfreestone/feed.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
---
author: cfreestone
layout: rss_feed
---
5 changes: 5 additions & 0 deletions cfreestone/index.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
author: cfreestone
title: Charlie Freestone
layout: default_author
---
Binary file added cfreestone/profile-pic.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading