-
Notifications
You must be signed in to change notification settings - Fork 103
Charlie Freestone Blog Post - Some Best Practices for Accessible Automation Testing #317
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: gh-pages
Are you sure you want to change the base?
Changes from all commits
c6a042d
e14ac3e
f37e213
d02e6c8
f92db1f
cbeba45
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,58 @@ | ||
--- | ||
author: cfreestone | ||
summary: While on my most recent project I had the unique experience of working closely with many testers and test minded individuals. This allowed me to learn some much-needed lessons about how to best implement automation testing with accessibility in mind, a sometimes-overlooked area of test automation. | ||
categories: | ||
- Testing | ||
--- | ||
# Some Best Practices for Accessible Automation Testing | ||
While on my most recent project I had the unique experience of working closely with many testers and test minded individuals. This allowed me to learn some much-needed lessons about how to best implement automation testing with accessibility in mind, a sometimes-overlooked area of test automation. | ||
chriswilty marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
## Introduction | ||
What I hope to share with you is how simple it can be to both think about and implement the approach to make sure your tests are accessible. This can be because you have strict reasons on the project to ensure compliance with certain metrics, because this can lead to improved readability of your code and its longevity, or because you want to read some wonderful thoughts "from certain testers perspective". | ||
|
||
## What to expect() | ||
Now it’s good to make note that the lessons learnt here are all from the work I have carried out while implementing Playwright tests. This will mean you will need some basic understanding of TypeScript and Playwright when I discuss some examples. So without further ado let's discuss the `expect()` function. | ||
|
||
This neat little function is the bread-and-butter basics of automating in playwright, it's what we want to see when we place our test into a certain configuration. This doesn’t mean it has to do handstands or backflips, but just like any gymnastics routine we do wait on it with bated breath, hoping for results we will cheer for. This anticipation makes it easy for us to write something which makes us wait for that big finale in our tests, even if the tests took the literal definition of breaking a leg. | ||
|
||
And therein lies the issue; during a test we usually assert for confirmation to ensure we are where we expect. That means we can `expect(response.status()).toBe(200)` which is a wonderful successful request that has found the requested resource of a page. Instead we get a `201 response`, which is just as equally a successful request but it led to the creation of a resource. Test results can’t always be perfect and that means that this stopping point during the test, before we check the larger critical parameters later on, will skip the test. Instead, we have to accept that not every response will be exciting and just `expect(response.status()).toBe(OK)` which allows us to still capture data from a test even if the status code is different. | ||
chriswilty marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
## Don’t just getBy, Role with it | ||
After correctly measuring our expectations, we want to roll into selecting all the necessary parameters we can get by with. Therefore, it comes as no surprise we use the `.getByRole function`, but in order for us to make sense of it then it's all about those pesky selectors. | ||
|
||
This is because there is an interface that runs behind the scenes of the UI and Playwright can pick selectors which will not be seen by someone who needs to use the keyboard to navigate around the system. For these users, the elements they have access to are either tabbed to or navigated to via arrow keys. If using a screen reader these selectors need to have a consistent and readable naming convention. Otherwise, any software that reads it aloud would read out jargon which will only confuse the User. | ||
|
||
Things like filters and drop-downs can be difficult to locate but they can be found. A good tip for finding this is by entering the webpage and open up DevTools, then clicking near the item you wish to find the selector of, but not on it. After that press tab and open up the console within DevTools. If you then type the command: `document.activeElement` you will find the info you need. | ||
|
||
By the same accessible reasoning it is good practice to not use `.getbyTestID`.This is due to the fact that names of those same ID’s will not be useful to someone with an accessibility issue, as it will sound like Jargon when navigated to. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I believe you may have undercooked the importance of selecting by role (or label or title), not just for the tester but for the developer also. It's impossible to select by role if the developer hasn't used semantic html or has made everything a Making a rule that the tester will select by role or label or title, or text where those are not possible, and never use the dreaded There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Ok what would you suggest this would need changes too. I'd say that this still talks about the point, what detail needs to be added here? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Well, you state this:
But then you don't elaborate on that naming convention, nor on how using getByRole has anything to do with this naming convention. Remember also that use of getByRole is entirely dependent on the developer using roles correctly when implementing the page, so it's all well and good to say "test by role" but you can only do that if the developers are on board with that. Similarly, finding an element using devtools will not necessarily give you the information you need, if the developer has used test IDs on divs instead of roles or labels, or generally non-semantic html. What I am suggesting is that you are missing the important interplay between developer and test engineer. Have a think about the conversation you would have with the developer, if you tabbed to an element, used devtools to identify it, and the result was a There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Ok I can add some more detail surrounding this. I don't want to make the conversation about how to have the conversation with the developer about what needs changing. I am purely saying and discussing why this way is a good implementation and how its something which links up with the skills I learnt about accessibility testing. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. For the record, I think you are looking at this the wrong way around. You are advising that testers not use |
||
|
||
Sometimes though this may not of been implemented on the project. This requires roles to be assigned correctly on the project you are on, things like test ID's or divs could of been used instead which makes this impossible to implement. When this is the case its always best to have that conversation with the team so that you can explain how useful and helpful having roles can be. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
|
||
|
||
## Some notable names worth tagging | ||
And after all of these suggestions, we find ourselves back at the top of a finished test, ready to move on and forget about all that good work we have done. If only there was a way to easily locate your previous work when you have some busy moments on the project and need to hand over what you’ve created. Well, to name a few reasons, let’s start with the Name of your test and follow with some tagging advice. | ||
|
||
When creating names of tests they tend to look like this: | ||
|
||
```test('should return a 400 when User is Valid')``` | ||
|
||
which uses single quotation marks `'` as the name is a string. However, it can be difficult when you wish to pass in a variable to that same string, as variables are noted by single quotation marks also. | ||
|
||
So, when creating Test Names its best practice to instead use the back tick ``` ` ``` as this will then still allow you to use a variable in the test name as seen in this example: | ||
|
||
```test(`should return a 400 when User Infomation is '${variable}'`)``` | ||
|
||
For those who don’t know where it is, the back tick can be located underneath the ESC key on the majority of keyboards. | ||
|
||
Going forward with the creation of our tests we want to ensure that they can be associated with the correct story or ticket from which they were created. When paired with the previous section for “Test Names”,. The tag function can be used in the test name and can be done like this: | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Comma is correct, not full-stop. |
||
|
||
```test(`a test example to show a test name for a test under ticket THW-000`, {tag: 'THW-000'}, async()``` | ||
|
||
Here you can see that the tag simply needs to be placed at the end of the test name and is separated with a comma. For good practice this should be the ticket number that the work is being generated from. | ||
|
||
If you want to go the extra mile its even better to use annotations for ticket numbers. That way tags can be used excusivly to filter what tests are ran, filtering test results. This is what is documented in the Playwright documenteation. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
|
||
|
||
## Conclusion | ||
Well, there have been some odd analogies and puns along the way but hopefully this peek into a tester’s mind has helped you understand the importance of, and some ways to implement some good accessible skills in your automation tests while also keeping it simple to implement. Maybe there are even more simple techniques out there so go out and find them, or better yet bring them up with others and let's get everything even more accessible! | ||
|
||
|
||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
--- | ||
author: cfreestone | ||
layout: atom_feed | ||
--- |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
--- | ||
author: cfreestone | ||
layout: rss_feed | ||
--- |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
--- | ||
author: cfreestone | ||
title: Charlie Freestone | ||
layout: default_author | ||
--- |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please remove this one, as your title is provided for you:
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Still need this correction.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry I have been busy with client work I shall make this adjustment.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Other way around, need a title in the head section, not in the body of the post: