Automated Accessibility testing for Android apps
Accessibility for apps implies that differently abled people can access the same information as people without the disability.
Accessibility analysis for your apps can unveil new opportunities to make your app more accessible and hence usable by wider audience. This can be done by manually scanning the app screens via Accessibility Scanner app. However, automating this process helps in covering all areas that we might miss during manual accessibility analysis. Firebase test lab comes to our rescue here and gives us a report of all accessibility bugs in our app without any manual intervention.
So, let’s start leveraging firebase test lab to make our android apps more accessible.
Step 1: Go to the firebase console here after login in your account.
Step 2: To start creating a new project, click on “Add project”. Then enter a project name.
You can choose to enable or disable Google analytics for the project.
Your Firebase project is ready now.
Step 3: In the left navigation menu, go to “Test Lab” under the “Release & Monitor” tab.
Step 4: In the “Android Robo test” section, click on “Browse” and upload the apk or adb file.
This will run the robo test on the apk or adb.
Step 5: On the Robo test screen, we get to see the device on which the test was run and whether it passed or failed.
Click on device details to see the results of the test on that device.
Click on the “Accessibility” tab to view accessibility issues.
Here we see accessibility issues that are warnings, minor issues and some tips.
We also get to see classification of these issues based on four accessibility guidelines -
1- Issues related to touch targets: These issues arise when the clickable element in consideration has width or height less than “48dp”.
2- Issues related to low contrast: These issues arise when the element in consideration has a contrast ratio that doesn’t abide by contrast ratio guidelines given by W3C.
3- Issues related to content labelling: These issues arise when the element under consideration doesn’t have a content description or has content description same as other elements on the same screen(which makes it difficult for assistive technologies to distinguish elements).
4- Issues related to Implementation: These issues arise when the element under consideration has implementation issues listed here.
From the screenshots, we get to know the screens and the elements that cause violation of a particular accessibility guideline. On viewing the screenshot, we get to know the element id, the issue with the element and how that can be fixed.
How to start Another Robo test on a different device(s)?
Step 1: Click on “Test Lab” in the left navigation menu in Firebase console. On the screen that opens, click on “Run a test”.
Step 2: In step 1(configure test), upload the apk or adb file and in “Select dimensions” tab, choose a device.
You can also filter the available devices list based on API Level or Form factor(Phone, Tablet, Wearable).
After that we can select the device by clicking on the “+” button, We can also select various devices and click on “Continue” button.
How to make the Robo test consider the user journey(or screens encountered) for a particular user?
Step 1: Click on “Test Lab” in the left navigation menu in Firebase console. On the screen that opens, click on “Run a test”.
Step 2: In step 1(configure test), upload the apk or adb file and in “Select dimensions” tab, choose a device and click on additional options. In the “Test Account Credentials” section, enter username resource name and value and password resource name and value.
This will help in capturing accessibility issues encountered for signed-in users(if most of your app screens work for signed-in users) or when the app is used by a particular user.
You can also switch to the “Robo” tab and get to understand whether the supplied credentials were used successfully by the Robo test crawler and the crawling path(screens traversed).
Conclusion
Leveraging Firebase Test Lab for automating accessibility testing has proved to be an effective & reliable method for finding potential accessibility issues in Android apps. We can get to know all kinds of potential accessibility flaws in just a few minutes without even needing to open the app.
This process can be used with our app builds of any build variant(debug or release), which helps us identify and fix accessibility issues earlier in our development cycle and hence aids in making the app accessible sooner to a wider audience.
Firebase Test Lab uses Accessibility Scanner app(and Accessibility Test Framework) for computing this list of accessibility issues, hence is a reliable and cost-effective method to identify accessibility issues in our app. We can understand more about Firebase Test Lab Accessibility issue categories here.
At the end of this blog, I would encourage all of you to try leveraging Firebase Test Lab for accessibility analysis of your Android apps, so that you can identify and fix accessibility issues sooner and build better and inclusive apps.
References:
1. https://developer.android.com/guide/topics/ui/accessibility
2. https://firebase.google.com/docs/test-lab/android/analyzing-results#interpret_accessibility_results