Accessibility Usability Test Protocol

ABC Airlines

Round 1

Building a barrier free web

Table of Contents

ABC Airlines 7 Core Functions Usability Test for Accessibility Methodology.................3

Think-Aloud Protocol............................................................................................................................................... 4
Usability Test Prep.......................................................................................................5
Task Based Usability Questions....................................................................................5
Q1) Core7: Contact ABC Airlines by phone.....................................................................................................6
Q2) Core7: Contact ABC Airlines by Email.......................................................................................................7
Q3) Core 6: Flight Schedule....................................................................................................................................8
Q4) Core 4: Flight Status..........................................................................................................................................9
Q5) Core 3: Personal Travel Itinerary.............................................................................................................10
Q6) Core 3: Personal Travel Itinerary/Request Assistance..................................................................11
Q7) Core 2: Check-in w Confirmation Number...........................................................................................12
Q8) Core 2: Check-in w Credit Card.................................................................................................................13
Q9) Core 1: Book a Flight – NOT READY........................................................................................................14
Q10) Core 5: Access Frequent Flyer Account – NOT READY................................................................14
Exit Questions and Wrap-Up......................................................................................15

2
 ABC Airlines 7 Core Functions
Usability Test for Accessibility Methodology

It takes only five users to uncover 80 percent of high-level usability problems.

Jakob Nielsen

The purpose of usability testing for the accessibility of the 7 core functions of the
ABC Airlines site is to qualitatively determine if the system is:

 Perceivable - people with disabilities can actually access the information

via sight, sound and touch. The information is sensed by the person's
brain.
 Operable - people with disabilities can actual interact with the functionality
of the system. Can use forms, controls and navigation.
 Understandable - people with disabilities can comprehend the content
and the interface. They describe the system as useful, predictable and
helpful.
 Robust - the content and functionality of the system work reliably with
assistive technology.
 Task Success Rate - people with disabilities were able to complete each
task.

The target categories of disabilities that will be tested are

 Visual
 Auditory
 Tactile
 Cognitive

During July 2015, Deque will test the ABC Airlines web site with 15-25 individuals
pulled from the defined categories of disabilities. We will administer an entrance
and exit survey before and after each test, and ask test subjects to sign a release
giving their permission for quoting their comments to be used for data-gathering
purposes. Test subjects will be asked to complete tasks read aloud to them by
the facilitator.

Our goals will be to determine if the 7 core functions (as defined by the Air
Carrier Access Act) can be successfully used by people with disabilities. We will
be gathering data on the following:

 Are there any accessibility barriers that prevent a test subject from
completing a task?
 Which tasks are completed successfully?

3
After each session, we include an open-ended general discussion period
allowing test subjects to share thoughts on any aspect of the accessibility of the
web site or testing with us.

Think-Aloud Protocol
Deque uses a task-based think-aloud protocol. We ask users to verbalize their
thought processes while attempting to complete the tasks. We ask them to say
what path they are taking to find information, what questions they have and what
surprises or confuses them as they go through the site. We keep questions
open-ended and neutral, such as “What do you mean by that?” or “What did you
expect to happen?” When users identify a problem, we ask them how they would
fix it. We observe body language and facial expressions as well.

4
Usability Test Prep
The test facilitator will instruct and observe test subjects performing fairly
simple, common tasks. Facilitators will verbally lead users through the series of
questions, encouraging them to think out loud and respond to what they are
looking at, and ask questions about their thought processes and decisions as
they work, without being intrusive or leading. The facilitator will also take in-depth
notes.

Test Facilitator: ___________________ Test Subject: __________________

Test Date: _______________________ Test Location: _________________

Specific Computer/Software used for this Test:

 Assistive Technology:
 PC or Mac:
 Operating System:
 Browser and Version:
 Screen Resolution:

Take user to the ABC Airlines home page before beginning this section.

Task Based Usability Questions

The task-based questions for this usability accessibility test begin on the next
page. Questions are laid out with only one question per page to allow for note
taking. The first question has notes (in blue) for the facilitator. The facilitator
should only read aloud the words that are highlighted in yellow.

5
Q1) Core7: Contact ABC Airlines by phone

Using the ABC Airline website, find the phone number to call ABC
Airlines.

[Facilitator asks this question out loud. Try to use the wording listed here.
Do not lead the user to the answer. Do not help the user answer the
question. Remind the user we are testing that software, not them.
Encourage them to think out loud. (“What words are going through your
mind?”, “What are you looking for?”)

Observe known click stream: [Facilitator checks each step user clicks]
___ Home Page
___ Contact Us (link)

If user does not follow the known path, what did s/he click on? [Facilitator notes different
paths/dead-ends]

User’s Verbal comments: [Facilitator notes user’s verbal comments]

Any accessibility barriers or issues discovered?

   
0 1 2 3

[Facilitator indicates difficulty rate of this task for this user based on Facilitator’s
opinion]

 0 = User completed task with zero difficulty. (Zero Frustration)

 1 = User completed task with only minor problem(s). (Little Frustration)
 2 = User completed task, but it required more effort/time/dead-ends than
the user expected. (Medium/High Frustration)
 3 = User did not complete task. (Point of Failure)

6
Q2) Core7: Contact ABC Airlines by Email

You've come up with a great idea for improving ABC Airlines by having
free apples on every flight. Starting from the Contact Us page, find a
way to submit your great idea electronically.
Observe known click stream:
___ Send us a message (link)

If user does not follow the known path, what did s/he click on?

User’s Verbal comments:

Any accessibility barriers or issues discovered?

   
0 1 2 3

7
Q3) Core 6: Flight Schedule

Fine the flight schedule available from AUS to SJC on September 4,

2015. How many non-stop flights are scheduled on that day?

Observe known click stream:

___ Flight (link) in main nav
___ Flight Schedule (link)
___ Complete form fields and active Search (button)
___ Flight Schedule page loads

If user does not follow the known path, what did s/he click on?

User’s Verbal comments:

Any accessibility barriers or issues discovered?

   
0 1 2 3

8
Q4) Core 4: Flight Status

Navigate back to the ABC Airlines home page. Check to see what gate
the first flight from AUS to DAL departed from yesterday.
Observe known click stream:
___ Home Page
___ Flight Status (link)
___ Complete form fields and active Search (button)
___ Flight Status Page loads

If user does not follow the known path, what did s/he click on?

User’s Verbal comments:

Any accessibility barriers or issues discovered?

   
0 1 2 3

9
Q5) Core 3: Personal Travel Itinerary

Find and review your travel itinerary for your reservation based on the
following information:
 Confirmation number:
 First Name:
 Last Name:
Observe known click stream:
___ Flight (link) in main nav
___ Manage Reservations (link)
___ Complete form fields and activate (Continue) button

If user does not follow the known path, what did s/he click on?

After reviewing your itinerary for this trip, do you know the dates of
travel, the number of flight segments, the departure and arrival times,
the flight numbers for each flight as well as the travel time for each
flight?

User’s Verbal comments:

Any accessibility barriers or issues discovered?

   
0 1 2 3

10
Q6) Core 3: Personal Travel Itinerary/Request Assistance

From this personal travel itinerary page, find a way to let ABC Airlines
know that you will be traveling with a trained assistance animal (a guide
dog).

Observe known click stream:

___ Add/Edit link
___ Complete form fields and activate (Continue) button

If user does not follow the known path, what did s/he click on?

User’s Verbal comments:

Do the options for assistance for people with disabilities meet your
needs? Is there anything else that needs to be added?

Any accessibility barriers or issues discovered?

   
0 1 2 3

11
Q7) Core 2: Check-in w Confirmation Number

Navigate back to the ABC Airlines home page. Get your boarding pass
for your upcoming flight. Using the following information:

 Confirmation number:
 First Name:
 Last Name:

Observe known click stream:

___ Home Page
___ Check-in (link)
___ Complete form fields and activate (Check In) button

If user does not follow the known path, what did s/he click on?

User’s Verbal comments:

Any accessibility barriers or issues discovered?

   
0 1 2 3

12
Q8) Core 2: Check-in w Credit Card

Navigate back to the ABC Airlines home page. This time, try to access
your boarding pass for your upcoming flight using only your credit card
number.

 Credit Card Number:

Observe known click stream:

___ Home Page
___ Check-in (link)
___ Complete form fields and activate (Check In) button

If user does not follow the known path, what did s/he click on?

User’s Verbal comments:

Any accessibility barriers or issues discovered?

   
0 1 2 3

13
Q9) Core 1: Book a Flight – NOT READY
This functionality is not ready for testing. It will be tested in a later sprint. It will
require more than one test/task question to cover all the different permutations of
booking a reservation and selecting flight amenities.

Q10) Core 5: Access Frequent Flyer Account – NOT READY

This functionality is not ready for testing. It will be tested in a later sprint. It ma
require more than one test/task question to cover the Frequent Flyer
functionality.

14
Exit Questions and Wrap-Up
When users are finished going through the set of task-based questions, give
them a few minutes to talk informally about their usability experience with you.
Ask them what they liked/disliked about the site most, if they have suggestions
for improvements, etc. If they’ve done or mentioned anything during the test itself
that you want to ask further questions about, now is the time to ask. In my
experience, users are very willing to talk with you when the test is over!

15