King Saud University
College of Computer and Information Sciences
Computer Science Department
CSC 440
First Semester
Human Computer Interaction
1437-1438
Assignment # 5
In this assignment, you will learn how to plan and conduct real usability evaluations. By now, your interactive prototypes should be
fully functional. This assignment involves creating a user study plan and piloting it with at least one person; then, testing your
Sketchflow or Justinmind prototype with four different persons, generating user feedback and insights, and using this to inform
revisions to your final prototype.
STEP 1: Evaluation Plan
First, develop a user test plan. Think about what you want to learn, and then design your test plan based on that. Things that you will
want to think about:
What tasks will you ask the user to perform that will help answer your questions?
Decide how you want the user to share their feedback. You might ask them to think aloud, have them fill out a questionnaire,
or conduct post-test interviews.
Prepare an outline of how the user test will be run, written instructions that you will read to the users, and any other materials (e.g.
questionnaires, interview questions) that will be used during the session. You will need consent forms for your participants, you can
use our template to create one if you find it helpful.
STEP 2: Run User Tests
Run user tests on your interactive prototype with 4 different people. You will test your prototype according to the plan you drafted up.
You should record that by Morea which allows you to re-view video and analyze it with your team.
Your user will not be writing down the problems they find for you. It's your job to learn what your users are thinking; the feedback
they provide you will be invaluable for your next iteration. Your high-level goal is to find ways to improve your interface. Look for
breakdowns and pain points in your interface and try to understand what the problems are and how you might fix them.
As much as possible, keep things consistent between your user tests: use the same script, follow the same protocol, answer questions
in the same way, test in a similar environment (desk in office, living room, etc). Immediately after each test, do a quick debrief with
your team and write down any reactions or thoughts that came up. You will most likely forget them, so it's important to write them
down right after the user test.
STEP 3: Go Over Test Results
After testing, take some time with your team to reflect on your findings. Go through all the notes and other recordings. Try to be
objective, don't write problems off. Discuss as a team and try and define some general patterns in user behavior. When you identify
some interesting points, talk deeply about them - ask each other question, analyze user decisions, other paths they could have taken,
and so on. Let your insights guide redesigns for your final prototype iteration. The end product of your discussion should be a list of
recommendations, changes that you will implement.
Submit
1.
A description of your evaluation plan (200 words max), including justification for your choices based on what you hope to
2.
3.
learn. (Use Template)
Any questionnaires or surveys you wrote (either as text or a scanned image).
Your interactive prototype that you used in all testing sessions, if you didn't make any update after dynamic prototyping
4.
submission no need to submit it again.
Four Video Clips of Interaction highlighting key design problems with a description of what's going on each clip and the
5.
6.
timing for the key design problem.
Consent form signed by all participants.
Report describing the tasks, participant demographics, user performance measures on the tasks that you selected (time-ontask, accuracy), and a list of changes you will implement in your next iteration (for each change, include a brief explanation
for why you selected it). (Use Template)
In the Tutorial: Presentation
Your group will give a 10 minute presentation of your experience with this assignment. Be prepared to
discuss the user testing and your findings. What were some major findings? What changes did they
translate to? How did this round of testing help you in refining your design? What your
recommendation that you have been presented for each issues?
Include in your presentation screenshots of your interface BEFORE and AFTER fixing the problems that
have been identified in your observations and analysis.
King Saud University
College of Computer and Information Science
Computer Science Department
CSC440 Human Computer Interaction
1st Semester 2016/2017
Tutorial Assignment 5 Template
Section Number
Assignment Number
5 Usability Evaluation
Due Date
Due Date Sunday Dec 11, 2016
Date of Submission
Group Members
No. of
Name
ID
meetings
attend
Percentage of
tasks achieved
433203471
50%
434200371
50%
Part 1: Evaluation Plan
In this stage, we test and evaluate our prototype. For do this task we use Morae" which is usability software
tools offers an all-encompassing testing experience for its users. Also, gives the ability to record a user's
interactions with application, site, or product.
we have many tasks to evaluate our design of Arab Readers application:
Users are able to enter her information to sign up by the correct way.
Users are able to enter her information to log in by the correct way.
Users can found the path it goes to her profile simplify.
5
Users can add a book to her favorite books page.
Users can offer the book for sale.
We choice 4 random users from different age to test our application by Morae software. The user will use our
prototype as any application he downloading in his phone and try it. the movement of interaction the user and
the prototype will be recorded. after test the prototype by users, we must know the feedback from them. Then
we will ask them to think aloud through doing tasks. we chose this way to know the feedback from users
instantaneous and continuous.
Part 2: Usability Tests
The main objectives of the project were:
1.
2.
3.
2.1 Participant Profiles
Participant
Age
Gender
Grade
name
Average
Use of
knowledge of
handheld
diet and
device(1-5)
exercise
system (1-5)
1 Atheer
21
Female
2 Areen
15
Female
3 Sara
40
Female
4 Abdulaziz
18
Male
2nd in High
3rd in College
Not Student
Done High
School
School
Ages
50
40
30
20
10
0
Atheer
Areen
Sara
Abdulaziz
Ages
2.2 Timeline
Participant
Day
Date
name
Start session
Finish session
Duration of
time
time
session
1 Atheer
Saturday
13/3/1438
9:00 pm
9:03 pm
3 minutes
2 Areen
Saturday
13/3/1438
9:06 pm
9:10 pm
4 minutes
3 Sara
Saturday
13/3/1438
9:24 pm
9:28 pm
4 minutes
4 Abdulaziz
Saturday
13/3/1438
9:20 pm
9:23 pm
2.5 minutes
2.3 Context of the Testing
participant
location
Atheer
Bed Room
Areen
Bed Room- in office space
Sara
Living Room
Abdulaziz
Living Room
2.4 Performance Measures and Feedback
Metric
Effectiveness
How was it measured?
Task success, Number of
Errors
How was the data created/captured?
Level of task success and faild.
Efficiency
Time on Task
Average time for tasks
Satisfaction
User surveys/interviews
Answer questions and think loud
9
2.5 Tasks
Task #
Task
Objective of the task
(as it is was presented to
(What was this task designed for capturing?)
users)
1
Enter their info for signup
Enter Username & password
for Login to the application
Go to profile .See their
information in profile page
Adding Book to favorite page
Entering data in text fields, readability
Entering data in text fields, readability
Click on profile icon, visual design.
Click on Books image then click on "
Button, visual design and structure.
Offering Book for buy/
Click on Books image then click on "
borrow
Button, visual design and structure.
Part 3: Usability Results
3.1 Critical Problems
Theres no any Critical Problems.
3.2 Major Problems
Theres no any Major Problems.
3.3 Minor Problems
Also theres no any Minor Problems.
10
Part 4: Recommendations
Usability Issue
Recommendation
Issue that you discovered.
Theres must be hint or explanation for how to add
Is Causing..User
know what the point for adding a book to favorite
frustration, confusion, etc.
Book to Favorite. Because must of the users didnt
(Critical/Major/Minor)
Major
and how to do it.
Users cant find abc
Category list for books. Because its better to
Is Causing..User
have books that suits with all user types.
Search bar, to search for certain book instead
frustration, confusion, etc
Severity
Minor
of scrolling all books in home page.
Part 5: Appendix
11
12
