r aoolong-established, empirical and exploratory technique to answer questions such as “how would an end user respond to our 9ftware or design under realistic conditions?” Usability testing is a technique used in user-centered interaction design to evaluate a product by testing it on users. ssearch methodology — Evaluation — Experimental design Observation and analysis of user behavior while users use a product or luct prototype to achieve a goal (Dumas and Reddish, 1993, p.22) identify usability problems and make recommendations for fixing and improving the design (Rubin, 1994) AwadioosResearch Methodology- EvaluationA/B TestingIndividual campaigns on these channels could include * The homepage of a website * A notification on a smartphone of a ne product offer * The initial email a new customer receives after signing up for a serviceWhy Is A/B Testing oe Important? Palatal A/B testing increases engagement Rees with customers Ceirereeeeoeny ean arith A/B testing enhances campaign effectiveness and optimizes Sc programs for a company’s target A B audience A/B testing enhances marketers’ awareness and expertise of oe rn audience preferences ee Geen ‘ea ricer eerWHAT TO TEST Messages Visuals Calls-to-Action (CTAs) Content Site Navigation Forms And so on...A/B TESTING WORKFLOW Choose an element to test Write a hypothesis Decide on the sample group Define what su iI100K Yer like Set up the Look at ti Ps Con you optimize interfaces’ ch online f ts are the best eeeUsability Testing: Web Design Criteria Links must be consistent and predictable Group like things on the same page Be consistent with language Most important information should be on the first screen fe keywords for quick reading/scanning Do not use animation or sounds (usually) Make links look like links Distinguish text from g Avoid jargon (Spool, 15 PoUsability Testing: Web Design Criteria + Ten Usability Heuristics (Niel y of system status he control and freedom tency and standard: rather than recall and minimalist design diagnoseUsability Testing: Methodology * Artificial environment (laboratory) Mmay onitor/admini — Empathetic Impartial pontaneously to situations that cannot be anticipated. user time for task th the plan if mistakes occurUsability Testing: Methodology * Design test materials he test to participant ds in understanding the participants’ performance — Data logger materials * Data collection instrument for categorizing participants’ action + Can note time to match with videotape recordingUsability Testing: Methodology + Design test materials — Non-disclosure and tape consent forms for legal protectionUsability Testing: Methodology * Design test materials — Posttest questionnaire + All participants asked the same q * Gather qualitative information and pre — Debriefing guide + Structure and protocols for ending the session ot apparent in actions » Rationale » Points of confu:Usability Testing: Methodology Test materials and equipment Conduct the test — Represent the actual work environment Users are asked to think aloud Observe users while using or reviewing the product + Probe + Controlled and extensive questioning Collect quantitative and qualitative data and measure: + Record comments or questions about the p *+ Observe and document users’ behaviorsUsability Testing: Methodology * Analyze the dataUsability Testing: Methodology * Analyze the data — Quantitati CJ BaesPeexa Awataoe 6"Usability Testing: Interpret Data * Interpret the data — Five factors for benchmarking the usability of an interfa (Shneiderman and Plaisant, 2004) + Time to learn Speed of pe Rate of err Retention subjectivea Usability Testing: Interpret Data * Interpret the data — Prioritize severity of problems + Severity ratings (Zimmerman and Akerelrea, 2004) Time required to complete t Number of users tered problem Negative impact on users’ perception ofthe product + Difficult if 70% of users cannot perform task + Error Severity + Probability of Occurrence (Rubin, 1994)Usability Testing: Interpret Data * Usable Web site: (Rubin, 1994) = Usefulness +e — Efficient ‘+ Maximum productivity with minimum effort — Learnability + Ea hat the user needs it to do of learning application and moving from being a novics illed user + User's attitude about the site—howUsability Testing: Report Results Executive summary + Profile + Detail d method: Succinctly explain the analysi + Pr + Include tables and + Provide example Identify strengths and weaknesse Recommend improvement rorsUsability Testing: Limitations * Two major limitations — Reliability + Testing of users who are not atypical us + Individual variation within the test popula — Validity * Test tasks, scenarios of the search processes, and testing environment are not accurate + Results not generalizable to the entire user population