Lecture 7:

Testing Web Applications

Muhammad Umair Naru

Overview
 Introduction
 Fundamentals
 Web Engineering Specifics
 Methods & Techniques
 Automating Testing
 Quiz 1
 The international ISO/IEC standard 9126 defines
a technology-independent model for software
quality which defines six quality characteristics,
each divided into a specific set of sub
characteristics. What are those six quality
characteristics? (Just write their names)
The Importance of Testing
 Traditionally, testing has focused on
functional requirements – not enough for Web
applications.
 On the Web, testing is a critical measure of
quality assurance.
 Meeting users’ expectations
 Finding errors and shortcomings
 Many users, many platforms
 Behavior of third-party software
Fundamentals
 Terminology
 Some definitions:
 Testing: An activity conducted to evaluate the
quality of a product to improve it by identifying
defects and problems.
 Error: the actual result deviates from the
expected.
 Our expected results should (theoretically) come
from our requirements definition.
 Most often, the goals/concerns/expectations of
stakeholders serve as the testing basis.
 Test case: a set of inputs, execution conditions,
and expected results for testing an object.
Test Objectives
 Main objective: find errors, NOT show that
none exist.
 Complete test coverage is impossible, so
testing focuses on mitigating the largest risks.
 Where’s the greatest potential for loss?
 What are the sources of this risk?
 Start testing as early as possible – even with
restricted resources and time.
Test Levels
 Unit tests:
 Testing the “atomic” units - classes, Web pages, etc. -
independently. (Developer)
 Integration tests:
 Test the interaction of units (Tester & Developer)
 System tests:
 Testing the whole, integrated system (Dedicated team)
 Acceptance tests:
 “Real-world” tests – testing under conditions that are as
close to the “live” environment as possible (Client)
 Beta tests:
 Informal, product-wide tests conducted by “friendly” users.
Example – Unit Test (in Java)
@Test public void simpleAdd() {
   Money m12CHF= new Money(12, "CHF");
    Money m14CHF= new Money(14, "CHF");
    Money expected= new Money(26, "CHF");     
Money result= m12CHF.add(m14CHF);     
assertTrue(expected.equals(result));
}

 Unit tests are preferred to debugging expressions

 Do not rely on human judgment
 Easy to run many at the same time
Unit Testing Frameworks
 Java :: JUnit
 http://junit.sourceforge.net
 Integrated into NetBeans & Eclipse
 PHP
 SimpleTest :: http://simpletest.sourceforge.net
 PHPUnit :: http://www.phpunit.de
 Note: these frameworks work best for object-
oriented code.
 Your Role as a Tester
 The ideal tester has a “destructive” attitude.
 Very difficult for developers to “destroy” their
own work.
 However, Web projects focus heavily on unit
tests, making them more prone to errors.
 Thus, some guidelines:
 Have others in the Web team perform tests.
 Best tester is the one who gets the most bugs
fixed.
Testing – Web
Engineering Specifics
Web Engineering Specifics - 1
 Errors in Web content
 Found mainly through proofreading - very costly
 Alternative tests: Spell-checking, meta-
information
 Hypertext structure
 Is each page accessible via a link?
 Does each page link to the hypertext structure?
 Are there any broken links?
 What happens when the user hits “Back” in their
browser?
 Web Engineering Specifics - 2
 Subjective requirements for presentation
 Often in the eye of the beholder (e.g., aesthetics).
 Tester must distinguish accepted behavior from
faulty.
 Presentation testing on the Web borrows from print
publishing.
 Multi-platform delivery
 Can you test on every device?
 Can you create test cases on every device?
 Simulators are frequently available, but may be
buggy.
Web Engineering Specifics - 3
 Global availability
 Testing dynamic content in multiple languages
 Testing for layout difficulties due to varying text
lengths.
 Juvenility & Multidisciplinarity of Web team
 Reluctance to accept testing methods.
 Lack of testing knowledge.
 Consensus-building is required.
 May do too much testing; just as bad as too little.
Web Engineering Specifics - 4
 Multiple System Components
 Third-party; different platforms.
 Testing of the components’ integration and
configuration is also required.
 Immaturity of test methods
 Suitable test suites for new technologies often
don’t exist, or are poorly designed.
 Continuous change
 Requirements, hardware, software changes.
 Retest following each major upgrade.
Test Methods &
Techniques
 Link Testing
 Finding broken links
 Can be automated through a spider
 Doesn’t help for pages with no incoming links.
 Finding orphan pages
 Orphans are pages with no links back to the
navigation structure.
 Users get frustrated and leave.
 Capturing statistics
 Depth & breadth of navigation.
 Distance between two related pages.
 # of links.
 Load times.
 Browser Testing
 Browsers vary by:
 Manufacturer
 Version
 Operating system
 Device
 Configuration (stylesheets, JavaScript on/off)
 W3C Standard compliance
 Important questions to ask:
 How is state managed?
 Can a (dynamic) web page be bookmarked?
 Can users open multiple windows?
 What happens when cookies and/or scripting is
turned off?
 Usability Testing
 What are the 2 main approaches to usability
testing (engineering)?

 We’ve also talked about accessibility and

evaluation approaches
 Visual, cognitive, physical, and age-related
should garner the most attention when testing.
 http://validator.w3.org
Load Testing
 Does the system meet required response
times and throughput?
 Load profile - expected access types, visits
per day, transaction types, transactions per
session, etc.
 Must determine the range of values for
response times and throughput.
 Evaluate the results to look for bottlenecks.
Stress Testing
 How does the system behave under
abnormal/extreme conditions?
 The test should tell you…
 If the system meets the target responses
times and throughputs
 If the system responds with an appropriate
error message. (i.e. graceful degradation)
 If the system crashes (it should NOT!)
 How quickly the system recovers to normal
operation.
Continuous Testing
 Simulates usage over a long period of time
 Testing for errors that “pop up” because
resources aren’t released by an operation.
 Unreleased database connections
 Other memory leaks
 Typically, running the operation a few times
doesn’t produce an error, hence the need for
continuous testing.
 Security Testing
 A systematic test scheme is strongly
encouraged.
 Testing for correctness is not sufficient
 Is confidential data inadvertently exposed?
 What happens if we input incomplete data?
 What happens if we inject malicious code?
 SSL-encrypted pages
 Is our SSL certificate working?
 What happens if I try to access a protected
page/site in a non-secure way (i.e., http://)?
 Test-Driven Development
 Inspired by the test-first approach used in XP;
can be used in any type of project.
 Tests must be written before implementation.
 Every unit has a test.
 When a test fails, the developer must only
change the code to successfully run the test.
 Developers can concentrate on small steps,
while still making clean code that works.
 More pressure leads to more testing.
Automating Testing
Automating Testing - Advantages
 Some tests are impossible to perform
manually.
 Load & stress tests.
 Link testing for large websites.
 More tests can be run in less time.
 When updating an application, can detect
errors caused by side-effects to unchanged
functionality.
 Automating Testing - Disadvantages
 Expectations of automated testing is often too
high.
 Automation does NOT improve effectiveness.
 If tests are poorly devised, automating them does
not magically improve them.
 Automation is expensive
 Test execution infrastructure must be maintained.
 License fees & training costs
 Authors’ suggestion: tools to enhance manual
testing (partial automation.)
Quiz 2
 For Web-centric modeling, we employ the
UML Web Engineering (UWE) notation but
UML is not the only modeling
method/language for WebE. Write name of
two modeling methods/languages other than
UML.
 What is the purpose of Presentation
Modeling?