Orthogonal Array Testing
Orthogonal Array Testing is a systematic and statistical way of testing, especially helpful when
dealing with multiple variables and their interactions. It allows for a reduced number of test
cases while ensuring coverage of significant combinations of input parameters.
▪ L represents the Orthogonal Array.
▪ Runs indicates the total number of test cases derived from the orthogonal array.
▪ Levels signifies the number of possible values each factor can take.
▪ Factors are the independent variables in the system that are being tested.
Example-1: Consider a login feature of a web application as an example.
We will focus on three factors:
1. Username Input (Factor 1)
2. Password Input (Factor 2)
3. Login Button Response (Factor 3)
Step 1: Define the Factors and Levels
• Factor 1: Username Input
o Level 1: Correct username
o Level 2: Incorrect username
• Factor 2: Password Input
o Level 1: Correct password
o Level 2: Incorrect password
• Factor 3: Login Button Response
o Level 1: Immediate response
o Level 2: Delayed response
Thus, we have 3 factors, each having 2 levels.
Step 2: Select the Orthogonal Array
Since we have 3 factors and each has 2 levels, the appropriate orthogonal array to use is
L4(2^3). This means:
• L4 represents 4 runs or test cases.
• 2 represents 2 levels for each factor.
• 3 represents 3 factors.
The orthogonal array for this configuration looks like this:
Test Factor 1 (Username Factor 2 (Password Factor 3 (Login
Case Input) Input) Response)
1 Correct Correct Immediate
2 Correct Incorrect Delayed
3 Incorrect Correct Delayed
4 Incorrect Incorrect Immediate
Step 3: Construct the Test Cases
Each row in the orthogonal array represents a unique combination of factors that you will test.
Below are the test cases corresponding to the array.
1. Test Case 1:
o Correct username, Correct password, Immediate response
o Expected result: Successful login.
2. Test Case 2:
o Correct username, Incorrect password, Delayed response
o Expected result: Login fails with error message after delay.
3. Test Case 3:
o Incorrect username, Correct password, Delayed response
o Expected result: Login fails with error message after delay.
4. Test Case 4:
o Incorrect username, Incorrect password, Immediate response
o Expected result: Login fails immediately with an error message.
Step 4: Execute the Test Cases
Perform the test cases as per the combinations defined in the orthogonal array. For each case,
check the expected behavior of the system against the actual result.
Step 5: Analyze the Results
After running all the test cases, analyze if the results conform to the expected outcomes. This
will help in identifying any issues related to specific factor combinations.
Note: Using the Orthogonal Array Testing method, for the given example we were able to
reduce the number of test cases from 8 (all combinations of 2^3) to just 4, covering all possible
critical combinations of factors. This significantly saves time and resources while maintaining
the effectiveness of testing for different interactions between factors.
Example-2: Consider that you want to test the Add to Cart feature across different browsers,
operating systems, and payment methods.
We will focus on three factors:
1. Browser (Factor 1)
2. Operating System (Factor 2)
3. Payment Methods (Factor 3)
Factor 1: Browser
• Level 1: Chrome
• Level 2: Firefox
• Level 3: Safari
Factor 2: Operating System
• Level 1: Windows
• Level 2: macOS
• Level 3: Linux
Factor 3: Payment Method
• Level 1: Credit Card
• Level 2: PayPal
Step 2: Select the Orthogonal Array
3 factors: Browser, Operating System, and Payment Method.
3 levels for the Browser and Operating System, and 2 levels for Payment Method.
To accommodate the different numbers of levels, we can use the L9(3^3 2) orthogonal array,
which means:
• L9 represents 9 runs (test cases).
• 3^3 indicates that three factors have 3 levels each.
• 2 indicates that one factor (Payment Method) has 2 levels.
The orthogonal array for this configuration is:
Test Case Factor 1 (Browser) Factor 2 (OS) Factor 3 (Payment Method)
1 Chrome Windows Credit Card
2 Chrome macOS PayPal
3 Chrome Linux Credit Card
4 Firefox Windows PayPal
5 Firefox macOS Credit Card
6 Firefox Linux PayPal
7 Safari Windows Credit Card
8 Safari macOS PayPal
9 Safari Linux Credit Card
Step 3: Construct the Test Cases
Each row in the orthogonal array corresponds to a test case. These test cases cover different
combinations of browsers, operating systems, and payment methods.
1. Test Case 1: Chrome, Windows, Credit Card
o Expected result: Product successfully added to cart and paid for using Credit
Card on Chrome in Windows.
2. Test Case 2: Chrome, macOS, PayPal
o Expected result: Product successfully added to cart and paid for using PayPal
on Chrome in macOS.
3. Test Case 3: Chrome, Linux, Credit Card
o Expected result: Product successfully added to cart and paid for using Credit
Card on Chrome in Linux.
4. Test Case 4: Firefox, Windows, PayPal
o Expected result: Product successfully added to cart and paid for using PayPal
on Firefox in Windows.
5. Test Case 5: Firefox, macOS, Credit Card
o Expected result: Product successfully added to cart and paid for using Credit
Card on Firefox in macOS.
6. Test Case 6: Firefox, Linux, PayPal
o Expected result: Product successfully added to cart and paid for using PayPal
on Firefox in Linux.
7. Test Case 7: Safari, Windows, Credit Card
o Expected result: Product successfully added to cart and paid for using Credit
Card on Safari in Windows.
8. Test Case 8: Safari, macOS, PayPal
o Expected result: Product successfully added to cart and paid for using PayPal
on Safari in macOS.
9. Test Case 9: Safari, Linux, Credit Card
o Expected result: Product successfully added to cart and paid for using Credit
Card on Safari in Linux.
Step 4: Execute the Test Cases
Run each test case as defined in the orthogonal array. This will allow you to observe the
behavior of the Add to Cart and Payment functionality across different browser, operating
system, and payment method combinations.
Step 5: Analyze the Results
Once all the test cases are executed, evaluate whether the functionality worked as expected
for each combination. This helps identify any issues specific to a browser, operating system,
or payment method.
Note: Using the Orthogonal Array Testing method here, we reduced the total number of test
cases from what would have been 18 test cases (3 browsers × 3 operating systems × 2 payment
methods = 18) to just 9 test cases. This greatly reduces the effort required while ensuring that
all critical combinations of browser, operating system, and payment method are covered.
