Test Driven Development.

ppt

30-05-07

Test Driven Development

Boydens Jeroen

Content
When use tests? Why automate testing? We have specifications? What is Test Driven Development? How to write tests? Unit Testing TestDriven.Net Nunit
http://www.khbo.be/~peuteman/TDD.html
2/44

Enterprise Programming research group

Test Driven Development.ppt

30-05-07

1. When use tests?

Whenever software changes
Bug fixes, to prevent regression
Whenever bugs are reported Whenever bugs are fixed

Upgrades, to detect we are not breaking something that used to work

If the test fails, we broke something If the test passes, we did ok

No Fear to break existing code

Contradictory to
Don't touch it, it might break Fragile system

Eventually whenever hardware changes

3/44

2. Why automate testing?

Because testing is
Boring Necessary Repetitive For humans: error-prone Better done while developing
Instead of done by the end user

In many cases only manual testing

4/44

Enterprise Programming research group

Test Driven Development.ppt

30-05-07

Manual testing
inserting output statements
To follow flow
Do we reach this statement?

To verify calculations
What is the current value of this variable

Insert assert statements

Only output in debug build

5/44

Manual testing..
Running the application in debugger
Insert breakpoint at start of suspicious code Step through code Watch evolution of local variables

Starting to use the application and..

Wait for strange effect to happen Wait for error to occur
6/44

Enterprise Programming research group

Test Driven Development.ppt

30-05-07

Manual testing..
With manual testing, testing only done with specific values
Those values causing the current error

If testing is manual
insert some extra work now to write the tests
You do not get it for free

Extra work to "tell" the computer what to do

Automate the boring, necessary, repetitive work
That's what computers are good at
7/44

Automated testing
Automated tests..
run as many times as needed
Without complaining

run whenever needed, e.g.:

When we suspect a problem When we check in code Before we deploy our code

always run in exactly the same way

They don't get headaches

run for many different values can be called by a smokebot

Automated systems that runs all your tests on checked in code hourly, nightly, ..
8/44

Enterprise Programming research group

Test Driven Development.ppt

30-05-07

Benefits
Benefits of automated tests
We test the base functionality of our application The tests are repeatable Our co-workers can verify how we tested Serves partly as "documentation"
Show several ways of how to call a method

Support regression testing

Run all tests anytime we change something Our tests become a safety net

We will never have the same bug again

Because when we find a bug, we add a test to detect the bug

We spend less time in debug mode

Probably more time in specification mode
To create undoubtful specifications
9/44

3. We have specifications?
Specifications are a very good source to write tests Strategy:
Select a post condition
Tests do not start from pre conditions and class invariants

Turn into a test If we are not able to turn the specification into a test
Then the specification is unclear
Rewrite the post condition

When the tests are running, based on the specifications

Then they serve as acceptance tests
10/44

Enterprise Programming research group

Test Driven Development.ppt

30-05-07

4. What is Test Driven Development?

Red bar/Green Bar mantra Part of eXtreme Programming

TDD with legacy code

11/44

Red bar/Green Bar mantra !

Write test first Write minimal skeleton to compile test
Red bar

Write code to make test run

Green bar

From time to time: Refactor Test code

Green bar

Repeat cycle
12/44

Enterprise Programming research group

Test Driven Development.ppt

30-05-07

Write test first

Write test first
Before you write the code Start from specifications
Specifications should define the interface Select values for tests

Or start from notes

13/44

Write test first..

Write tests as tryouts of the code
Constructor numerator, denominator Class Rational Method Add other operand Property Numerator Property Denominator

Modify the interface if needed

Easy since no code exists

Tests improve the design

They make the interface better: They make the methods more atomic:
Do one thing Do it good

You think about how the objects will be used

The code written is testable

14/44

Enterprise Programming research group

Test Driven Development.ppt

30-05-07

TDD by Example, K. Beck

<quote> By writing a test before implementing the item under test, attention is focussed on the items interface and observable behavior </quote>

15/44

Part of eXtreme Programming

Extreme Programming, way of doing things:
Simple Design Pair Programming Test Driven Development Design Improvement Continuous Integration Collective Code Ownership Coding Standards Metaphor Sustainable Pace Whole Team Planning Game Customer Tests Small Releases
16/44

Enterprise Programming research group

Test Driven Development.ppt

30-05-07

XP most used
XP most used principles:
Simple Design Pair Programming Test Driven Development Design Improvement

TDD can be used without practicing XP

17/44

Extreme Programming
Simplicity is the key
Structure your software system such that it can handle all current issues - and only those - in an elegant way
Do not try to anticipate future elements
Example: do not partition a class of persons into subclasses of men and women, if such a separation is not needed right now

Never add functionality that you do not need yet

Do not add methods related to the marital state of persons, if that property is of no relevance right now

18/44

Enterprise Programming research group

Test Driven Development.ppt

30-05-07

Extreme Programming
Pair programming
All code is produced by pairs of two programmers working on the same machine
The best way to pair program is to just sit side by side in front of the monitor
Slide the key board and mouse back and forth

One person types and thinks tactically about the method being created
The other person thinks strategically about how that method fits into the class

Pair programming increases software quality without impacting time to deliver

2 people working at a single computer add as much functionality as two working separately A pair of programmers delivers software of superior quality

19/44

Extreme Programming
Refactor mercilessly, Design Improvement
Restructure your software system as soon as it no longer handles all current issues in the most elegant way
Split a class into two or more classes if it turns out to define groups of different objects
Turn a class of persons into a hierarchy involving an abstract class of persons and concrete subclasses of men and women

Split a complex method into sub-methods

Extract parts of the body of a complex method into auxiliary methods

Move methods up and down a hierarchy of classes

20/44

Enterprise Programming research group

10

Test Driven Development.ppt

30-05-07

TDD with legacy code

Typically there are no tests at all Start small
Find a testable section Write a test for it Gradually add more tests
First write the test that detects the bug
Make sure the test fails The test should pass

The code is probably not written for easy testing

Before each bug fix

Correct the code

If the code is not testable

Then refactor the code
Change source Create adapters

21/44

5. How to Write Tests?

Getting started Black box testing White box testing What can we test? Keep all tests passing Some guidelines

22/44

Enterprise Programming research group

11

Test Driven Development.ppt

30-05-07

Getting started writing tests

Get some training
You're getting it

Use a testing framework

Do not try to reinvent the wheel

Shoot for 100% coverage

More about coverage later

Pragmatic Unit Testing in C# with NUnit, 2nd Ed. Andy Hunt and Dave Thomas

Review your results

Look how good you did

Make your results visible

Be proud about what you did

23/44

Black Box Testing

Start with black box tests
Based on the test object's external perspective No knowledge of its internal structure

These tests are written in accordance to the specification These tests focus on the end-user These tests typically lead to 80% coverage
24/44

Enterprise Programming research group

12

Test Driven Development.ppt

30-05-07

White Box Testing

Also Known As Glass Box Testing
Based on the test object's internal perspective

These tests are written in accordance to the implementation These tests focus on the developer With these tests we will focus on the 20% code not covered by black box tests
25/44

What can we test?

Postconditions with valid inputs Postconditions with invalid inputs
Look for exceptions Expected exceptions thrown?

Test on boundary values

A lot of errors occur on boundary values

26/44

Enterprise Programming research group

13

Test Driven Development.ppt

30-05-07

Keep all tests passing

Broken window syndrome
In a abandoned building:
One broken window Leads to another

Don't even allow one test to fail

Whenever we change functionality

Update the test Did the interface change?
We might have broken clients

Whenever we add functionality

Add extra tests Make all tests pass So we do not fix one, and break something else
27/44

Some guidelines on writing tests

Make your tests run fast
Or they won't be used Mocking might help
Insert a fake object with the same interface

Make sure your tests are non destructive Make your tests repeatable

Do not change persistent state so other tests can't run, app behaviour change All state added during the test should be discarded for further tests Implement tearDown Everyone who changes the code can run the tests Do one thing And do it good

Make your tests available Make your tests atomic

28/44

Enterprise Programming research group

14

Test Driven Development.ppt

30-05-07

6. Unit Testing
What is unit testing?
Programmer tests Tests on small parts, units Tests below the user interface layer
Interface based (Not GUI) Lead to component decoupling

Test the internals of a class

By looking at external behaviour By looking at internal implementation

29/44

Unit Test Framework

xUnit Framework
Implementations:
JUnit
Java

NUnit
.Net languages

Unit

Simple to use
Command Line NUnit GUI Visual Studio Add-In
TestDriven.Net csUnit

30/44

Enterprise Programming research group

15

Test Driven Development.ppt

30-05-07

7. TestDriven.Net
Allows Ad Hoc Testing of code Uses internally Nunit Includes Cover Explorer

Line coverage
Simple to measure
Weakest form of coverage

Path coverage would be better

Examine many paths Test logic of the code

Coverage leads to white box tests

31/44

8. NUnit
How does NUnit work? Features of NUnit Generating Unit tests: Pex

32/44

Enterprise Programming research group

16

Test Driven Development.ppt

30-05-07

How does NUnit Work? Class

namespace bank { public class Account { private float balance; public void Deposit(float amount) { balance += amount; } public void Withdraw(float amount) { balance -= amount; } public void TransferFunds( Account destination, float amount) { } public float Balance { get { return balance; } } } }
-balance : float +Deposit(in amount : float) +Withdraw(in amount : float) +TransferFunds(in destination : Account, in amount : float) +Balance() : float

Account

33/44

How does NUnit Work? Test

namespace bank { using NUnit.Framework; [TestFixture] public class AccountTest { [Test] public void TransferFunds() { Account source = new Account(); source.Deposit(200.00F); Account destination = new Account(); destination.Deposit(150.00F); source.TransferFunds(destination, 100.00F); Assert.AreEqual(250.00F, destination.Balance); Assert.AreEqual(100.00F, source.Balance); } } }

34/44

Enterprise Programming research group

17

Test Driven Development.ppt

30-05-07

How does NUnit Work?

Reflection
Used internally

Attributes [ ]
The test-code is specially annotated using custom attributes From nunit.framework.dll

Fixtures [ TestFixture ]
Marks a class as containing tests Restrictions: public class + default constructor Constructor without side effects
is called multiple times

Optionally [ Setup ] and [ TearDown ] method

35/44

How does NUnit Work?

Tests [ Test ]
Marks a specific method as a test method Precondition: class marked as [ TestFixture ] No Parameters

Assertions
Our code contains assertions To demonstrate the correct working of the application

Configuration files
Different settings for test as settings in production

Multiple Assemblies tested all at once Visual Studio support

Through TestDriven.Net Add In Through csUnit Add In

36/44

Enterprise Programming research group

18

Test Driven Development.ppt

30-05-07

Features of NUnit
Attributes Assertions

37/44

Features of NUnit: Attributes

Attributes:
[ [ [ [ [ [ [ [ [ [ TestFixture ] Test ] SetUp ] TearDown ] TestFixtureSetup ] TestFixtureTearDown ] SetUpFixture ] ExpectedException ] Platform ] Category ] class method method, before each test method, after each test method, once before tests in class method, once after tests class, once for namespace method, specify Exception Type class, platform for which test should run class or method, setting named properties on any test case or fixture class or method, always ignored unless explicitly selected to run class, group test fixtures backward compatibility class, method, do not run test, explicitly report ignored

[ Explicit ] [ Suite ] [ Ignore ]

38/44

Enterprise Programming research group

19

Test Driven Development.ppt

30-05-07

Features of Nunit: Assertions

Assertions:
Assert class:
Equality Asserts Identity Asserts Comparison Asserts Type Asserts Condition tests Utility Methods

Extra classes:
StringAssert CollectionAssert FileAssert
39/44

Features of NUnit:
Equality Asserts (value) Identity Asserts (reference) Comparison Asserts

Assert Class

Assert.Are[Not]Equal( int expected, int actual ); Assert.Are[Not]Equal( object expected, object actual ); Assert.AreSame( object expected, object actual ); Assert.Contains( object anObject, IList collection ); Assert.Greater[OrEqual]( int arg1, int arg2 ); Assert.Greater[OrEqual]( IComparable arg1, IComparable arg2 ); Assert.Less[OrEqual]( int arg1, int arg2 ); Assert.Less[OrEqual]( IComparable arg1, IComparable arg2 );

Type Asserts

Condition tests

Assert.Is[Not]InstanceOfType( Type expected, object actual ); Assert.Is[Not]AssignableFrom( Type expected, object actual ); Assert.Is{True,False}( bool condition ); Assert.Is[Not]Null( object anObject ); Assert.IsNaN( double aDouble ); Assert.Is[Not]Empty( string aString ); Assert.Is[Not]Empty( ICollection collection );

Utility Methods

Assert.Fail(); Assert.Ignore();

40/44

Enterprise Programming research group

20

Test Driven Development.ppt

30-05-07

Features of NUnit:
StringAssert CollectionAssert

extra classes

StringAssert.Contains( string expected, string actual ); StringAssert.{Starts,Ends}With( string expected, string actual ); StringAssert.AreEqualIgnoringCase( string expected, string actual ); CollectionAssert.AllItemsAreInstancesOfType( Collection collection, Type expectedType ); CollectionAssert.AllItemsAreNotNull( Collection collection ); CollectionAssert.AllItemsAreUnique( Collection collection ); CollectionAssert.Are[Not]Equal( Collection expected, Collection actual ); CollectionAssert.Are[Not]Equivalent( Collection expected, Collection actual); CollectionAssert.[DoesNot]Contain[s]( Collection expected, object actual ); CollectionAssert.Is[Not]SubsetOf( Collection subset, Collection superset ); CollectionAssert.Is[Not]Empty( Collection collection ); FileAssert.Are[Not]Equal( Stream expected, Stream actual ); FileAssert.Are[Not]Equal( FileInfo expected, FileInfo actual ); FileAssert.Are[Not]Equal( string expected, string actual );

FileAssert

41/44

Generating unit tests: Pex

Pex: Dynamic Analysis and Test Generation for .NET Pex (Program EXploration)
intelligent assistant Automatically generates unit tests suggests how to fix the bugs

http://research.microsoft.com/pex/

42/44

Enterprise Programming research group

21

Test Driven Development.ppt

30-05-07

csUnit: Other Nunit add In

http://www.csunit.org Inspired on JUnit 2.4 support TestNG features of parameterized tests

43/44

Remember
Program testing can be a very effective way to show the presence of bugs, but is hopelessly inadequate for showing their absence.
Edsger W. Dijkstra EWD 340: The humble programmer published in Commun. ACM 15 (1972)
44/44

Enterprise Programming research group

22