Testing and Documenting Perl Code
Scott Wiersdorf
NTT/Verio
Why Developers Should Test and Document Their
Code
or "Why Something is Better Than Nothing"
Assertion
I want to convince you that just as building a house without plans is
cobbling, not engineering, writing software without knowing in advance
how the software should work is also cobbling and not software
engineering.
This is not to say that you must
the software will precisely work
Because of the complex nature of
non-linearly, which allows us to
adjustments as we go.
know all of the methods
at the beginning of the
software writing, we do
discover new things and
or even how
project.
it
make
Writing tests and documentation (which is what this presentation is
about) are essential to the discovery process. By writing tests and
documentation before you begin coding, you are creating plans for
your code. Tests and documentation are the architectural plans for a
piece of software, without which the engineering process is simply an
exercise in trial-and-error and the software itself is highly fragile
and likely poorly-designed. Testing and documenting before coding
allows you to achieve a more succinct, stable, and readable API than
might otherwise be realized.
In addition to providing plans for new code, tests and documentation
provide a reference for existing code. Regression tests give you as
the programmer a tool that can tell you almost instantly if any
refactoring you've done has broken something. This lets you go back
and clean up that section of code you've got marked as 'FIXME' that
you've been meaning to go back and fix for a year but haven't had
time. Without regression tests, the FIXME's stay forever.
Roadmap: Testing and Documenting Code
About Me (briefly)
Guilt Trip: Why Not Test/Document
A Testing Example
Fleshing Out Our Example
Tests
Documentation
Writing Code
Future Improvements
The Take-away
What you should know about me
I am not a testing expert!
I am a programmer
I do not always write tests or documentation for all my code
I do write tests and documentation for nearly all of my code
I have (good) intentions to go back when it's time to refactor
old code and write tests and documentation for them, too
Why Not Testing
Roadmap: Why Not Testing
Ignorance
Someone Else's Problem
Looming deadline/short on time
Extra work
Testing is for weenies!
Why Not Test: Ignorance
I'm supposed to test?
I don't make mistakes (denial)
Yes!
Software engineering has come a long way since "top-down
modular design" was innovative
Testing is less effort than debugging
Learning to test your code will make you a more portable
programmer
You'll have more friends if you test your code
Yeah, right.
You write your programs once...and then write them all over again
when a change needs to be made (you're a real hard worker)
Meskimen's Law: There's never time to do it right, but there's
always time to do it over.
Work smarter, not harder: regression tests are your friends
Why Not Test: Ignorance (cont'd)
I think someone else in our department tests (see next slide)
Could be. After all, what are users for?
The best unit tests are done by the developer who wrote the unit
I don't know how to set up tests
That's why you're paying careful attention!
We'll cover that later
Why Not Test: Someone Else's Problem (SEP)
We already have a department that does that
Ok, that's a valid point
But here are a few more valid points:
It's also your problem because it's your code
No one knows the code like you do
No one cares about your code like you do
If you don't care about your code as you write it, you are a curse to your
employer
You won't care much about any of your code after a couple of weeks
anyway
Point: the best time to write tests is before and while you are
coding, not after (though after is better than nothing). You're
the best person to do that
Why Not Test: Looming Deadline
"We try to solve the problem by rushing through the design
process so that enough time will be left at the end of the
project to uncover errors that were made because we rushed
through the design process." --Glenford Myers
This also applies to testing
The best time to write tests is at the beginning of a project
You'll save time either at the beginning of the project (by not
writing tests) or at the end of the project (less debugging)
With regression tests, you will save time at the end of and
throughout the project
You will have more provably correct code sooner if you test
You will save oodles of time in the future with regression tests
There really aren't many good time-based arguments for not
writing tests
Why Not Test: Extra Work
You will always be afraid of touching a finished, working piece
of code until you have tests
Fourth Law of Code: Code that has no regression or unit tests
is either dead code or very fragile code that breaks at
inconvenient times (doubly so for production code)
Corollary: Working code with a complete suite of regression
and unit tests is living, growing code and will be useful to many
people over a longer period of time and will tolerate a high
degree of change and internal reworking
You will never have real confidence in your code until you
have tests
Your software will never be world-class quality until you have
tests (almost all CPAN modules have tests and many CPAN
modules have very thorough tests)
Why Not Test: Only Weenies Test!
Some weenies:
Larry Wall
and a cast of thousands, including
Tom Christiansen
Gurusamy Sarathy
Damian Conway
Simon Cozens
and more: consider CPAN
Don't you wish you were a weenie?
Why Not Documentation
Roadmap: Why Not Documentation
Same reasons as testing
Ignorance
Someone Else's Problem (documentation team)
Looming deadline/short on time
Extra work
Documenting is for weenies!
We'll go through this quickly
Why Not Document: Ignorance
What good is an undocumented module?
No one can use it except you
Do you really want to own this module for eternity?
By documenting your module, you free the module to be
owned by someone else
You also free yourself from having to explain to others how it
works
Why Not Document: Someone Else's Problem
Isn't this why we have a documentation team?
Can they understand your code?
(No, they can't. For the love of Pete, have a heart and
document your code)
(And another thing... if documenting your code is not part of
your job description, it should be. Generally speaking, tech
writers are not programmers. To think they should document
your code is evidence of misunderstanding their function. -B.
Heaton)
You really are the only one who knows how this code works--if
you document it, then someone else will understand it too (and
you will be free!)
If you're really a poor writer, do the best you can and let the
doc team clean it up later
The doc team loves it when developers document their code
Why Not Document: Looming Deadline
I'm short on time--I'm not going to document
Good point: given a choice between documenting and coding,
choose coding
This is why we document before we code!
It will shorten the amount of time you spend coding by more
than the time spent documenting
You will be more productive (only anecdotal evidence to
support this, but I believe in it)
Why Not Document: Extra Work
You have fallen victim to false laziness
False laziness says that less work up front yields less total
work in the long run
True laziness says that more work testing and documenting up
front yields less total work in the long run
If you document your code, you'll spend less time in the
following areas:
explaining how your code works to others (this lasts as long as the
module and is the #1 benefit in my book)
rewriting code to get the API right
"We try to solve the problem by rushing through the design process
so that enough time will be left at the end of the project to uncover
errors that were made because we rushed through the design
process." --Glenford Myers
time spent documenting is time spent designing and is well spent
mental exertion
Why Not Document: Extra Work (cont'd)
Substanitated Rumor: People who document well also think
well
Writing solidifies nebulous thoughts: "How can I know what I
think till I see what I say?" --E. M. Forster
If you take the time to document, you'll resolve most of the
really hard logic problems and interface issues early, saving
you labor later
Why Not Document: Only Weenies Document
Some more weenies:
Larry Wall
and a cast of thousands
and more: consider CPAN
A Testing Example
Roadmap: An Example of Writing Tests
Creating a module
Editing test.pl
Running tests
Fixing mistakes
Creating a Module
Many ways to create a module
Some more popular than others
We're going to use 'h2xs' because it's easy and found
everywhere
Creating a Module (cont'd)
create a module:
h2xs -AX Foo::Bar
This will create the following hierarchy:
Foo/Bar/Bar.pm
Foo/Bar/Changes
Foo/Bar/MANIFEST
Foo/Bar/Makefile.PL
Foo/Bar/test.pl
Notice the 'test.pl' file
Example 1: Create a module
Goal: a module that can parse an Apache log file and give
summary information in the form of an object
Start with h2xs:
h2xs -AX Log::Apache::Object
Writing Log/Apache/Object/Object.pm
Writing Log/Apache/Object/Makefile.PL
Writing Log/Apache/Object/test.pl
Writing Log/Apache/Object/Changes
Writing Log/Apache/Object/MANIFEST
The first thing we do is edit 'test.pl'
Depending on your version of Perl, you'll see varying amounts
of stuff cluttering your file
We'll cover what's in 'test.pl' next
Example 1: sample test.pl
A sample test.pl: this much will be created for you
#########################
# change 'tests => 1' to 'tests => last_test_to_print';
use Test;
BEGIN { plan tests => 1 };
use Log::Apache::Object;
ok(1); # If we made it this far, we're ok.
#########################
Example 1: Edit test.pl
now we hack on test.pl
we write one or two tests for each "feature" we're about to add
use Test;
BEGIN { plan tests => 1 };
use Log::Apache::Object;
ok(1); # If we made it this far, we're ok.
## first test
my $obj;
ok( $obj = new Log::Apache::Object );
Example 1: Running test.pl
tests are run automatically when we type 'make test'
% perl Makefile.PL
Checking if your kit is complete...
Looks good
Writing Makefile for Log::Apache::Object
% make
cp Object.pm blib/lib/Log/Apache/Object.pm
Manifying blib/man3/Log::Apache::Object.3pm
% make test
PERL_DL_NONLAZY=1 /usr/bin/perl -Iblib/arch -Iblib/lib \
-I/usr/local/lib/perl5/5.6.1/i386-freebsd \
-I/usr/local/lib/perl5/5.6.1 test.pl
1..1
ok 1
Can't locate object method "new" via package \
"Log::Apache::Object" (perhaps you forgot to load \
"Log::Apache::Object"?) at test.pl line 7.
*** Error code 255
Stop in /usr/home/scottw/testing/Log/Apache/Object.
we forgot to update the number of tests
we haven't written a "new" method yet
Example 1: Fixing Mistakes
Let's update the tests:
BEGIN { plan tests => 2 };
and write a 'new' method:
sub new {
my $self = { };
my $proto = shift;
my $class = ref($proto) || $proto;
bless $self, $class;
return $self;
}
and run the tests again
% make test
cp Object.pm blib/lib/Log/Apache/Object.pm
PERL_DL_NONLAZY=1 /usr/bin/perl -Iblib/arch -Iblib/lib test.pl
1..2
ok 1
ok 2
Success!
Wasn't That Satisfying?
We now have a module that works and we can prove it
We can get instant feedback when we make changes
We'll never have to write a test for the constructor again
...unless we add/change functionality in the constructor
We can be bold in making wide-impact changes such as
refactoring entire methods or changing global defaults, etc.
Fleshing Out Our Example
Roadmap: More Tests, Features and Documentation
Documenting What We're About To Do
Adding New Functionality
Writing Code
What We Just Did
Future Improvements
Documenting What We're About To Do
What good is a module with no documentation?
Documenting is easy with POD (it's already there)
Documenting before coding also helps solidify the API in the
same way testing does, but even better because putting
abstract concepts into concrete language (e.g., English) forces
your brain to conceptualize it; you quickly find ambiguities and
contradictions
Documenting: SYNOPSIS
=head1 SYNOPSIS
use Log::Apache::Object;
my $obj =
new Log::Apache::Object(file => '/www/logs/access_log');
$obj->parse();
print "I have had " . $obj->count(status => 404) .
" errors since the log was rotated\n";
Documenting: DESCRIPTION
=head1 DESCRIPTION
B<Log::Apache::Object> is an OO class for parsing Apache log
files. The following methods are available:
=over 4
=item B<new>
Creates a new B<Log::Apache::Object>. Valid arguments are
key/value pairs:
file => "filename.log"
=item B<init>
Initializes the object; this is called when the object is
instantiated, but may be called any time thereafter to
re-initialize the object (e.g., with new data). Takes the same
arguments as B<new>.
Documenting: DESCRIPTION (cont'd)
=item B<file>
Sets/returns the current log file to process.
Example:
$obj = new Log::Apache::Object;
$obj->file('/www/logs/access_log');
print "Working with log " . $obj->log . "\n";
Documenting: DESCRIPTION (cont'd)
=item B<parse>
Parses the log file and stores the log data internally to the
object. Future method calls will access this storage so the
log will not have to be re-parsed.
Example:
$obj->parse();
=item B<count([$field [, $value]])>
If no argument is specified, returns the number of lines in
the log. If B<$field> is specified, the total
number of unique keys in that field will be returned (e.g.,
'host' will return unique hosts in the log). If B<$value>
is specified, the number of entries in B<$field> for
B<$value> will be returned.
Example:
print "I have " . $obj->count . " lines\n";
print "I have " . $obj->count('host') . " unique hosts\n";
print "10.0.11.1 made " . $obj->count('host' => '10.0.11.1')
" requests\n";
=back
Documenting: BUGS
BUGS gives us a public "to do" list (nobody wants to admit to
having bugs)
=head1 BUGS
=over 4
=item *
Only parses Apache combined log format
=item *
Stores the entire file in memory
=item *
Does not cache data outside of the object (re-parse each time)
=back
What We Just Did: Documentation
We wrote the documentation before the code
This helped us solidify the API before we wrote it
Now our module is completely documented (to date)
When we do 'make install', it will "manify" the POD and make a
real manpage
We did a good enough job in the docs for somebody not
familiar with the code to use it
This is the Perl Virtue of laziness
We don't have to answer any questions about our code
We just say, "RTFM"
Admission of Guilt
I didn't write all of the documentation before I began coding
'init' and 'file' were after-thoughts
But I documented them after realizing I needed them
Documenting, testing, coding is not linear but recursive and
sometimes random: that's ok (as long as it gets done)
Adding New Functionality in test.pl
Let's edit our test.pl file (remember to update test count)
## create a file
open TMP, ">.tmp_$$" or die "$!\n";
print TMP <<'_FILE_';
10.1.1.11 - - [24/Mar/2003:08:25:10 -0700] "GET /nonexistent HTTP/1.1" \
404 300 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; (R1 1.3))"
10.1.1.11 - - [24/Mar/2003:08:25:15 -0700] "GET /nonexistentential HTTP/1.1" \
404 300 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; (R1 1.3))"
10.1.1.11 - - [24/Mar/2003:08:25:24 -0700] "GET /foo/bar.pl HTTP/1.1" \
200 9328 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; (R1 1.3))"
_FILE_
close TMP;
## now further tests
undef $obj;
ok( $obj = new Log::Apache::Object(file => ".tmp_$$") );
ok( $obj->parse() );
ok( $obj->count, 3 );
ok( $obj->count('status'), 2 ); ## two 404's and one 200
ok( $obj->count('status' => '404'), 2 );
ok( $obj->count('host'
=> '10.1.1.11'), 3 );
## clean up
END { unlink ".tmp_$$" }
What We Just Did: Testing
We wrote the tests before the code
This helped us clarify the API before we wrote it
Our test interface also serves as a mock-up or reference for
development on the code
Tests help us gauge our progress on the actual module
Writing tests is easy and fast!
...usually.
Writing good tests is always satisfying and productive
Writing Code: 'new' and 'init'
Now let's write some code!
our %agg
= ();
our @lines = ();
sub new {
my $self = { };
my $proto = shift;
my $class = ref($proto) || $proto;
bless $self, $class;
$self->init(@_);
return $self;
}
sub init {
my $self = shift;
my %args = @_;
$self->file($args{'file'});
}
Writing Code: 'file' method
This could also be done with $AUTOLOAD
sub file {
my $self = shift;
my $file = shift;
return $self->{'_file'} = ( defined $file
? $file
: $self->{'_file'} );
}
Writing Code: 'parse' method
sub parse {
my $self = shift;
my %data = ();
die "Must specify file to parse\n"
unless $self->file;
open FILE, $self->file
or die "Could not open " . $self->file . ": $!\n";
while( <FILE> ) {
chomp;
next unless ($data{host}, $data{user}, $data{timestamp}, $data{method},
$data{uri}, $data{protocol}, $data{status}, $data{bytes},
$data{referer}, $data{agent}) = m!
^(\S+)\s+\S+\s+
## host, logname (not captured)
(\S+)\s+\[(.+?)\]\s+
## user, timestamp
"(\S+)\s+(\S+)\s+(\S+)"\s+ ## request URI, method, protocol
(\d+)\s+(\d+)\s+
## status, bytes sent
"(.+?)"\s+"(.+?)"$!x;
## referer, agent
## some aggregate data
$agg{host}->{$data{host}}++;
$agg{uri}->{$data{uri}}++;
$agg{status}->{$data{status}}++;
$agg{agent}->{$data{agent}}++;
## store this line
push @lines, \%data;
}
close FILE;
}
$agg{user}->{$data{user}}++;
$agg{protocol}->{$data{protocol}}++;
$agg{referer}->{$data{referer}}++;
Writing Code: 'count' method
sub count {
my $self = shift;
my $field = shift;
my $value = shift;
return scalar @lines unless $field;
return scalar keys %{$agg{$field}} unless $value;
return $agg{$field}->{$value};
}
Some Future Improvements
Future improvements:
split URI data into uri and arguments
split referer data into host and uri
split agent data into browser/version/realname/os
make 'count' method handle SQL-like syntax
write request data to disk as we read it (less memory)
store request data in a binary format (less disk space)
Future benefits of having regression tests:
we won't have to change our tests!
we'll be able to tell if adding functionality breaks anything
immediately and we'll likely know why
Conclusion
Practical Take-away
Testing is rewarding and gives instant feedback to how you're
progressing and how accurate your code is
Having regression tests enables you to refactor your code
whenever you feel like it without fear of breaking something
Writing documentation forces your brain to move from abstract
to concrete; this process uncovers potential logic problems
Concentrating on the API in testing and documentation helps
you create a cleaner API before you begin coding
You will save more total time by writing tests and
documentation up front and you will free your code to be
owned by someone else later
Philosophical Take-Away
The true quality of a module lies not in clever algorithms,
speed, or obscure use of the Perl flip-flop operator, but in
clean design and in a well-thought-out API
Testing and documenting before coding gives you a concrete
plan you can turn to while coding; you also are forced to face
and resolve logic problems and API conflicts long before you
even code. This saves the most time possible because you
hardly ever have to backward engineer anything
Testing and documenting before coding provides you with a
suite of regression tests that allows you to refactor your code
anytime and as often as you want; you get to keep your
confidence that the module still works because you have
instant feedback when something breaks. This lets you make
bold, sweeping changes or allows you to take advantage of
new technology confidently.
You'll have more friends if you test and document your code
