1N/ATest::Tutorial - A tutorial about writing really basic tests
1N/AI<AHHHHHHH!!!! NOT TESTING! Anything but testing!
1N/ABeat me, whip me, send me to Detroit, but don't make
1N/AI<Besides, I don't know how to write the damned things.>
1N/AIs this you? Is writing tests right up there with writing
1N/Adocumentation and having your fingernails pulled out? Did you open up
1N/A ######## We start with some black magic
1N/Aand decide that's quite enough for you?
1N/AIt's ok. That's all gone now. We've done all the black magic for
1N/Ayou. And here are the tricks...
1N/A=head2 Nuts and bolts of testing.
1N/AHere's the most basic test program.
1N/A print 1 + 1 == 2 ? "ok 1\n" : "not ok 1\n";
1N/Asince 1 + 1 is 2, it prints:
1N/AWhat this says is: C<1..1> "I'm going to run one test." [1] C<ok 1>
1N/A"The first test passed". And that's about all magic there is to
1N/Atesting. Your basic unit of testing is the I<ok>. For each thing you
1N/Atest, an C<ok> is printed. Simple. B<Test::Harness> interprets your test
1N/Aresults to determine if you succeeded or failed (more on that later).
1N/AWriting all these print statements rapidly gets tedious. Fortunately,
1N/Athere's B<Test::Simple>. It has one function, C<ok()>.
1N/A use Test::Simple tests => 1;
1N/Aand that does the same thing as the code above. C<ok()> is the backbone
1N/Aof Perl testing, and we'll be using it instead of roll-your-own from
1N/Ahere on. If C<ok()> gets a true value, the test passes. False, it
1N/A use Test::Simple tests => 2;
1N/A # Looks like you failed 1 tests of 2.
1N/AC<1..2> "I'm going to run two tests." This number is used to ensure
1N/Ayour test program ran all the way through and didn't die or skip some
1N/Atests. C<ok 1> "The first test passed." C<not ok 2> "The second test
1N/Afailed". Test::Simple helpfully prints out some extra commentary about
1N/AIt's not scary. Come, hold my hand. We're going to give an example
1N/Aof testing a module. For our example, we'll be testing a date
1N/Alibrary, B<Date::ICal>. It's on CPAN, so download a copy and follow
1N/A=head2 Where to start?
1N/AThis is the hardest part of testing, where do you start? People often
1N/Aget overwhelmed at the apparent enormity of the task of testing a
1N/Awhole module. Best place to start is at the beginning. Date::ICal is
1N/Aan object-oriented module, and that means you start by making an
1N/Aobject. So we test C<new()>.
1N/A use Test::Simple tests => 2;
1N/A my $ical = Date::ICal->new; # create an object
1N/A ok( defined $ical ); # check that we got something
1N/A ok( $ical->isa('Date::ICal') ); # and it's the right class
1N/Arun that and you should get:
1N/Acongratulations, you've written your first useful test.
1N/AThat output isn't terribly descriptive, is it? When you have two
1N/Atests you can figure out which one is #2, but what if you have 102?
1N/AEach test can be given a little descriptive name as the second
1N/A use Test::Simple tests => 2;
1N/A ok( defined $ical, 'new() returned something' );
1N/A ok( $ical->isa('Date::ICal'), " and it's the right class" );
1N/A ok 1 - new() returned something
1N/A ok 2 - and it's the right class
1N/A=head2 Test the manual
1N/ASimplest way to build up a decent testing suite is to just test what
1N/Athe manual says it does. [3] Let's pull something out of the
1N/A use Test::Simple tests => 8;
1N/A $ical = Date::ICal->new( year => 1964, month => 10, day => 16,
1N/A hour => 16, min => 12, sec => 47,
1N/A ok( defined $ical, 'new() returned something' );
1N/A ok( $ical->isa('Date::ICal'), " and it's the right class" );
1N/A ok( $ical->sec == 47, ' sec()' );
1N/A ok( $ical->min == 12, ' min()' );
1N/A ok( $ical->hour == 16, ' hour()' );
1N/A ok( $ical->day == 17, ' day()' );
1N/A ok( $ical->month == 10, ' month()' );
1N/A ok( $ical->year == 1964, ' year()' );
1N/Arun that and you get:
1N/A ok 1 - new() returned something
1N/A ok 2 - and it's the right class
1N/A # Failed test (- at line 16)
1N/A # Looks like you failed 1 tests of 8.
1N/AWhoops, a failure! [4] Test::Simple helpfully lets us know on what line
1N/Athe failure occured, but not much else. We were supposed to get 17,
1N/Abut we didn't. What did we get?? Dunno. We'll have to re-run the
1N/Atest in the debugger or throw in some print statements to find out.
1N/AInstead, we'll switch from B<Test::Simple> to B<Test::More>. B<Test::More>
1N/Adoes everything B<Test::Simple> does, and more! In fact, Test::More does
1N/Athings I<exactly> the way Test::Simple does. You can literally swap
1N/ATest::Simple out and put Test::More in its place. That's just what
1N/ATest::More does more than Test::Simple. The most important difference
1N/Aat this point is it provides more informative ways to say "ok".
1N/AAlthough you can write almost any test with a generic C<ok()>, it
1N/Acan't tell you what went wrong. Instead, we'll use the C<is()>
1N/Afunction, which lets us declare that something is supposed to be the
1N/Asame as something else:
1N/A use Test::More tests => 8;
1N/A $ical = Date::ICal->new( year => 1964, month => 10, day => 16,
1N/A hour => 16, min => 12, sec => 47,
1N/A ok( defined $ical, 'new() returned something' );
1N/A ok( $ical->isa('Date::ICal'), " and it's the right class" );
1N/A is( $ical->sec, 47, ' sec()' );
1N/A is( $ical->min, 12, ' min()' );
1N/A is( $ical->hour, 16, ' hour()' );
1N/A is( $ical->day, 17, ' day()' );
1N/A is( $ical->month, 10, ' month()' );
1N/A is( $ical->year, 1964, ' year()' );
1N/A"Is C<$ical-E<gt>sec> 47?" "Is C<$ical-E<gt>min> 12?" With C<is()> in place,
1N/Ayou get some more information
1N/A ok 1 - new() returned something
1N/A ok 2 - and it's the right class
1N/A # Failed test (- at line 16)
1N/A # Looks like you failed 1 tests of 8.
1N/Aletting us know that C<$ical-E<gt>day> returned 16, but we expected 17. A
1N/Aquick check shows that the code is working fine, we made a mistake
1N/Awhen writing up the tests. Just change it to:
1N/A is( $ical->day, 16, ' day()' );
1N/Aand everything works.
1N/ASo any time you're doing a "this equals that" sort of test, use C<is()>.
1N/AIt even works on arrays. The test is always in scalar context, so you
1N/Acan test how many elements are in a list this way. [5]
1N/A is( @foo, 5, 'foo has 5 elements' );
1N/A=head2 Sometimes the tests are wrong
1N/AWhich brings us to a very important lesson. Code has bugs. Tests are
1N/Acode. Ergo, tests have bugs. A failing test could mean a bug in the
1N/Acode, but don't discount the possibility that the test is wrong.
1N/AOn the flip side, don't be tempted to prematurely declare a test
1N/Aincorrect just because you're having trouble finding the bug.
1N/AInvalidating a test isn't something to be taken lightly, and don't use
1N/Ait as a cop out to avoid work.
1N/A=head2 Testing lots of values
1N/AWe're going to be wanting to test a lot of dates here, trying to trick
1N/Athe code with lots of different edge cases. Does it work before 1970?
1N/AAfter 2038? Before 1904? Do years after 10,000 give it trouble?
1N/ADoes it get leap years right? We could keep repeating the code above,
1N/A use Test::More tests => 32;
1N/A # An ICal string And the year, month, date
1N/A # hour, minute and second we expect.
1N/A '19971024T120000' => # from the docs.
1N/A [ 1997, 10, 24, 12, 0, 0 ],
1N/A '20390123T232832' => # after the Unix epoch
1N/A [ 2039, 1, 23, 23, 28, 32 ],
1N/A '19671225T000000' => # before the Unix epoch
1N/A [ 1967, 12, 25, 0, 0, 0 ],
1N/A '18990505T232323' => # before the MacOS epoch
1N/A [ 1899, 5, 5, 23, 23, 23 ],
1N/A while( my($ical_str, $expect) = each %ICal_Dates ) {
1N/A my $ical = Date::ICal->new( ical => $ical_str );
1N/A ok( defined $ical, "new(ical => '$ical_str')" );
1N/A ok( $ical->isa('Date::ICal'), " and it's the right class" );
1N/A is( $ical->year, $expect->[0], ' year()' );
1N/A is( $ical->month, $expect->[1], ' month()' );
1N/A is( $ical->day, $expect->[2], ' day()' );
1N/A is( $ical->hour, $expect->[3], ' hour()' );
1N/A is( $ical->min, $expect->[4], ' min()' );
1N/A is( $ical->sec, $expect->[5], ' sec()' );
1N/ASo now we can test bunches of dates by just adding them to
1N/AC<%ICal_Dates>. Now that it's less work to test with more dates, you'll
1N/Abe inclined to just throw more in as you think of them.
1N/AOnly problem is, every time we add to that we have to keep adjusting
1N/Athe C<use Test::More tests =E<gt> ##> line. That can rapidly get
1N/Aannoying. There's two ways to make this work better.
1N/AFirst, we can calculate the plan dynamically using the C<plan()>
1N/A # For each key in the hash we're running 8 tests.
1N/A plan tests => keys %ICal_Dates * 8;
1N/AOr to be even more flexible, we use C<no_plan>. This means we're just
1N/Arunning some tests, don't know how many. [6]
1N/A use Test::More 'no_plan'; # instead of tests => 32
1N/Anow we can just add tests and not have to do all sorts of math to
1N/Afigure out how many we're running.
1N/A=head2 Informative names
1N/ATake a look at this line here
1N/A ok( defined $ical, "new(ical => '$ical_str')" );
1N/Awe've added more detail about what we're testing and the ICal string
1N/Aitself we're trying out to the name. So you get results like:
1N/A ok 25 - new(ical => '19971024T120000')
1N/A ok 26 - and it's the right class
1N/Aif something in there fails, you'll know which one it was and that
1N/Awill make tracking down the problem easier. So try to put a bit of
1N/Adebugging information into the test names.
1N/ADescribe what the tests test, to make debugging a failed test easier
1N/Afor you or for the next person who runs your test.
1N/A=head2 Skipping tests
1N/APoking around in the existing Date::ICal tests, I found this in
1N/A use Test::More tests => 7;
1N/A # Make sure epoch time is being handled sanely.
1N/A my $t1 = Date::ICal->new( epoch => 0 );
1N/A is( $t1->epoch, 0, "Epoch time of 0" );
1N/A # XXX This will only work on unix systems.
1N/A is( $t1->ical, '19700101Z', " epoch to ical" );
1N/A is( $t1->year, 1970, " year()" );
1N/A is( $t1->month, 1, " month()" );
1N/A is( $t1->day, 1, " day()" );
1N/A # like the tests above, but starting with ical instead of epoch
1N/A my $t2 = Date::ICal->new( ical => '19700101Z' );
1N/A is( $t2->ical, '19700101Z', "Start of epoch in ICal notation" );
1N/A is( $t2->epoch, 0, " and back to ICal" );
1N/AThe beginning of the epoch is different on most non-Unix operating
1N/Asystems [8]. Even though Perl smooths out the differences for the most
1N/Apart, certain ports do it differently. MacPerl is one off the top of
1N/Amy head. [9] We I<know> this will never work on MacOS. So rather than
1N/Ajust putting a comment in the test, we can explicitly say it's never
1N/Agoing to work and skip the test.
1N/A use Test::More tests => 7;
1N/A # Make sure epoch time is being handled sanely.
1N/A my $t1 = Date::ICal->new( epoch => 0 );
1N/A is( $t1->epoch, 0, "Epoch time of 0" );
1N/A skip('epoch to ICal not working on MacOS', 6)
1N/A is( $t1->ical, '19700101Z', " epoch to ical" );
1N/A is( $t1->year, 1970, " year()" );
1N/A is( $t1->month, 1, " month()" );
1N/A is( $t1->day, 1, " day()" );
1N/A # like the tests above, but starting with ical instead of epoch
1N/A my $t2 = Date::ICal->new( ical => '19700101Z' );
1N/A is( $t2->ical, '19700101Z', "Start of epoch in ICal notation" );
1N/A is( $t2->epoch, 0, " and back to ICal" );
1N/AA little bit of magic happens here. When running on anything but
1N/AMacOS, all the tests run normally. But when on MacOS, C<skip()> causes
1N/Athe entire contents of the SKIP block to be jumped over. It's never
1N/Arun. Instead, it prints special output that tells Test::Harness that
1N/Athe tests have been skipped.
1N/A ok 1 - Epoch time of 0
1N/A ok 2 # skip epoch to ICal not working on MacOS
1N/A ok 3 # skip epoch to ICal not working on MacOS
1N/A ok 4 # skip epoch to ICal not working on MacOS
1N/A ok 5 # skip epoch to ICal not working on MacOS
1N/A ok 6 # skip epoch to ICal not working on MacOS
1N/A ok 7 # skip epoch to ICal not working on MacOS
1N/AThis means your tests won't fail on MacOS. This means less emails
1N/Afrom MacPerl users telling you about failing tests that you know will
1N/Anever work. You've got to be careful with skip tests. These are for
1N/Atests which don't work and I<never will>. It is not for skipping
1N/Agenuine bugs (we'll get to that in a moment).
1N/AThe tests are wholly and completely skipped. [10] This will work.
1N/A skip("I don't wanna die!");
1N/A die, die, die, die, die;
1N/AThumbing through the Date::ICal man page, I came across this:
1N/A $ical_string = $ical->ical;
1N/A Retrieves, or sets, the date on the object, using any
1N/A"Retrieves or sets". Hmmm, didn't see a test for using C<ical()> to set
1N/Athe date in the Date::ICal test suite. So I'll write one.
1N/A use Test::More tests => 1;
1N/A my $ical = Date::ICal->new;
1N/A $ical->ical('20201231Z');
1N/A is( $ical->ical, '20201231Z', 'Setting via ical()' );
1N/A not ok 1 - Setting via ical()
1N/A # Failed test (- at line 6)
1N/A # got: '20010814T233649Z'
1N/A # expected: '20201231Z'
1N/A # Looks like you failed 1 tests of 1.
1N/AWhoops! Looks like it's unimplemented. Let's assume we don't have
1N/Athe time to fix this. [11] Normally, you'd just comment out the test
1N/Aand put a note in a todo list somewhere. Instead, we're going to
1N/Aexplicitly state "this test will fail" by wrapping it in a C<TODO> block.
1N/A use Test::More tests => 1;
1N/A local $TODO = 'ical($ical) not yet implemented';
1N/A my $ical = Date::ICal->new;
1N/A $ical->ical('20201231Z');
1N/A is( $ical->ical, '20201231Z', 'Setting via ical()' );
1N/ANow when you run, it's a little different:
1N/A not ok 1 - Setting via ical() # TODO ical($ical) not yet implemented
1N/A # got: '20010822T201551Z'
1N/A # expected: '20201231Z'
1N/ATest::More doesn't say "Looks like you failed 1 tests of 1". That '#
1N/ATODO' tells Test::Harness "this is supposed to fail" and it treats a
1N/Afailure as a successful test. So you can write tests even before
1N/Ayou've fixed the underlying code.
1N/AIf a TODO test passes, Test::Harness will report it "UNEXPECTEDLY
1N/ASUCCEEDED". When that happens, you simply remove the TODO block with
1N/AC<local $TODO> and turn it into a real test.
1N/A=head2 Testing with taint mode.
1N/ATaint mode is a funny thing. It's the globalest of all global
1N/Afeatures. Once you turn it on it effects I<all> code in your program
1N/Aand I<all> modules used (and all the modules they use). If a single
1N/Apiece of code isn't taint clean, the whole thing explodes. With that
1N/Ain mind, it's very important to ensure your module works under taint
1N/AIt's very simple to have your tests run under taint mode. Just throw
1N/Aa C<-T> into the C<#!> line. Test::Harness will read the switches
1N/Ain C<#!> and use them to run your tests.
1N/A use Test::More 'no_plan';
1N/ASo when you say C<make test> it will be run with taint mode and
1N/AThe first number doesn't really mean anything, but it has to be 1.
1N/AIt's the second number that's important.
1N/AFor those following along at home, I'm using version 1.31. It has
1N/Asome bugs, which is good -- we'll uncover them with our tests.
1N/AYou can actually take this one step further and test the manual
1N/Aitself. Have a look at B<Test::Inline> (formerly B<Pod::Tests>).
1N/AYes, there's a mistake in the test suite. What! Me, contrived?
1N/AWe'll get to testing the contents of lists later.
1N/ABut what happens if your test program dies halfway through?! Since we
1N/Adidn't say how many tests we're going to run, how can we know it
1N/Afailed? No problem, Test::More employs some magic to catch that death
1N/Aand turn the test into a failure, even if every test passed up to that
1N/AI cleaned it up a little.
1N/AMost Operating Systems record time as the number of seconds since a
1N/Acertain date. This date is the beginning of the epoch. Unix's starts
1N/Aat midnight January 1st, 1970 GMT.
1N/AMacOS's epoch is midnight January 1st, 1904. VMS's is midnight,
1N/ANovember 17th, 1858, but vmsperl emulates the Unix epoch so it's not a
1N/AAs long as the code inside the SKIP block at least compiles. Please
1N/Adon't ask how. No, it's not a filter.
1N/ADo NOT be tempted to use TODO tests as a way to avoid fixing simple
1N/AMichael G Schwern E<lt>schwern@pobox.comE<gt> and the perl-qa dancers!
1N/ACopyright 2001 by Michael G Schwern E<lt>schwern@pobox.comE<gt>.
1N/AThis documentation is free; you can redistribute it
and/or modify it
1N/Aunder the same terms as Perl itself.
1N/AIrrespective of its distribution, all code examples in these files
1N/Aare hereby placed into the public domain. You are permitted and
1N/Aencouraged to use this code in your own programs for fun
1N/Aor for profit as you see fit. A simple comment in the code giving
1N/Acredit would be courteous but is not required.