TAP::Parser - Parse TAP output

  1. VERSION
  2. SYNOPSIS
  3. DESCRIPTION
  4. METHODS
    1. Class Methods
    2. Instance Methods
  5. INDIVIDUAL RESULTS
    1. Result types
    2. Common type methods
    3. plan methods
    4. pragma methods
    5. comment methods
    6. bailout methods
    7. unknown methods
    8. test methods
  6. TOTAL RESULTS
    1. Individual Results
    2. Pragmas
    3. Summary Results
    4. ignore_exit
  7. CALLBACKS
  8. TAP GRAMMAR
  9. BACKWARDS COMPATIBILITY
    1. Differences
  10. SUBCLASSING
    1. Parser Components
  11. ACKNOWLEDGMENTS
  12. AUTHORS
  13. BUGS
  14. COPYRIGHT & LICENSE

VERSION

Version 3.44

SYNOPSIS

use TAP::Parser;

my $parser = TAP::Parser->new( { source => $source } );

while ( my $result = $parser->next ) {
    print $result->as_string;
}

DESCRIPTION

TAP::Parser is designed to produce a proper parse of TAP output. For an example of how to run tests through this module, see the simple harnesses examples/.

There's a wiki dedicated to the Test Anything Protocol:

http://testanything.org

It includes the TAP::Parser Cookbook:

http://testanything.org/testing-with-tap/perl/tap::parser-cookbook.html

METHODS

Class Methods

new

my $parser = TAP::Parser->new(\%args);

Returns a new TAP::Parser object.

The arguments should be a hashref with one of the following keys:

The following keys are optional.

Instance Methods

next

my $parser = TAP::Parser->new( { source => $file } );
while ( my $result = $parser->next ) {
    print $result->as_string, "\n";
}

This method returns the results of the parsing, one result at a time. Note that it is destructive. You can't rewind and examine previous results.

If callbacks are used, they will be issued before this call returns.

Each result returned is a subclass of TAP::Parser::Result. See that module and related classes for more information on how to use them.

run

$parser->run;

This method merely runs the parser and parses all of the TAP.

make_grammar

Make a new TAP::Parser::Grammar object and return it. Passes through any arguments given.

The grammar_class can be customized, as described in "new".

make_result

Make a new TAP::Parser::Result object using the parser's TAP::Parser::ResultFactory, and return it. Passes through any arguments given.

The result_factory_class can be customized, as described in "new".

make_iterator_factory

NEW to 3.18.

Make a new TAP::Parser::IteratorFactory object and return it. Passes through any arguments given.

iterator_factory_class can be customized, as described in "new".

INDIVIDUAL RESULTS

If you've read this far in the docs, you've seen this:

while ( my $result = $parser->next ) {
    print $result->as_string;
}

Each result returned is a TAP::Parser::Result subclass, referred to as result types.

Result types

Basically, you fetch individual results from the TAP. The six types, with examples of each, are as follows:

Each result fetched is a result object of a different type. There are common methods to each result object and different types may have methods unique to their type. Sometimes a type method may be overridden in a subclass, but its use is guaranteed to be identical.

Common type methods

type

Returns the type of result, such as comment or test.

as_string

Prints a string representation of the token. This might not be the exact output, however. Tests will have test numbers added if not present, TODO and SKIP directives will be capitalized and, in general, things will be cleaned up. If you need the original text for the token, see the raw method.

raw

Returns the original line of text which was parsed.

is_plan

Indicates whether or not this is the test plan line.

is_test

Indicates whether or not this is a test line.

is_comment

Indicates whether or not this is a comment. Comments will generally only appear in the TAP stream if STDERR is merged to STDOUT. See the merge option.

is_bailout

Indicates whether or not this is bailout line.

is_yaml

Indicates whether or not the current item is a YAML block.

is_unknown

Indicates whether or not the current line could be parsed.

is_ok

if ( $result->is_ok ) { ... }

Reports whether or not a given result has passed. Anything which is not a test result returns true. This is merely provided as a convenient shortcut which allows you to do this:

my $parser = TAP::Parser->new( { source => $source } );
while ( my $result = $parser->next ) {
    # only print failing results
    print $result->as_string unless $result->is_ok;
}

plan methods

if ( $result->is_plan ) { ... }

If the above evaluates as true, the following methods will be available on the $result object.

plan

if ( $result->is_plan ) {
   print $result->plan;
}

This is merely a synonym for as_string.

directive

my $directive = $result->directive;

If a SKIP directive is included with the plan, this method will return it.

1..0 # SKIP: why bother?

explanation

my $explanation = $result->explanation;

If a SKIP directive was included with the plan, this method will return the explanation, if any.

pragma methods

if ( $result->is_pragma ) { ... }

If the above evaluates as true, the following methods will be available on the $result object.

pragmas

Returns a list of pragmas each of which is a + or - followed by the pragma name.

comment methods

if ( $result->is_comment ) { ... }

If the above evaluates as true, the following methods will be available on the $result object.

comment

if ( $result->is_comment ) {
    my $comment = $result->comment;
    print "I have something to say:  $comment";
}

bailout methods

if ( $result->is_bailout ) { ... }

If the above evaluates as true, the following methods will be available on the $result object.

explanation

if ( $result->is_bailout ) {
    my $explanation = $result->explanation;
    print "We bailed out because ($explanation)";
}

If, and only if, a token is a bailout token, you can get an "explanation" via this method. The explanation is the text after the mystical "Bail out!" words which appear in the tap output.

unknown methods

if ( $result->is_unknown ) { ... }

There are no unique methods for unknown results.

test methods

if ( $result->is_test ) { ... }

If the above evaluates as true, the following methods will be available on the $result object.

ok

my $ok = $result->ok;

Returns the literal text of the ok or not ok status.

number

my $test_number = $result->number;

Returns the number of the test, even if the original TAP output did not supply that number.

description

my $description = $result->description;

Returns the description of the test, if any. This is the portion after the test number but before the directive.

directive

my $directive = $result->directive;

Returns either TODO or SKIP if either directive was present for a test line.

explanation

my $explanation = $result->explanation;

If a test had either a TODO or SKIP directive, this method will return the accompanying explanation, if present.

not ok 17 - 'Pigs can fly' # TODO not enough acid

For the above line, the explanation is not enough acid.

is_ok

if ( $result->is_ok ) { ... }

Returns a boolean value indicating whether or not the test passed. Remember that for TODO tests, the test always passes.

Note: this was formerly passed. The latter method is deprecated and will issue a warning.

is_actual_ok

if ( $result->is_actual_ok ) { ... }

Returns a boolean value indicating whether or not the test passed, regardless of its TODO status.

Note: this was formerly actual_passed. The latter method is deprecated and will issue a warning.

is_unplanned

if ( $test->is_unplanned ) { ... }

If a test number is greater than the number of planned tests, this method will return true. Unplanned tests will always return false for is_ok, regardless of whether or not the test has_todo (see TAP::Parser::Result::Test for more information about this).

has_skip

if ( $result->has_skip ) { ... }

Returns a boolean value indicating whether or not this test had a SKIP directive.

has_todo

if ( $result->has_todo ) { ... }

Returns a boolean value indicating whether or not this test had a TODO directive.

Note that TODO tests always pass. If you need to know whether or not they really passed, check the is_actual_ok method.

in_todo

if ( $parser->in_todo ) { ... }

True while the most recent result was a TODO. Becomes true before the TODO result is returned and stays true until just before the next non- TODO test is returned.

TOTAL RESULTS

After parsing the TAP, there are many methods available to let you dig through the results and determine what is meaningful to you.

Individual Results

These results refer to individual tests which are run.

passed

my @passed = $parser->passed; # the test numbers which passed
my $passed = $parser->passed; # the number of tests which passed

This method lets you know which (or how many) tests passed. If a test failed but had a TODO directive, it will be counted as a passed test.

failed

my @failed = $parser->failed; # the test numbers which failed
my $failed = $parser->failed; # the number of tests which failed

This method lets you know which (or how many) tests failed. If a test passed but had a TODO directive, it will NOT be counted as a failed test.

actual_passed

# the test numbers which actually passed
my @actual_passed = $parser->actual_passed;

# the number of tests which actually passed
my $actual_passed = $parser->actual_passed;

This method lets you know which (or how many) tests actually passed, regardless of whether or not a TODO directive was found.

actual_ok

This method is a synonym for actual_passed.

actual_failed

# the test numbers which actually failed
my @actual_failed = $parser->actual_failed;

# the number of tests which actually failed
my $actual_failed = $parser->actual_failed;

This method lets you know which (or how many) tests actually failed, regardless of whether or not a TODO directive was found.

todo

my @todo = $parser->todo; # the test numbers with todo directives
my $todo = $parser->todo; # the number of tests with todo directives

This method lets you know which (or how many) tests had TODO directives.

todo_passed

# the test numbers which unexpectedly succeeded
my @todo_passed = $parser->todo_passed;

# the number of tests which unexpectedly succeeded
my $todo_passed = $parser->todo_passed;

This method lets you know which (or how many) tests actually passed but were declared as "TODO" tests.

todo_failed

# deprecated in favor of 'todo_passed'.  This method was horribly misnamed.

This was a badly misnamed method. It indicates which TODO tests unexpectedly succeeded. Will now issue a warning and call todo_passed.

skipped

my @skipped = $parser->skipped; # the test numbers with SKIP directives
my $skipped = $parser->skipped; # the number of tests with SKIP directives

This method lets you know which (or how many) tests had SKIP directives.

Pragmas

pragma

Get or set a pragma. To get the state of a pragma:

if ( $p->pragma('strict') ) {
    # be strict
}

To set the state of a pragma:

$p->pragma('strict', 1); # enable strict mode

pragmas

Get a list of all the currently enabled pragmas:

my @pragmas_enabled = $p->pragmas;

Summary Results

These results are "meta" information about the total results of an individual test program.

plan

my $plan = $parser->plan;

Returns the test plan, if found.

good_plan

Deprecated. Use is_good_plan instead.

is_good_plan

if ( $parser->is_good_plan ) { ... }

Returns a boolean value indicating whether or not the number of tests planned matches the number of tests run.

Note: this was formerly good_plan. The latter method is deprecated and will issue a warning.

And since we're on that subject ...

tests_planned

print $parser->tests_planned;

Returns the number of tests planned, according to the plan. For example, a plan of '1..17' will mean that 17 tests were planned.

tests_run

print $parser->tests_run;

Returns the number of tests which actually were run. Hopefully this will match the number of $parser->tests_planned.

skip_all

Returns a true value (actually the reason for skipping) if all tests were skipped.

start_time

Returns the wall-clock time when the Parser was created.

end_time

Returns the wall-clock time when the end of TAP input was seen.

start_times

Returns the CPU times (like "times" in perlfunc when the Parser was created.

end_times

Returns the CPU times (like "times" in perlfunc when the end of TAP input was seen.

has_problems

if ( $parser->has_problems ) {
    ...
}

This is a 'catch-all' method which returns true if any tests have currently failed, any TODO tests unexpectedly succeeded, or any parse errors occurred.

version

$parser->version;

Once the parser is done, this will return the version number for the parsed TAP. Version numbers were introduced with TAP version 13 so if no version number is found version 12 is assumed.

exit

$parser->exit;

Once the parser is done, this will return the exit status. If the parser ran an executable, it returns the exit status of the executable.

wait

$parser->wait;

Once the parser is done, this will return the wait status. If the parser ran an executable, it returns the wait status of the executable. Otherwise, this merely returns the exit status.

ignore_exit

$parser->ignore_exit(1);

Tell the parser to ignore the exit status from the test when determining whether the test passed. Normally tests with non-zero exit status are considered to have failed even if all individual tests passed. In cases where it is not possible to control the exit value of the test script use this option to ignore it.

parse_errors

my @errors = $parser->parse_errors; # the parser errors
my $errors = $parser->parse_errors; # the number of parser_errors

Fortunately, all TAP output is perfect. In the event that it is not, this method will return parser errors. Note that a junk line which the parser does not recognize is not an error. This allows this parser to handle future versions of TAP. The following are all TAP errors reported by the parser:

get_select_handles

Get an a list of file handles which can be passed to select to determine the readiness of this parser.

delete_spool

Delete and return the spool.

my $fh = $parser->delete_spool;

CALLBACKS

As mentioned earlier, a "callback" key may be added to the TAP::Parser constructor. If present, each callback corresponding to a given result type will be called with the result as the argument if the run method is used. The callback is expected to be a subroutine reference (or anonymous subroutine) which is invoked with the parser result as its argument.

my %callbacks = (
    test    => \&test_callback,
    plan    => \&plan_callback,
    comment => \&comment_callback,
    bailout => \&bailout_callback,
    unknown => \&unknown_callback,
);

my $aggregator = TAP::Parser::Aggregator->new;
for my $file ( @test_files ) {
    my $parser = TAP::Parser->new(
        {
            source    => $file,
            callbacks => \%callbacks,
        }
    );
    $parser->run;
    $aggregator->add( $file, $parser );
}

Callbacks may also be added like this:

$parser->callback( test => \&test_callback );
$parser->callback( plan => \&plan_callback );

The following keys allowed for callbacks. These keys are case-sensitive.

TAP GRAMMAR

If you're looking for an EBNF grammar, see TAP::Parser::Grammar.

BACKWARDS COMPATIBILITY

The Perl-QA list attempted to ensure backwards compatibility with Test::Harness. However, there are some minor differences.

Differences

SUBCLASSING

If you find you need to provide custom functionality (as you would have using Test::Harness::Straps), you're in luck: TAP::Parser and friends are designed to be easily plugged-into and/or subclassed.

Before you start, it's important to know a few things:

  1. All TAP::* objects inherit from TAP::Object.

  2. Many TAP::* classes have a SUBCLASSING section to guide you.

  3. Note that TAP::Parser is designed to be the central "maker" - ie: it is responsible for creating most new objects in the TAP::Parser::* namespace.

    This makes it possible for you to have a single point of configuring what subclasses should be used, which means that in many cases you'll find you only need to sub-class one of the parser's components.

    The exception to this rule are SourceHandlers & Iterators, but those are both created with customizable IteratorFactory.

  4. By subclassing, you may end up overriding undocumented methods. That's not a bad thing per se, but be forewarned that undocumented methods may change without warning from one release to the next - we cannot guarantee backwards compatibility. If any documented method needs changing, it will be deprecated first, and changed in a later release.

Parser Components

Sources

A TAP parser consumes input from a single raw source of TAP, which could come from anywhere (a file, an executable, a database, an IO handle, a URI, etc..). The source gets bundled up in a TAP::Parser::Source object which gathers some meta data about it. The parser then uses a TAP::Parser::IteratorFactory to determine which TAP::Parser::SourceHandler to use to turn the raw source into a stream of TAP by way of "Iterators".

If you simply want TAP::Parser to handle a new source of TAP you probably don't need to subclass TAP::Parser itself. Rather, you'll need to create a new TAP::Parser::SourceHandler class, and just plug it into the parser using the sources param to "new". Before you start writing one, read through TAP::Parser::IteratorFactory to get a feel for how the system works first.

If you find you really need to use your own iterator factory you can still do so without sub-classing TAP::Parser by setting "iterator_factory_class".

If you just need to customize the objects on creation, subclass TAP::Parser and override "make_iterator_factory".

Note that make_source & make_perl_source have been DEPRECATED and are now removed.

Iterators

A TAP parser uses iterators to loop through the stream of TAP read in from the source it was given. There are a few types of Iterators available by default, all sub-classes of TAP::Parser::Iterator. Choosing which iterator to use is the responsibility of the iterator factory, though it simply delegates to the Source Handler it uses.

If you're writing your own TAP::Parser::SourceHandler, you may need to create your own iterators too. If so you'll need to subclass TAP::Parser::Iterator.

Note that "make_iterator" has been DEPRECATED and is now removed.

Results

A TAP parser creates TAP::Parser::Results as it iterates through the input stream. There are quite a few result types available; choosing which class to use is the responsibility of the result factory.

To create your own result types you have two options:

option 1

Subclass TAP::Parser::Result and register your new result type/class with the default TAP::Parser::ResultFactory.

option 2

Subclass TAP::Parser::ResultFactory itself and implement your own TAP::Parser::Result creation logic. Then you'll need to customize the class used by your parser by setting the result_factory_class parameter. See "new" for more details.

If you need to customize the objects on creation, subclass TAP::Parser and override "make_result".

Grammar

TAP::Parser::Grammar is the heart of the parser. It tokenizes the TAP input stream and produces results. If you need to customize its behaviour you should probably familiarize yourself with the source first. Enough lecturing.

Subclass TAP::Parser::Grammar and customize your parser by setting the grammar_class parameter. See "new" for more details.

If you need to customize the objects on creation, subclass TAP::Parser and override "make_grammar"

ACKNOWLEDGMENTS

All of the following have helped. Bug reports, patches, (im)moral support, or just words of encouragement have all been forthcoming.

AUTHORS

Curtis "Ovid" Poe <[email protected]>

Andy Armstong <[email protected]>

Eric Wilhelm @ <ewilhelm at cpan dot org>

Michael Peters <mpeters at plusthree dot com>

Leif Eriksen <leif dot eriksen at bigpond dot com>

Steve Purkis <[email protected]>

Nicholas Clark <[email protected]>

Lee Johnson <notfadeaway at btinternet dot com>

Philippe Bruhat <[email protected]>

BUGS

Please report any bugs or feature requests to [email protected], or through the web interface at http://rt.cpan.org/NoAuth/ReportBug.html?Queue=Test-Harness. We will be notified, and then you'll automatically be notified of progress on your bug as we make changes.

Obviously, bugs which include patches are best. If you prefer, you can patch against bleed by via anonymous checkout of the latest version:

git clone git://github.com/Perl-Toolchain-Gang/Test-Harness.git

Copyright 2006-2008 Curtis "Ovid" Poe, all rights reserved.

This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.