TAP::Harness improvements

[Andy Armstrong]

On 16 Nov 2007, at 12:20, Ovid wrote:
>  257/340 t/acceptance-tests/customer.t
>  258/340 t/acceptance-tests/brand.t
>  259/340 t/acceptance-tests/products.t
>  260/340 t/acceptance-tests/services.t
>  261/340 t/acceptance-tests/master.t
>  262/340 t/acceptance-tests/authentication.t
>
> I'm running Devel::Cover over a large test suite right now and I  
> find that I need to write a separate script which showed the test  
> count and current test.  It's easy and useful to incorporate this.   
> Thoughts?  If we like this, I'd be happy to write it (after all,  
> AndyA has been doing virtually everything lately and I'd like to let  
> him know he's not *completely* abandoned :)

Yeah, I like that. Go for it :)

> Each YAML file has a separate, but identical, .t file which loads  
> the YAML and runs the tests in it.  This is useful as the tests are  
> data driven, but the libraries can be expensive to load and it takes  
> almost 10 minutes.  We could get a nice speedup by having a master  
> 'yaml_runner.t' which loads the libraries once, but each YAML file  
> has its own test count.  We talked in the past about nested TAP  
> streams and this would make conversion *much* easier.  Thoughts?  I  
> can work around this by having the code load the tests and sum the  
> test counts, but rather than custom code which does this every time,  
> nested TAP would make this problem go away.


That's the second recent use case for structured TAP of some sort. The  
other being the idea of Test Steering Protocol - which aims to solve a  
problem which partially disappears if you can have a test script that  
spawns others tests conditionally and outputs structured TAP as a  
result:

1..2
begin 1 tests from xt/opt/plimsole.t
1..2
ok 1
ok 2
end 1
begin 2 tests from xt/opt/sneaker.t
1..3
ok 1
not ok 2
ok 3
end 2

So I'm +1 on that.

The outstanding question was which of the two proposals:

http://testanything.org/wiki/index.php/Test_Groups
http://testanything.org/wiki/index.php/Test_Blocks

I'm in favour of Test_Blocks - but I'd be interested to hear  
objections :)

-- 
Andy Armstrong, Hexten