[tapx-dev] Thoughts about Test::Harness 3.0

[Michael G Schwern]

Shlomi Fish wrote:
> Secondly, my point was not that we[1] shouldn't work on a better, more modular 
> and more extensible TAP harness, just that trying to do that with 
> Test::Harness is not such a good idea. I wouldn't recommend anyone to try to 
> extend, or embed Test::Harness because the code does not give way to this. If 
> you have something that does that with Test::Harness (probably in evil, 
> unspeakable ways), that code should be converted to TAP::Parser or whatever.

First, the discussion is largely moot as most of the work has quietly been done.
http://search.cpan.org/dist/TAP-Parser/lib/TAP/Harness/Compatible.pm
http://search.cpan.org/dist/TAP-Parser/bin/prove

TH's functionality and tests already cover a wide range of use cases.  It
provides a complex TAP display situation for TAP::Parser to try and drive.  A
number of bugs where shaken out and features added to stretch TAP::Parser to
cover them.  Even if nobody uses it, it was good exercise.

If the intention is to just replace Straps with TAP then most of the work was
done in the TH 1.x -> 2.x conversion which introduced Straps.  A new backened
should not be difficult to drop in.

If the intention is rewrite all of TH using TAP then its easier to just junk
TH and rewrite it.  TH's code is 20 years old leading directly back to t/TEST
in Perl 1.0.  Its a nightmare.  If you're used to working with it then your
perceptions of how hard it would be to rewrite are clouded by it.  I know mine
were.

Calling this resulting rewrite Test::Harness allows us to retire all of the 20
year old TH code as well as slip TAP::Parser into the core.


> Now for the fact that there are few customisations to TAP consumers now, and 
> what the future may hold. I agree that more customisations can be written and 
> possibly should be written. However, you all agree that there are very few 
> customisations against Test::Harness *now* due to its lack of mudolarity and 
> extensibility. So, we can assume that these customisations and extensions are 
> not too critical, and that they can be re-implemented above a different TAP 
> consumer.

I agree with that.  I'm dedicated to that by junking Straps.  Straps was an
experiment and has failed.  Anyone who depended on Straps should switch to
TAP.  TH 3 should not include or try to emulate Straps.

OTOH the TH functions, environment and display style is widely used.  It has
to be kept around but, as mentioned above, its easier to throw out that 20
year old code then keep it on life-support.  I, for one, do not relish
updating the code for the new TAP extensions.


> Finally, the number of CGI scripts in the world far exceeds the number of 
> different implementations of CGI consumers (i.e: web-servers, etc.) and 
> always will. Thus, so does the number of TAP producing scripts will always be 
> greater than the number of different TAP consumers and their different 
> customisations. While there's still a lot of exciting development you can do 
> in the realm of TAP consumers, it will still remain out of the direct 
> involvement of Random-J-QA-Hacker, who will just read:
> 
> http://search.cpan.org/dist/Task-Test-Run-AllPlugins/lib/Task/Test/Run/AllPlugins.pm
> 
> (or its TAP::Parser / TAP::Harness / whatever equivalent)
> 
> and happily run it.

This sounds like going to Craftsman Tool Company and saying, "People want
furniture, why are you making saws?!  You should be making furniture!"

Somebody has to make the tools that make what everybody else uses.  Somebody
has to maintain the toolchain.