Thinking about Oslo (was Re: Useful Test Utility)

[Ovid]

--- Andy Armstrong <andy at hexten.net> wrote:
 
> Big planned things?
> 
> * TAP blocks of some type
> * Doing something sensible with YAML diagnostics
> 
> I can't remember anything else off hand.

In addition to the above ...

I made a few notes last night.  There are quite a few (why does "quite
a few" mean "many?") features that seem marginal but are really needed
to help make TAP::Harness a good tool for large scale test suite
management.  These aren't needed by the vast majority of users, but
those who need fine-grained control over what their test suites do need
direct support for this rather than trying to hack their own
implementations.

1.  Code that runs before and after a test suite (starting up and
shutting down a test database?)  This is one of the most requested
features.

  prove --startup some_code.pl --shutdown more_code.pl

2.  Code that runs before and after every test program (note that I'm
deliberately reusing terms from Test::Class):

  prove --setup some_code.pl --teardown more_code.pl

3.  Test suite analysis tools (similar to my 'slow tests' code
analysis)?

  prove --analyze

4.  Ability to specify *which* tests get run in parallel (it's not an
"all or nothing" condition).  Only those tests get run in parallel and
others get run sequentially.  "-j 9" would be a default it a parallel
directory is specified).

  prove --parallel t/parallel -j 9 t/

5.  Make the harness output TAP instead of "t/foo.......ok":

  prove --tap t/

  ok 1 - t/foo.t passed

6.  .proverc only used if prove is run with no arguments.

7.  Saved state the default?  This makes analysis better.  I'd also
like optional SQLite support.  This makes history easier to track.

Cheers,
Ovid

--
Buy the book  - http://www.oreilly.com/catalog/perlhks/
Perl and CGI  - http://users.easystreet.com/ovid/cgi_course/
Personal blog - http://publius-ovidius.livejournal.com/
Tech blog     - http://use.perl.org/~Ovid/journal/