[tapx-dev] Test::Harness 3.0 roadmap

[Ovid]

--- Andy Armstrong <andy at hexten.net> wrote:

> (from Perl QA)
> 
> On 13 Mar 2007, at 01:15, Ovid wrote:
> > That's not to say we can't discuss it or navel-gaze all day long. 
> I
> > actually like those things.  However, I want Test::Harness 3.0 to
> get
> > done.  If that works and it's stable, then we can worry about the  
> > other
> > stuff.  In fact, as much as I hate to say it, even making things
> > parrellellizable (is that a word?) should be off the table until
> > Test::Harness 3.0 is as seamless an upgrade as possible.

Just to be clear:  I don't mind new features like parallelization or
new diagnostics working their way into T::H 3, but they just can't be
the first priority.  We need to guarantee that we can provide people
with what they already have and make it possible to do the things that
they want.  I think we're almost there.
 
> Can we summarise what still needs to be done for 3.0?
> 
> * Test::Harness compatibility is probably as good as it's going to  
> get until I get some feedback about
>    anything that it's missing.

Interestingly, that's probably the number one issue.  Here's a TODO
list :)

  http://rt.cpan.org/Public/Dist/Display.html?Name=Test-Harness

There are 65 items, many of them are spam.  Andy, can you clear any of
those or at least get rid of the spam?  If we are unsure about a
particular bug, the key would be to see if we can replicate it in the
current harness just so that we at least understand what the bug is.
 
> * I'd say the parser is pretty solid now.

Yup.  I'm very happy about that.

> * The one open ticket, 24728 is fixed. Could you mark it closed  
> please Curtis?

Are you sure it's fixed?  I was going to close it a while ago, but it
wasn't really fixed.  Taking Schwern's sample program:

  #!/usr/bin/perl -w

  print "1..1\n";
  print "ok 1\n";
  print "# comment\n";

And running it:

  trunk $ prove diagnostic.plx 
  diagnostic....ok                                                     
       
  All tests successful.
  Files=1, Tests=1,  0 wallclock secs ( 0.00 cusr +  0.00 csys =  0.00
CPU)

  trunk $ runtests diagnostic.plx 
  diagnostic........1/1 # comment
  diagnostic........ok   
  All tests successful.
  Files=1, Tests=1,  0 wallclock secs ( 0.00 cusr +  0.00 csys =  0.00
CPU)

'runtests' still spits out the diagnostic unless we include the '-q'
switch.  I do not think, with the current implementation, that we can
"fix" this to exactly mimic the behavior of the current test harness. 
Mark it as 'resolved' but with said explanation?

> * You're planning a change to the execrc mechanism before we're done,
> yes?

Yes.  I'm planning on eliminating it.  I can't think of any reason for
it.  --exec does everything we need, it's more flexible and easier to
explain.  Eric, I think you had some comments about a problem with its
implementation but I can't find that email right now.  Can you
summarize?

> What else?

I think we can table the 'runtestsrc' file for now because an alias
will handle that.  It's a wishlist item.

Summary:

* Check the RT T:H queue
* Eliminate execrc
* Default --exec file
* M::B patch for early adopters
* 'runtests' should be the new 'prove'?  What about the 'prove' 
  implementation that we have?

With that, we can break T:H:C into a separate T::H distro (when Andy is
comfortable with it).  We'll need to put a "devel" version number on it
so people don't accidentally download it.

Again, I really don't mind parallelization or the new diagnostic syntax
and I would be happy if we can get them into T:H 3, so long as the
above is resolved.

Did I miss anything?  I think we're really, really close.

And thanks to all of you for all of your work.  This never would have
gotten here with just me.

Cheers,
Ovid

--

Buy the book -- http://www.oreilly.com/catalog/perlhks/
Perl and CGI -- http://users.easystreet.com/ovid/cgi_course/