[960] Fault unknown TAP tokens in strict mode (V13+)

[Andy Armstrong]

On 17 Feb 2008, at 23:05, Michael G Schwern wrote:
>> OK. It's TAP 13 which was intended to be stricter. TAP 12 is  
>> unaffected.
>
> I don't like the idea that specifying a TAP version means I'm going  
> to have
> trouble testing stuff that prints to STDOUT.

OK. I can see that.

>>> Historically, a TAP parser should ignore any unknown output for two
>>> reasons:
>>>
>>> 1) It allows old parsers to still understand a subset of new TAP.
>>
>> I guess we could say that a TAP version N parser will be non-strict
>> when it sees TAP version N+M
>
> That's a good idea.

I prefer pragmas now though :)

>>> 2) It allows one to test things that print to STDOUT.
>>
>> Yup. Hmm. Dunno. That's always struck me as kinda dangerous :)
>
> It's not by choice, I'll tell you that, but it is practical.  You  
> just don't
> have that kind of control and being able to test anything trumps  
> cleanliness.
>  It is the Test ANYTHING Protocol.  We've lived with it for a long  
> time now
> and it only causes the very occasional glitch.
>
> I have a feeling it's going to cause more trouble to be strict then  
> it is to
> just ignore it and it doesn't seem like there's a great clamoring  
> for a strict
> TAP parser.

The use case was Nicholas asking on P5P whether it might be possible  
to make the TAP parser strict. You're right though - that's the only  
clamouring I've heard.

>>> It doesn't solve #2.  One often doesn't have control over the code
>>> they're
>>> testing, so it makes that impossible.
>>>
>>> At minimum I'd bump the TAP version #.  And there needs to be a way
>>> to shut it
>>> off.  Problem is, the test itself knows best whether there's going
>>> to be
>>> garbage on STDOUT.  So the control would lie in the test.
>>
>> I quite like the idea of having to explicitly declare your test as
>> stdout-dirty.
>
> That would seem to introduce the idea of test pragmas which Eric  
> seemed to be
> suggesting.


So how about?

TAP version 13
ok 1
barf some random crap		<-- no problem
pragma +strict
barf more 			<-- parse error
pragma -some_other_pragma
ok 2

And have the parser default to non-strict for least-surprises  
compatibility with how Perl works?

-- 
Andy Armstrong, Hexten