the parsing process is not inherently slow

[Andy Armstrong]

On 2 Oct 2007, at 21:31, Eric Wilhelm wrote:

> # from Andy Armstrong
> # on Tuesday 02 October 2007 10:10:
>
>> On 2 Oct 2007, at 17:49, Eric Wilhelm wrote:
>>> Even if an improved parser speed becomes a bottleneck (at around 900
>>> cores), we can deal with that in better ways than adding
>>> conditionals to the harness.
>>
>> For that to be true the parser would have to be capable of consuming
>> TAP 900 times faster than the tests produced it. Do you believe
>> that's possible?
>
> Of course.  The dotReader test suite has 7326 subtests and takes 83
> seconds of CPU time to run.  That's 88 subtests output per second.

But many tests generate TAP far faster than that...

> (for comparison, Scalar::Util outputs 380/1.8=211 subtests per second
> but has only two modules, so won't spend as much time compiling the  
> way
> large test suites tend to do (and some of the dotReader tests are
> loading Wx.))
>
> `parses_tap.pl` time:       0.08s  => 91,575 subtests/s
> `runtests --exec cat` time: 1.72s  =>  4,259 subtests/s
>
> 900 cores * 88 subtests / s        => 79,200 subtests/s
>
> So, my stupid parser starts blocking 900 cores pretty soon because I
> need to add correctness, but we're talking about 900 cores here!
>
> Let's pretend I can make a proper parser which hits the requisite 80k
> subtests/s.  At Scalar::Util's 211 subtests/s, I can still keep up  
> with
> about 380 cores.

Nobody's suggesting faster wouldn't be better. Are you proposing  
specific changes?

> How many cores can my stupid parser handle at that rate?  Using the  
> slow
> 80k st/s, rather than the actual 455k st/s, we get 11 cores, but part
> of the reason that it parses that faster than dotReader is because  
> each
> test file is dumping a huge load of subtests (~4k st/test), so the
> parser is in the "next ok" loop a lot, which implies that it might be
> fair to apply the 455k st/t number, which gives us 455000/7442 = 61
> cores.  Anyway, we only had 56 tests, so some cores are bored ;-)

That's rather hypothetical though. Your parser doesn't support our  
interface, isn't rigorous about TAP syntax etc etc. What are you  
suggesting we do about it?

If it's going to be a crowd pleaser I'll quite happily write  
TAP::Parser::XS. I don't think we're going to wring much more speed  
out of the current code - certainly not an order of magnitude.

I hear what you're saying but I don't really understand your point.  
Certainly if the parser was much faster the multiplexing parser would  
be viable in a larger range of conditions. We already know that. If  
that's what you're saying you're pushing at an open door.

>> Insisting that all the parsing happen on a single core is always
>> going to hit a brick wall at some point.
>
> My observations imply that the wall is way over there
> ---------------------------------------------------------------------- 
> --------------------------->.

Yes, for tests that don't generate TAP very quickly. We don't know  
how typical that is though.

> And I'm not *insisting* on a single-threaded parser.  I'm suggesting
> that cleaner parallel architecture is better even with the currently
> dirt-slow parser (because the parse time is already negligible on most
> real-world tests.)

Ah right! And what is this cleaner parallel architecture? What's  
unclean about what we have now?

> These numbers are ridiculous only in that 900 cores would present a
> ridiculous amount of other challenges.  However, taking the 80k st/s
> parse rate as an upper bound on speed, I can think we can easily shoot
> for filling 40 cores with "normal" suites (40 cores * 500 st/s = 20k
> st/s parse rate) and (conservatively) 5 with pathological ones like
> Regexp::Common.

How?

> So, a forking parser optimizes for *pathological* test suites (7k  
> st/s)
> if you have about 8 cores.  Given that a vast install-base of more  
> than
> two cores is a year or more away (the quads currently being the  
> pricier
> "flagship" cpu), I say messing with a forking parser this year is not
> worth it.

Well it's pretty much done. That given, I don't think I understand  
your argument against it.

> The brick wall I see on the not-so-distant horizon is the one where
> TAP::Harness is not as extensible or flexible as promised and
> subclasses or plugins (and users) will be dealing with API
> compatibility troubles while it tries to adapt.

Have we lost some extensibility or flexibility lately?

> BTW, A simple distributed harness subclass was only a short hop away
> from TAP::Harness::Parallel, requiring a bit of one-time Unionfs+NFS
> setup and sticking ssh $node before $^X (or roughly so.)  It might  
> have
> even been a quite simple subclass of T::H::P.

That's good news. I assume the same will be true of the multiplexing  
parser we have now then - given that it's the same overall shape as  
your implementation. The main difference is the use of select to poll  
the iterators. Are you aware of any problematic differences?

-- 
Andy Armstrong, Hexten