the parsing process is not inherently slow
n at rciss.us
Tue Oct 2 22:06:14 BST 2007
On 2 Oct 2007, at 21:31, Eric Wilhelm wrote:
> # from Andy Armstrong
> # on Tuesday 02 October 2007 10:10:
>> On 2 Oct 2007, at 17:49, Eric Wilhelm wrote:
>>> Even if an improved parser speed becomes a bottleneck (at around 900
>>> cores), we can deal with that in better ways than adding
>>> conditionals to the harness.
>> For that to be true the parser would have to be capable of consuming
>> TAP 900 times faster than the tests produced it. Do you believe
>> that's possible?
> Of course. The dotReader test suite has 7326 subtests and takes 83
> seconds of CPU time to run. That's 88 subtests output per second.
But many tests generate TAP far faster than that...
> (for comparison, Scalar::Util outputs 380/1.8=211 subtests per second
> but has only two modules, so won't spend as much time compiling the
> large test suites tend to do (and some of the dotReader tests are
> loading Wx.))
> `parses_tap.pl` time: 0.08s => 91,575 subtests/s
> `runtests --exec cat` time: 1.72s => 4,259 subtests/s
> 900 cores * 88 subtests / s => 79,200 subtests/s
> So, my stupid parser starts blocking 900 cores pretty soon because I
> need to add correctness, but we're talking about 900 cores here!
> Let's pretend I can make a proper parser which hits the requisite 80k
> subtests/s. At Scalar::Util's 211 subtests/s, I can still keep up
> about 380 cores.
Nobody's suggesting faster wouldn't be better. Are you proposing
> How many cores can my stupid parser handle at that rate? Using the
> 80k st/s, rather than the actual 455k st/s, we get 11 cores, but part
> of the reason that it parses that faster than dotReader is because
> test file is dumping a huge load of subtests (~4k st/test), so the
> parser is in the "next ok" loop a lot, which implies that it might be
> fair to apply the 455k st/t number, which gives us 455000/7442 = 61
> cores. Anyway, we only had 56 tests, so some cores are bored ;-)
That's rather hypothetical though. Your parser doesn't support our
interface, isn't rigorous about TAP syntax etc etc. What are you
suggesting we do about it?
If it's going to be a crowd pleaser I'll quite happily write
TAP::Parser::XS. I don't think we're going to wring much more speed
out of the current code - certainly not an order of magnitude.
I hear what you're saying but I don't really understand your point.
Certainly if the parser was much faster the multiplexing parser would
be viable in a larger range of conditions. We already know that. If
that's what you're saying you're pushing at an open door.
>> Insisting that all the parsing happen on a single core is always
>> going to hit a brick wall at some point.
> My observations imply that the wall is way over there
Yes, for tests that don't generate TAP very quickly. We don't know
how typical that is though.
> And I'm not *insisting* on a single-threaded parser. I'm suggesting
> that cleaner parallel architecture is better even with the currently
> dirt-slow parser (because the parse time is already negligible on most
> real-world tests.)
Ah right! And what is this cleaner parallel architecture? What's
unclean about what we have now?
> These numbers are ridiculous only in that 900 cores would present a
> ridiculous amount of other challenges. However, taking the 80k st/s
> parse rate as an upper bound on speed, I can think we can easily shoot
> for filling 40 cores with "normal" suites (40 cores * 500 st/s = 20k
> st/s parse rate) and (conservatively) 5 with pathological ones like
> So, a forking parser optimizes for *pathological* test suites (7k
> if you have about 8 cores. Given that a vast install-base of more
> two cores is a year or more away (the quads currently being the
> "flagship" cpu), I say messing with a forking parser this year is not
> worth it.
Well it's pretty much done. That given, I don't think I understand
your argument against it.
> The brick wall I see on the not-so-distant horizon is the one where
> TAP::Harness is not as extensible or flexible as promised and
> subclasses or plugins (and users) will be dealing with API
> compatibility troubles while it tries to adapt.
Have we lost some extensibility or flexibility lately?
> BTW, A simple distributed harness subclass was only a short hop away
> from TAP::Harness::Parallel, requiring a bit of one-time Unionfs+NFS
> setup and sticking ssh $node before $^X (or roughly so.) It might
> even been a quite simple subclass of T::H::P.
That's good news. I assume the same will be true of the multiplexing
parser we have now then - given that it's the same overall shape as
your implementation. The main difference is the use of select to poll
the iterators. Are you aware of any problematic differences?
Andy Armstrong, Hexten
More information about the tapx-dev