Parallel thoughts

[Eric Wilhelm]

# from Andy Armstrong
# on Tuesday 02 October 2007 00:43:

>That's a benefit in the way that only running a single process is a  
>benefit because it's easier :)

Easier to maintain and extend too.

>> The problem with #3 is that you cannot distribute it across machines
>> without having (matching) Storable and TAP::Parser installed on the
>> nodes.  There's probably also something to do with running a
>> different perl version as the harness vs the tests.
>
>Yes, but the problem with the other two is that they're CPU bound  
>before we've done anything interesting.

it would be nice to just make the grammar run faster and quit trying to
throw cores at it.

I realize the below implementation is an oversimplification, but given 
that we seem to be stuck on the subject of the parser speed, I think it 
is worth noting that "parses_tap.pl" parses the Regexp::Common archive 
in 0.4s while the `runtests -r --exec cat $spooldir` takes 38s.  That's 
missing some features, but it's about 7500% faster.

  http://scratchcomputing.com/tmp/parses_tap.pl

Does it have 7500% more features?

The simple parser is still 3500% faster than the --fork -j 9.  Given the 
complexity of the IPC and the growing complexity of Harness.pm with all 
of these embedded features (and it still has no nonblock mode for the 
gui subclass), I'm not liking the direction that things seem to be 
headed.

And for the dotreader test suite, the --fork slows it down by about 11%.

--Eric
-- 
So malloc calls a timeout and starts rummaging around the free chain,
sorting things out, and merging adjacent small free blocks into larger
blocks. This takes 3 1/2 days.
--Joel Spolsky
---------------------------------------------------
    http://scratchcomputing.com
---------------------------------------------------