[659] Still fork even if jobs == 1.

[Andy Armstrong]

On 3 Oct 2007, at 00:35, Michael G Schwern wrote:
> Andy Armstrong wrote:
>> On 3 Oct 2007, at 00:12, Michael G Schwern wrote:
>>>> Still fork even if jobs == 1.
>>>> With jobs == 0 it executes the forking code path but no forking
>>>> takes place inside Parallel::Iterator.
>>> So --fork -j1 means what processes are doing what again?
>>
>> Jobs == 0      : no forking
>> Jobs == 1 .. n : fork that many workers
>
> Sorry, I'm still confused.  I'm a bit lost in how the new parallel
> implementation works.  What part is forking?  I presumed the TAP  
> parser is
> forking but then I don't understand why having one fork is useful.

For now it's using Parallel::Iterator. P::I is like a parallel  
version of map. It forks however many workers you specify and uses  
them to process the elements of a list (or iterator, hash) in parallel.

So it's conceptually like

my @parsers = map { run_test_and_return_parser( $_ ) } @tests;

but with run_test_and_return_parser run in one of the forked  
processes rather than the parent.

So the parent process feeds test names into a cluster of processes that

* construct a parser (which launches the test script)
* loops over the results
* returns the used parser to the parent

The returned parsers are serialized using Storable so the parsers  
that wind up getting aggregated in the parent process are effectively  
dclones of the real parsers.

Without --fork that code isn't used at all.

With --fork -j0 the P::I code is used - but P::I doesn't fork if you  
specify 0 workers - it does all its work in the parent process.

With --fork -j1 (and up) P::I starts that many workers and hands  
tests to them as they complete the previous test.

So with --fork -j1 you'll typically have the master process, a single  
forked worker and a third process for the current test script. The  
single forked worker will handle all of the tests sequentially.

-- 
Andy Armstrong, Hexten