Simplistic tokenizer cheat?

[Michael G Schwern]

Here's a thought.  Since most tests are of the form...

	/^(not\ )?
          ok\
          (\d+)
          (?:\ ([^#]+) )?
         $/x

That is...

	ok 1
	not ok 1
	ok 1 - description
	not ok 1 - description

Can we put in a simplistic tokenizer cheat to bypass the more expensive full
grammar check?

	if( $line =~ /$simple_regex/ ) {
		proceed directly to parsing
	}
	else {
		full tokenizer
	}


-- 
Robrt:   People can't win
Schwern: No, but they can riot after the game.