Generates more ES5 tests

Generates more ES5 tests

To be massively refactored before ready for real review.
For a taste, try  http://www.erights.org/tests/testjs/  on modern browsers.

[MarkM, 2011-02-19 10:02:50]

Hi Christian,

This isn't ready for an actual review yet since it doesn't yet do anything. But
it seems a good way to represent thousands of little micro-features, like
"global.Nan is not writable", from which we can either automatically generate
micro tests, or from which we can run a big macro test. Which do you think we
should do?

[christian.plesner.hansen, 2011-02-21 05:43:31]

On 2011/02/19 10:02:50, MarkM wrote:
> Hi Christian,
> 
> This isn't ready for an actual review yet since it doesn't yet do anything.
But
> it seems a good way to represent thousands of little micro-features, like
> "global.Nan is not writable", from which we can either automatically generate
> micro tests, or from which we can run a big macro test. Which do you think we
> should do?

I think you should do this as you see fit, these are just thoughts.

One thing you might want to factor into this is that it is useful to be able to
control test expectations and monitor test output with high granularity.  When
running these tests people aren't just interested in pass/fail, they're just as
interested in comparisons with previous runs, comparisons with expected-failure
status files corresponding to open bugs, etc.  The bigger the test the less
obvious it is what has changed if suddenly starts passing/failing, and the lower
granularity you have to express that a given wrong behavior is well-known and
tracked by a bug, and should hence be ignored when running the tests.  That's
one reason I like having many small tests each with a unique ID which you can
attach meta-information to.

Another thought is that so far I've been using python as the meta language
because, unlike JS, there is solid stand-alone support on all platforms and it
has a well-defined OS interface.  I see why you might be drawn to use JS for
this but it does have some significant limitations as a meta language for
working with tests.