Add reference graph scanner for finding mutability and host object leak bugs.

The scanner runs as a guest test and examines every object reachable by
property and prototype traversal and invoking functions (based on
hardcoded definitions of possible argument lists). It checks for:
* Objects which are unfrozen and not specifically expected to be.
* Objects which are from the host frame, including exceptions.
* Domado taming constructors.
* toString methods which do not return strings.

Supporting changes:
* browser-test-case.js exposes an interface to the guest to modify URL
  parameters.
* jsunitFail is available to the guests and passes along a reason.
* New directAccess facilities to eval in other frames.
* A special case to extend the BrowserTestCase timeout for lack of
  progress, because the scanner does not quickly make progress as
  measured by number of not-yet-finished test cases.

Possible future work which is NOT in this revision:
* There is a test that toString methods are always non-enumerable,
  which has been disabled because there are many failures which ought to
  be fixed separately.
* Meeting Domado taming constructors is an error, but this is a
  'blacklist' strategy. Replace it with a 'whitelist' strategy which
  somehow is not excessively noisy about the myriad functions available.
  This is what innocuous() in Domado was introduced to support. (Or,
  use a blacklist based on inertCtor marking taming ctors as dangerous,
  which would be better than the current string-matching strategy.)
* More coverage of mis-invoking methods.

@r5411

[felix8a, 2013-05-17 23:47:34]

lgtm++

https://codereview.appspot.com/9197045/diff/1/tests/com/google/caja/plugin/es...
File tests/com/google/caja/plugin/es53-test-scan-guest.js (right):

https://codereview.appspot.com/9197045/diff/1/tests/com/google/caja/plugin/es...
tests/com/google/caja/plugin/es53-test-scan-guest.js:14: 
TODO, lint this and add @requires

https://codereview.appspot.com/9197045/diff/1/tests/com/google/caja/plugin/es...
tests/com/google/caja/plugin/es53-test-scan-guest.js:30: function
simpleEval(env, expr) {
simpleEval is a distracting name, it triggers my "eval needs extra attention"
sensors, but this is only used for property lookup. maybe call it something like
"lookupValue" or "propertyValue"?

or are you intending to extend it with more eval forms later?

https://codereview.appspot.com/9197045/diff/1/tests/com/google/caja/plugin/es...
tests/com/google/caja/plugin/es53-test-scan-guest.js:33: return simpleEval(env,
match[1])[match[2]];
recursing from the right here seems odd to me, this can't become a tail-call.
recursing from the left gets the same answer and can be a tail-call.

https://codereview.appspot.com/9197045/diff/1/tests/com/google/caja/plugin/es...
tests/com/google/caja/plugin/es53-test-scan-guest.js:49: none: function none(c)
{},
c -> callback

https://codereview.appspot.com/9197045/diff/1/tests/com/google/caja/plugin/es...
tests/com/google/caja/plugin/es53-test-scan-guest.js:60: any: function
any(var_args) {
either 'tupleg' is an odd name or 'any' is an odd name lacking a 'g' suffix. I
think since all the generator functions except 'value' expect to recursively
expand generators in their args, might as well omit the 'g' suffix in 'tupleg'
and 'applyg'.

https://codereview.appspot.com/9197045/diff/1/tests/com/google/caja/plugin/es...
tests/com/google/caja/plugin/es53-test-scan-guest.js:90: return function
GapplygGen(c) {
c -> callback

https://codereview.appspot.com/9197045/diff/1/tests/com/google/caja/plugin/es...
tests/com/google/caja/plugin/es53-test-scan-guest.js:100: return
G.applyg(function() {
note var_args

https://codereview.appspot.com/9197045/diff/1/tests/com/google/caja/plugin/es...
tests/com/google/caja/plugin/es53-test-scan-guest.js:131: if
(cajaVM.sharedImports) {
this seems like a bad way of testing for es5. es53 has a sharedImports, which is
currently only on ___, but I don't see why we wouldn't want it also on cajaVM
for regularity.

https://codereview.appspot.com/9197045/diff/1/tests/com/google/caja/plugin/es...
tests/com/google/caja/plugin/es53-test-scan-guest.js:156: } else if (m =
/^function (\w+)/.exec(
add parens around assignment used as boolean
maybe pattern should be ([\w$]+)

https://codereview.appspot.com/9197045/diff/1/tests/com/google/caja/plugin/es...
tests/com/google/caja/plugin/es53-test-scan-guest.js:223: // Note identityGens
is an object-keyed map; doesn't need to be weak.
what's identityGens?

https://codereview.appspot.com/9197045/diff/1/tests/com/google/caja/plugin/es...
tests/com/google/caja/plugin/es53-test-scan-guest.js:268: if (value =
identityTable.get(obj)) {
add parens

https://codereview.appspot.com/9197045/diff/1/tests/com/google/caja/plugin/es...
tests/com/google/caja/plugin/es53-test-scan-guest.js:416: } else if
(/^-?[0-9]+$/i.test(p)) {
useless /i

https://codereview.appspot.com/9197045/diff/1/tests/com/google/caja/plugin/es...
tests/com/google/caja/plugin/es53-test-scan-guest.js:588:
seenTable.set(o.prototype, globalName + '.prototype');
o.prototype -> p

https://codereview.appspot.com/9197045/diff/1/tests/com/google/caja/plugin/es...
tests/com/google/caja/plugin/es53-test-scan-guest.js:593: this.scan = function()
{
scan is long and complicated and hard to read, but other than that seems ok to
me.

https://codereview.appspot.com/9197045/diff/1/tests/com/google/caja/plugin/es...
tests/com/google/caja/plugin/es53-test-scan-guest.js:683: marker = ' thrown ';
marker is set but not used

https://codereview.appspot.com/9197045/diff/1/tests/com/google/caja/plugin/es...
tests/com/google/caja/plugin/es53-test-scan-guest.js:690: var thrown;
confusing location for var, move to function scope

https://codereview.appspot.com/9197045/diff/1/tests/com/google/caja/plugin/es...
tests/com/google/caja/plugin/es53-test-scan-guest.js:848: finished(problemCount,
gapCount, totalDone);
not something to fix now, but just want to note that large amounts of code in
nested local scopes makes it harder for me to verify what identifiers mean, like
which 'finished' this is referring to.

[kpreid2, 2013-05-18 00:35:59]

The scanner runs as a guest test and examines every object reachable by
property and prototype traversal and invoking functions (based on
hardcoded definitions of possible argument lists). It checks for:
* Objects which are unfrozen and not specifically expected to be.
* Objects which are from the host frame, including exceptions.
* Domado taming constructors.
* toString methods which do not return strings.

Supporting changes:
* browser-test-case.js exposes an interface to the guest to modify URL
  parameters.
* jsunitFail is available to the guests and passes along a reason.
* New directAccess facilities to eval in other frames.
* A special case to extend the BrowserTestCase timeout for lack of
  progress, because the scanner does not quickly make progress as
  measured by number of not-yet-finished test cases.

Possible future work which is NOT in this revision:
* There is a test that toString methods are always non-enumerable,
  which has been disabled because there are many failures which ought to
  be fixed separately.
* Meeting Domado taming constructors is an error, but this is a
  'blacklist' strategy. Replace it with a 'whitelist' strategy which
  somehow is not excessively noisy about the myriad functions available.
  This is what innocuous() in Domado was introduced to support. (Or,
  use a blacklist based on inertCtor marking taming ctors as dangerous,
  which would be better than the current string-matching strategy.)
* More coverage of mis-invoking methods.

[kpreid2, 2013-05-18 00:36:53]

New snapshot for the record, submitting.

https://codereview.appspot.com/9197045/diff/1/tests/com/google/caja/plugin/es...
File tests/com/google/caja/plugin/es53-test-scan-guest.js (right):

https://codereview.appspot.com/9197045/diff/1/tests/com/google/caja/plugin/es...
tests/com/google/caja/plugin/es53-test-scan-guest.js:14: 
On 2013/05/17 23:47:34, felix8a wrote:
> TODO, lint this and add @requires

Linted. Linter found an actual problem, too: the function invocation logic was
not always initializing 'thrown' (but instead the unused 'marker' variable, a
leftover).

https://codereview.appspot.com/9197045/diff/1/tests/com/google/caja/plugin/es...
tests/com/google/caja/plugin/es53-test-scan-guest.js:30: function
simpleEval(env, expr) {
On 2013/05/17 23:47:34, felix8a wrote:
> simpleEval is a distracting name, it triggers my "eval needs extra attention"
> sensors, but this is only used for property lookup. maybe call it something
like
> "lookupValue" or "propertyValue"?
> 
> or are you intending to extend it with more eval forms later?

Well, it was invented as a kludge to work around the lack of eval. It's only
accidentally less powerful than eval, and I might well add function calls if I
later found I needed to refer to an object only available as
"someGlobal.getThing()" or "Object.getPrototypeOf(foo)" or some such.

I don't like the idea of renaming to be more specific because that implies there
is something meaningful about that more-specific concept.

https://codereview.appspot.com/9197045/diff/1/tests/com/google/caja/plugin/es...
tests/com/google/caja/plugin/es53-test-scan-guest.js:33: return simpleEval(env,
match[1])[match[2]];
On 2013/05/17 23:47:34, felix8a wrote:
> recursing from the right here seems odd to me, this can't become a tail-call.
> recursing from the left gets the same answer and can be a tail-call.

That recursion pattern doesn't work for eval and I want this to be a
straightforward implementation of part of eval.

https://codereview.appspot.com/9197045/diff/1/tests/com/google/caja/plugin/es...
tests/com/google/caja/plugin/es53-test-scan-guest.js:49: none: function none(c)
{},
On 2013/05/17 23:47:34, felix8a wrote:
> c -> callback

Done.

https://codereview.appspot.com/9197045/diff/1/tests/com/google/caja/plugin/es...
tests/com/google/caja/plugin/es53-test-scan-guest.js:60: any: function
any(var_args) {
On 2013/05/17 23:47:34, felix8a wrote:
> either 'tupleg' is an odd name or 'any' is an odd name lacking a 'g' suffix. I
> think since all the generator functions except 'value' expect to recursively
> expand generators in their args, might as well omit the 'g' suffix in 'tupleg'
> and 'applyg'.

Done.

https://codereview.appspot.com/9197045/diff/1/tests/com/google/caja/plugin/es...
tests/com/google/caja/plugin/es53-test-scan-guest.js:90: return function
GapplygGen(c) {
On 2013/05/17 23:47:34, felix8a wrote:
> c -> callback

Done.

https://codereview.appspot.com/9197045/diff/1/tests/com/google/caja/plugin/es...
tests/com/google/caja/plugin/es53-test-scan-guest.js:100: return
G.applyg(function() {
On 2013/05/17 23:47:34, felix8a wrote:
> note var_args

Done.

https://codereview.appspot.com/9197045/diff/1/tests/com/google/caja/plugin/es...
tests/com/google/caja/plugin/es53-test-scan-guest.js:131: if
(cajaVM.sharedImports) {
On 2013/05/17 23:47:34, felix8a wrote:
> this seems like a bad way of testing for es5.

So it is. Replaced with inES5Mode.

> es53 has a sharedImports, which is currently only on ___, but
> I don't see why we wouldn't want it also on cajaVM for regularity.

Neither do I, hence
https://code.google.com/p/google-caja/issues/detail?id=1719

https://codereview.appspot.com/9197045/diff/1/tests/com/google/caja/plugin/es...
tests/com/google/caja/plugin/es53-test-scan-guest.js:156: } else if (m =
/^function (\w+)/.exec(
On 2013/05/17 23:47:34, felix8a wrote:
> add parens around assignment used as boolean
> maybe pattern should be ([\w$]+)

Done.

https://codereview.appspot.com/9197045/diff/1/tests/com/google/caja/plugin/es...
tests/com/google/caja/plugin/es53-test-scan-guest.js:223: // Note identityGens
is an object-keyed map; doesn't need to be weak.
On 2013/05/17 23:47:34, felix8a wrote:
> what's identityGens?

Former name of identityTable. Comment rewritten.

https://codereview.appspot.com/9197045/diff/1/tests/com/google/caja/plugin/es...
tests/com/google/caja/plugin/es53-test-scan-guest.js:268: if (value =
identityTable.get(obj)) {
On 2013/05/17 23:47:34, felix8a wrote:
> add parens

Done.

https://codereview.appspot.com/9197045/diff/1/tests/com/google/caja/plugin/es...
tests/com/google/caja/plugin/es53-test-scan-guest.js:416: } else if
(/^-?[0-9]+$/i.test(p)) {
On 2013/05/17 23:47:34, felix8a wrote:
> useless /i

Done.

https://codereview.appspot.com/9197045/diff/1/tests/com/google/caja/plugin/es...
tests/com/google/caja/plugin/es53-test-scan-guest.js:588:
seenTable.set(o.prototype, globalName + '.prototype');
On 2013/05/17 23:47:34, felix8a wrote:
> o.prototype -> p

Done.

https://codereview.appspot.com/9197045/diff/1/tests/com/google/caja/plugin/es...
tests/com/google/caja/plugin/es53-test-scan-guest.js:683: marker = ' thrown ';
On 2013/05/17 23:47:34, felix8a wrote:
> marker is set but not used

Done.

https://codereview.appspot.com/9197045/diff/1/tests/com/google/caja/plugin/es...
tests/com/google/caja/plugin/es53-test-scan-guest.js:690: var thrown;
On 2013/05/17 23:47:34, felix8a wrote:
> confusing location for var, move to function scope

Done. Note that thrown was not, but should have been, et in the places where the
unused 'marker' was.