workaround for chrome testing flakiness

chromedriver is randomly flaky. symptom: test run hangs forever when
trying to load the next browser test. this is a workaround that
avoids most of that flakiness.

for a while I was concerned that the hang might be a side effect of
the complex things we do to js, but I have a test run that hung after
browser-expectations, which is the first test, and there's nothing
particularly complicated in it, so it seems likely to be a bug in
chromedriver.

the bug seems to be mainly triggered by navigating to about:blank.
but I think I have a few instances of it failing elsewhere. I spent
some time trying to figure out the bug, but didn't get very far,
and this workaround seems adequate, so I'm deferring understanding
the real problem until later.

the workaround is to get a fresh webdriver instance for each test.
this is the way most people use webdriver, so it's likely to work
robustly in the future too.

It's slightly slower than reusing the same browser, but
not significantly slower. (The reuse-a-browser code was created
a long time ago when we had much fewer browser tests, and
browser startup time was a significant fraction of the time.
Browser startup is now an insignificant fraction of the time.)

[kpreid2, 2013-07-16 16:54:54]

LGTM, but see below for alternatives I'd like to explore.

https://codereview.appspot.com/11311043/diff/1/tests/com/google/caja/plugin/W...
File tests/com/google/caja/plugin/WebDriverHandle.java (right):

https://codereview.appspot.com/11311043/diff/1/tests/com/google/caja/plugin/W...
tests/com/google/caja/plugin/WebDriverHandle.java:130:
driver.get("about:blank");
Two thoughts:

1. Why do we need to visit about:blank at all, rather than going directly to the
next test page? If we don't, wouldn't that fix the hang?

2. Can we make BROWSER_REUSE the default if running under Firefox rather than
Chrome? In particular, Firefox will start up in safe mode if the option key is
held down, even if it's not foreground, which breaks testing and so is
problematic if you're trying to do something else while the tests are running.

[felix8a, 2013-07-16 17:08:56]

https://codereview.appspot.com/11311043/diff/1/tests/com/google/caja/plugin/W...
File tests/com/google/caja/plugin/WebDriverHandle.java (right):

https://codereview.appspot.com/11311043/diff/1/tests/com/google/caja/plugin/W...
tests/com/google/caja/plugin/WebDriverHandle.java:130:
driver.get("about:blank");
On 2013/07/16 16:54:54, kpreid2 wrote:
> Two thoughts:
> 
> 1. Why do we need to visit about:blank at all, rather than going directly to
the
> next test page? If we don't, wouldn't that fix the hang?

iirc, taking out the nav to about:blank didn't fix the problem, but I didn't
really keep careful notes at the time, so I might be remembering wrong. It takes
like an hour or so of continual testing to check if the flakiness is still
there, and since I had no idea why it was flaky I was stabbing randomly in the
dark. now that I have some better idea, I'll attack it more systematically, next
time I look at it.

> 2. Can we make BROWSER_REUSE the default if running under Firefox rather than
> Chrome? In particular, Firefox will start up in safe mode if the option key is
> held down, even if it's not foreground, which breaks testing and so is
> problematic if you're trying to do something else while the tests are running.

hm, that seems like not the right fix since even with BROWSER_REUSE true it will
start a new firefox at unpredictable times. I'll think about it.

[felix8a, 2013-07-16 17:10:51]

@r5495