Retry D3DCompile with different compile settings if it fails.

Retry D3DCompile with different compile settings if it fails.

Now also tries with D3DCOMPILE_AVOID_FLOW_CONTROL, and
D3DCOMPILE_PREFER_FLOW_CONTROL.
Committed: https://code.google.com/p/angleproject/source/detail?r=1289

[apatrick1, 2012-10-03 01:22:29]

PTAL. I'm a bit surprised the D3D compiler doesn't do this itself.

[dgkoch, 2012-10-03 01:33:44]

can't. wait. to. use. SM4. shaders.. they shouldn't have this problem. 

This certainly won't do any wonders for compilation times either.

https://codereview.appspot.com/6593069/diff/1/src/libGLESv2/ProgramBinary.cpp
File src/libGLESv2/ProgramBinary.cpp (right):

https://codereview.appspot.com/6593069/diff/1/src/libGLESv2/ProgramBinary.cpp...
src/libGLESv2/ProgramBinary.cpp:1080: result = D3DCompile(hlsl, strlen(hlsl),
g_fakepath, NULL, NULL, "main", profile, flags | extraFlags[i], 0, &binary, i ==
0 ? &errorMessage : NULL);
I understand that you want to keep the first compilation log to report in case
all permutations fail, however this log also contains warning, etc that will get
added to the program info log even if the shader compilation is successful, so
if one of the later one succeeds, the log will have incorrect information.

[nicolas, 2012-10-03 01:58:46]

We've contemplated doing this before with similar shader compilation issues, but
concluded that we should instead attempt to identify the code the HLSL compiler
will choke on and either rewrite it into something more digestible or use the
[loop] and [unroll] attributes...

That said, we could probably use this as a quick fix for now, and leave it in as
a fallback even after we fixed the root cause (however long that will take)?

Patch looks good to me aside from Daniel's remark. A clarifying comment in the
code would be nice too, so we know what this was for when we revisit it every
time we encounter new shader compilation issues.

[dgkoch, 2012-10-03 17:59:30]

One other request: In the cases where we fall back to the alternate compiles,
please be sure to  include something in the info log that explains what is going
on.  That way if a developer looks at the log, they'll be able to figure out
that their shader is on the border of not-compiling, or if it takes a long time
to fail, they'll know.

[apatrick1, 2012-10-03 21:00:54]

Retry D3DCompile with different compile settings if it fails.

Now also tries with D3DCOMPILE_AVOID_FLOW_CONTROL, and
D3DCOMPILE_PREFER_FLOW_CONTROL.

[apatrick1, 2012-10-03 21:03:41]

Agreed on shader compilation time not getting better but in most cases it will
only slow things down for shaders that wouldn't otherwise work so I think its
okay. I ran this through the WebGL conformance tests and there were no
regressions.

[dgkoch, 2012-10-04 00:24:46]

https://codereview.appspot.com/6593069/diff/6001/src/libGLESv2/ProgramBinary.cpp
File src/libGLESv2/ProgramBinary.cpp (right):

https://codereview.appspot.com/6593069/diff/6001/src/libGLESv2/ProgramBinary....
src/libGLESv2/ProgramBinary.cpp:1107: infoLog.append(" Retrying with ");
Can we add in the error message from this compilation as well, in case someone
wants to try to fix the first case?

LGTM with that added.

[apatrick1, 2012-10-04 01:13:24]

Like this? The Ascii Street View demo produces this info log before succeeding:

(30,5): error X4505: maximum temp register index exceeded
(30,5): error X4505: maximum temp register index exceeded
(88,9): error X4505: maximum temp register index exceeded
(88,9): error X4505: maximum temp register index exceeded
(90,9): error X4505: maximum temp register index exceeded
(90,9): error X4505: maximum temp register index exceeded

Warning: D3D shader compilation failed with defaultflags. Retrying with avoid
flow control.
(30,5): error X4505: maximum temp register index exceeded
(30,5): error X4505: maximum temp register index exceeded
(88,9): error X4505: maximum temp register index exceeded
(88,9): error X4505: maximum temp register index exceeded
(90,9): error X4505: maximum temp register index exceeded
(90,9): error X4505: maximum temp register index exceeded

Warning: D3D shader compilation failed with avoid flow controlflags. Retrying
with prefer flow control.

It takes 10 seconds or so to finish!

[dgkoch, 2012-10-04 01:25:07]

Yes.  LGTM.

https://codereview.appspot.com/6593069/diff/4003/src/libGLESv2/ProgramBinary.cpp
File src/libGLESv2/ProgramBinary.cpp (right):

https://codereview.appspot.com/6593069/diff/4003/src/libGLESv2/ProgramBinary....
src/libGLESv2/ProgramBinary.cpp:1126: infoLog.append("flags.");
need a space before 'flags'.