Using yy_scan_string, which flushes the old buffer does not work. GLSL requires that each input str…

Using yy_scan_string, which flushes the old buffer does not work. GLSL requires that each input string is concatenated, but yy_scan_string treats each string individually. Added a custom YY_INPUT which maintains the continuity between each string.
Committed: https://code.google.com/p/angleproject/source/detail?r=1066

[Alok Priyadarshi, 2012-04-29 11:28:47]

With this patch, all location tests pass as well. LocationTest.TokenStraddling*
tests were failing earlier.

[kbr1, 2012-04-30 18:31:17]

LGTM overall. Couple of minor comments.

http://codereview.appspot.com/6130045/diff/2018/src/compiler/preprocessor/new...
File src/compiler/preprocessor/new/Input.cpp (right):

http://codereview.appspot.com/6130045/diff/2018/src/compiler/preprocessor/new...
src/compiler/preprocessor/new/Input.cpp:29: if (mLength[i] == -1)
The test against -1 seems fragile. Perhaps < 0 instead?

http://codereview.appspot.com/6130045/diff/2018/src/compiler/preprocessor/new...
File src/compiler/preprocessor/new/Input.h (right):

http://codereview.appspot.com/6130045/diff/2018/src/compiler/preprocessor/new...
src/compiler/preprocessor/new/Input.h:20: Input(int count, const char* const
string[], const int length[]);
Could you document the semantics of the "length" array?

http://codereview.appspot.com/6130045/diff/2018/tests/preprocessor_tests/inpu...
File tests/preprocessor_tests/input_test.cpp (right):

http://codereview.appspot.com/6130045/diff/2018/tests/preprocessor_tests/inpu...
tests/preprocessor_tests/input_test.cpp:14: EXPECT_FALSE(preprocessor.init(-1,
0, 0));
Could you use NULL instead of 0 to indicate null pointers throughout this test?

http://codereview.appspot.com/6130045/diff/2018/tests/preprocessor_tests/inpu...
tests/preprocessor_tests/input_test.cpp:60: }
Could you add a test which verifies lexing of two strings with explicitly
specified lengths?

[Alok Priyadarshi, 2012-05-01 09:43:41]

Thanks!

http://codereview.appspot.com/6130045/diff/2018/src/compiler/preprocessor/new...
File src/compiler/preprocessor/new/Input.cpp (right):

http://codereview.appspot.com/6130045/diff/2018/src/compiler/preprocessor/new...
src/compiler/preprocessor/new/Input.cpp:29: if (mLength[i] == -1)
On 2012/04/30 18:31:17, kbr1 wrote:
> The test against -1 seems fragile. Perhaps < 0 instead?

Done.

http://codereview.appspot.com/6130045/diff/2018/src/compiler/preprocessor/new...
File src/compiler/preprocessor/new/Input.h (right):

http://codereview.appspot.com/6130045/diff/2018/src/compiler/preprocessor/new...
src/compiler/preprocessor/new/Input.h:20: Input(int count, const char* const
string[], const int length[]);
On 2012/04/30 18:31:17, kbr1 wrote:
> Could you document the semantics of the "length" array?

Added documentation to the main function that takes these arguments:
Preprocessor::init().

http://codereview.appspot.com/6130045/diff/2018/tests/preprocessor_tests/inpu...
File tests/preprocessor_tests/input_test.cpp (right):

http://codereview.appspot.com/6130045/diff/2018/tests/preprocessor_tests/inpu...
tests/preprocessor_tests/input_test.cpp:14: EXPECT_FALSE(preprocessor.init(-1,
0, 0));
On 2012/04/30 18:31:17, kbr1 wrote:
> Could you use NULL instead of 0 to indicate null pointers throughout this
test?

Done everywhere.

http://codereview.appspot.com/6130045/diff/2018/tests/preprocessor_tests/inpu...
tests/preprocessor_tests/input_test.cpp:60: }
On 2012/04/30 18:31:17, kbr1 wrote:
> Could you add a test which verifies lexing of two strings with explicitly
> specified lengths?

Added ReadStringsWithLength, which also verifies that some of the characters are
ignored if a length < strlen(str) is specified.