Data Pipelining rewriter

ContentRewriter that executes <script type="text/os-data"> blocks and replaces them with inline content that sets the DataContext.

To use:
- Add the PipelineDataContentRewriter to the List<ContentRewriter>, at a minimum before the sanitizing content rewriter
- Bind GadgetHtmlParser to SocialMarkupHtmlParser

[awiner, 2009-02-10 00:13:26]

Added support for interdependent batches, only deleting successfully processed
scripts

[awiner, 2009-02-10 00:17:57]

The data pipelining rewriter has a couple of new improvements:
- support for multiple batches, e.g. one data element can introduce variables
used for the EL in a second element.  This is arbitrarily limited to hardcoded 3
batches, but could trivially be made configurable.
- elements that cannot be successfully processed aren't deleted

I'd like to get this checked in the next day or so, so please review if you can.

[louiscryan, 2009-02-11 02:27:59]

Looks good to me. Minor points

http://codereview.appspot.com/14063/diff/3001/33
File
java/gadgets/src/main/java/org/apache/shindig/gadgets/parse/nekohtml/SocialMarkupHtmlParser.java
(right):

http://codereview.appspot.com/14063/diff/3001/33#newcode59
Line 59: private class MarkupDocumentHandler extends DocumentHandler {
rename to SocialMarkupDocumentHander?

http://codereview.appspot.com/14063/diff/3001/33#newcode86
Line 86: scanner.evaluateInputSource(scriptSource);
Its pretty cool that the scanner supports nested input sources :)

http://codereview.appspot.com/14063/diff/3001/33#newcode103
Line 103: scriptContent = new StringBuilder();
a little wasteful, how about using a boolean to represent the state and just
clearing on completion. If builder gets large thats alot of alloc.

http://codereview.appspot.com/14063/diff/3001/27
File
java/gadgets/src/test/java/org/apache/shindig/gadgets/spec/PipelinedDataTest.java
(right):

http://codereview.appspot.com/14063/diff/3001/27#newcode231
Line 231: public void testBatching() throws Exception {
consider making this test cover a little more ground..
more than one round of batching, an entry with no dependents, a dependency bound
into the initial context.

[awiner, 2009-02-11 19:11:45]

Thanks for the comments.

http://codereview.appspot.com/14063/diff/3001/33
File
java/gadgets/src/main/java/org/apache/shindig/gadgets/parse/nekohtml/SocialMarkupHtmlParser.java
(right):

http://codereview.appspot.com/14063/diff/3001/33#newcode59
Line 59: private class MarkupDocumentHandler extends DocumentHandler {
On 2009/02/11 02:27:59, louiscryan wrote:
> rename to SocialMarkupDocumentHander?

Done.

http://codereview.appspot.com/14063/diff/3001/33#newcode86
Line 86: scanner.evaluateInputSource(scriptSource);
On 2009/02/11 02:27:59, louiscryan wrote:
> Its pretty cool that the scanner supports nested input sources :)

Enormously so.

http://codereview.appspot.com/14063/diff/3001/33#newcode103
Line 103: scriptContent = new StringBuilder();
On 2009/02/11 02:27:59, louiscryan wrote:
> a little wasteful, how about using a boolean to represent the state and just
> clearing on completion. If builder gets large thats alot of alloc.

Done.

http://codereview.appspot.com/14063/diff/3001/27
File
java/gadgets/src/test/java/org/apache/shindig/gadgets/spec/PipelinedDataTest.java
(right):

http://codereview.appspot.com/14063/diff/3001/27#newcode231
Line 231: public void testBatching() throws Exception {
On 2009/02/11 02:27:59, louiscryan wrote:
> consider making this test cover a little more ground..
> more than one round of batching, an entry with no dependents, a dependency
bound
> into the initial context.

Wrote a couple more tests - one with two batches (including an initial
dependency), one with a missing dependency so no batches can run.  Also verified
that os-data sections were deleted (or not, depending on the test).