Implemented syntax-enhancements for SELECT FIRST and let().

Implemented syntax-enhancements for SELECT FIRST and let().

AST changes:
 - Added ast.Let, which is like ast.Map, but for a single value.

stdlib changes:
 - Added a 'first' function which returns the first value from an
   IRepeated.

Syntax changes:
 - let keyword in DottySQL: let(x = 10, y = 20) x + y # -> 30
 - FIRST keyword in DottySQL: SELECT FIRST FROM ... WHERE ...
   maps to first(SELECT * FROM ... WHERE ...).

BUG=
R=bgalehouse@google.com

Committed: f18cb1b63f1bcba166c23214c8cbaec9469bc93e

[scudette, 2016-04-24 19:19:41]

How would one use a let in an sql statement?

https://codereview.appspot.com/295920043/diff/40001/efilter_tests/unit/parser...
File efilter_tests/unit/parsers/dottysql/parser.py (right):

https://codereview.appspot.com/295920043/diff/40001/efilter_tests/unit/parser...
efilter_tests/unit/parsers/dottysql/parser.py:104: "let x = 5, y = 10 x + y",
I feel that there needs to be a syntactic break between the let statement and
the use of it (e.g. :) - are you sure the transition between let expression to
use expression is unambiguous? Are you sure the parser wont get confused in some
cases?

[the80srobot, 2016-04-28 09:23:51]

I will submit a patchset today adding more docs and maybe making the syntax
slightly more human friendly (see reply below).

https://codereview.appspot.com/295920043/diff/40001/efilter_tests/unit/parser...
File efilter_tests/unit/parsers/dottysql/parser.py (right):

https://codereview.appspot.com/295920043/diff/40001/efilter_tests/unit/parser...
efilter_tests/unit/parsers/dottysql/parser.py:104: "let x = 5, y = 10 x + y",
On 2016/04/24 19:19:41, scudette wrote:
> I feel that there needs to be a syntactic break between the let statement and
> the use of it (e.g. :) - are you sure the transition between let expression to
> use expression is unambiguous? Are you sure the parser wont get confused in
some
> cases?

It's not ambiguous in the parser, because what follows after the let word is
EXPRESSION [ { "," EXPRESSION } ] EXPRESSION . so there are two ways to be sure
- firstly, after the first EXPRESSION, if you don't encounter a "," then you're
parsing the nested expression (the last one). Secondly, even without the comma
to disambiguate, you would still have 2+ expressions after the "let" and could
simply say that the last one is the nested expression.

From a human-readability standpoint I am not sure whether it looks alright or
not. I kind of think it would work better with a line break or something
similar.

Here is an example of how you would use it (I will add documentation in the
project as well):

# Find children of "init":
LET (init_proc = SELECT proc FROM pslist WHERE proc.command == "init") SELECT *
FROM pslist WHERE proc.ppid == init_proc.pid

# Or, more involved, find children of children of init:

LET
  init_proc = SELECT proc FROM pslist WHERE proc.command == "init",
  init_children_pids = SELECT proc.pid FROM pslist WHERE proc.ppid ==
init_proc.pid

SELECT * FROM pslist WHERE proc.pid IN inin_children_pids

[the80srobot, 2016-04-29 09:56:14]

So I actually decided to get rid of SELECT FIRST because it's a crutch and the
solver should just do the right thing and support the same semantics with LIMIT
1. I'll be adding tests in the next patchset.

[the80srobot, 2016-04-29 13:40:24]

The latest patchset enforces the expectation as to whether a value is repeated
or not in different contexts, but has a performance cost. In the worst case, the
benchmarks are about 50% slower with the change. I'm investigating why that is.

[the80srobot, 2016-04-29 13:45:13]

Looks like the performance cost is down to needing to materialize lazy results
early if we need to determine their cardinality. Don't see an obvious way around
that unless we want to have a compiler.

For the common use cases of EFILTER this is probably fine.

[bgalehouse, 2016-04-29 21:26:36]

LGTM

https://codereview.appspot.com/295920043/diff/80001/efilter/scope.py
File efilter/scope.py (right):

https://codereview.appspot.com/295920043/diff/80001/efilter/scope.py#newcode67
efilter/scope.py:67: "IStructured; got %r." % (scope,))
? (Leftover from an edit?)

https://codereview.appspot.com/295920043/diff/100001/efilter_tests/unit/trans...
File efilter_tests/unit/transforms/solve.py (right):

https://codereview.appspot.com/295920043/diff/100001/efilter_tests/unit/trans...
efilter_tests/unit/transforms/solve.py:91: # Returning multiple resutls from
SELECT should work with set
typo

[the80srobot, 2016-05-02 07:43:10]

Committed patchset #7 (id:120001) manually as
f18cb1b63f1bcba166c23214c8cbaec9469bc93e (presubmit successful).

[the80srobot, 2016-05-02 07:43:17]

https://codereview.appspot.com/295920043/diff/80001/efilter/scope.py
File efilter/scope.py (right):

https://codereview.appspot.com/295920043/diff/80001/efilter/scope.py#newcode67
efilter/scope.py:67: "IStructured; got %r." % (scope,))
On 2016/04/29 21:26:36, bgalehouse wrote:
> ? (Leftover from an edit?)

One of the wonders of Python: if the type of 'scope' ever is a tuple then this
would blow up, because the % operator then expects one % per member of the
tuple. Always giving it a tuple makes sure the TypeError is formatted correctly.

https://codereview.appspot.com/295920043/diff/100001/efilter_tests/unit/trans...
File efilter_tests/unit/transforms/solve.py (right):

https://codereview.appspot.com/295920043/diff/100001/efilter_tests/unit/trans...
efilter_tests/unit/transforms/solve.py:91: # Returning multiple resutls from
SELECT should work with set
On 2016/04/29 21:26:36, bgalehouse wrote:
> typo

Done.