Reduce memory use of lilypond-book

Reduce memory use of lilypond-book

The snippet extraction was using re.search(string[start:end]). This
created many copies of the input string, such that processing a 12M
would use over 6G of memory.

Instead, use regex.search(string, start, end) which avoids the
string slicing.

Do some general cleanups while we're at it:

* move find_toplevel_snippets to book_base

* add unittest for find_toplevel_snippets

* remove wildcard imports

* add some doc strings

[hahnjo, 2020-02-24 10:05:04]

LGTM, thanks for the cleanup!

In the future we might even consider putting all the book_*.py files into a
directory / package. That should make it more natural to import them.

https://codereview.appspot.com/583570043/diff/545630043/python/book_base_test.py
File python/book_base_test.py (right):

https://codereview.appspot.com/583570043/diff/545630043/python/book_base_test...
python/book_base_test.py:48: self.assertEqual(chunks[-2].line_number, 18)
I'd start counting from the beginning (ie, chunks[7]): When extending the test
in the future, we don't have to recount from the end.

[hanwenn, 2020-02-24 10:15:40]

memory

[hanwenn, 2020-02-24 10:17:59]

jonas

[hanwenn, 2020-02-24 10:18:44]

PTAL

[hahnjo, 2020-02-24 10:22:49]

LGTM

[hanwenn, 2020-02-28 17:30:20]

commit fa1ec0c100859f7490f524870743f03b54f89ab7
Author: Han-Wen Nienhuys <hanwen@lilypond.org>
Date:   Mon Feb 24 09:40:08 2020 +0100

    Reduce memory use of lilypond-book