Adds a site search to website and improves doc search

See Issue 1405 - this allows users to search the site as
well as the docs, and improves the way that Google is
used to search for docs.  In the first instance this will
mean that the English site gets a different search from
translated sites. Once this is pushed, we will need to get
translations for the "Search site" and "Search docs"
phrases, and I will try to put the language phrase into
the search box automatically, rather than having a lot 
of near identical search-box.ihtml files.

[PhilEHolmes, 2011-08-18 16:59:07]

Updates website search.  Please review.

[janek, 2011-08-22 21:06:25]

Hi,

i've downloaded the patch and compiled website (with 'make website'),
but there is a problem: there are things missing in the resulting
website, so i cannot judge if it will be ok.  See in the attachment
how /lilypond-git/build/out-website/website/index.html looks on my
machine.  (maybe i'm looking at the wrong file?)
Phil, could you compile the website with your changes and host it
somewhere? (or tell me how to do this properly)
At the very least i'd like to see some screenshots, but screenshots
won't show me how the top bar behaves when window is resized.

cheers,
Janek

[Graham Percival, 2011-08-23 18:04:39]

On Tue, Aug 23, 2011 at 03:32:40PM +0100, Phil Holmes wrote:
> ----- Original Message ----- From: "Graham Percival"
> <graham@percival-music.ca>
> >Partial instructions are buried in
> >http://code.google.com/p/lilypond/issues/detail?id=1663
> >It would be nice if those instructions could migrate to the CG.
> 
> How about this (in a new section - Building the website locally)
> 
> Building the website locally
> 
> Start by making the binary files with make (typical time - 10
> minutes)

Not necessary unless you want to regenerate the pictures and
examples.  Hmm, this step kind-of relies on GOP 11 git
repositories, which is delayed for 2 weeks.  Growl.

ok, for now maybe make a separate "generating pictures and
examples" section?

> Run make website (takes a minute or so).
> 
> This creates a number of files, starting with:
> 
> lilypond-git/build/out-website/index.html
...
> lilypond-git/build/out-website/website/index.html

I don't think it's worth pointing out that first one at all.  Just
point directly to the second one, since it's built at the same
time anyway.

> To compile the documentation run make doc.  This typically
> takes a couple of hours or so.
> 
> After this has been run, we get a number more copies of
> index.html, including:
> 
> /lilypond-git/build/out-www/offline-
> root/Documentation/web/index.html

This is **not** the website.  That's a copy of the "web" manual,
but it's built with a completely different mechanism.


Phil, I may be misremembering.  Do you currently have
  ~/lilypond/media/pictures/
  ~/lilypond/media/ly-examples/
directories in your home directory?  I thought that you had that
set up, but if not then we should do that now.  Basically, you
follow the instructions in "initial setup" in CG 6.2, but instead
of uploading those directories to a separate server, you just copy
them on your local computer instead.

Cheers,
- Graham

[email_philholmes.net, 2011-08-23 20:15:20]

----- Original Message ----- 
From: "Graham Percival" <graham@percival-music.ca>
To: "Phil Holmes" <mail@philholmes.net>
Cc: "Janek Warchol" <janek.lilypond@gmail.com>; 
<reply@codereview.appspotmail.com>; <PhilEHolmes@googlemail.com>; 
<lilypond-devel@gnu.org>; <percival.music.ca@gmail.com>
Sent: Tuesday, August 23, 2011 7:04 PM
Subject: Re: Adds a site search to website and improves doc search 
(issue4894053)


> On Tue, Aug 23, 2011 at 03:32:40PM +0100, Phil Holmes wrote:
>> ----- Original Message ----- From: "Graham Percival"
>> <graham@percival-music.ca>
>> >Partial instructions are buried in
>> >http://code.google.com/p/lilypond/issues/detail?id=1663
>> >It would be nice if those instructions could migrate to the CG.
>>
>> How about this (in a new section - Building the website locally)
>>
>> Building the website locally
>>
>> Start by making the binary files with make (typical time - 10
>> minutes)
>
> Not necessary unless you want to regenerate the pictures and
> examples.  Hmm, this step kind-of relies on GOP 11 git
> repositories, which is delayed for 2 weeks.  Growl.

Yeah - you're almost certainly right.  I've got so used to the need to run 
this first.  I'll redo and check.

> ok, for now maybe make a separate "generating pictures and
> examples" section?
>
>> Run make website (takes a minute or so).
>>
>> This creates a number of files, starting with:
>>
>> lilypond-git/build/out-website/index.html
> ...
>> lilypond-git/build/out-website/website/index.html
>
> I don't think it's worth pointing out that first one at all.  Just
> point directly to the second one, since it's built at the same
> time anyway.

I'd like to state it's necessary to ignore the other, but happy to do in 
that way.

>> To compile the documentation run make doc.  This typically
>> takes a couple of hours or so.
>>
>> After this has been run, we get a number more copies of
>> index.html, including:
>>
>> /lilypond-git/build/out-www/offline-
>> root/Documentation/web/index.html
>
> This is **not** the website.  That's a copy of the "web" manual,
> but it's built with a completely different mechanism.

AFAICS it's the only version of the web material that looks like the real 
website - images, css, etc, included.

> Phil, I may be misremembering.  Do you currently have
>  ~/lilypond/media/pictures/
>  ~/lilypond/media/ly-examples/
> directories in your home directory?  I thought that you had that
> set up, but if not then we should do that now.  Basically, you
> follow the instructions in "initial setup" in CG 6.2, but instead
> of uploading those directories to a separate server, you just copy
> them on your local computer instead.

No - I don't.  But wouldn't this be setting up my system in order to build a 
"correct" website?  What we're trying to do here is write instructions on 
how to check that changes to the website that contributors make can be 
easily checked.

--
Phil Holmes

[mail_philholmes.net, 2011-08-23 20:32:58]

----- Original Message ----- 
From: "Phil Holmes" <email@philholmes.net>
To: "Graham Percival" <graham@percival-music.ca>
Cc: <reply@codereview.appspotmail.com>; <PhilEHolmes@googlemail.com>; 
<lilypond-devel@gnu.org>; <percival.music.ca@gmail.com>
Sent: Tuesday, August 23, 2011 9:15 PM
Subject: Re: Adds a site search to website and improves doc 
search(issue4894053)


> ----- Original Message ----- 
> From: "Graham Percival" <graham@percival-music.ca>
> To: "Phil Holmes" <mail@philholmes.net>

>> This is **not** the website.  That's a copy of the "web" manual,
>> but it's built with a completely different mechanism.
>
> AFAICS it's the only version of the web material that looks like the real 
> website - images, css, etc, included.

Thinking about this a little further - part of the problem is that make 
website doesn't do anything like make the website.  It just creates the html 
files.  It really ought to put the images and css files in the right place, 
fix up the links etc.  I'm sure this would cause the normal difficulties 
about getting it to work, but a) I don't think doing this would add much to 
the run time of make website and b) it's really what should happen.  You 
agree?

--
Phil Holmes

[Graham Percival, 2011-08-23 21:11:33]

On Tue, Aug 23, 2011 at 09:32:50PM +0100, Phil Holmes wrote:
> ----- Original Message ----- From: "Phil Holmes"
> <email@philholmes.net>
> >----- Original Message ----- From: "Graham Percival"
> ><graham@percival-music.ca>
> >To: "Phil Holmes" <mail@philholmes.net>
> 
> >>This is **not** the website.  That's a copy of the "web" manual,
> >>but it's built with a completely different mechanism.
> >
> >AFAICS it's the only version of the web material that looks like
> >the real website - images, css, etc, included.
> 
> Thinking about this a little further - part of the problem is that
> make website doesn't do anything like make the website.  It just
> creates the html files.  It really ought to put the images and css
> files in the right place, fix up the links etc.

It does do this!  -- if you have stuff in $HOME/lilypond/media/

> I'm sure this would cause the normal difficulties about getting
> it to work, but a) I don't think doing this would add much to
> the run time of make website and b) it's really what should
> happen.  You agree?

Problem:
- webserver does not have imagemagick
- webserver does not have lilypond
- webserver certainly cannot compile lilypond

Hence the $HOME/lilypond/media/ stuff.


I would like to put those in a separate git repository, so that it
would be easier to create the website (just point a variable at
your version of that other repository, instead of having to
manually create $HOME/lilypond/media/ yourself).  But that step
will be discussed in GOP-PROP 11, which is still 2 weeks off
(depending on how much time I spend on maintenance and build
issues).

In the short-term, I think it would be extremely helpful if you
could create the $HOME/lilypond/media stuff, then create the
website from a blank build tree -- convince yourself that you
can/will get the full website without running make or make doc.

Cheers,
- Graham

[Graham Percival (old account), 2011-08-26 04:26:32]

I have grave doubts about adding a second search box.
1) it makes the top bar uncomfortably squashed in my default web browser window
2) it requires users to choose which type of search they want to do.


#1 isn't about me forcing my desktop browser preferences on anybody, but rather
I'm making the point that almost all other websites work well on my desktop.  I
think we should be very cautious about making lilypond.org less accessible than
the average websote.  (or perhaps I should specify "average geek website", since
I'm sure that certain other genres of websites are much less accessible than
places that I visit)

Do we have any compelling evidence that we _need_ to separate these searches? 
Didn't the previous system just search the website and stable docs; wasn't that
sufficient?

Also, IIRC there was a suggestion that we search for something like
  site:lilypond.org/Documentation/v2.14/
instead of adding +2.14.  Am I misremembering / would that work / etc?

[mail_philholmes.net, 2011-08-26 08:15:12]

----- Original Message ----- 
From: <percival.music.ca@gmail.com>
To: <PhilEHolmes@googlemail.com>; <janek.lilypond@gmail.com>; 
<graham@percival-music.ca>; <email@philholmes.net>; <mail@philholmes.net>
Cc: <lilypond-devel@gnu.org>; <reply@codereview.appspotmail.com>
Sent: Friday, August 26, 2011 5:26 AM
Subject: Re: Adds a site search to website and improves doc search (issue 
4894053)


>I have grave doubts about adding a second search box.
> 1) it makes the top bar uncomfortably squashed in my default web browser
> window
> 2) it requires users to choose which type of search they want to do.
>
>
> #1 isn't about me forcing my desktop browser preferences on anybody, but
> rather I'm making the point that almost all other websites work well on
> my desktop.  I think we should be very cautious about making
> lilypond.org less accessible than the average websote.  (or perhaps I
> should specify "average geek website", since I'm sure that certain other
> genres of websites are much less accessible than places that I visit)
>
> Do we have any compelling evidence that we _need_ to separate these
> searches?  Didn't the previous system just search the website and stable
> docs; wasn't that sufficient?

No.  It doesn't search the website at all.  That's the original problem - 
there was a report that if you are looking for "frogs" in the search box, it 
returns nothing.  That's not in any way good.  So we need to be able to 
search the site.  But if you modify the current search to search the site, 
then you get lots of hits from non-stable doc information if you're looking 
for information in the documentation.  Having 2 boxes seemed the best way to 
me to overcome this particular problem.

> Also, IIRC there was a suggestion that we search for something like
>   site:lilypond.org/Documentation/v2.14/
> instead of adding +2.14.  Am I misremembering / would that work / etc?
>
>
> http://codereview.appspot.com/4894053/
>

Good point.  I believe that change that I originally tested got lost in one 
of my build nukes.

--
Phil Holmes

[janek, 2011-09-06 20:40:26]

2011/8/26  <percival.music.ca@gmail.com>:
> I have grave doubts about adding a second search box.
> 1) it makes the top bar uncomfortably squashed in my default web browser
> window
> 2) it requires users to choose which type of search they want to do.
>
>
> #1 isn't about me forcing my desktop browser preferences on anybody, but
> rather I'm making the point that almost all other websites work well on
> my desktop.  I think we should be very cautious about making
> lilypond.org less accessible than the average websote.  (or perhaps I
> should specify "average geek website", since I'm sure that certain other
> genres of websites are much less accessible than places that I visit)
>
> Do we have any compelling evidence that we _need_ to separate these
> searches?  Didn't the previous system just search the website and stable
> docs; wasn't that sufficient?
>
> Also, IIRC there was a suggestion that we search for something like
>  site:lilypond.org/Documentation/v2.14/
> instead of adding +2.14.  Am I misremembering / would that work / etc?

What about searching all docs and website by default, but adding
something like "advanced search" where one can set what he wants to
search?
If not, i guess that searching stable docs + website would be ok (if
it's possible).

cheers,
Janek