Introduce local GUI releases.

Introduce local GUI releases.

Implemented the new "local" juju-gui-source option.
By default the GUI release is now retrieved from
the charm itself, so that in the deployment process
the charm can avoid connecting to Launchpad.

This should result in the charm being able to be
deployed behind a firewall without the traditional
workarounds. Note that this is not yet demonstrated
and must be QAed in a real firewalled environment.

The process tries is like the following:
- if juju-gui-source is "local", the last tarball
  in the releases dir is installed;
- if juju-gui-source is a stable or development version,
  the charm checks if that version is present in the
  local repository before downloading it from Launchpad.
- the rest of the options should still work as usual
  (but some QA could help, e.g. deploying from a branch).

Also updated the charm documentation and added some
missing tests.

Tests: `make unittest` from the branch root
(I am currently running the functional tests).

QA (assuming 0.10.1 is the latest GUI release):
- `juju bootstrap --debug`;
- `make deploy`;
- check the logs: no PPAs are used, the local release
  is installed;
- `juju set juju-gui juju-gui-source=0.10.0`;
- check the logs: the release is downloaded from
  Launchpad;
- `juju set juju-gui juju-gui-source=0.10.1`;
- check the logs: the charm successfully find
  the 0.10.1 release in the local repository and
  avoids downloading it from Launchpad.
- keep trying to break the charm in all the ways
  you can imagine...

Thank you.

https://code.launchpad.net/~frankban/charms/precise/juju-gui/local-releases/+merge/189896

(do not edit description out of merge proposal)

[frankban, 2013-10-08 15:52:04]

Please take a look.

[matthew.scott, 2013-10-08 18:13:45]

LGTM, QA okay.

[gary.poster, 2013-10-08 18:29:31]

LGTM with many trivials and one slightly-more-than-trivial-but-not-much. 
Relying on Makyo's QA.  Great, thank you!

Gary

https://codereview.appspot.com/14545044/diff/1/config.yaml
File config.yaml (right):

https://codereview.appspot.com/14545044/diff/1/config.yaml#newcode20
config.yaml:20: Where to install Juju GUI from. Possible values are:
Possible values are the following.

[changes below will largely be punctuation and capitalization, shifting
semicolons to periods and capitalizing the start of new sentences.]

https://codereview.appspot.com/14545044/diff/1/config.yaml#newcode21
config.yaml:21: - 'local' (default): the latest local release will be deployed.
Releases
Capitalize "The"

https://codereview.appspot.com/14545044/diff/1/config.yaml#newcode22
config.yaml:22: are stored in the releases directory of this charm;
...charm; -> ...charm.

https://codereview.appspot.com/14545044/diff/1/config.yaml#newcode23
config.yaml:23: - 'stable' the latest stable release will be deployed;
The latest release from the "stable" series will be deployed.

https://codereview.appspot.com/14545044/diff/1/config.yaml#newcode24
config.yaml:24: - 'trunk': the latest trunk release will be deployed;
The latest release from the "trunk" series will be deployed.  Please note that
this is not a build of the current Juju GUI trunk.  For that functionality, use
"lp:juju-gui", as described below.

https://codereview.appspot.com/14545044/diff/1/config.yaml#newcode25
config.yaml:25: - a stable version (e.g '0.1.0'): the specified stable version
will be
The...

https://codereview.appspot.com/14545044/diff/1/config.yaml#newcode28
config.yaml:28: release will be downloaded from Launchpad;
...Launchpad.

https://codereview.appspot.com/14545044/diff/1/config.yaml#newcode29
config.yaml:29: - a trunk version (e.g '0.1.0+build.1'): the specified trunk
version
The...

https://codereview.appspot.com/14545044/diff/1/config.yaml#newcode32
config.yaml:32: release will be downloaded from Launchpad;
...Launchpad.

https://codereview.appspot.com/14545044/diff/1/config.yaml#newcode33
config.yaml:33: - a Bazaar branch (e.g. 'lp:juju-gui'): a release will be
created and
A release...

https://codereview.appspot.com/14545044/diff/1/config.yaml#newcode36
config.yaml:36: revision, e.g. 'lp:juju-gui:42' will checkout revno 42;
...revno 42.

https://codereview.appspot.com/14545044/diff/1/config.yaml#newcode37
config.yaml:37: - a 'url:' prefixed url (ex: url:http://...) of a specific
location
- a 'url:' prefixed url: The release found at the given URL (ex: url:http://...
or url:file://...) will be deployed.

https://codereview.appspot.com/14545044/diff/1/hooks/utils.py
File hooks/utils.py (right):

https://codereview.appspot.com/14545044/diff/1/hooks/utils.py#newcode165
hooks/utils.py:165: \.tgz  # File extension.
would you mind changing this to

    \.(tgz|xz)

per https://bugs.launchpad.net/juju-gui/+bug/1218888 ?

...No, never mind.  We can do that in a separate branch, later.  It would be
good to decrease the charm size, now that releases will be included, but not
something we have to tackle right now.  Stop me! :-)

https://codereview.appspot.com/14545044/diff/1/hooks/utils.py#newcode221
hooks/utils.py:221: *release_version* is None, the URL and name of the latest
release will be
...and file name...

maybe?

https://codereview.appspot.com/14545044/diff/1/hooks/utils.py#newcode690
hooks/utils.py:690: return version_path_map[-1][1]
Maybe we ought to sort the releases more carefully, if this is the expected
behavior?  otherwise 0.8 will sort before 0.10, right?

https://codereview.appspot.com/14545044/diff/1/hooks/utils.py#newcode724
hooks/utils.py:724: url, filename = get_launchpad_release(project, origin,
version)
In theory you could try release files here too. <shrug>

https://codereview.appspot.com/14545044/diff/1/tests/20-functional.test
File tests/20-functional.test (right):

https://codereview.appspot.com/14545044/diff/1/tests/20-functional.test#newco...
tests/20-functional.test:220: # Also make sure the correct cache headers are
sent.
:-)

https://codereview.appspot.com/14545044/diff/1/tests/test_utils.py
File tests/test_utils.py (right):

https://codereview.appspot.com/14545044/diff/1/tests/test_utils.py#newcode324
tests/test_utils.py:324: # The last release is correctly retrieved.
I'd suggest adding a variant of this test that includes 0.12 and 0.9 and make
sure that 0.12 is used.  Or similar.

[frankban, 2013-10-09 09:25:34]

*** Submitted:

Introduce local GUI releases.

Implemented the new "local" juju-gui-source option.
By default the GUI release is now retrieved from
the charm itself, so that in the deployment process
the charm can avoid connecting to Launchpad.

This should result in the charm being able to be
deployed behind a firewall without the traditional
workarounds. Note that this is not yet demonstrated
and must be QAed in a real firewalled environment.

The process tries is like the following:
- if juju-gui-source is "local", the last tarball
  in the releases dir is installed;
- if juju-gui-source is a stable or development version,
  the charm checks if that version is present in the
  local repository before downloading it from Launchpad.
- the rest of the options should still work as usual
  (but some QA could help, e.g. deploying from a branch).

Also updated the charm documentation and added some
missing tests.

Tests: `make unittest` from the branch root
(I am currently running the functional tests).

QA (assuming 0.10.1 is the latest GUI release):
- `juju bootstrap --debug`;
- `make deploy`;
- check the logs: no PPAs are used, the local release
  is installed;
- `juju set juju-gui juju-gui-source=0.10.0`;
- check the logs: the release is downloaded from
  Launchpad;
- `juju set juju-gui juju-gui-source=0.10.1`;
- check the logs: the charm successfully find
  the 0.10.1 release in the local repository and
  avoids downloading it from Launchpad.
- keep trying to break the charm in all the ways
  you can imagine...

Thank you.

R=matthew.scott, gary.poster
CC=
https://codereview.appspot.com/14545044

https://codereview.appspot.com/14545044/diff/1/config.yaml
File config.yaml (right):

https://codereview.appspot.com/14545044/diff/1/config.yaml#newcode20
config.yaml:20: Where to install Juju GUI from. Possible values are:
On 2013/10/08 18:29:31, gary.poster wrote:
> Possible values are the following.
> 
> [changes below will largely be punctuation and capitalization, shifting
> semicolons to periods and capitalizing the start of new sentences.]

Done.

https://codereview.appspot.com/14545044/diff/1/config.yaml#newcode21
config.yaml:21: - 'local' (default): the latest local release will be deployed.
Releases
On 2013/10/08 18:29:31, gary.poster wrote:
> Capitalize "The"

Done.

https://codereview.appspot.com/14545044/diff/1/config.yaml#newcode22
config.yaml:22: are stored in the releases directory of this charm;
On 2013/10/08 18:29:31, gary.poster wrote:
> ...charm; -> ...charm.

Done.

https://codereview.appspot.com/14545044/diff/1/config.yaml#newcode23
config.yaml:23: - 'stable' the latest stable release will be deployed;
On 2013/10/08 18:29:31, gary.poster wrote:
> The latest release from the "stable" series will be deployed.

Done.

https://codereview.appspot.com/14545044/diff/1/config.yaml#newcode24
config.yaml:24: - 'trunk': the latest trunk release will be deployed;
On 2013/10/08 18:29:31, gary.poster wrote:
> The latest release from the "trunk" series will be deployed.  Please note that
> this is not a build of the current Juju GUI trunk.  For that functionality,
use
> "lp:juju-gui", as described below.

Good clarification! Done.

https://codereview.appspot.com/14545044/diff/1/config.yaml#newcode25
config.yaml:25: - a stable version (e.g '0.1.0'): the specified stable version
will be
On 2013/10/08 18:29:31, gary.poster wrote:
> The...

Done.

https://codereview.appspot.com/14545044/diff/1/config.yaml#newcode28
config.yaml:28: release will be downloaded from Launchpad;
On 2013/10/08 18:29:31, gary.poster wrote:
> ...Launchpad.

Done.

https://codereview.appspot.com/14545044/diff/1/config.yaml#newcode29
config.yaml:29: - a trunk version (e.g '0.1.0+build.1'): the specified trunk
version
On 2013/10/08 18:29:31, gary.poster wrote:
> The...

Done.

https://codereview.appspot.com/14545044/diff/1/config.yaml#newcode32
config.yaml:32: release will be downloaded from Launchpad;
On 2013/10/08 18:29:31, gary.poster wrote:
> ...Launchpad.

Done.

https://codereview.appspot.com/14545044/diff/1/config.yaml#newcode33
config.yaml:33: - a Bazaar branch (e.g. 'lp:juju-gui'): a release will be
created and
On 2013/10/08 18:29:31, gary.poster wrote:
> A release...

Done.

https://codereview.appspot.com/14545044/diff/1/config.yaml#newcode36
config.yaml:36: revision, e.g. 'lp:juju-gui:42' will checkout revno 42;
On 2013/10/08 18:29:31, gary.poster wrote:
> ...revno 42.

Done.

https://codereview.appspot.com/14545044/diff/1/config.yaml#newcode37
config.yaml:37: - a 'url:' prefixed url (ex: url:http://...) of a specific
location
On 2013/10/08 18:29:31, gary.poster wrote:
> - a 'url:' prefixed url: The release found at the given URL (ex:
url:http://...
> or url:file://...) will be deployed.

Done.

https://codereview.appspot.com/14545044/diff/1/hooks/utils.py
File hooks/utils.py (right):

https://codereview.appspot.com/14545044/diff/1/hooks/utils.py#newcode165
hooks/utils.py:165: \.tgz  # File extension.
On 2013/10/08 18:29:31, gary.poster wrote:
> would you mind changing this to
> 
>     \.(tgz|xz)
> 
> per https://bugs.launchpad.net/juju-gui/+bug/1218888 ?
> 
> ...No, never mind.  We can do that in a separate branch, later.  It would be
> good to decrease the charm size, now that releases will be included, but not
> something we have to tackle right now.  Stop me! :-)

Good idea. I agree we can do that quickly in a separate branch later. FWIW we
also need to change get_launchpad_release.

https://codereview.appspot.com/14545044/diff/1/hooks/utils.py#newcode221
hooks/utils.py:221: *release_version* is None, the URL and name of the latest
release will be
On 2013/10/08 18:29:31, gary.poster wrote:
> ...and file name...
> 
> maybe?

Done.

https://codereview.appspot.com/14545044/diff/1/hooks/utils.py#newcode690
hooks/utils.py:690: return version_path_map[-1][1]
On 2013/10/08 18:29:31, gary.poster wrote:
> Maybe we ought to sort the releases more carefully, if this is the expected
> behavior?  otherwise 0.8 will sort before 0.10, right?

Indeed! Thank you for spotting this bug! Fixed.

https://codereview.appspot.com/14545044/diff/1/hooks/utils.py#newcode724
hooks/utils.py:724: url, filename = get_launchpad_release(project, origin,
version)
On 2013/10/08 18:29:31, gary.poster wrote:
> In theory you could try release files here too. <shrug>

Yes, I preferred not to do that, at least on this branch, because I wanted to
tell a simpler "stable == download from Launchpad" story. But I agree this could
be an optimization for the future.

https://codereview.appspot.com/14545044/diff/1/tests/test_utils.py
File tests/test_utils.py (right):

https://codereview.appspot.com/14545044/diff/1/tests/test_utils.py#newcode324
tests/test_utils.py:324: # The last release is correctly retrieved.
On 2013/10/08 18:29:31, gary.poster wrote:
> I'd suggest adding a variant of this test that includes 0.12 and 0.9 and make
> sure that 0.12 is used.  Or similar.

Done.

[frankban, 2013-10-09 09:30:01]

Thank you both for the reviews!