Fix deployer integration.

Fix deployer integration.

Fix the deployer error when the bundle includes
services with constraints.

Updated the deployer dependency. The new tarball
is generated from lp:~frankban/juju-deployer/guienv-fixes
I will take care of proposing that branch later, however,
the diff can be found here: 
http://pastebin.ubuntu.com/6376113/

This branch includes the work Benji did for fixing
this issue.

TODO: but these are not release blockers IMHO:
- improve the way we handle the set up of
  testing/production environments (as Rick suggested);
- improve the GUI server bundle validation, e.g. disallow
  the deployment of local charms, or better check that the
  YAML structure is what we expect;
- investigate and fix the bundle deployment error handling:
  why errors in the deployer process are not propagated?
  Why concurrent.futures explodes with an error while trying
  to set up the exception to be propagated in the Future?
  This is important because right now an error in the
  deployer is not notified, so the deployment is forever
  considered in progress and all the other deployments
  will be just queued... Actually this is "almost" a release
  blocker.

Tests: make unittest

QA:
I used this bundle: http://pastebin.ubuntu.com/6376098/
to live test the branch.
Bootstrap a juju environment, run `make deploy` and then
switch the gui source to lp:juju-gui.
When everything is ready, try to deploy a bundle
which includes num_units != 1, customized config and
constraints. Check the bundle is deployed correctly
and the resulting services have the defined
number of units, options and constraints.
Now try to deploy a bundle with invalid constraints: right
after the drag and drop, the GUI should notify an
invalid constraints error.

https://code.launchpad.net/~frankban/charms/precise/juju-gui/sanitize-constraints-and-other-fun/+merge/194343

(do not edit description out of merge proposal)

[frankban, 2013-11-07 14:03:16]

Please take a look.

[rharding, 2013-11-07 14:23:39]

LGTM with comments below. The only one is the requirements using the bzr'd
version of the deployer still.

https://codereview.appspot.com/22810043/diff/1/hooks/utils.py
File hooks/utils.py (right):

https://codereview.appspot.com/22810043/diff/1/hooks/utils.py#newcode123
hooks/utils.py:123: 'juju-deployer-0.2.8.tar.gz',
is this 0.2.8 your custom build of the deployer then? Can we add a -gui or
something to the end to distinguish it from the real release?

https://codereview.appspot.com/22810043/diff/1/server/guiserver/bundles/base.py
File server/guiserver/bundles/base.py (right):

https://codereview.appspot.com/22810043/diff/1/server/guiserver/bundles/base....
server/guiserver/bundles/base.py:54: IMPORTER_OPTIONS =
deployer.cli.setup_parser().parse_args([])
brilliant, <3 this to help prevent that breakage later on.

https://codereview.appspot.com/22810043/diff/1/server/guiserver/bundles/utils.py
File server/guiserver/bundles/utils.py (right):

https://codereview.appspot.com/22810043/diff/1/server/guiserver/bundles/utils...
server/guiserver/bundles/utils.py:131: if not isinstance(constraints,
collections.Mapping):
interesting, hadn't seen this before

https://codereview.appspot.com/22810043/diff/1/server/guiserver/bundles/utils...
server/guiserver/bundles/utils.py:174: service_data['constraints'] =
_prepare_constraints(constraints)
so this throws a ValueError and it's caught somewhere above this prepare_bundle?

https://codereview.appspot.com/22810043/diff/1/server/guiserver/bundles/views.py
File server/guiserver/bundles/views.py (right):

https://codereview.appspot.com/22810043/diff/1/server/guiserver/bundles/views...
server/guiserver/bundles/views.py:120: except ValueError as err:
ah, got it. Caught here.

https://codereview.appspot.com/22810043/diff/1/tests/requirements.pip
File tests/requirements.pip (right):

https://codereview.appspot.com/22810043/diff/1/tests/requirements.pip#newcode37
tests/requirements.pip:37: -e
bzr+http://launchpad.net/juju-deployer/darwin#egg=juju-deployer
shouldn't this be using the deployer in deps vs the launchpad dir?

[rharding, 2013-11-07 14:24:17]

oh, where are the hard coded production versions?

[frankban, 2013-11-07 14:43:11]

Please take a look.

https://codereview.appspot.com/22810043/diff/1/hooks/utils.py
File hooks/utils.py (right):

https://codereview.appspot.com/22810043/diff/1/hooks/utils.py#newcode123
hooks/utils.py:123: 'juju-deployer-0.2.8.tar.gz',
On 2013/11/07 14:23:40, rharding wrote:
> is this 0.2.8 your custom build of the deployer then? Can we add a -gui or
> something to the end to distinguish it from the real release?

Very good point. As discussed on IRC I added an XXX comment here instead.

https://codereview.appspot.com/22810043/diff/1/server/guiserver/bundles/base.py
File server/guiserver/bundles/base.py (right):

https://codereview.appspot.com/22810043/diff/1/server/guiserver/bundles/base....
server/guiserver/bundles/base.py:54: IMPORTER_OPTIONS =
deployer.cli.setup_parser().parse_args([])
On 2013/11/07 14:23:40, rharding wrote:
> brilliant, <3 this to help prevent that breakage later on.

Indeed, this is cool, thanks Benji!

https://codereview.appspot.com/22810043/diff/1/tests/requirements.pip
File tests/requirements.pip (right):

https://codereview.appspot.com/22810043/diff/1/tests/requirements.pip#newcode37
tests/requirements.pip:37: -e
bzr+http://launchpad.net/juju-deployer/darwin#egg=juju-deployer
On 2013/11/07 14:23:40, rharding wrote:
> shouldn't this be using the deployer in deps vs the launchpad dir?

As mentioned in the MP description, this is another card.

[rharding, 2013-11-07 15:06:20]

QA ok, tested a successful deploy and the error for a bundle with invalid
constraints on a service.

There was one layout qa issue where the two services were stacked and it wasn't
obvious mysql was beneath the service block for wordpress. That's not an issue
with these changes though so LGTM.

[benji, 2013-11-07 15:18:58]

LGTM, QA passed

Thanks for the work

[frankban, 2013-11-07 15:39:53]

Thank you both!

[frankban, 2013-11-07 15:47:33]

*** Submitted:

Fix deployer integration.

Fix the deployer error when the bundle includes
services with constraints.

Updated the deployer dependency. The new tarball
is generated from lp:~frankban/juju-deployer/guienv-fixes
I will take care of proposing that branch later, however,
the diff can be found here: 
http://pastebin.ubuntu.com/6376113/

This branch includes the work Benji did for fixing
this issue.

TODO: but these are not release blockers IMHO:
- improve the way we handle the set up of
  testing/production environments (as Rick suggested);
- improve the GUI server bundle validation, e.g. disallow
  the deployment of local charms, or better check that the
  YAML structure is what we expect;
- investigate and fix the bundle deployment error handling:
  why errors in the deployer process are not propagated?
  Why concurrent.futures explodes with an error while trying
  to set up the exception to be propagated in the Future?
  This is important because right now an error in the
  deployer is not notified, so the deployment is forever
  considered in progress and all the other deployments
  will be just queued... Actually this is "almost" a release
  blocker.

Tests: make unittest

QA:
I used this bundle: http://pastebin.ubuntu.com/6376098/
to live test the branch.
Bootstrap a juju environment, run `make deploy` and then
switch the gui source to lp:juju-gui.
When everything is ready, try to deploy a bundle
which includes num_units != 1, customized config and
constraints. Check the bundle is deployed correctly
and the resulting services have the defined
number of units, options and constraints.
Now try to deploy a bundle with invalid constraints: right
after the drag and drop, the GUI should notify an
invalid constraints error.

R=rharding, benji
CC=
https://codereview.appspot.com/22810043