Fix TypeError with unicode tokens from keystone

Fix TypeError with unicode tokens from keystone

One line change to make tokens from v2 keystone auth use the str type
rather than unicode. In mysterious circumstances, that breaks when
twisted tries to construct the request.

Also adds some testing to cover this case and enforce the type.

https://code.launchpad.net/~gz/juju/os_token_string_type_1030897/+merge/127008

(do not edit description out of merge proposal)

[gz, 2012-09-28 15:59:37]

Please take a look.

[hazmat, 2012-10-01 14:42:48]

https://codereview.appspot.com/6573068/diff/1/juju/providers/openstack/client.py
File juju/providers/openstack/client.py (right):

https://codereview.appspot.com/6573068/diff/1/juju/providers/openstack/client...
juju/providers/openstack/client.py:255: self.token = str(token_details["id"])
pls explictly encode utf8. str('some unicode char') is environment dependent.

[gz, 2012-10-01 17:20:44]

On 2012/10/01 14:42:48, hazmat wrote:
> 
> pls explictly encode utf8. str('some unicode char') is environment dependent.

So, this is the messy detail I was trying to avoid, but a summary:


Behaviour of str()
==================

Assuming a unicode object (otherwise __str__ is called), this is basically
equivalent to:

    obj.encode(sys.getdefaultencoding())

This is not locale dependent, and in modern Pythons the knob for changing the
default encoding is hard to access, in the context of the juju client we are
basically assured equivalence to:

    obj.encode('ascii')

Which we could do instead, as the str() spelling is generally a bit of a red
flag as people like using it to try papering over actual encoding problems.


HTTP header encoding
====================

Header field content is specified in HTTP 1.1 as being either ISO-8859-1 or
encoded as per MIME headers:

<http://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4.2>
<http://www.w3.org/Protocols/rfc2616/rfc2616-sec2.html#sec2.2>
<http://www.ietf.org/rfc/rfc2047.txt>

This is not a very common or easy to use encoding scheme, and in practice, no
one really does it. The in progress revision of the specification is changing
the requirements around this:

<http://trac.tools.ietf.org/wg/httpbis/trac/ticket/74>


So, keystone should never give a non-ascii token but for documentation purposes
encoding as utf-8 is confusing given that's not the correct scheme in this
context.

[hazmat, 2012-10-03 14:58:28]

On 2012/10/01 17:20:44, gz wrote:
> On 2012/10/01 14:42:48, hazmat wrote:
> > 
> > pls explictly encode utf8. str('some unicode char') is environment
dependent.
> 
> So, this is the messy detail I was trying to avoid, but a summary:
> 
> 
> Behaviour of str()
> ==================
> 
> Assuming a unicode object (otherwise __str__ is called), this is basically
> equivalent to:
> 
>     obj.encode(sys.getdefaultencoding())
> 
> This is not locale dependent, and in modern Pythons the knob for changing the
> default encoding is hard to access, in the context of the juju client we are
> basically assured equivalence to:
> 
>     obj.encode('ascii')
> 

It may be hard, but people still do it, and its global for the interpreter. Its
sadly common for py2 web apps to make this utf8 via sitecustomize. If we want
ascii and we know, it we should just be explicit about it then relying on
implicit conversion. 

> Which we could do instead, as the str() spelling is generally a bit of a red
> flag as people like using it to try papering over actual encoding problems.
> 

yup. sounds good.


> 
> HTTP header encoding
> ====================
> 
> Header field content is specified in HTTP 1.1 as being either ISO-8859-1 or
> encoded as per MIME headers:
> 
> <http://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4.2>
> <http://www.w3.org/Protocols/rfc2616/rfc2616-sec2.html#sec2.2>
> <http://www.ietf.org/rfc/rfc2047.txt>
> 
> This is not a very common or easy to use encoding scheme, and in practice, no
> one really does it. The in progress revision of the specification is changing
> the requirements around this:
> 
> <http://trac.tools.ietf.org/wg/httpbis/trac/ticket/74>
> 
> 
> So, keystone should never give a non-ascii token but for documentation
purposes
> encoding as utf-8 is confusing given that's not the correct scheme in this
> context.

fair enough.