Rietveld Code Review Tool
Help | Bug tracker | Discussion group | Source code | Sign in
(207)

Issue 12136043: Escape dollar and dot in and out of MongoDB (Closed)

Can't Edit
Can't Publish+Mail
Start Review
Created:
10 years, 9 months ago by sidnei.da.silva
Modified:
10 years, 9 months ago
Reviewers:
mp+177727, thumper, rog
Visibility:
Public.

Description

Escape dollar and dot in and out of MongoDB Both dollar sign and dot are not allowed as keys. In fact, dot creates a sub-document of some sort, which breaks compatibility with pyjuju and causes a panic (LP:1205112). The developer manual suggests escaping those with their unicode full width equivalents: http://docs.mongodb.org/manual/faq/developers/#faq-dollar-sign-escaping https://code.launchpad.net/~sidnei/juju-core/encode-dollar-and-dot/+merge/177727 (do not edit description out of merge proposal)

Patch Set 1 #

Patch Set 2 : Escape dollar and dot in and out of MongoDB #

Total comments: 6
Unified diffs Side-by-side diffs Delta from patch set Stats (+121 lines, -6 lines) Patch
A [revision details] View 1 1 chunk +2 lines, -0 lines 0 comments Download
M state/settings.go View 1 7 chunks +37 lines, -6 lines 5 comments Download
M state/settings_test.go View 2 chunks +82 lines, -0 lines 1 comment Download

Messages

Total messages: 4
sidnei.da.silva
Please take a look.
10 years, 9 months ago (2013-07-31 00:12:01 UTC) #1
sidnei.da.silva
Please take a look.
10 years, 9 months ago (2013-07-31 00:13:35 UTC) #2
thumper
LGTM
10 years, 9 months ago (2013-07-31 01:40:20 UTC) #3
rog
10 years, 9 months ago (2013-07-31 10:00:02 UTC) #4
This is looking good, thanks!
A few suggestions below.

https://codereview.appspot.com/12136043/diff/4001/state/settings.go
File state/settings.go (right):

https://codereview.appspot.com/12136043/diff/4001/state/settings.go#newcode17
state/settings.go:17: var escapeReplacer = strings.NewReplacer(".", "\uff0e",
"$", "\uff04")

Perhaps a comment to the effect that the characters are full-width "." and "$".
Alternatively actually use the (gasp!) unicode in the source.
Alternatively define constants fullWidthDot = "\uff0e" etc.

Choose one :-)

https://codereview.appspot.com/12136043/diff/4001/state/settings.go#newcode144
state/settings.go:144: }

rawKey := escapeReplacer.Replace(key)

and then use rawKey below?

https://codereview.appspot.com/12136043/diff/4001/state/settings.go#newcode197
state/settings.go:197: delete(in, "txn-queue")

Perhaps this should unescape the settings too?
That way you'd also catch other uses (for instance the call in megawatcher.go
which is currently missing an unescape call)

https://codereview.appspot.com/12136043/diff/4001/state/settings.go#newcode201
state/settings.go:201: func unescapeSettingsMap(in map[string]interface{}) {
Instead of two functions, how about:

func replaceKeys(m map[string]interface{}, replacer *strings.Replacer) {
    for key, value := range in {
        if new := replacer.Replace(key); new != key {
              delete(m, key)
              m[new] = value
        }
    }
}

Then you can call replaceKeys(m, unescapeReplacer) to unescape the keys, etc, or
define the below two functions in terms of it.

https://codereview.appspot.com/12136043/diff/4001/state/settings.go#newcode314
state/settings.go:314: newValues := copyMap(values)
A thought: if you made copyMap take a func(string)string to
transform keys, you could create the desired map
in one pass.

Then you'd do:

newValues := copyMap(values, escapeSettingsKey)

assuming a suitable definition for escapeSettingsKey.
Or, if we are allowed go 1.1 features)

newValues := copyMap(values, escapeReplacer.Replace)

https://codereview.appspot.com/12136043/diff/4001/state/settings_test.go
File state/settings_test.go (right):

https://codereview.appspot.com/12136043/diff/4001/state/settings_test.go#newc...
state/settings_test.go:208: func (s *SettingsSuite) TestSetItemEscape(c *C) {

We're testing lots of escaping here, but none of the unescaping AFAICS.
I'd like to see at least one test that just uses the public methods of Settings
and checks that we can read and write settings containing dots and dollars.
Sign in to reply to this message.

Powered by Google App Engine
RSS Feeds Recent Issues | This issue
This is Rietveld f62528b