code review 11211045: encoding/json: coerce invalid UTF-8 to valid UTF-8 duri...

encoding/json: coerce invalid UTF-8 to valid UTF-8 during Marshal

In practice, rejecting an entire structure due to a single invalid byte
in a string is just too picky, and too hard to track down.
Be consistent with the bulk of the standard library by converting
invalid UTF-8 into UTF-8 with replacement runes.

[rsc, 2013-07-12 18:39:10]

Hello golang-dev@googlegroups.com,

I'd like you to review this change to
https://code.google.com/p/go/

[crawshaw1, 2013-07-12 21:28:52]

LGTM

[rsc, 2013-07-12 21:37:14]

*** Submitted as https://code.google.com/p/go/source/detail?r=f6ae6596a571 ***

encoding/json: coerce invalid UTF-8 to valid UTF-8 during Marshal

In practice, rejecting an entire structure due to a single invalid byte
in a string is just too picky, and too hard to track down.
Be consistent with the bulk of the standard library by converting
invalid UTF-8 into UTF-8 with replacement runes.

R=golang-dev, crawshaw
CC=golang-dev
https://codereview.appspot.com/11211045

[r, 2013-07-12 21:38:09]

LGTM

https://codereview.appspot.com/11211045/diff/6001/src/pkg/encoding/json/decod...
File src/pkg/encoding/json/decode_test.go (right):

https://codereview.appspot.com/11211045/diff/6001/src/pkg/encoding/json/decod...
src/pkg/encoding/json/decode_test.go:395: s := "hello\xffworld"
this is a pretty weak test case but the code is easy enough it's probably ok.
if you're in the mood, make it a table with at least

{"",""}
{"\xff", "\uffd"}
{"\xff\xff", "\uffd\uffd"}
{"a\xffb", "a\uffdb"}
{"\xe6\x97\xa5\xe6\x9c\xac\xff\xaa\x9e", "日本\uffd\uffd\uffd"}