code review 23220043: go.text/transform: added RemoveFunc transform for remov...

go.text/transform: added RemoveFunc transform for removing individual
runes from the input. This corresponds to ICU's Remove transform.
For example, to remove accents from characters one could use RemoveFunc
as follows:
        nonspacingMark := func(r rune) bool {
                return unicode.Is(unicode.Mn, r)
        }
        transform.Chain(norm.NFD, transform.RemoveFunc(nonspacingMark), norm.NFC)
(Once norm.Form implements Transformer; guess what will be my next CL.)

[mpvl, 2013-11-07 21:45:22]

Hello r@golang.org (cc: golang-dev@googlegroups.com, nigeltao@golang.org),

I'd like you to review this change to
https://code.google.com/p/go.text

[mpvl, 2013-11-22 11:17:06]

Hi Rob,

Did you have a chance to look at this yet? The norm blog post refers to
this. I'm not convinced transform is the best place to put this, btw,
perhaps go.text/unicode/runes or something alike works better. There won't
be too many generic transforms, though. ICU only has this one, I believe.
 A search and replace transform is another possible transform but it would
be better placed in the search package.

Marcel


On Thu, Nov 7, 2013 at 10:45 PM, <mpvl@golang.org> wrote:

> Reviewers: r,
>
> Message:
> Hello r@golang.org (cc: golang-dev@googlegroups.com,
> nigeltao@golang.org),
>
> I'd like you to review this change to
> https://code.google.com/p/go.text
>
>
> Description:
> go.text/transform: added RemoveFunc transform for removing individual
> runes from the input. This corresponds to ICU's Remove transform.
> For example, to remove accents from characters one could use RemoveFunc
> as follows:
>         nonspacingMark := func(r rune) bool {
>                 return unicode.Is(unicode.Mn, r)
>         }
>         transform.Chain(norm.NFD, transform.RemoveFunc(nonspacingMark),
> norm.NFC)
> (Once norm.Form implements Transformer; guess what will be my next CL.)
>
> Please review this at https://codereview.appspot.com/23220043/
>
> Affected files (+198, -2 lines):
>   A transform/examples_test.go
>   M transform/transform.go
>   M transform/transform_test.go
>
>
>

[r, 2013-11-25 15:11:41]

LGTM

https://codereview.appspot.com/23220043/diff/40001/transform/transform.go
File transform/transform.go (right):

https://codereview.appspot.com/23220043/diff/40001/transform/transform.go#new...
transform/transform.go:420: // is true from the input. Illegal bytes in the
input are replaced by RuneError.
RemoveFunc returns a Transformer that removes from the input all runes r for
which f(r) is true.

[mpvl, 2013-11-26 07:29:29]

*** Submitted as
https://code.google.com/p/go/source/detail?r=7c893d4d9458&repo=text ***

go.text/transform: added RemoveFunc transform for removing individual
runes from the input. This corresponds to ICU's Remove transform.
For example, to remove accents from characters one could use RemoveFunc
as follows:
        nonspacingMark := func(r rune) bool {
                return unicode.Is(unicode.Mn, r)
        }
        transform.Chain(norm.NFD, transform.RemoveFunc(nonspacingMark),
norm.NFC)
(Once norm.Form implements Transformer; guess what will be my next CL.)

R=r
CC=golang-dev, nigeltao
https://codereview.appspot.com/23220043

[mpvl, 2013-11-26 08:18:09]

https://codereview.appspot.com/23220043/diff/40001/transform/transform.go
File transform/transform.go (right):

https://codereview.appspot.com/23220043/diff/40001/transform/transform.go#new...
transform/transform.go:420: // is true from the input. Illegal bytes in the
input are replaced by RuneError.
On 2013/11/25 15:11:41, r wrote:
> RemoveFunc returns a Transformer that removes from the input all runes r for
> which f(r) is true.

Done.