code review 5689054: doc: add The Laws of Reflection article

doc/articles/laws_of_reflection.tmpl

Index: doc/articles/laws_of_reflection.tmpl
===================================================================
new file mode 100644
--- /dev/null
+++ b/doc/articles/laws_of_reflection.tmpl
@@ -0,0 +1,654 @@
+<!--{
+	"Title": "The Laws of Reflection"
+}-->
+{{donotedit}}
+
+<p>
+Reflection in computing is the
+ability of a program to examine its own structure, particularly
+through types; it's a form of metaprogramming. It's also a great
+source of confusion.
+</p>
+
+<p>
+In this article we attempt to clarify things by explaining how
+reflection works in Go. Each language's reflection model is
+different (and many languages don't support it at all), but
+this article is about Go, so for the rest of this article the word
+"reflection" should be taken to mean "reflection in Go".
+</p>
+
+<p><b>Types and interfaces</b></p>
+
+<p>
+Because reflection builds on the type system, let's start with a
+refresher about types in Go.
+</p>
+
+<p>
+Go is statically typed. Every variable has a static type, that is,
+exactly one type known and fixed at compile time: <code>int</code>,
+<code>float32</code>, <code>*MyType</code>, <code>[]byte</code>,
+and so on. If we declare
+</p>
+
+{{code "progs/interface.go" `/type MyInt/` `/STOP/`}}
+
+<p>
+then <code>i</code> has type <code>int</code> and <code>j</code>
+has type <code>MyInt</code>. The variables <code>i</code> and
+<code>j</code> have distinct static types and, although they have
+the same underlying type, they cannot be assigned to one another
+without a conversion.
+</p>
+
+<p>
+One important category of type is interface types, which represent
+fixed sets of methods. An interface variable can store any concrete
+(non-interface) value as long as that value implements the
+interface's methods. A well-known pair of examples is
+<code>io.Reader</code> and <code>io.Writer</code>, the types
+<code>Reader</code> and <code>Writer</code> from the <a href=
+"http://golang.org/pkg/io/">io package</a>:
+</p>
+
+{{code "progs/interface.go" `/// Reader/` `/STOP/`}}
+
+<p>
+Any type that implements a <code>Read</code> (or
+<code>Write</code>) method with this signature is said to implement
+<code>io.Reader</code> (or <code>io.Writer</code>). For the
+purposes of this discussion, that means that a variable of type
+<code>io.Reader</code> can hold any value whose type has a
+<code>Read</code> method:
+</p>
+
+{{code "progs/interface.go" `/func readers/` `/STOP/`}}
+
+<p>
+It's important to be clear that whatever concrete value
+<code>r</code> may hold, <code>r</code>'s type is always
+<code>io.Reader</code>: Go is statically typed and the static type
+of <code>r</code> is <code>io.Reader</code>.</p>
+
+<p>
+An extremely important example of an interface type is the empty
+interface:
+</p>
+
+<pre>
+interface{}
+</pre>
+
+<p>
+It represents the empty set of methods and is satisfied by any
+value at all, since any value has zero or more methods.
+</p>
+
+<p>
+Some people say that Go's interfaces are dynamically typed, but
+that is misleading. They are statically typed: a variable of
+interface type always has the same static type, and even though at
+run time the value stored in the interface variable may change
+type, that value will always satisfy the interface.
+</p>
+
+<p>
+We need to be precise about all this because reflection and
+interfaces are closely related.
+</p>
+
+<p><b>The representation of an interface</b></p>
+
+<p>
+Russ Cox has written a <a href=
+"http://research.swtch.com/2009/12/go-data-structures-interfaces.html">
+detailed blog post</a> about the representation of interface values
+in Go. It's not necessary to repeat the full story here, but a
+simplified summary is in order.
+</p>
+
+<p>
+A variable of interface type stores a pair: the concrete value
+assigned to the variable, and that value's type descriptor.
+To be more precise, the value is the underlying concrete data item
+that implements the interface and the type describes the full type
+of that item. For instance, after
+</p>
+
+{{code "progs/interface.go" `/func typeAssertions/` `/STOP/`}}
+
+<p>
+<code>r</code> contains, schematically, the (value, type) pair,
+(<code>tty</code>, <code>*os.File</code>). Notice that the type
+<code>*os.File</code> implements methods other than
+<code>Read</code>; even though the interface value provides access
+only to the <code>Read</code> method, the value inside carries all
+the type information about that value. That's why we can do things
+like this:
+</p>
+
+{{code "progs/interface.go" `/var w io.Writer/` `/STOP/`}}
+
+<p>
+The expression in this assignment is a type assertion; what it
+asserts is that the item inside <code>r</code> also implements
+<code>io.Writer</code>, and so we can assign it to <code>w</code>.
+After the assignment, <code>w</code> will contain the pair
+(<code>tty</code>, <code>*os.File</code>). That's the same pair as
+was held in <code>r</code>. The static type of the interface
+determines what methods may be invoked with an interface variable,
+even though the concrete value inside may have a larger set of
+methods.
+</p>
+
+<p>
+Continuing, we can do this:
+</p>
+
+{{code "progs/interface.go" `/var empty interface{}/` `/STOP/`}}
+
+<p>
+and our empty interface value <code>e</code> will again contain
+that same pair, (<code>tty</code>, <code>*os.File</code>). That's
+handy: an empty interface can hold any value and contains all the
+information we could ever need about that value.
+</p>
+
+<p>
+(We don't need a type assertion here because it's known statically
+that <code>w</code> satisfies the empty interface. In the example
+where we moved a value from a <code>Reader</code> to a
+<code>Writer</code>, we needed to be explicit and use a type
+assertion because <code>Writer</code>'s methods are not a
+subset of <code>Reader</code>'s.)
+</p>
+
+<p>
+One important detail is that the pair inside an interface always
+has the form (value, concrete type) and cannot have the form
+(value, interface type). Interfaces do not hold interface
+values.
+</p>
+
+<p>
+Now we're ready to reflect.
+</p>
+
+<p><b>The first law of reflection</b></p>
+
+<p><b>1. Reflection goes from interface value to reflection object.</b></p>
+
+<p>
+At the basic level, reflection is just a mechanism to examine the
+type and value pair stored inside an interface variable. To get
+started, there are two types we need to know about in
+<a href="http://golang.org/pkg/reflect">package reflect</a>:
+<a href="http://golang.org/pkg/reflect/#Type">Type</a>and
+<a href="http://golang.org/pkg/reflect/#Value">Value</a>. Those two types
+give access to the contents of an interface variable, and two
+simple functions, called <code>reflect.TypeOf</code> and
+<code>reflect.ValueOf</code>, retrieve <code>reflect.Type</code>
+and <code>reflect.Value</code> pieces out of an interface value.
+(Also, from the <code>reflect.Value</code> it's easy to get
+to the <code>reflect.Type</code>, but let's keep the
+<code>Value</code> and <code>Type</code> concepts separate for
+now.)
+</p>
+
+<p>
+Let's start with <code>TypeOf</code>:
+</p>
+
+{{code "progs/interface2.go" `/package main/` `/STOP main/`}}
+
+<p>
+This program prints
+</p>
+
+<pre>
+type: float64
+</pre>
+
+<p>
+You might be wondering where the interface is here, since the
+program looks like it's passing the <code>float64</code>
+variable <code>x</code>, not an interface value, to
+<code>reflect.TypeOf</code>. But it's there; as <a href=
+"http://golang.org/pkg/reflect/#Type.TypeOf">godoc reports</a>, the
+signature of <code>reflect.TypeOf</code> includes an empty
+interface:
+</p>
+
+<pre>
+// TypeOf returns the reflection Type of the value in the interface{}.
+func TypeOf(i interface{}) Type
+</pre>
+
+<p>
+When we call <code>reflect.TypeOf(x)</code>, <code>x</code> is
+first stored in an empty interface, which is then passed as the
+argument; <code>reflect.TypeOf</code> unpacks that empty interface
+to recover the type information.
+</p>
+
+<p>
+The <code>reflect.ValueOf</code> function, of course, recovers the
+value (from here on we'll elide the boilerplate and focus just on
+the executable code):
+</p>
+
+{{code "progs/interface2.go" `/var x/` `/STOP/`}}
+
+<p>
+prints
+</p>
+
+<pre>
+value: &lt;float64 Value&gt;
+</pre>
+
+<p>
+Both <code>reflect.Type</code> and <code>reflect.Value</code> have
+lots of methods to let us examine and manipulate them. One
+important example is that <code>Value</code> has a
+<code>Type</code> method that returns the <code>Type</code> of a
+<code>reflect.Value</code>. Another is that both <code>Type</code>
+and <code>Value</code> have a <code>Kind</code> method that returns
+a constant indicating what sort of item is stored:
+<code>Uint</code>, <code>Float64</code>, <code>Slice</code>, and so
+on. Also methods on <code>Value</code> with names like
+<code>Int</code> and <code>Float</code> let us grab values (as
+<code>int64</code> and <code>float64</code>) stored inside:
+</p>
+
+{{code "progs/interface2.go" `/START f1/` `/STOP/`}}
+
+<p>
+prints
+</p>
+
+<pre>
+type: float64
+kind is float64: true
+value: 3.4
+</pre>
+
+<p>
+There are also methods like <code>SetInt</code> and
+<code>SetFloat</code> but to use them we need to understand
+settability, the subject of the third law of reflection, discussed
+below.
+</p>
+
+<p>
+The reflection library has a couple of properties worth singling
+out. First, to keep the API simple, the "getter" and "setter"
+methods of <code>Value</code> operate on the largest type that can
+hold the value: <code>int64</code> for all the signed integers, for
+instance. That is, the <code>Int</code> method of
+<code>Value</code> returns an <code>int64</code> and the
+<code>SetInt</code> value takes an <code>int64</code>; it may be
+necessary to convert to the actual type involved:
+</p>
+
+{{code "progs/interface2.go" `/START f2/` `/STOP/`}}
+
+<p>
+The second property is that the <code>Kind</code> of a reflection
+object describes the underlying type, not the static type. If a
+reflection object contains a value of a user-defined integer type,
+as in
+</p>
+
+{{code "progs/interface2.go" `/START f3/` `/START/`}}
+
+<p>
+the <code>Kind</code> of <code>v</code> is still
+<code>reflect.Int</code>, even though the static type of
+<code>x</code> is <code>MyInt</code>, not <code>int</code>. In
+other words, the <code>Kind</code> cannot discriminate an int from
+a <code>MyInt</code> even though the <code>Type</code> can.
+</p>
+
+<p><b>The second law of reflection</b></p>
+
+<p><b>2. Reflection goes from reflection object to interface
+value.</b></p>
+
+<p>
+Like physical reflection, reflection in Go generates its own
+inverse.
+</p>
+
+<p>
+Given a <code>reflect.Value</code> we can recover an interface
+value using the <code>Interface</code> method; in effect the method
+packs the type and value information back into an interface
+representation and returns the result:
+</p>
+
+<pre>
+// Interface returns v's value as an interface{}.
+func (v Value) Interface() interface{}
+</pre>
+
+<p>
+As a consequence we can say
+</p>
+
+{{code "progs/interface2.go" `/START f3b/` `/START/`}}
+
+<p>
+to print the <code>float64</code> value represented by the
+reflection object <code>v</code>.
+</p>
+
+<p>
+We can do even better, though. The arguments to
+<code>fmt.Println</code>, <code>fmt.Printf</code> and so on are all
+passed as empty interface values, which are then unpacked by the
+<code>fmt</code> package internally just as we have been doing in
+the previous examples. Therefore all it takes to print the contents
+of a <code>reflect.Value</code> correctly is to pass the result of
+the <code>Interface</code> method to the formatted print
+routine:
+</p>
+
+{{code "progs/interface2.go" `/START f3c/` `/START/`}}
+
+<p>
+(Why not <code>fmt.Println(v)</code>? Because <code>v</code> is a
+<code>reflect.Value</code>; we want the concrete value it holds.)
+Since our value is a <code>float64</code>, we can even use a
+floating-point format if we want:
+</p>
+
+{{code "progs/interface2.go" `/START f3d/` `/STOP/`}}
+
+<p>
+and get in this case
+</p>
+
+<pre>
+3.4e+00
+</pre>
+
+<p>
+Again, there's no need to type-assert the result of
+<code>v.Interface()</code> to <code>float64</code>; the empty
+interface value has the concrete value's type information inside
+and <code>Printf</code> will recover it.
+</p>
+
+<p>
+In short, the <code>Interface</code> method is the inverse of the
+<code>ValueOf</code> function, except that its result is always of
+static type <code>interface{}</code>.
+</p>
+
+<p>
+Reiterating: Reflection goes from interface values to reflection
+objects and back again.
+</p>
+
+<p><b>The third law of reflection</b></p>
+
+<p><b>3. To modify a reflection object, the value must be settable.</b></p>
+
+<p>
+The third law is the most subtle and confusing, but it's easy
+enough to understand if we start from first principles.
+</p>
+
+<p>
+Here is some code that does not work, but is worth studying.
+</p>
+
+{{code "progs/interface2.go" `/START f4/` `/STOP/`}}
+
+<p>
+If you run this code, it will panic with the cryptic message
+</p>
+
+<pre>
+panic: reflect.Value.SetFloat using unaddressable value
+</pre>
+
+<p>
+The problem is not that the value <code>7.1</code> is not
+addressable; it's that <code>v</code> is not settable. Settability
+is a property of a reflection <code>Value</code>, and not all
+reflection <code>Values</code> have it.
+</p>
+
+<p>
+The <code>CanSet</code> method of <code>Value</code> reports the
+settability of a <code>Value</code>; in our case,
+</p>
+
+{{code "progs/interface2.go" `/START f5/` `/STOP/`}}
+
+<p>
+prints
+</p>
+
+<pre>
+settability of v: false
+</pre>
+
+<p>
+It is an error to call a <code>Set</code> method on an non-settable
+<code>Value</code>. But what is settability?
+</p>
+
+<p>
+Settability is a bit like addressability, but stricter. It's the
+property that a reflection object can modify the actual storage
+that was used to create the reflection object. Settability is
+determined by whether the reflection object holds the original
+item. When we say
+</p>
+
+{{code "progs/interface2.go" `/START f6/` `/START/`}}
+
+<p>
+we pass a <em>copy</em> of <code>x</code> to
+<code>reflect.ValueOf</code>, so the interface value created as the
+argument to <code>reflect.ValueOf</code> is a <em>copy</em> of
+<code>x</code>, not <code>x</code> itself. Thus, if the
+statement
+</p>
+
+{{code "progs/interface2.go" `/START f6b/` `/STOP/`}}
+
+<p>
+were allowed to succeed, it would not update <code>x</code>, even
+though <code>v</code> looks like it was created from
+<code>x</code>. Instead, it would update the copy of <code>x</code>
+stored inside the reflection value and <code>x</code> itself would
+be unaffected. That would be confusing and useless, so it is
+illegal, and settability is the property used to avoid this
+issue.
+</p>
+
+<p>
+If this seems bizarre, it's not. It's actually a familiar situation
+in unusual garb. Think of passing <code>x</code> to a
+function:
+</p>
+
+<pre>
+f(x)
+</pre>
+
+<p>
+We would not expect <code>f</code> to be able to modify
+<code>x</code> because we passed a copy of <code>x</code>'s value,
+not <code>x</code> itself. If we want <code>f</code> to modify
+<code>x</code> directly we must pass our function the address of
+<code>x</code> (that is, a pointer to <code>x</code>):</p>
+
+<p>
+<code>f(&amp;x)</code>
+</p>
+
+<p>
+This is straightforward and familiar, and reflection works the same
+way. If we want to modify <code>x</code> by reflection, we must
+give the reflection library a pointer to the value we want to
+modify.
+</p>
+
+<p>
+Let's do that. First we initialize <code>x</code> as usual
+and then create a reflection value that points to it, called
+<code>p</code>.
+</p>
+
+{{code "progs/interface2.go" `/START f7/` `/START/`}}
+
+<p>
+The output so far is
+</p>
+
+<pre>
+type of p: *float64
+settability of p: false
+</pre>
+
+<p>
+The reflection object <code>p</code> isn't settable, but it's not
+<code>p</code> we want to set, it's (in effect) <code>*p</code>. To
+get to what <code>p</code> points to, we call the <code>Elem</code>
+method of <code>Value</code>, which indirects through the pointer,
+and save the result in a reflection <code>Value</code> called
+<code>v</code>:
+</p>
+
+{{code "progs/interface2.go" `/START f7b/` `/START/`}}
+
+<p>
+Now <code>v</code> is a settable reflection object, as the output
+demonstrates,
+</p>
+
+<pre>
+settability of v: true
+</pre>
+
+<p>
+and since it represents <code>x</code>, we are finally able to use
+<code>v.SetFloat</code> to modify the value of
+<code>x</code>:
+</p>
+
+{{code "progs/interface2.go" `/START f7c/` `/STOP/`}}
+
+<p>
+The output, as expected, is
+</p>
+
+<pre>
+7.1
+7.1
+</pre>
+
+<p>
+Reflection can be hard to understand but it's doing exactly what
+the language does, albeit through reflection <code>Types</code> and
+<code>Values</code> that can disguise what's going on. Just keep in
+mind that reflection Values need the address of something in order
+to modify what they represent.
+</p>
+
+<p><b>Structs</b></p>
+
+<p>
+In our previous example <code>v</code> wasn't a pointer itself, it
+was just derived from one. A common way for this situation to arise
+is when using reflection to modify the fields of a structure. As
+long as we have the address of the structure, we can modify its
+fields.
+</p>
+
+<p>
+Here's a simple example that analyzes a struct value,
+<code>t</code>. We create the reflection object with the address of
+the struct because we'll want to modify it later. Then we set
+<code>typeOfT</code> to its type and iterate over the fields using
+straightforward method calls (see 
+<a href="http://golang.org/pkg/reflect/">package reflect</a> for details).
+Note that we extract the names of the fields from the struct type,
+but the fields themselves are regular <code>reflect.Value</code>
+objects.
+</p>
+
+{{code "progs/interface2.go" `/START f8/` `/STOP/`}}
+
+<p>
+The output of this program is
+</p>
+
+<pre>
+0: A int = 23
+1: B string = skidoo
+</pre>
+
+<p>
+There's one more point about settability introduced in
+passing here: the field names of <code>T</code> are upper case
+(exported) because only exported fields of a struct are
+settable.
+</p>
+
+<p>
+Because <code>s</code> contains a settable reflection object, we
+can modify the fields of the structure.
+</p>
+
+{{code "progs/interface2.go" `/START f8b/` `/STOP/`}}
+
+<p>
+And here's the result:
+</p>
+
+<pre>
+t is now {77 Sunset Strip}
+</pre>
+
+<p>
+If we modified the program so that <code>s</code> was created from
+<code>t</code>, not <code>&amp;t</code>, the calls to
+<code>SetInt</code> and <code>SetString</code> would fail as the
+fields of <code>t</code> would not be settable.
+</p>
+
+<p><b>Conclusion</b></p>
+
+<p>
+Here again are the laws of reflection:
+</p>
+
+<ol>
+<li>Reflection goes from interface value to reflection
+object.</li>
+<li>Reflection goes from reflection object to interface
+value.</li>
+<li>To modify a reflection object, the value must be settable.</li>
+</ol>
+
+<p>
+Once you understand these laws reflection in Go becomes much easier
+to use, although it remains subtle. It's a powerful tool that
+should be used with care and avoided unless strictly
+necessary.
+</p>
+
+<p>
+There's plenty more to reflection that we haven't covered &mdash;
+sending and receiving on channels, allocating memory, using slices
+and maps, calling methods and functions &mdash; but this post is
+long enough. We'll cover some of those topics in a later
+article.
+</p>