Rietveld Code Review Tool
Help | Bug tracker | Discussion group | Source code | Sign in
(449)

Unified Diff: doc/how-to-write-tests.txt

Issue 96110043: Add some developer docs for writing tests.
Patch Set: Add some developer docs for writing tests. Created 9 years, 10 months ago
Use n/p to move between diff chunks; N/P to move between comments. Please Sign in to add in-line comments.
Jump to:
View side-by-side diff with in-line comments
Download patch
« no previous file with comments | « [revision details] ('k') | no next file » | no next file with comments »
Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
Index: doc/how-to-write-tests.txt
=== added file 'doc/how-to-write-tests.txt'
--- doc/how-to-write-tests.txt 1970-01-01 00:00:00 +0000
+++ doc/how-to-write-tests.txt 2014-05-08 22:32:25 +0000
@@ -0,0 +1,304 @@
+How to write tests
+==================
+
+On the whole, new or updated code will not pass review unless there are tests
+associated with the code. For code additions, the tests should cover as much
+of the new code as practical, and for code changes, either the tests should be
+updated, or at least the tests that already exist that cover the refactored
+code should be identified when requesting a review to show that there is already
+test coverage, and that the refactoring didn't break anything.
+
+
+go test and gocheck
+-------------------
+
+The `go test` command is used to run the tests. Juju uses the `gocheck` package
+("launchpad.net/gocheck") to provide a checkers and assert methods for the test
+writers. The use of gocheck replaces the standard `testing` library.
+
+Across all of the tests in juju-core, the gocheck package is imported
+with a shorter alias, because it is used a lot.
+
+```go
+import (
+ // system packages
+
+ gc "launchpad.net/gocheck"
+
+ // juju packages
+)
+```
+
+
+setting up tests for new packages
+---------------------------------
+
+Lets say we are creating a new provider for "magic" cloud, and we have a package
+called "magic" that lives at "launchpad.net/juju-core/provider/magic". The
+general approach for testing in juju is to have the tests in a separate package.
+Continuing with this example the tests would be in a package called "magic_test".
+
+A common idiom that has occurred in juju is to setup to gocheck hooks in a special
+file called `package_test.go` that would look like this:
+
+
+```go
+// Copyright 2014 Canonical Ltd.
+// Licensed under the AGPLv3, see LICENCE file for details.
+
+package magic_test
+
+import (
+ "testing"
+
+ gc "launchpad.net/gocheck"
+)
+
+func Test(t *testing.T) {
+ gc.TestingT(t)
+}
+```
+
+or
+
+```go
+// Copyright 2014 Canonical Ltd.
+// Licensed under the AGPLv3, see LICENCE file for details.
+
+package magic_test
+
+import (
+ stdtesting "testing"
+
+ "launchpad.net/juju-core/testing"
+)
+
+func Test(t *stdtesting.T) {
+ testing.MgoTestPackage(t)
+}
+```
+
+The key difference here is that the first one just hooks up `gocheck`
+so it looks for the `gocheck` suites in the package. The second makes
+sure that there is a mongo available for the duration of the package tests.
+
+A general rule is not to setup mongo for a package unless you really
+need to as it is extra overhead.
+
+
+writing the test files
+----------------------
+
+Normally there will be a test file for each file with code in the package.
+For a file called `config.go` there should be a test file called `config_test.go`.
+
+The package should in most cases be the same as the normal files with a "_test" suffix.
+In this way, the tests are testing the same interface as any normal user of the
+package. It is reasonably common to want to modify some internal aspect of the package
+under test for the tests. This is normally handled by a file called `export_test.go`.
+Even though the file ends with `_test.go`, the package definition is the same as the
+normal source files. In this way, for the tests and only the tests, additional
+public symbols can be defined for the package and used in the tests.
+
+Here is an annotated extract from `provider/local/export_test.go`
+
+```go
+// The package is the "local" so it has access to the package symbols
+// and not just the public ones.
+package local
+
+import (
+ "github.com/juju/testing"
+ gc "launchpad.net/gocheck"
+
+ "launchpad.net/juju-core/environs/config"
+)
+
+var (
+ // checkIfRoot is a variable of type `func() bool`, so CheckIfRoot is
+ // a pointer to that variable so we can patch it in the tests.
+ CheckIfRoot = &checkIfRoot
+ // providerInstance is a pointer to an instance of a private structure.
+ // Provider points to the same instance, so public methods on that instance
+ // are available in the tests.
+ Provider = providerInstance
+)
+
+// ConfigNamespace is a helper function for the test that steps through a
+// number of private methods or variables, and is an alternative mechanism
+// to provide functionality for the tests.
+func ConfigNamespace(cfg *config.Config) string {
+ env, _ := providerInstance.Open(cfg)
+ return env.(*localEnviron).config.namespace()
+}
+```
+
+Suites and Juju base suites
+---------------------------
+
+With gocheck tests are grouped into Suites. Each suite has distinct
+set-up and tear-down logic. Suites are often composed of other suites
+that provide specific set-up and tear-down behaviour.
+
+There are three main suites:
+
+ * /testing/testbase.LoggingSuite (testing/testbase/log.go)
+ * /testing.FakeHomeSuite (testing/environ.go)
+ * /juju/testing.JujuConnSuite (juju/testing/conn.go)
+
+The second two have the LoggingSuite functionality included through
+composition. The LoggingSuite is also composed of the LoggingSuite from
+github.com/juju/testing, which brings in the CleanupSuite from the same.
+The CleanupSuite has the functionality around patching environment
+variables and normal variables for the duration of a test. It also
+provides a clean-up stack that gets called when the test teardown happens.
+
+The FakeHomeSuite creates a temporary directory and sets the HOME environment
+variable to it. It also creates ~/.juju and a simple environments.yaml file,
+~/.ssh with a fake id_rsa.pub key, it isolates the test from the JUJU_HOME,
+JUJU_ENV, and JUJU_LOGGING_CONFIG environment variables.
+
+The JujuConnSuite does this and more. It also sets up a state server and api
+server. This is one problem with the JujuConnSuite, it almost always does a
+lot more than you actually want or need. This should really be broken into
+smaller component parts that make more sense. If you can get away with not
+using the JujuConnSuite, you should try.
+
+To create a new suite composed of one or more of the suites above, you can do
+something like:
+
+```go
+type ToolsSuite struct {
+ testbase.LoggingSuite
+ dataDir string
+}
+
+var _ = gc.Suite(&ToolsSuite{})
+
+```
+
+If there is no extra setup needed, then you don't need to specify any
+set-up or tear-down methods as the LoggingSuite has them, and they are
+called by default.
+
+If you did want to do something, say, create a directory and save it in
+the dataDir, you would do something like this:
+
+```go
+func (t *ToolsSuite) SetUpTest(c *gc.C) {
+ t.LoggingSuite.SetUpTest(c)
+ t.dataDir = c.MkDir()
+}
+```
+
+If the test suite has multiple contained suites, please call them in the
+order that they are defined, and make sure something that is composed from
+the LoggingSuite is first. They should be torn down in the reverse order.
+
+Even if the code that is being tested currently has no logging in it, it
+is a good idea to use the LoggingSuite as a base for two reasons:
+ * it brings in something composed of the CleanupSuite
+ * if someone does add logging later, it is captured and doesn't polute
+ the logging output
+
+
+Patching variables and the environment
+--------------------------------------
+
+Inside a test, and assuming that the Suite has a CleanupSuite somewhere
+in the composition tree, there are a few very helpful functions.
+
+```go
+
+var foo int
+
+func (s *someTest) TestFubar(c *gc.C) {
+ // The TEST_OMG environment value will have "new value" for the duration
+ // of the test.
+ s.PatchEnvironment("TEST_OMG", "new value")
+
+ // foo is set to the value 42 for the duration of the test
+ s.PatchValue(&foo, 42)
+}
+```
+
+PatchValue works with any matching type. This includes function variables.
+
+
+Checkers
+--------
+
+Checkers are a core concept of `gocheck` and will feel familiar to anyone
+who has used the python testtools. Assertions are made on the gocheck.C
+methods.
+
+```go
+c.Check(err, gc.IsNil)
+c.Assert(something, gc.Equals, somethingElse)
+```
+
+The `Check` method will cause the test to fail if the checker returns
+false, but it will continue immediately cause the test to fail and will
+continue with the test. `Assert` if it fails will cause the test to
+immediately stop.
+
+For the purpose of further discussion, we have the following parts:
+
+ `c.Assert(observed, checker, args...)`
+
+The key checkers in the `gocheck` module that juju uses most frequently are:
+
+ * `IsNil` - the observed value must be `nil`
+ * `NotNil` - the observed value must not be `nil`
+ * `Equals` - the observed value must be the same type and value as the arg,
+ which is the expected value
+ * `DeepEquals` - checks for equality for more complex types like slices,
+ maps, or structures. This is DEPRECATED in favour of the DeepEquals from
+ the `github.com/juju/testing/checkers` covered below
+ * `ErrorMatches` - the observed value is expected to be an `error`, and
+ the arg is a string that is a regular expression, and used to match the
+ error string
+ * `Matches` - a regular expression match where the observed value is a string
+ * `HasLen` - the expected value is an integer, and works happily on nil
+ slices or maps
+
+
+Over time in the juju project there were repeated patterns of testing that
+were then encoded into new and more complicated checkers. These are found
+in `github.com/juju/testing/checkers`, and are normally imported with the
+alias `jc`.
+
+The matchers there include (not an exclusive list):
+
+ * `IsTrue` - just an easier way to say `gc.Equals, true`
+ * `IsFalse` - observed value must be false
+ * `GreaterThan` - for integer or float types
+ * `LessThan` - for integer or float types
+ * `HasPrefix` - obtained is expected to be a string or a `Stringer`, and
+ the string (or string value) must have the arg as start of the string
+ * `HasSuffix` - the same as `HasPrefix` but checks the end of the string
+ * `Contains` - obtained is a string or `Stringer` and expected needs to be
+ a string. The checker passes if the expected string is a substring of the
+ obtained value.
+ * `DeepEquals` - works the same way as the `gocheck.DeepEquals` except
+ gives better errors when the values do not match
+ * `SameContents` - obtained and expected are slices of the same type,
+ the checker makes sure that the values in one are in the other. They do
+ not have the be in the same order.
+ * `Satisfies` - the arg is expected to be `func(observed) bool`
+ often used for error type checks
+ * `IsNonEmptyFile` - obtained is a string or `Stringer` and refers to a
+ path. The checker passes if the file exists, is a file, and is not empty
+ * `IsDirectory` - works in a similar way to `IsNonEmptyFile` but passes if
+ the path element is a directory
+ * `DoesNotExist` - also works with a string or `Stringer`, and passes if
+ the path element does not exist
+
+
+
+Good tests
+----------
+
+Good tests should be:
+ * small and obviously correct
+ * isolated from any system or environment values that may impact the test
« no previous file with comments | « [revision details] ('k') | no next file » | no next file with comments »

Powered by Google App Engine
RSS Feeds Recent Issues | This issue
This is Rietveld f62528b