Organizing Test Fixtures into Layers
New in version 0.4
Layers allow more flexible organization of test fixtures than test-, class- and module- level fixtures. Layers in nose2 are inspired by and aim to be compatible with the layers used by Zope’s testrunner.
Using layers, you can do things like:
Implement package-level fixtures by sharing a layer among all test cases in the package.
Share fixtures across tests in different modules without having them run multiple times.
Create a fixture tree deeper than three levels (test, class and module).
Make fixtures available for other packages or projects to use.
A layer is a new-style class that implements at least a
class Layer(object): @classmethod def setUp(cls): # ...
It may also implement
testTearDown, all as
To assign a layer to a test case, set the test case’s
class Test(unittest.TestCase): layer = Layer
Note that the layer class is assigned, not an instance of the layer. Typically layer classes are not instantiated.
Layers may subclass other layers:
class SubLayer(Layer): @classmethod def setUp(cls): # ...
In this case, all tests that belong to the sub-layer also belong to the base layer. For example for this test case:
class SubTest(unittest.TestCase): layer = SubLayer
setUp methods from both
Layer will run before
any tests are run. The superclass’s setup will always run before the subclass’s
teardown, the reverse: the subclass’s
teardown runs before
One important thing to note: layers that subclass other layers must
not call their superclass’s
tearDown, etc. The test
runner will take care of organizing tests so that the superclass’s
methods are called in the right order:
Layer.setUp -> SubLayer.setUp -> Layer.testSetUp -> SubLayer.testSetUp -> TestCase.setUp TestCase.run TestCase.tearDown SubLayer.testTearDown <- Layer.testTearDown <- SubLayer.tearDown <- Layer.tearDown <-
If a sublayer calls it superclass’s methods directly, those methods will be called twice.
Layer method reference
- class Layer
Not an actual class, but reference documentation for the methods layers can implement. There is no layer base class. Layers must be subclasses of
objector other layers.
- classmethod setUp(cls)
setUpmethod is called before any tests belonging to that layer are executed. If no tests belong to the layer (or one of its sub-layers) then the
setUpmethod will not be called.
- classmethod tearDown(cls)
tearDownmethod is called after any tests belonging to the layer are executed, if the layer’s
setUpmethod was called and did not raise an exception. It will not be called if the layer has no
setUpmethod, or if that method did not run or did raise an exception.
- classmethod testSetUp(cls[, test])
testSetUpmethod is called before each test belonging to the layer (and its sub-layers). If the method is defined to accept an argument, the test case instance is passed to the method. The method may also be defined to take no arguments.
- classmethod testTearDown(cls[, test])
testTearDownmethod is called after each test belonging to the layer (and its sub-layers), if the layer also defines a
setUpTestmethod and that method ran successfully (did not raise an exception) for this test case.
nose2 includes a DSL for setting up layer-using tests called “such”. Read all about it here: Such: a Functional-Test Friendly DSL.
The layers plugin module includes a second plugin that alters test
report output to make the layer groupings more clear. When activated
--layer-reporter command-line option (or via a config
file), test output that normally looks like this:
test (test_layers.NoLayer) ... ok test (test_layers.Outer) ... ok test (test_layers.InnerD) ... ok test (test_layers.InnerA) ... ok test (test_layers.InnerA_1) ... ok test (test_layers.InnerB_1) ... ok test (test_layers.InnerC) ... ok test2 (test_layers.InnerC) ... ok ---------------------------------------------------------------------- Ran 8 tests in 0.001s OK
Will instead look like this:
test (test_layers.NoLayer) ... ok Base test (test_layers.Outer) ... ok LayerD test (test_layers.InnerD) ... ok LayerA test (test_layers.InnerA) ... ok LayerB LayerC test (test_layers.InnerC) ... ok test2 (test_layers.InnerC) ... ok LayerB_1 test (test_layers.InnerB_1) ... ok LayerA_1 test (test_layers.InnerA_1) ... ok ---------------------------------------------------------------------- Ran 8 tests in 0.002s OK
The layer reporter plugin can also optionally colorize the keywords (by default, ‘A’, ‘having’, and ‘should’) in output from tests defined with the such DSL.
If you would like to change how the layer is displayed, set the
class LayerD(Layer): description = '*** This is a very important custom layer description ***'
Now the output will be the following:
test (test_layers.NoLayer) ... ok Base test (test_layers.Outer) ... ok *** This is a very important custom layer description *** test (test_layers.InnerD) ... ok LayerA test (test_layers.InnerA) ... ok LayerB LayerC test (test_layers.InnerC) ... ok test2 (test_layers.InnerC) ... ok LayerB_1 test (test_layers.InnerB_1) ... ok LayerA_1 test (test_layers.InnerA_1) ... ok ---------------------------------------------------------------------- Ran 8 tests in 0.002s OK
Warnings and Caveats
Test case order and module isolation
Test cases that use layers will not execute in the same order as test cases that do not. In order to execute the layers efficiently, the test runner must reorganize all tests in the loaded test suite to group those having like layers together (and sub-layers under their parents). If you share layers across modules this may result in tests from one module executing interleaved with tests from a different module.
Mixing layers with
setUpClass and module fixtures
Don’t cross the streams.
The implementation of class- and module-level fixtures in unittest2
depends on introspecting the class hierarchy inside of the
unittest.TestSuite. Since the suites that the
layers plugin uses to
organize tests derive from
unittest.BaseTestSuite (instead of
unittest.TestSuite), class- and module- level fixtures in
TestCase classes that use layers will be ignored.
Mixing layers and multiprocess testing
In the initial release, test suites using layers are incompatible with the multiprocess plugin. This should be fixed in a future release.
Enable this Plugin
This plugin is built-in, but not loaded by default.
Even if you specify
always-on = True in the configuration, it will not run unless you also enable it. You can do so by putting the following in a
[unittest] plugins = nose2.plugins.layers
plugins parameter may contain a list of plugin names, including
[‘A’, ‘having’, ‘should’]
The default configuration is equivalent to including the following in a
[layer-reporter] always-on = False colors = False highlight-words = A having should indent =
- --layer-reporter DEFAULT
Add layer information to test reports