Publishing nosetests results in Jenkins CI

While trying to publish test results in Jenkins CI via the xUnit plugin, I’ve set up the Post-build Actions section in Jenkins according to what the nosetests documentation suggests, that is, to “tick the box named “Publish JUnit test result report”” and referenced the correct file.

However, I was repeatedly stumbling upon an error:

WARNING: The file '/var/lib/jenkins/workspace/Project/nosetests.xml' is an invalid file.
WARNING: At line 1 of file:/var/lib/jenkins/workspace/Project/nosetests.xml:cvc-complex-type.3.2.2: Attribute 'skip' is not allowed to appear in element 'testsuite'.
ERROR: The result file '/var/lib/jenkins/workspace/Project/nosetests.xml' for the metric 'JUnit' is not valid. The result file has been skipped.

Turns out, my nosetests.xml file does indeed contain the skip attribute. And that’s also in line with the example in the official documentation.
Much to my surprise, though, running a few web searches made me realize there aren’t many other users of Jenkins + xUnit plugin + nosetests who have the same problem.

To fix this, it looked like I had to write my one XLS file. All you need to do is create two templates, one matching the “skip” attribute, and the other matching everything else.

The “skip”-matching template will simply get rid of the attribute altogether. The other will pass everything else as it is.

By taking cue from this StackOverflow answer, the result is as follows:

<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
    <xsl:output method="xml" version="1.0" encoding="UTF-8" omit-xml-declaration="no" indent="yes"/>
    <xsl:strip-space elements="*"/>

    <xsl:template match="@skip" />
    <xsl:template match="@*|node()">
        
            <xsl:apply-templates select="@*|node()"/>
        
    


Save it as nosetests.xsl. Now, armed with your shiny XSL file, put it in the $JENKINS_HOME/userContent directory, say in userContent/xunit_xsl/nosetests.xsl.

In the job config page, under the Post-Build Actions section, change the type from JUnit to Custom, and provide a reference to the XSD file you’ve created, in the form of $JENKINS_HOME/userContent/xunit_xsl/nosetests.xsl.

Save, run, and happily enjoy your test results!

Advertisements

Graceful Harakiri

In the past few days I’ve been trying to overcome a problem we saw on our CI environment: tests being abruptly cut if they hang.

If a build takes too long, our CI tool stops it by sending it a SIGTERM signal. That’s fine in general, but if it’s a test (run by the nosetests driver) that’s taking too long to finish, a SIGTERM would cause it to immediately stop, without leaving any trace on the output where it hanged.

What I coded was a plugin, Graceful Harakiri, that intercepts the SIGTERM signal, converts it to an interrupt and, in the process, prints out the current frame, giving away some information about where the test got stuck.

The code is on GitHub – have a look at the description and use it. Feedback is most welcome.

Nosetests test generators – make your bool values more explicit

I had to write a test generator in Python + nostests to drive the same test in two different conditions.

The way I initially wrote the generator initially resembled something like this:

def testChildIsPopulatedCorrectly(self):
    for case in [True, False]:
        yield self.checkChildIsPopulatedCorrectly, case

Although concise, the test as it’s written is not really easy to follow: there is no way to know what the test does without delving into the checkChildIsPopulatedCorrectly method and see which paths case triggers.

A second problem happens when the test runs. The output will be:

testChildIsPopulatedCorrectly(True,) ... ok
testChildIsPopulatedCorrectly(False,) ... ok

Again, it is not very clear what True and False mean in that context.

A solution is to create two simple classes, within the generator[1]:

def testChildIsPopulatedCorrectly(self):
    class WaitForChildren(object):
        def __bool__(self): return True  # [2]
        def __repr__(self): return 'Wait until all the children in the tree are populated'
    class DontWaitForChildren(object):
        def __bool__(self): return False
        def __repr__(self): return 'Do not wait until all the children in the tree are populated'
    for case in [WaitForChildren(), DontWaitForChildren()]:
        yield self.checkChildIsPopulatedCorrectly, case

The next time you run the tests, you’ll get:

testChildIsPopulatedCorrectly(Wait until all the children in the tree are populated,) ... ok
testChildIsPopulatedCorrectly(Do not wait until all the children in the tree are populated,) ... ok

Now both the test generator and the output are clearer.

[1] This is a proof of concept. The complete solution I implemented involves the definition of a more generic class in a different module.

[2] just a heads-up that you should include __nonzero__ = __bool__ if you are using Python 2.x.

Nosetests and the docstrings

For some reason when you use the -v switch in nosetests, instead of using the test name, the runner uses the docstring (and only first line, for that matter).

Solution: install the “nose-ignore-docstring” plugin:
sudo easy_install nose-ignore-docstring
and then enable it by adding:
[nosetests]
with-ignore-docstrings=1

to ~/.noserc

For additional information, I’ll point you to the plugin web page on PyPy.