Python Continuous Integration (CI) with Jenkins and Mercurial

admin on 2012/12/15

Since my Python projects are growing more and more, I’d like to start having them automatically tested.

I remembered that at my last student’s job we had a Bamboo server to manage builds and tests. For my current task this kind of financial outlay was out of the question however.

So I scourged the interwebs for alternatives and eventually found Jenkins - an open source build server that forked from Sun’s Hudson after Oracle took over.

It has a myriad of plugins and all the tasks I had in mind for the future seemed to be supported.

  • Mercurial Integration - I tested it and it works with minor issues. So far, I have to insert the password in clear text and it always clones the full repository instead of just updating as I described in this bug report.

    As I was told there: the full clone can be avoided by pulling over ssh and not via https. Took me a little bit to set this up, but now it works 🙂

  • Gradle Build Support. Maven is the main option, but there’s a plugin for gradle as well.

  • Android Development support is there as well.

For the Python integration I mainly followed this guide: https://github.com/dataflow/devdocs/wiki/Configuringjenkins

If you need the ssh functionality,consider the guide from bitbucket and a good description on stackoverflow:
https://confluence.atlassian.com/pages/viewpage.action?pageId=270827678
http://stackoverflow.com/questions/6234016/jenkins-user-on-apt-get-install-installation

And here’s some more pointers that slowed me down a bit:

  • Key needs to be created without a password
  • Repeating: After adding the key, you need to pull at least once from the console to add xp-dev / bitbucket to your known hosts.
  • ssh-agent /bin/bash needs to be started (and might have to be restarted after a reboot)

I pulled up Eclipse/PyDev, made a quick main file with some unit tests (see below) and uploaded the entire thing to a private bitbucket repository.

import random
import unittest

class Test(unittest.TestCase):

    def testName(self):
        name = "Ich"
        self.assertEqual("Ich", name)

    def testFalse(self):
        self.assertEqual(1, 2, "nope")

    def testSort(self):
        myseq = range(10)
        random.shuffle(myseq)
        myseq.sort()
        self.assertEqual(range(10),myseq , "sorting didn't work")

if __name__ == "__main__":
    #import sys;sys.argv = ['', 'Test.testName']
    unittest.main()

The result is unsurprising.

Overall, I am quite pleased with the quality of Jenkins. Some things are not immediately obvious, but then again, Bamboo also took quite some configuration and CI users are probably not the average crowd anyway. Voilà, hopefully this will help me improve the quality of my programs.