Monthly Archives: November 2013

Comparison of py.test and nose for Python testing

I happened upon this useful comparison of py.test and nose at the testing-in-python mailing list, by Kenny (theotherwhitemeat at gmail). He spent some time evaluating testing tools for Python with a focus on py.test and nose . This article is a reformat of his mailing list post. I assume no credit for the content. The list of references [1] … [13] is at the end of article.

py.test

  • parallelizable: threading + SMP support [3] [4]
  • better documentation: [1] [3]
  • can generate script that does all py.test functions, obviating the need to distribute py.test [1][10]
  • integrate tests into your distribution (py.test –genscript=runtests.py), to create a standalone version of py.test [10]
  • can run nose, unittest, doctest style tests [1] [2]
  • test detection via globs, configurable [3]
  • test failure output more discernible than nose [3] [9]
  • easier, more flexible assertions than nose [8]
  • setup speed is sub-second different from nose, also test speeds can be managed via distribution (threads + SMP via xdist) [9] [11]
  • provides test isolation, if needed [9]
  • dependency injection via funcargs [10] [12] [13]
  • coverage plugin [11]

nose

  • documentation concerns, this may be outdated [3]
  • parallelization issues [3] [8]
  • slightly faster than py.test [4] [11]
  • test detection via regex (setup in cmdline or config file) [3]
  • can run unittest, doctest style tests [1] [2]
  • cannot run py.test style tests [1]

Conclusions

  • test formats are so similar, that nose or py.test can be used without much consequence until you’re writing more exotic tests (you could swap with little consequence)
  • nose is sub-second faster than py.test in execution time; this is typically unimportant
  • community seems to slightly favor py.test

References

  1. http://mail.scipy.org/pipermail/astropy/2011-July/001673.html
  2. http://pytest.org/latest/nose.html
  3. http://fedoraproject.org/wiki/User:Tflink/AutoQA_nose_pytest_comparison
  4. http://www.libcrack.so/2012/01/09/a-brief-analysis-of-python-testing-software/
  5. http://pythontesting.net/framework/nose/nose-introduction/
  6. http://wiki.python.org/moin/PythonTestingToolsTaxonomy
  7. http://docs.python-guide.org/en/latest/writing/tests.html#tools
  8. http://stackoverflow.com/questions/191673/preferred-python-unit-testing-framework
  9. http://thread.gmane.org/gmane.comp.python.testing.general/3748
  10. http://article.gmane.org/gmane.comp.python.testing.general/3752
  11. http://article.gmane.org/gmane.comp.python.testing.general/3765
  12. http://pytest.org/latest/funcargs.html
  13. http://holgerkrekel.net/2009/05/13/parametrizing-python-tests-generalized/