Re: python: testing; and locales

[Thomas Klausner <tk%giga.or.at@localhost>]

I've asked an acquaintance of mine who I know to be a great Python
hacker, and here's his reply.

-- begin quote --
> Recently I've started trying to run the tests for our python packages.
> For some of them it worked nicely (i.e., plain setup.py calls are
> fine) but I learned that there at least two other common methods:
> py.test and tox.
> 
> py.test seems ok so far, just needs a do-test target added.
> 
> tox seems a bit more annoying, since it seems to build up a virtualenv
> and install all dependencies it thinks it needs from the network.
> 
> Does anyone here know enough about tox to suggest how to make this
> integrate better with pkgsrc? In particular, at least to just use
> installed dependencies and complain about missing stuff instead of
> just installing it.

Tox is really great for development, but simply inappropriate for integration
testing done by packagers, and completely intentionally so. Tox aims to
eradicate differences between environments to make setting up a dev environment
easier. But your task is to test your package against the current environment,
not run the tests in a way such that they will be immune to env differences.

Tox does have some options to install global packages into the env, but you'd
probably have to use that option for all dependencies you installed via pkgsrc,
and then you still can't be sure that no deps are missing, since otherwise tox
will just provide them within the virtualenv.

Read tox.ini, write your own test running code based on it.  E.g. if tox just
calls pytest, just call pytest directly.

> Then I noticed another problems. Python-3.4, probably because of its
> supposedly better unicode support, is very anxious to get a proper
> locale setting from the environment.
> 
> I've seen errors like this (py-lxml):
> UnicodeEncodeError: 'ascii' codec can't encode characters in position 5-18:...
> 
> where I assume the problem is just that the locale is wrong. Or is
> that a bug in the test?
> 
> Should we set a locale in the pkgsrc build environment?

Yeah, install a UTF-8 locale in your test env and enable it. Still coordinate
with upstream, since it really depends on the actual test that failed. Some
accidentally use the default encodings in places where it's not appropriate.

Also, if you were using tox to run the tests: Tox wipes the env for
subprocesses for isolation purposes, and only preserves those specified via
passenv in tox.ini. Another reason NOT to use tox for testing while packaging,
but if you're going to do it anyway, make sure LANG is on that list.

-- end quote --