I have written a little Python script for my personal purposes and scheduled it to be run on Raspberry Pi by cron. After some polishing work I was pretty sure it worked well and was successfully tested. What I mean by to be tested is to be executed manually from the shell and to observe the expected results. So far so good.
However when the script was run by cron, it failed at the line where it prints some string containing Unicode characters. The line executes normally when running from the shell. I suspected there is some issue with the standard output of the processes run by the cron, because in such case there is no meaningful notion of standard output.
As it turns out, programs executed by cron have no terminal attached to their standard output. According to this Stack Exchange post the standard output is sent to the system’s mail system and thus delivered to a user this way. One can easily verify this by issuing tty command in the cron and redirecting the output to a file. Something similar to this is not a terminal (message translated directly from my system with Polish locale) should be observed.
The further explanation goes as follows: if there is no terminal attached to the process, the Python interpreter cannot detect the encoding of the terminal (it has nothing to do with the system’s environmental variables describing locale. They are all global and are part of the process’ environment, but they do not affect the non-terminal device being attached to standard output). You can verify this by running a Python script that tries to output terminal’s encoding: print sys.stdout.encoding. The None can be observed. So, the interpreter falls back to Ascii encoding and crashes when printing Unicode characters.
The solution in this case was to enforce the interpreter to use Unicode encoder for standard output.
UTF8Writer = codecs.getwriter('utf8')
sys.stdout = UTF8Writer(sys.stdout)
The output is discarded at all, but these lines prevent the interpreter from using default Ascii encoder which is not appropriate for printing Unicode string.
