Saturday 6 November 2010

Lupa on OS X with MacPorts Python 2.6

Lupa doesn't Just Work on OS X 10.6 stock Python nor on MacPorts Python 2.6; it fails with:

Python 2.6.1 (r261:67515, Feb 11 2010, 00:51:29)
[GCC 4.2.1 (Apple Inc. build 5646)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import lupa
>>> from lupa import LuaRuntime
>>> lua = LuaRuntime()
Segmentation fault


LuaJIT seems to depend on being mapped within a certain address window (not sure why). The install page mentions that you have to use these linker options on any binary that embeds LuaJIT:
-pagezero_size 10000 -image_base 100000000

to satisfy this requirement. Neither Snow Leopard's nor MacPorts' Python uses those flags (why would they?) so you need a modified Python in order to support the lupa binary module. If you build your own Python from source, it's simply:
./configure LIBS="-pagezero_size 10000 -image_base 100000000"

In MacPorts it's a simple edit to the Portfile; for Python 2.6 (MacPorts package python26) I changed the file /opt/local/var/macports/sources/rsync.macports.org/release/ports/lang/python26/Portfile like so:

configure.args --enable-framework=${frameworks_dir} \
--enable-ipv6

became:

configure.args --enable-framework=${frameworks_dir} \
--enable-ipv6 \
LIBS="-pagezero_size 10000 -image_base 100000000"


After installing lupa, if you get:

Python 2.6.6 (r266:84292, Nov 6 2010, 10:18:50)
[GCC 4.2.1 (Apple Inc. build 5664)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import lupa
Traceback (most recent call last):
File "", line 1, in
File "lupa/__init__.py", line 25, in
from lupa._lupa import *
ImportError: No module named _lupa
>>>

then check you're not still in the lupa source directory; Python picks up the lupa subdirectory from the source tree which has no binary module.

While I was poking around I also noticed lupa has a bug in its __init__.py relating to loading binary lua modules. Instead of:
import DLFCN

use:
import ctypes

Instead of:
sys.setdlopenflags(DLFCN.RTLD_NOW|DLFCN.RTLD_GLOBAL)

use:
sys.setdlopenflags(ctypes.RTLD_GLOBAL)


Perhaps one day somebody will write a Lua implementation in pure Python. I wouldn't care if it was dog-slow. Lua has become the de facto embeddable scripting language, and the lower the barrier to embedding, the better — even when embedding into existing scripting languages :)

Wednesday 3 November 2010

Relax: write some selfish software

I recently decided to improve my organisation a bit, and there's no better way to start such a project than by procrastinating — by designing and writing that absolutely essential piece of perfectly tailored software that will implement your system, while the situation you'd hoped to apply it to devolves even further into chaos. Just before git init, though, I stopped and thought.

For the ten years or so that I've been capable of writing code worth sharing, I've never given a second thought to making anything I've written available under an open source licence. Zealots have gushed about the benefits sharing brings to the community and to the quality of the resulting software, and I've largely agreed.

While I've had my fair share of minor patches accepted (and rejected) upstream, no standalone project I've ever written has got very far nor found any audience to speak of; I generally stay up 20 hours of 24 for a fortnight hacking away, then I lose interest. I thought about why that was, and came to the conclusion it's mostly because I prematurely pander to my non-existent audience — and that's hard work. My phantom customers sit on my shoulder while I code, and in my mind they're impossible to please. Before I know it I'm writing preference panes and portable self-documenting file formats and littering the code with FIXMEs for all the things I want some imaginary co-maintainer to forgive me for (and filling a bugtracker with them too). As a handful of potential users contacts me, I'm so eager for actual users that I agree to implement the features they ask for, even though they want things I don't, then get bored implementing them and stop.

I decided to try this one the opposite way: a deliberately selfish program that would do exactly what I'd want it to do, so would need no preferences; that would only ever store my data, eliminating the need to worry about file formats and field names; that would run only on my platforms and devices, and so have no portability concerns; that would have absolutely no test cases because the only one it would need would be "works for me?"; and whose code I could litter with disgusting hacks.

It went really well. It's simple software anyway, but it was up and running in a fraction of the time it usually takes and I still have the energy for the inevitable first re-write. Some wise sage once wrote: "Plan to throw one away. You will anyway." With all previous such projects I think I've given up at exactly the point where you realise you have to re-write, but you're so worn out you can't be bothered any more.

That isn't to say I won't open-source this or future code. There are definitely too many bits of personal organiser software out there already, for instance — ones that aren't tailored inflexibly to a single pedant's whims — but as it's free to share, I might as well.

Unless your aim is specifically to release a popular piece of open-source software or you have lots of free time to add features for others, ignore "release early, release often". Concentrate on creating something that does a good job of addressing your variety of your problem. When it comes to releasing it to the world, just write a few paragraphs about the true state of play and git push it somewhere public. Even then, maintain staunch indifference about whether it gets used by anyone but you.