Deadlock when using bg module loading under nose testing

[paulie-g]

I patched @gleitz 's howdoi to load pygments in a background thread with lazyasd. The code worked. The unit tests passed when run by hand. Speed-up was achieved. I rejoiced. @gleitz merged my two-line PR. Then Travis failed.

Looking at the strace -f, it appears the background thread is blocking on a futex and the main doesn't progress either. Deadlock. Specifically, it's deadlocking at the top of BackgroundModuleLoader.run(). In an attempt to debug this and see if it can be avoided, I implemented all the stuff in PR #8 .

As it happens, the only thing that works to avoid the deadlock is avoiding the time.sleep() there. It bothers me that I still don't understand why nose is triggering this. Even stranger, nosetests-2.7 deadlocks on my machine but not on Travis. On the other hand, this whole thing was meant to be a 5 minute 2 sloc PR to speed up a tool I use on occasion. When I found myself looking through cpython to see if the GIL is being released on time.sleep(), I realized that this rabbit hole is in danger of swallowing the rest of my week ;)

In an ideal world, I'd like:

• Implement fine-grained control of bg module load delay #8 merged and a new point release made so I can submit a PR to howdoi fixing the deadlock under nose test
• A less vague idea of how and why this is happening

Sadly, we live in an imperfect world. Cheers for your work on lazyasd and xonsh ;)

[gleitz]

@paulie-g thanks for the in-depth exploration

[scopatz]

Thanks for this @paulie-g!