Running dispy inside uWSGI environment?

[elogy]

Hi everybody,
sorry to bother you with weird problems again, but maybe someone already did what I'm trying to do.

I'm using Bottle to handle HTTP requests inside my application, a request to a certain URL will normally trigger a function that calls FFmpeg via subprocess.Popen(), creates files on disk and then shows an HTTP response. I then phased out the function that creates the files and calls FFmpeg to my cluster nodes.
The machine that is handling the HTTP requests is also the dispy client.

apiserver = Bottle()

dispy_cluster = dispy.JobCluster(ffmpeg_encode_and_mux,
                                     nodes=DISPYNODES,
                                     recover_file=DISPYRECOVERY,
                                     secret=DISPYSECRET,
                                     ip_addr=DISPYSERVER
                                     )

[…]

@apiserver.route('/entry')
def single():
[...]
    # start an ffmpeg process for all resolutions needed
    for res in needed_resolutions:
        pid = None
        if DISPY:
            pid = dispy_cluster.submit(timms_obj, res) # submit ffmpeg_encode_and_mux to cluster
        else:
            pid = ffmpeg_encode_and_mux(timms_obj, res) # compute locally

This works perfectly fine if I run my application via the built-in test server, i.e. by running

if __name__ == '__main__':
    httpserver.run(host='localhost', port=8080, debug=True)

I can see the computation being registered and passed to the nodes, and I see a response in my browser.

However, when I try to run the same application using nginx and uWSGI (which is how I would want to deploy my application), I can see the computation being registered on the nodes, but when I call the URL that's routed to the method (/entry in this case), the page loads endlessly until the browser times it out.

Does anyone have a clue what's going on here? I could only guess that it has something to do with the way uWSGI starts processes, and maybe there is no feasible solution for this. I'd be glad if anyone has a solution for this, however. 💪