EvServer, Introduction: The tale of a forgotten feature

Long long time ago there was a WSGI spec. This document described a lot of interesting stuff. Between other very important paragraphs you could find a hidden gem:

[...] applications will usually return an iterator (often a generator-iterator) that produces the output in a block-by-block fashion. These blocks may be broken to coincide with mulitpart boundaries (for “server push”), or just before time-consuming tasks (such as reading another block of an on-disk file). [...]

It means that all WSGI conforming servers should be able to send multipart http responses. WSGI clock application theoretically could be written like that:

def clockdemo(environ, startresponse):
    startresponse("200 OK", [('Content-type','text/plain')])
    for i in range(100):
        yield "%s\n" % (datetime.datetime.now(),)
        time.sleep(1)

The problem is that way of programming just doesn’t work well. It’s not scalable, requires a lot of threads and can eat a lot of resources. That’s why the feature has been forgotten.

Until May 2008, when Christopher Stawarz reminded us this feature and proposed an enhancement to it. He suggested, that instead of blocking, like time.sleep(1), inside the code WSGI application should return a file descriptor to server. When an event happens on this descriptor, the WSGI app will be continued. Here’s equivalent of the previous code, but using the extension. With appropriate server this could be scalable and work as expected:

def clockdemo(environ, startresponse):
    startresponse("200 OK", [('Content-type','text/plain')])
    sd = socket.socket(socket.AFINET, socket.SOCKDGRAM)
    try:
        for i in range(100):
            yield environ['x-wsgiorg.fdevent.readable'](sd, 1.0)
            yield "%s\n" % (datetime.datetime.now(),)
    except GeneratorExit:
        pass
    sd.close()

So I created a server that supports it: EvServer the Asynchronous Python WSGI Server


Implementation

I did my best to implement the latest of the three versions of Chris proposal. The code is based on my hacked together implementation of a very similar project django-evserver, which was created way before the extension was invented and before I knew about the WSGI multipart feature.

EvServer is very small and lightweight , the core is about 1000 lines of Python code. Apparently, due to the fact that EvServer is using ctypes bindings to libevent, it’s quite fast.

I did a basic test to see how fast it is. The methodology is very dumb, I just measure the number of handled WSGI requests per second, so as a result I receive only the server speed. The difference is clearly visible:

Server	Fetches/sec
evserver	4254
spawning with threads	1237
spawning without threads	2200
cherrypy wsgi server	1700

  
Description
 
So what really EvServer is?

• It’s yet another WSGI server.
• It’s very low levelish, the WSGI application has control on almost every http header.
• It’s great for building COMET applications.
• It’s fast and lightweight.
• It’s feature complete.
• Internally it’s asynchronous.
• It’s simple to use.
• It’s 100% written in Python, though it uses libevent library, which is in C.

What EvServer is not?

• Unfortunately, it’s not mature yet.
• It’s Linux and Mac only.
• It’s not fully blown, Apache-like web server.
• Currently it’s Python 2.5 only.

Examples

Admittedly using raw WSGI for regular web applications is a bit inconvenient. Fortunately decent web frameworks support passing iterators from the web application down to the WSGI server, throughout all the framework. On my list of frameworks that support iterators you can find: Django and Web.py.

Django

Django 1.0 supports returning iterators from views. This is Django code for the clock example:

def djangoclock(request):
    def iterator():
        sd = socket.socket(socket.AFINET, socket.SOCKDGRAM)
        try:
            while True:
                yield request.environ['x-wsgiorg.fdevent.readable'](sd, 1.0)
                yield '%s\n' % (datetime.datetime.now(),)
        except GeneratorExit:
            pass
        sd.close()
    return HttpResponse(iterator(), mimetype="text/plain")

The problem is that this code is not going to work using the standard ./manage runserver development server. Fortunately, it’s very easy to integrate EvServer with Django, you only need to put that into settings.py:

INSTALLEDAPPS = (
    [...]
    'django.contrib.sites',
    'evserver',             # <<< THIS LINE enables runevserver command)

Now you can test your app using ./manage runevserver.
Full source code for the example django application is in the EvServer examples directory.

Web.py

From the 0.3 version Web.py supports returning iterators. You can see it in action here:

class webpyclock:
    def GET(self, name):
        web.header('Content-Type','text/plain', unique=True)
        environ = web.ctx.environ
        def iterable():
            sd = socket.socket(socket.AFINET, socket.SOCKDGRAM) # any udp socket
            try:
                while True:
                    yield environ['x-wsgiorg.fdevent.readable'](sd, 1.0)
                    yield "%s\n" % (datetime.datetime.now(),)
            except GeneratorExit:
                pass
            sd.close()
        return iterable()

The full source code is included in EvServer example directory . You can run this code using command:

evserver --exec "import examples.frameworkwebpy; application = examples.framework_webpy.application"

Summary

I haven’t discussed any useful scenario yet, I’ll try to do that in the future post. I’m thinking of some interesting uses for EvServer – pushing the data to the browser using COMET.

LShift is recruiting!