16 October 2012
A couple of week ago I attended my first PyConUK conference. It was a really interesting and useful event that left me enthused to do more with Python and get involved with the fantastic community around it.
I enjoyed Saturday’s lightning talks so much that it inspired me to try and give one for myself on Sunday. I stayed up until 3am working on the slides and then spent most of Sunday feeling apprehensive, especially given the quality of the other talks. When the time came I rushed through my topic fuelled only by nervous energy.
The video of the Sunday afternoon has now been posted here. My talk is around 1hr45m in. I’d swear I was talking faster than that.
The topic of my talk was the topic of this post. Hopefully I can capture it in a little more detail here.
Taking Bantr to 2.0
Over the past 18 months I’ve been working on a social football site called Bantr. A big priority for Bantr was that their site stay up to date with the latest scores, in game action and user discussion without having to reload the page and preferably as soon as there is new data available.
We’ll soon be launching Bantr 2.0, which is built from the ground up. Our server side is now purely a RESTful API. I found working within the constraints of REST was initially frustrating but ultimately led to a simpler and more modular API that prevented leaking of frontend functionality further back up the stack.
It’s my first project using the Flask microframework and I’ve been pleased with the simplicity and performance of the code. If possible I was really keen to stick with Python as the only server-side language this time.
Client side, we can fire events off to the server at any time with a simple XHR request. The only piece missing is the ability for the server to fire events off to the client. And that’s exactly what Server-Sent Events allow us to do.
To use them client side, we just need to create an
EventSource object is
onmessage callback to handle the received event.
Here’s a super-simple page that’ll add a new list item to an unordered list for each server event.
Adding support for Internet Explorer is as simple a dropping in a tiny polyfill you can find on GitHub.
The actual protocol for Server-Sent Events is very simple. The client
will open a standard connection to the server and make a GET
request. It expects the server to hold open the socket and send new
events by prefixing them with
data: and terminating with two
Below is a simple server that returns a incrementing counter. Note the use of a generator passed to the Flask Response function.
#!/usr/bin/env python import gevent import gevent.monkey from gevent.pywsgi import WSGIServer gevent.monkey.patch_all() from flask import Flask, request, Response, render_template app = Flask(__name__) def event_stream(): count = 0 while True: gevent.sleep(2) yield 'data: %s\n\n' % count count += 1 @app.route('/my_event_source') def sse_request(): return Response( event_stream(), mimetype='text/event-stream') @app.route('/') def page(): return render_template('sse.html') if __name__ == '__main__': http_server = WSGIServer(('127.0.0.1', 8001), app) http_server.serve_forever()
It’s also important to send back a mimetype of
text/event-stream or the
browser won’t generate our events.
The W3C specification for Server-Sent Events states that they should be compatible with Cross Origin Resource Sharing (CORS) policies. However this does not currently appear to be the case for Chrome and Safari. Therefore you will need to serve your SSE URL from the same domain as the site consuming it unless you make use of a polyfill.
One particularly handy feature of SSE over WebSockets is their ability to be proxied by nginx. Do make sure to disable proxy buffering if you do this though, else your events might bunch up at the proxy before being sent on to the client.
Also be aware of proxy read timeout. I counter these by generating
ping events every 120 seconds that go ignored by my client code.
You can find the code above and my PyCon slides at http://github.com/stevenewey/ssedemo.