Application scaling and memcache
We’ve known about the secret scaling powers of memcache for some time (Here’s a list of sites using it) , BUT … I have just read some slides from a presentation which provides an overview of how facebook is using memcache. (Slide 21)
Facebook uses
- 200 dedicated memcache servers
- Each server is 16GB quad core AMD64
- Over 3Tb of memcache
Wow 3 Terabytes of memcache ! – It looks like memcache is the application stack for facebook. Memcache and equivalent tools will change the way that people think about design and structure of their application. Just because you’ve got a blazingly fast memory doesn’t mean that you shouldn’t tune your database !
We’ve just noticed that there is an pgmemcache API for memcache and Postgresql. Now we have to think about where do you put your cache ? at the application level or the database level ?
FYI: The facebook team have been contributing to the memcache source and they recommend that you use v 1.2.x rather than 1.1.x . (Note Debian and ubuntu has 1.1.3 by default, you’ll need to manually install the new package)
September 17th, 2007 at 4:30 pm
You may find this interesting, if you haven’t read it already: http://koziarski.net/archives/2007/5/28/clever-caching . Get your keys right and the rest follows.
My first instinct is to cache from the user end back. The less deep into the stack the request has to go, the better.
September 18th, 2007 at 11:09 am
Django also recently sprouted a built in way to store session data in memcache instead of hitting the database even harder (http://code.djangoproject.com/changeset/6333). It appears to be merely a case of putting “SESSION_ENGINE=django.contrib.sessions.backends.cache” in settings.py. And setting up memcache, obviously.