Tuesday, October 24th, 2006Tuesday, October 24th, 2006

Use Server-Side Caching When Possible (memcached)

Purely out of necessity, I've become a system administrator/architect for digitalpoint.com servers... and a few people have been asking me for general admin tips to make things stable and scalable, so here's a good one for you...

Use memcached... no really, use it.

memcached is a distributed memory caching system that allows multiple servers to access the same shared memory (you can use it just for single local server too of course). You can cache pretty much anything... SQL query results (especially ones that can't hit indexes), dynamically generated web pages, dynamic RSS feeds, etc.

If you compile the memcache() functions/class directly into PHP, you have a an easy (yet powerful) way to incorporate it.

For example, let's say I have some existing code that makes a call to some sort of API (I use memcached for the keyword suggestion tool in this way)... you can add a couple lines of code to cache the results.

$data = @file_get_contents ("http://some_api_call_url?keywords=$keywords");

could become this:

$memcache = new Memcache;
$memcache->addServer('192.168.1.1');
$memcache->addServer('192.168.1.2');
if (!$data = $memcache->get($keywords)) {
	$data = @file_get_contents ("http://some_api_call_url?keywords=$keywords");
	$memcache->set($keywords, $data, false, 86400); // 24 hour cache
}
$memcache->close();

Now we are only hitting the API for the same keywords once every 24 hours... that setup utilizes 2 memcached servers working as a single entity for pooling and failover.

I use memcache for all sorts of things (this blog's pages are cached for 60 seconds as another example because WordPress sucks, and protects against a massive influx of traffic [front-page digg for example]).

Here's some things I personally use it for...

  • Caching this blog's pages (for 60 seconds)
  • Caching Keyword Suggestion Tool queries (for 7 days)
  • Caching AdSense Sandbox queries (for 24 hours)
  • Temporary counters in Keyword Tracker (using decrement() for displaying remaining keywords on a "Check All")
  • Recent forum topics that show on all the webmaster tools
  • vBulletin forum datastore

Any high-traffic site that is dynamically generated (especially if it includes SQL queries) could benefit from it... you could manage which memcache items need to be updated when you insert/update to the database, or an easy way to do it is just check the cache for the pages that potentially have high traffic... if they aren't in the cache, put them in the cache with a certain expiration date. Then the page is dynamically generated no more often than the cache expiration time (in the case of this blog, the pages are generated no more than once every 60 seconds).

P.S. - Make sure you block outside access to your memcached port since there's no authentication used!

9 Responses to “Use Server-Side Caching When Possible (memcached)”

  1. Patrick Says:

    We have been using memcached on Article Dashboard for about a month now and it really is awesome. My favorite feature is the ram access from all servers in the cluster. I won’t go into details about the performance increases but it was nothing short of amazing!

    -Patrick

    P.S. Here is a great article from Brad Fitzpatrick about using memcached on livejournal. http://www.linuxjournal.com/article/7451

  2. chrish Says:

    You know how I know you are white and nerdy?

    See above blog post.

  3. Johan Says:

    Does this require recompiling PHP? Or is it native?

  4. Shawn Says:

    You can compile it as a PHP extension and just add it to your php.ini file (you don’t need to recompile all of PHP).

  5. Ferrarislave Says:

    Shawn, do you need a spare server? I run a pretty intensive DB site (about 500k page views per day) off one server with 2GB of ram, could I use memcache()?

  6. Search Engines WEB Says:

    This is a great idea - had heard of memcached but never met anyone with first hand experience/

    So two recommendations in one post - may be worth testing

  7. Website Performance Tools: How I Installed Memcached | Jangro - Affiliate Marketing, SEO, etc. Heavy on the etc. Says:

    […] Distrubuted Caching with Memcached Use Server-Side Caching When Possible (memcached) If You’re Not Using memcached You’re Doing It Wrong Share and Enjoy: These icons link to social bookmarking sites where readers can share and discover new web pages. […]

  8. Brent Wilson Says:

    Hi Shawn,

    Just wondering if it was possible for you to share the Wordpress cache that you created? I’d love to cache my wordpress pages for 60 seconds (or even longer :) ). Thanks for your vBulletin Stats plugin for Memcached by the way. Very nice to have that information in a place I visit daily!

  9. A Few Updates - Brent Wilson Says:

    […] As far as the server optimization goes I upgraded to APC 3.0.13 today not realizing it had been released for a good bit. I also sat down and tried to get the memcache extension for PHP to work which I was successful in doing. So I finally got my memcached server running and right now all I am using it for is the datastore. I created a thread to see if I could drum up discussion on caching more of vBulletin into the memcached server so hopefully that will contain some good discussion. I was reading where the server admins of HowardSterns vBulletin forum were able to off load 70% of vBulletin into memcached which is amazing. I am also interested in Shawn Hogans use of memcached in Wordpress as a good bit of my traffic is hitting a wordpress blog. […]

Leave a Reply