Caching Mephisto with lighttpd and mod_magnet

2008-05-15T13:49:00Z

This blog uses Mephisto, a rails application, and Thin as backend webserver. The frontend webserver is Lighttpd.

While Mephisto has a good caching system, producing a static html file for every visited page, the file is still served via Thin, reducing the speed and the scalability of the system and introducing an unnecessary step.

Lighttpd is very good at serving static files, thus I searched a way to serve a file from the cache directory of Mephisto, if present; I found a hack using mod_magnet.

Hereafter I wrote the relevant snippets of configuration files and how to put every piece in the right position.

Note: the same procedure should apply if you use Mongrel instead of Thin, for the relevant point is to bypass the backend rails webserver and serve all files (whether they exist) from the Mephisto cache.

First I configure Lighttpd to proxy to Thin for the rails application. I assume Thin (or Mongrel) is listening on port 3000 on two hosts.

  HTTP[''host"] == "virtualhost.example.org" {
        magnet.attract-physical-path-to = ( "/var/www/cache_mephisto.lua" ) 
        proxy.balance = "hash"
        proxy.server  = ( "" => ( ( "host" => "10.0.0.1", "port" =>  3000 ),
                                ( "host" => "10.0.0.2", "port" =>  3000 ) ) )
  }

The magnet.attract-physical-path-to = ( "/var/www/cache_mephisto.lua" ) is the magic trick. cache_mephisto.lua is a script written in lua programming language. The whole script is:

  local dir =  "/path/to/mephisto/public/cache/" .. lighty.request["Host"]

  if ( lighty.env["request.uri"] == nil or lighty.env["request.uri"] == "/") then
    file = dir .. "/index.html"
  else
    file = dir .. ".html"
  end

  if lighty.stat(file) then
  lighty.header["Content-Type"] = "text/html"
  lighty.content = { { filename = file } }
  return 200
  end

 return lighty.RESTART_REQUEST

Inside the script you should adapt the variable dir to your Mephisto installation path. I need to append the virtual host name lighty.request["Host"] to the directory, for I run a multisite Mephisto installation; if you use a single site mephisto installation, probably you need to remove this append.

Now for some benchmarks:

serving the file via Thin:

   Requests per second:    1.40 [#/sec] (mean)
   Time per request:       716.007 [ms] (mean)

serving the file from the cache

   Requests per second:    11.46 [#/sec] (mean)
   Time per request:       87.247 [ms] (mean)

Almost a tenfold improvement!

Future works: serve from the cache the image files.

Caching Mephisto with lighttpd and mod_magnet

2008-05-15T12:56:49Z

This blog is using Mephisto, a rails application, and Thin as backend webserver. The frontend webserver is lighttpd.

Whilst Mephisto has a good caching system, producing a static html file for every visited page, the file is still served anyway via Thin, reducing the speed and the scalability of the system and introducing an unnecessary step, for Lighttpd is very good at serving static files.

Thus I searched a way to serve a file from the cache directory of Mephisto if it is present and I found an answer using mod_magnet.

Hereafter I wrote the relevant snippets of configuration files and how to put every piece in the right position.

Note: the same procedure should apply if you use Mongrel instead of Thin, for the relevant point is to bypass the backend webserver and serve all files (whether they exist) from the Mephisto cache.

First I configure Lighttpd to proxy to Thin for the rails application lighttpd configuration

Equars People - Caching Mephisto with lighttpd and mod_magnet Changes

Caching Mephisto with lighttpd and mod_magnet

Caching Mephisto with lighttpd and mod_magnet