#TechScribbles ubuntu sysadmin tasks for webserver – lighttpd, fcgi, thin, monit

29 Mar
One of my must-do activities during my time off has been sysadmin and I did some work on my server setup this week – slicehost-powered lighttpd web server talking to FastCGI PHP server and the Thin Ruby server.

** PHP with fastcgi server **

relevant snippets of lighttpd.conf for a PHP app:

## files to check for if …/ is requested
index-file.names ?? ?? ?? ?? ?? = ( "index.php", "index.html", "index.phtml",
???? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? "index.htm", "default.htm",
???? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? "index.lighttpd.html" )
$HTTP["host"] =~ "^ie6isolderthanyourgrandpa.com.*$" {
??server.document-root = "/path/to/app/public"
??server.errorlog = "/path/to/app/log/error.log"
??accesslog.filename = "/home/ie6/log/access.log"
??server.error-handler-404 = "/e404.php"
}

including in mod_fastcgi.conf:

fastcgi.map-extensions = ( ".phtml" => ".php" )
server.modules += ("mod_fastcgi")
fastcgi.server = ( ".php" =>
????( "localhost" =>
???? ??(
???? ?? ??"bin-path" => "/path/to/php.cgi",??### normally "/usr/bin/php5-cgi",
???? ?? ??"socket" => "/tmp/php.socket"
???? ??)
????)
)

Notice the bin-path delegates to a (trivial) shell script I made. This is recommended in several online discussions because apparently the standard way to configure php5-cgi is with environment variables. (I'm not sure why those can't be specified with a flag or why people don't use "A=B commandname" syntax but I went with the flow.) The important thing is not how it's run, but what this does. I was getting a lot of hanging with the lighttpd-fastcgi combination, and likewise I've read nginx-fastcgi has the same problem. Thankfully someone pointed out you can limit number of requests before the server re-starts itself:

export PHP_CGI=/path/to/php5-cgi
export PHP_FCGI_MAX_REQUESTS=1000
exec $PHP_CGI

** RUBY (Sinatra) with Thin server **

Simple setup with just one app server for now:

$HTTP["host"] =~ "(www.)?webwait.com" ??{
????proxy.balance = "fair"
????proxy.server = ??("/" =>
???? ?? ?? ?? ?? ?? ?? ?? ?? ??(
???? ?? ?? ?? ?? ?? ?? ?? ?? ?? ??( "host" => "127.0.0.1", "port" => 4567 ),
???? ?? ?? ?? ?? ?? ?? ?? ?? ?? ??# ( "host" => "127.0.0.1", "port" => 4568 ) if using multiple servers
???? ?? ?? ?? ?? ?? ?? ?? ?? ??)
???? ?? ?? ?? ?? ?? ?? ?? ??)
}

Thin itself is a standard init.d setup. I installed thin with "aptitude install thin" then "thin install". In the generated /etc/thin dir, I included my app's thin config:

environment: production
chdir: /path/to/webwait/sinatra/dir
address: 127.0.0.1
user: webwait
group: webwait
port: 4567
pid: /path/to/webwait/sinatra/dir/webwait.pid
rackup:??/path/to/webwait/sinatra/dir/config.ru
log: //path/to/webwait/sinatra/dir/thin.log
max_conns: 1024
timeout: 30
max_persistent_conns: 512
daemonize: true

And this points to the sinatra config, which is a rackup script. The sinatra app is called webwait.ru, so the rackup script is just:

require 'rubygems'
require 'sinatra'
require 'webwait'

run Sinatra::Application

** MONIT **

monit is doing well so far. One problem I had was with gmail – need monit version >= ?4.10 to handle SSL, which gmail requires for SMTP. So I had to rebuild it from source (v 5), which ended up changing some directories etc, but it worked fine and the config was unaltered.

Relevant snippets

I set up mail alerts against a special gmail account, which is great because I don't have or need a mail server on this server:

set mailserver smtp.gmail.com port 587 username "gmail-username" password "gmail-pass" using tlsv1 with timeout 30 seconds

I also set up a password-protected monit status page:

set httpd port port-number and
???? ??allow user:password
???? ??use address mydomain.com

The lighttpd check is quite similar to??

check process lighttpd with pidfile /var/run/lighttpd.pid
???? ??start program = "/etc/init.d/lighttpd start"
???? ??stop program ??= "/etc/init.d/lighttpd stop"
???? ??if cpu > 60% for 2 cycles then alert
???? ??if cpu > 80% for 5 cycles then restart
???? ??if totalmem > 200.0 MB for 5 cycles then restart
???? ??if children > 250 then restart
???? ??if loadavg(5min) greater than 10 for 8 cycles then stop
???? ??if failed host ajaxpatterns.org port 80 protocol http
???? ?? ?? and request "/"
???? ?? ?? then restart
???? ??if failed host webwait.com port 80 protocol http
???? ?? ?? and request "/"
???? ?? ?? then restart
???? ??if failed host ajaxify.com port 80 protocol http
???? ?? ?? and request "/"
???? ?? ?? then restart
???? ??# if failed port 443 type tcpssl protocol http
???? ?? ?? # with timeout 15 seconds
???? ?? ?? # then restart
???? ??if 3 restarts within 5 cycles then timeout
???? ??# depends on apache_bin
???? ??group server

Right now, I'm not monitoring fcgi separately – it's a hassle as there no .pid file and it's not necessarily running always. I *can* however check thin:

????check process thin with pidfile /path/to/webwait/sinatra/dir/webwait.pid
???? ??start program = "/etc/init.d/thin start"
???? ??stop program ??= "/etc/init.d/thin stop"
???? ??if cpu > 60% for 2 cycles then alert
???? ??if cpu > 80% for 5 cycles then restart
???? ??if totalmem > 200.0 MB for 5 cycles then restart
???? ??if children > 250 then restart
???? ??if loadavg(5min) greater than 10 for 8 cycles then stop
???? ??if failed host localhost port 4567 protocol http
???? ?? ?? and request "/"
???? ?? ?? then restart

** Forced restarts and reboots **

I also set up to restart the servers manually a few times a day, by way of cronjob, and reboot the server every few days. Now that I have monit set up, this may be overkill, but for hobby stuff like this I'd prefer to bias things toward less micromanagement, and less risk of long downtime when I'm otherwise occupied, at the cost of a little guaranteed downtime. In fact, restarting the servers is almost instantaneous; it's only the reboot every few days that will send things down for a minute or so … least of my worries.
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: