Archive | March, 2010

#TechScribbles ubuntu sysadmin tasks for webserver – lighttpd, fcgi, thin, monit

29 Mar
One of my must-do activities during my time off has been sysadmin and I did some work on my server setup this week – slicehost-powered lighttpd web server talking to FastCGI PHP server and the Thin Ruby server.

** PHP with fastcgi server **

relevant snippets of lighttpd.conf for a PHP app:

## files to check for if …/ is requested
index-file.names ?? ?? ?? ?? ?? = ( "index.php", "index.html", "index.phtml",
???? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? "index.htm", "default.htm",
???? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? "index.lighttpd.html" )
$HTTP["host"] =~ "^ie6isolderthanyourgrandpa.com.*$" {
??server.document-root = "/path/to/app/public"
??server.errorlog = "/path/to/app/log/error.log"
??accesslog.filename = "/home/ie6/log/access.log"
??server.error-handler-404 = "/e404.php"
}

including in mod_fastcgi.conf:

fastcgi.map-extensions = ( ".phtml" => ".php" )
server.modules += ("mod_fastcgi")
fastcgi.server = ( ".php" =>
????( "localhost" =>
???? ??(
???? ?? ??"bin-path" => "/path/to/php.cgi",??### normally "/usr/bin/php5-cgi",
???? ?? ??"socket" => "/tmp/php.socket"
???? ??)
????)
)

Notice the bin-path delegates to a (trivial) shell script I made. This is recommended in several online discussions because apparently the standard way to configure php5-cgi is with environment variables. (I'm not sure why those can't be specified with a flag or why people don't use "A=B commandname" syntax but I went with the flow.) The important thing is not how it's run, but what this does. I was getting a lot of hanging with the lighttpd-fastcgi combination, and likewise I've read nginx-fastcgi has the same problem. Thankfully someone pointed out you can limit number of requests before the server re-starts itself:

export PHP_CGI=/path/to/php5-cgi
export PHP_FCGI_MAX_REQUESTS=1000
exec $PHP_CGI

** RUBY (Sinatra) with Thin server **

Simple setup with just one app server for now:

$HTTP["host"] =~ "(www.)?webwait.com" ??{
????proxy.balance = "fair"
????proxy.server = ??("/" =>
???? ?? ?? ?? ?? ?? ?? ?? ?? ??(
???? ?? ?? ?? ?? ?? ?? ?? ?? ?? ??( "host" => "127.0.0.1", "port" => 4567 ),
???? ?? ?? ?? ?? ?? ?? ?? ?? ?? ??# ( "host" => "127.0.0.1", "port" => 4568 ) if using multiple servers
???? ?? ?? ?? ?? ?? ?? ?? ?? ??)
???? ?? ?? ?? ?? ?? ?? ?? ??)
}

Thin itself is a standard init.d setup. I installed thin with "aptitude install thin" then "thin install". In the generated /etc/thin dir, I included my app's thin config:

environment: production
chdir: /path/to/webwait/sinatra/dir
address: 127.0.0.1
user: webwait
group: webwait
port: 4567
pid: /path/to/webwait/sinatra/dir/webwait.pid
rackup:??/path/to/webwait/sinatra/dir/config.ru
log: //path/to/webwait/sinatra/dir/thin.log
max_conns: 1024
timeout: 30
max_persistent_conns: 512
daemonize: true

And this points to the sinatra config, which is a rackup script. The sinatra app is called webwait.ru, so the rackup script is just:

require 'rubygems'
require 'sinatra'
require 'webwait'

run Sinatra::Application

** MONIT **

monit is doing well so far. One problem I had was with gmail – need monit version >= ?4.10 to handle SSL, which gmail requires for SMTP. So I had to rebuild it from source (v 5), which ended up changing some directories etc, but it worked fine and the config was unaltered.

Relevant snippets

I set up mail alerts against a special gmail account, which is great because I don't have or need a mail server on this server:

set mailserver smtp.gmail.com port 587 username "gmail-username" password "gmail-pass" using tlsv1 with timeout 30 seconds

I also set up a password-protected monit status page:

set httpd port port-number and
???? ??allow user:password
???? ??use address mydomain.com

The lighttpd check is quite similar to??

check process lighttpd with pidfile /var/run/lighttpd.pid
???? ??start program = "/etc/init.d/lighttpd start"
???? ??stop program ??= "/etc/init.d/lighttpd stop"
???? ??if cpu > 60% for 2 cycles then alert
???? ??if cpu > 80% for 5 cycles then restart
???? ??if totalmem > 200.0 MB for 5 cycles then restart
???? ??if children > 250 then restart
???? ??if loadavg(5min) greater than 10 for 8 cycles then stop
???? ??if failed host ajaxpatterns.org port 80 protocol http
???? ?? ?? and request "/"
???? ?? ?? then restart
???? ??if failed host webwait.com port 80 protocol http
???? ?? ?? and request "/"
???? ?? ?? then restart
???? ??if failed host ajaxify.com port 80 protocol http
???? ?? ?? and request "/"
???? ?? ?? then restart
???? ??# if failed port 443 type tcpssl protocol http
???? ?? ?? # with timeout 15 seconds
???? ?? ?? # then restart
???? ??if 3 restarts within 5 cycles then timeout
???? ??# depends on apache_bin
???? ??group server

Right now, I'm not monitoring fcgi separately – it's a hassle as there no .pid file and it's not necessarily running always. I *can* however check thin:

????check process thin with pidfile /path/to/webwait/sinatra/dir/webwait.pid
???? ??start program = "/etc/init.d/thin start"
???? ??stop program ??= "/etc/init.d/thin stop"
???? ??if cpu > 60% for 2 cycles then alert
???? ??if cpu > 80% for 5 cycles then restart
???? ??if totalmem > 200.0 MB for 5 cycles then restart
???? ??if children > 250 then restart
???? ??if loadavg(5min) greater than 10 for 8 cycles then stop
???? ??if failed host localhost port 4567 protocol http
???? ?? ?? and request "/"
???? ?? ?? then restart

** Forced restarts and reboots **

I also set up to restart the servers manually a few times a day, by way of cronjob, and reboot the server every few days. Now that I have monit set up, this may be overkill, but for hobby stuff like this I'd prefer to bias things toward less micromanagement, and less risk of long downtime when I'm otherwise occupied, at the cost of a little guaranteed downtime. In fact, restarting the servers is almost instantaneous; it's only the reboot every few days that will send things down for a minute or so … least of my worries.
Advertisements

“But instead of going from server to server, Coppola decided to hack the scoreboard and give himself the most points. Naturally, he won.” nice hack!

22 Mar

Educating Elite Hackers
http://www.newsweek.com/id/234678/output/print

But instead of going from server to server, Coppola decided to hack the scoreboard and give himself the most points. Naturally, he won.

(via Instapaper)

quick overview and convenience functions for Tokyo Cabinet

22 Mar
I was using MongoDB, but decided to switch to Tokyo because I wanted something that's embedded, ie not running a separate server. It will make deployment and maintenance easier that way, and if I ever have massive I/O problems, I can switch over to Tokyo Tyrant with the same API. Win for the Happy Path.

On OSX I did this:

(1) port install tokyocabinet

(2) gem install tokyocabinet

(3 Use it like this:

#!/usr/bin/env ruby
require 'rubygems'
require 'tokyocabinet'

def db()
????store = TokyoCabinet::HDB.new
????store.open("countries.tch", TokyoCabinet::HDB::OWRITER | TokyoCabinet::HDB::OCREAT)
????result = yield store
????store.close
????result
end

def fetch(key)
????db { |store| Marshal.load(store.get(key)) }
end

def save(key, val)
????db { |store| store.put(key, Marshal.dump(val)) }
end

save("japan", { "prefectures" => 47 })
fetch("japan") ### returns { "prefectures" => 47 }

playing with jQuery custom events, good tip from @ajpiano

21 Mar

I’m rewriting WebWait, which includes major refactoring to use a pubsub model, hence jQuery custom events. Best summary of custom events is at http://fuelyourcoding.com/jquery-custom-events-they-will-rock-your-world/ :

—-

If you are the “just the facts” type of person, here is the 20 second overview. You can trigger custom events on any DOM object of your choosing using jQuery. Just trigger it using the following code:

1.$("#myelement").trigger('fuelified');

You can subscribe to that event using either bind or live functions in jQuery:

1.$("#myelement").bind('fuelified',function(e){ ... });
2.$(".cool_elements").live('fuelified',function

The element the trigger has been called on, is available to listener’s callback function as the variable this.

—–

But one odd thing about all these intros-to-custom-events is there’s always an element in these examples. Fine for mouse clicks and the like, but the main reason to use custom events, I would have thought, is to add a layer of logic around semantic events, things detached from UI concepts. So in WebWait, it’s things like “trialUpdate” and “trialDelete”. WebWait is a humble single-page app, so why would I want to namespace events? And even if I was namespacing events, why would I want to make the namespace a CSS selector, given that they are semantic (non-UI) in nature?

I tried using $().bind() and $.trigger(), but no luck (and prior to that, I accidentally used $.bind() and $.trigger(), which don’t actually exist). So I jumped onto #jQuery API and @ajpiano immediately informed me you can use $(document).bind() and $(document).trigger(). Problem solved. Of course, I could have used any old element on the page, but (a) doing so would have tied potentially reusable stuff to my application; (b) it’s a cleaner expression of my intent. The less gratuitous details, the better.

A plancast “widget” – parsing @plancast RSS data

16 Mar

Media_httpfarm3static_laknz

I’ve been enjoying plancast – similar to foursquare, which I don’t really use, but instead of saying where you *are*, you say where you *will be*. And for meaningful places, ie industry events, not just checking in because you have to be at the dry cleaners.

In my homepage revamp, I decided to integrate my next event using my plancast feed. However, the feed doesn’t provide much in the way of timing details. There’s no human-friendly format (“in 3 days”) and in fact, there’s no structured date format (i.e. 12092840482 seconds after 1970), and in fact, there’s not even a specific field for the time at all. It just appears in the “description” field, and it appears in different ways, i.e. a single date vs. a date range. (And different again if it spans two months, and if it spans two years, hmmm, it doesn’t seem to be working yet for that – ).

So I wrote a custom parser to deal with it. It accepts the “description” string from the RSS feed:

  function plancastDaysAgo($descriptionString) {

    $match = preg_match(“/When: (.*?)<br/”, $descriptionString, $matches);
    if (!$match) return;
    $whenString = $matches[1];

    # “Monday, March 14, 2010” or “March 14-15, 2010”
    $match = preg_match(“/^(?:w+, )?(w+) (d+)(?:-d+)?, (d+)$/”, $whenString, $matches);

    # “May 30- June 2, 2010”;
    if (! $match) $match = preg_match(“/^(w+) (d+)- w+ d+, (d+)/”, $whenString, $matches);

    date_default_timezone_set(‘Europe/London’);
    $inputDateString = $matches[1].” “.$matches[2].”, “.$matches[3];
    $days = floor((time() – strtotime($inputDateString))/86400);

    if ($days==0) return “today”;
    elseif ($days==1) return “yesterday”;
    elseif ($days==-1) return “tomorrow”;
    elseif ($days>1) return “$days days ago”;
    return “in “.abs($days).” days”;

  }

It’s called daysAgo for compatibility with standards “timeAgo”/”daysAgo” libraries, even though most of the time, the date will be in the future in this case.

Incidentally, a single item looks like this:

<item> <title><![CDATA[2010: The Year We Broadcast Our Credit Card at Radisson Austin, Tuesday, March 16, 2010, 12:30pm (Robert Scoble)]]></title> <link>http://plancast.com/a/1g51?utm_source=rss&amp;utm_medium=feed&amp;utm_campaign=feed</link> <description><![CDATA[Robert Scoble shared a new plan.<br /><br />What: 2010: The Year We Broadcast Our Credit Card<br />When: Tuesday, March 16, 2010, 12:30pm<br />Where: Radisson Austin]]></description> <pubDate>Mon, 08 Mar 2010 19:05:08 +0000</pubDate> <guid isPermaLink="false">l3095567@plancast.com</guid> <author>Robert Scoble (Scobleizer)</author> </item>

I’m using MagpieRSS, so it looks something like:
     if ($rss) {
      $next = $rss->items[0];
      print “on the calendar: “.plancastDaysAgo($next[description]);
    }

author-specific wordpress feed for ajaxian

15 Mar

A while ago, a mate of mine asked me if there was a way to see just my articles on Ajaxian. I finally looked into it and discovered it's possible:

(And also the more raw, canonical URL – http://ajaxian.com/?author=7)

I'm updating my homepage today and wanted to include my Ajaxian posts alongside my personal blog, posterous (you are here), and twittering.

Harmony drawing tool

10 Mar
Photo