Bash Linux server monitoring script

Part of my day-job is based around monitoring a business critical group of servers at work, so it seemed natural to try and implement some monitoring on a server I help look after for LSU CompSoc.

Given the major differences between the two environments and their importance, I thought a basic shell script should do the job for now (tested on Ubuntu 8.04.3 LTS).  The check’s I’ve implemented are Disk space checks, NIC errors, user changes (add/delete), package check, and checks for apache & mysql.  Also I’ve made sure that the script will run as a normal user so no need for sudo access to anything.

I’ve attached the script incase anyone else finds it of use  (You’ll need to modify the settings section as appropriate for your own server and create the relevant paths referenced in the settings.  The script is also dependent on this Perl SMTP CLI mailer (you’ll need to download a copy onto your server).  I plan to modify the script to support other mail methods but at present it just does what I needed from it.

I’ve also got another script to notify of any reboots and will add that here shortly.

[Download not found]

Open Flash Charts

Open Flash Chart Logo

Open Flash Chart Logo

I Thought I should try and give Open Flash Charts (OFC2) some publicity given how much I’m making use of this brilliant bit of software. I’m not going to cover what it can or can’t do as the OFC2 website does that already, but instead will cover why you should consider OFC for any charting needs.

OFC2 is a free, and open source, system for creating charts in Flash. It also comes with a collection of libraries to generate the charts using, PHP, Perl, Ruby, .NET, ColdFusion, and Javascript.

OFC2 supports a wide range of charts, and is constantly being improved my its creator, monk.e.boy and a few other dedicated contributors, recently adding animations which are always useful to please ‘PowerPoint’ obsessed end-users.

Why should you use OFC2 for webcharts? Well firstly its free, secondly its almost as good as any professional package I’ve found out there, and thirdly the small but active community on the OFC forums will try and help you with any issues you’ve found, as well as giving tips and often posting sample code they think could be useful to others.

I work for a very large international company in the financial services sector, and for various reasons some IT teams in the company are using amCharts for their graphing, and works very well for their needs. But for the work I’m doing we’ve gone with OFC2 as its far more flexible, no licensing/cost involved so no purchasing bureaucracy to deal with, and also personally I like to try to use and support open source projects where I can. Also by being free you can play with it as much as you need first to see if it meets your needs, no time limited or limited functionality demos.

The charts we use OFC for are used in reports for senior management, for daily visibility of our environment and also for other teams to view data about their usage of our environment and reduce the support burden on us. One key feature of OFC2 for us, is that we have the interactiveness of the flash chart but at the same time can easily save a chart as an image to embed in a document/report.

I’ve now been using OFC2 at work for well over nine months and recently implemented an extra abstraction layer between my data and OFC. This allows me to create a custom array structure for all my charts, and then use a set of predefined custom functions alongside the OFC2 api to turn this into JSON. This means I can very quickly adjust the code for any API changes, in one single file and makes it a lot harder to break charts while creating/updating the style of others.

One of OFC2′s greatest strengths is that it is constantly being improved, however this does cause a few problems for it, with slight changes to APIs and documentation lag caused partly by this. However on balance the pros far outweigh the cons, as it allows the software to evolve and develop in a way that is best for it and us users, without having to work changes around a fixed data structure or API unlike most commercial products. This means that messy/confusing data structures are able to be reduced to a n absolute minimum.

Pros

  • New releases are made often with bug fixes/new features
  • Free
  • Flexible
  • Friendly community support such as DZ’s branches/patches
  • A developer that’s willing to listen to suggestions and requests
  • Professional output
  • Open source, so you can change it as much you want

Cons

  • API can change with each new release
  • Small development team so can take a while for some things to get done
  • Seems to have a few memory leaks (more noticeable in IE than FF)
  • Documentation/examples can be a bit sparse in some areas and slightly out of sync with realeases for short periods

Drupal Modules

I thought I might as well ‘publish’ two Drupal modules that I’ve been working on.

Account Reactivation (Drupal 5)
This was created for LSU CompSoc’s new site, to enable members to request that their account is re-activated at the start of the new year, once they have signed up again.

Download the module here:

Timetable (Drupal 6)
This is a basic module I created as a test module to get back into Drupal development. It creates a timetable, with the ability to add and remove items via a form, and to set the slots available during the day on the timetable.

Download the module here:

————————————————————-

Please note that both these modules are provided ‘as is’ and may not be completely stable/include bugs . Do not use on a production site without first testing thoroughly.

However I will provide help/improvements/fixes if time allows for the modules.

Windows error/crash debugging

Recently my pc has been crashing a bit, which has caused it to reboot suddenly, and the error messages haven’t shown any useful details of the cause.

After a bit of googling I came across this guide which guides you through using the windows debugging tools to find the cause of the crash.

It will pinpoint the driver at fault etc. so you can update your drivers or look into ways of working round the issue.

Website Access Restrictions

I manage/admin a fair few sites covering an oil exploration equipment company to a church website. Unfortunately over time I’ve had a some issues with spammers posting random unwanted guestbook messages and so on.

Therefore I decided to go ahead and implement some measures to start blocking them. The main site this occurs on is the church site I run, which has an option to blacklist IP addresses. However as the spam comes from a range of IP addresses each time the site is ‘hit’ this would involve a tedious process of adding each address.

Therefore I decided to use the .htaccess allow/deny restrictions. While these are simple to use, getting the information required isn’t always as easy.

The method I took used three steps:

  1. Determining the IP Range involved
  2. Calculating the network prefix of the range
  3. Banning them

For the first step I noticed the IP addresses each time I was hit were similar, so I looked them up on http://software77.net/cgi-bin/ip-country/geo-ip.pl to see if they were in a single range assigned to a particular organisation or country. In this case they were.

Then I converted the range from a start and end IP address to a single address with a prefix. In my case it was 123.112.0.0/12. (you can check your conversion using http://www.csgnetwork.com/ipinfocalc.html) [The look-up link above now gives you the CIDR range]

Finally I added the following lines to my .htaccess file, including a comment for future reference

Order Allow,Deny
##Chinese IP range banned for guestbook spam 28/4/08
Deny from 123.112.0.0/12
Allow from all

As a side note banned users will get a HTTP 403 error page. I also created a custom page to be loaded for banned users, defined by the following line in my .htaccess file:

ErrorDocument 403 /errordocs/403.html

You also need to test that they have access to view this page, so I added another .htaccess file in the errordocs folder with the following lines:

order deny,allow
allow from all

PHP Server Monitoring Script

We’ve been having some issues with the availability of the LSUCS server recently, so I decided to try and put some monitoring in place. Unfortunately we’re not in the position to deploy a full monitoring suite with heartbeat monitoring etc.

So I developed a PHP script to check the status of the server remotely. This is done by checking the IP address, and connections on specified ports. The script can be configured to sends status emails, and to send error emails.

The script also accesses data from a remote PHP script on the monitored server to access disk usage data if available.

The script can be downloaded from here.

The script will probably be changed over time to support multiple servers and extra functionality.

26 Jan 2008: I’ve added support for multiple servers and extra features (see code for full details). The link above now points to the latest version.