This hurt my work creditcard

I'm trying to get the desktop support officer to name them all after diseases.
Can you imagine: someone trying to connect via wireless to a fellow in a room full of people -"who's got syphilis? - I'm trying to get onto syphilis".

Proof that I am a bigger dork than you

Screen caps of me in a video conference - I don't normally work in a singlet - I'd just come back from the gym. That's a peach by the way

Work

I have talked previously about what I do but if any of you out there look after websites or work in a uni for a living I thought you may be interested in the scale of online learning in a modern uni.
The graph above show hits per hour for an average day, at 4AM it plummets to nearly 300K hits but never much below that. We gets students logging in from all over the world. In one month alone we had 600k logins with an average stay of over an hour - that's over six hundred thousand hours of contact with the online learning system in one month. My staff took over 10k support calls and emails this year from academic staff alone. We podcast over 700 hrs of lectures a week. It has been a big year - I'm going to enjoy the xmass break - how about you.

Business time

The students are finishing up their last exams and now it is time for us to roll up our shirt sleeves and get cracking on some IT work that we couldn't do when they are here in force.

Meanwhile I'm trying to write some web pages to replace a bunch of paper forms and reduce our data entry requirements. For the most part it looks as if it will work, but I'm doing some cheeky screen scraping to read other peoples web pages to get the info I need. The uni is comically inefficient at making that kind of information available.

Alex (wife) is heading off to the US for a whirlwind visit, mostly the mid north east and Chicago with a day in Memphis. She's looking at some gas facilities for a big project she is working on here. She wont have time for sight seeing tho. :-(

That of course means that I have to look after the kidlets by myself for a week. Think I may need a few wines to get through that one.

Selling a boat on ebay

Interviewing next week for one of my System Administrator positions - give me a call if your a smoking hot linux sys admin who knows PERL and can install and manage open source apps such as media wiki. A unix ticket would be a plus.

Found them - you can stop looking everybody

In the gym bag. I must have vagued out after the 2h spin / pump back to back gym marathon on Monday that left me gasping and feeling as if I had bled out. I usually cope with being a scatter brain by attaching my keys to my belt loop via a 'bina - oh so high fashion I know but that's a price I'm willing to pay for being able to open the front door when I get home.

In other news I'm having more trouble deciding what to wear today than figuring out what to wear to a swingers party. I've been invited to a graduation for one of my staff; to tie or not to tie, that is the question. French cuffs or dress down?

I need your help people.

So, did I ever tell you about the time...

I'm not sure whether to tell you about the new gym instructor, the one way glass windows or the girl whom I accidentally disrobed..

Reverse order then.

So, did I ever tell you about the time I accidentally stripped a complete stranger out of her dress?
No, well OK then..
It seems like it was only yesterday, which is not a complete surprise as it was in fact, only yesterday.
I was on my way to a meeting in another part of the uni. The building is about half way across campus and, unsurprisingly, the room is up a long staircase.
About halfway up said staircase I encountered a young(ish) woman walking down the other way. Normal people move the the left, just like driving, it's the basic collision avoidance mechanism, part of the social contract - I move a bit out of the way, you move a bit out of the way and we all get along together.
But no.
Not this lass.
Tall, kind of skinny, browny blond hair, alternate rustic hippie look - cheese cloth blouse and long ankle length flowing Balinese batik thin cotton skirt and sandals.
I noticed the silver toe ring.
I noticed it because she was coming right down the center of the fucking staircase.
I moved so far to the side my left arm was against the wall. I uttered something like FFS. Then there was an almighty 'rip'. I had trodden on the train of her skirt. Quick look down. two bare legs and in a pair of daggy nickers. Eyes back forward I said something like "Jesus lady, it's not like you made it easy, I couldn't get any further out of you way.. your bad - not mine. I shan't look down, and how about next time you have the courtesy to move to the side on the footpath like every other living human, (perhaps excepting the yanks)".
She said nothing.
I walked up and took the next left exiting the stairs and quickly stepped up to the window to see if she exited the building... nope.

I'm a nerd herder

So this is the first week that students are back for Sem 2 classes.

I'm ultimately responsible for the running of the online teaching websites - wikies, blogs the online classrooms and a plethora of other technologies. We record in the order of a thousand hrs of lectures a week and podcast them. We get data feeds from the student system and emails from some other system and staff details from yet another system. The online tools via which students communicate with each other, read their lecture notes and submit assignments and the integration code between all these things is all mime to make go and provide staff support. We train the academics in how to use the tools and have a help desk to answer their questions when they didn't listen in class. I also have to look after desktop support for the section of the university in which we are located. I've a good team, some systems administrators, programmers, trainers and miscellaneous IT workers who all work well outside the box of their job descriptions.

On the slowest of days we are busy. Students are accessing their online units 24x7. We are busy on xmass day. Some students do courses that are entirely online and for them the systems I own are the university. This is the time of year that we hire extra support and put the KPI's and strategy meetings aside and knuckle down under the load, do the unpaid overtime and eat lunch at our desks. Academics get stressed and somehow it is our fault when academics do unusual and creative things with their online units. If we fuck up I get 20 thousand emails letting me know exactly how bad we fucked up. Nothing like real time feedback - it is like working naked - everybody can see everything you do and every little mistake.

So far this semester we haven't had a singe piece of hate mail - which after 40+ K enrollments into many hundreds of online units is, I feel, no small achievement.

How was your day?

Geek post

I'm having a problem at work. Students using a ADSL2+ with TPG (an ISP) are experiencing slow connections when downloading recordings (pod casts) of their lectures. Some students have submitted problem tickets to the database we maintain to help diagnose exactly this type of problem.

I need to provide TPG with details of the students who have this problem to help them sort out what is going wrong within their network. When the students submit their problem we record their IP address. Anyway, here is a useful little unix bash shell script I wrote which will read through a list of ip addresses and then do a whois lookup on each one and will print out all the ones from a particular provider.

#!/bin/bash
# script that iterates through a list of ip's in a file provided through stdin 
# does a whois look up on each of them and then prints out all the ones with
# a particular ISP - in this case TPG
#
#

FILE=""    # variable to hold the file name
if [ "$1" == "" ]; then  # if there is no file as an arg
   FILE="/dev/stdin"  #   then get it from stdin
else
   FILE="$1" 
   if [ ! -f $FILE ]; then 
   echo "$FILE : does not exists"
   exit 1
   elif [ ! -r $FILE ]; then
   echo "$FILE: can not be read"
   exit 2
   fi
fi



cat $FILE | while read LINE # pass the file line by line
do
 ISP=`whois $LINE | grep netname | awk '{print $2}'`
 if [ "$ISP" == "TPG-AU" ]; then
  echo "$LINE" 
  sleep 2
 fi
done

I always find examples like this found on other peoples blogs useful when searching the web for snipits of tested code.

Back at work today

I've been looking after the kids but they went back to child care today, so daddy goes back to work.

276 emails

23 phone messages

15 days to recrute a Perl programmer / sys admin / web admin who lives and breaths LDAP

3 months of purchace orders to do (I really should have doen them last year)

1 new boss

1 less direct reports

Sick kidlets

After learning of my almost certain retrenchment from what I think is a pretty good job, most of last week passed in an unmotivated blur. I'm probably a bit depressed but I'll get over myself.
Then on Sunday I noticed Annalise - the three year old, had a temperature and was complaining of a sore belly which was quickly followed by -- well, lets leave out the details shall we. I had to take Monday off to look after her; you can't send them to school when they need to be put on the toilet every half hour. Turns out it's tonsillitis. She didn't have a temperature on Tuesday but she was still sick. We normally have a carer - one of the great aunts, but she couldn't really look after Adrian and Annalise with Annalise needing that much attention. All seamed good Tuesday evening until I picked up Adrian from the floor and he was burning. Temp of 38.6° on the skin on his back (you try sticking a thermometer under a wriggling toddlers arm pit). So it's Wednesday now. I'm still not back at work and I have another sick kid to look after.
The same think is happening to the cousins.

Damocles

I have had a wonderful weekend, thanks for asking. House dad on Saturday building a cubby hose from couch cushions whilst Alex did the weekly shopping and chose some shoes and then a short bush walk in the National Part out the back of the house in the afternoon. Sunday we trekked to the zoo, late due to a conspiracy between Adrian not sleeping most of the early early morning and refusing to let daylight savings thieve an hour of snoozes.

And now I am at work.

Waiting for the all staff meeting at 1:30 to discuss the merger of the division of the university I work for with another and find out whether I will still have a job and thus whether I will be looking for work over Christmas or enjoying the time off.

How to plot how busy you server is from the user perspective

Do you run a mission critical application on an Apache web server?
if not stop reading - I'll post cute picks of kids or my MRI scan later.

click here if your a gheek and want to read more

How to plot how busy you server is from the user perspective.
The assumption I am making here is that the slower the server is the more user connections I will see because they spend more time connecting because the server is slower etc. etc.

In effect I'm going to plot the "requests being processed" against time.

So you've got mod_status installed and you have a cron job gziping and taring wgets of the server_status page

on a typical day head -n30 looks like this:

Apache Server Status for XXXXX.XXXXX.edu.au

Server Version: Apache/2.0.55 (Unix) DAV/2 mod_ssl/2.0.55 OpenSSL/0.9.6m mod_jk/1.2.6
Server Built: Jan 4 2006 17:25:42

Current Time: Wednesday, 10-Oct-2007 15:23:08 EST
Restart Time: Tuesday, 09-Oct-2007 17:24:28 EST
Parent Server Generation: 1
Server uptime: 21 hours 58 minutes 40 seconds
Total accesses: 1756415 - Total Traffic: 28.1 GB
CPU Usage: u1452.51 s1680.81 cu6053.03 cs0 - 11.6% CPU load
22.2 requests/sec - 372.4 kB/second - 16.8 kB/request
129 requests currently being processed, 41 idle workers

K_KCKKKWKWKKRWCKKKKKKKKKKKCK_KKRK_KKKKK_K._K.K.KKKKKWKCK.._KKKK_
.K_._._CKK__KK_K.KKK_KKWKKKK_KK..KK._.KK.K___KK.WK_KKCK_K_K_.KK_
.KKKKKK__WR_K.KK.K_KKWKKR.K._K__K..KKK_K_K_W.KK.K...............
....................................._..........................
....................._.................................K........
................................................................
..........................................K................_...K
......_.._..........................................K...........
................................................................
................K........K......................................
...........W....................................................
..............W.................................................
................................................................
.........._.................................................._..
.................................K..............................
........K...........................................K...........

And when you are being subjected to a DOS attack it looks more like this

Current Time: Monday, 08-Oct-2007 15:59:21 EST
Restart Time: Sunday, 09-Sep-2007 14:19:25 EST
Parent Server Generation: 29
Server uptime: 29 days 1 hour 39 minutes 56 seconds
Total accesses: 38331840 - Total Traffic: 326.1 GB
CPU Usage: u749.13 s842.35 cu2791.89 cs0 - .175% CPU load
15.3 requests/sec - 136.1 kB/second - 8.9 kB/request
1024 requests currently being processed, 0 idle workers

GRRRGRRRRRRRRRRGRRRRRRRRRRRRRRRRRKRRRRRRRRRRRRKRRKRRRRGRRRRRRRRR
RRRWGRGRRRRRRRRRRRRRRRRKRRRRRRRRRRRRGRKRRRRRRRRRRRGRRRRRRRRRWRRR
RRRGRRRRRGRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRGRGRKRRRCRRRKKRRRRRR
RRRRRRRRRRRRRRRRRRRRRRRRRRRRGRRRRRRRRRRRRRRRRRRKRRRRRRRRRRKRRRRR
RRRRRRRRRRRRRRRRRRRKKRRRRRRKRRRRKRRRRRRRRRRRRRRKRRRRRRRGRRRKRRRR
RRGRRRWRRGRRRRRRRRRKRGRRRRRRGGRRRRRRRRRRRRRRRRKRRRRRRRRRRKRRRRKR
RRKRRRRRRRRRRRRRRRRRRRRRRRRGRRRRRRRRRRRRRRRKRRRRKRRRRRRRRKRRRRRR
KRRRRRRRRRRKRRRRRRRRRRRRRRRRKRKRKRRRRRRRRRRRRRRRKKRRCRRRRRRRRKRW
RRKRCRKRRKRRRRRRRRRRRRRWRKKRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRR
RRRKRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRKRRRRRRRRRRRRRRRCRR
RRRKRRRKRRRRRRRRWRRRRRKRRRRRRRRRRRRRRRRRRRRRRWRRRRRRRRRRRRRRRRRR
KRRRRRRGRRRKKRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRKKRRRRRRRRRRRR
RRRRRRRRRRRRRRKRKRRRRKRRRRRRRRRRRRRRRRRRRRRRRRRRRRRWRKRRRRRRRRRR
KRRRRRKRRRKRRKRRRKRKRRRKRRRRRKRKRKRRRKRRRRRRRRRRRRRRRRRRRRRRRRRR
RRRRKKRRRRRRRRRRRRRRRRKRRRRKRRRRRRRRRRRRKKRRRRRRGRRRRRRRRRRRKRRK
KRKRRKRKRRRRRRRKRKRRRRKWKRRRRRRRKRRKRKRKRKRRRCRRRCKRCKRRRKRKRKKC

where all those R's are ..Reading..

Method

Write a script that loops through all your tar files untars and ungzips them then greps the head to get the date and the requests line. We gzip and tar the original pages into a directory - opt/servstat and then tar together whole days so mine looks like this but your mileage may vary:

#!/bin/bash

# Clear the screen in preparation for script output
clear
echo "--------------------------------------------------"

dir=`pwd`
startdir=$dir

if [ -d "tmp" ] ; then
 rm -r tmp
fi

if [ -d "~/Desktop/out" ] ; then
 rm -r ~/Desktop/out
fi
mkdir tmp
mkdir ~/Desktop/out

for file in $*; do
 (
  cd tmp

  tar zxvf ../$file
  (
   cd opt/servstat
   gunzip *.gz
   sudo chown stephen\: *
   mv *.txt ~/Desktop/out   
   head -n17 /home/stephen/Desktop/out/*.txt | grep -i Current >> ~/Desktop/outputFile
   rm ~/Desktop/out/*.txt 
  )
 )
 
done

cd $startdir

exit 0

end result being I get a file some 100K lines ling that looks like this;

<dt>Current Time: Wednesday, 10-Oct-2007 08:35:00 EST</dt>
<dt>69 requests currently being processed, 36 idle workers</dt>
<dt>Current Time: Wednesday, 10-Oct-2007 08:40:00 EST</dt>
<dt>103 requests currently being processed, 21 idle workers</dt>
<dt>Current Time: Wednesday, 10-Oct-2007 08:45:00 EST</dt>
<dt>78 requests currently being processed, 46 idle workers</dt>
<dt>Current Time: Wednesday, 10-Oct-2007 08:50:00 EST</dt>
<dt>80 requests currently being processed, 44 idle workers</dt>
<dt>Current Time: Wednesday, 10-Oct-2007 08:55:01 EST</dt>
<dt>111 requests currently being processed, 52 idle workers</dt>

you can get rid of the HTML if you want sed -e 's/<[^>]*>//g' outputFile > outputFileNoHTML

but chances are you want to keep it so that you can use it with sed to get the time, requests currently being processed, and idle workers all on the same line

if you want a csv to put into excel use this almighty "one liner"


cat outputFile | sed -e 's/<dt>Current Time: /\n/g' | sed -e :a -e '$!N;s/\n</ /;ta' -e 'P;D'| sed '/^$/d' | sed -e 's/ EST<\/dt> dt>/,/g' | sed -e 's/ requests currently being processed, /,/g' | sed -e 's/idle workers<\/dt>//g' > outputFile.csv


To Generate a tab seperated file use this:
cut -d ',' -f 2,3,4 aug.csv | sed 's/2007 /2007-/g' | sed 's/,/\t/g' > aug.tab
cut -d ',' -f 2,3,4 sep.csv | sed 's/2007 /2007-/g' | sed 's/,/\t/g' > sep.tab

or do the whole thing
cut -d ',' -f 2,3,4 outputFile.csv | sed 's/2007 /2007-/g' | sed 's/,/\t/g' > out.tab

now grep out the months if you want or deal with the whole file, i.e for September
grep Sep outputFile.csv > sep.csv
or
grep Sep outputFile.tab > out.csv

I'm using gnuplot to generate the plots. Here is the load file you want, in this case for all of August

# Gnuplot script file for plotting data from server_status
# This file is called   aug.plot
unset log    # remove any previous log-scaling
unset key    # remove any previous key
unset label    # remove any previous labels
unset title    # remove any previous title
set xtic auto    # set xtics automatically
set ytic auto    # set ytics automatically
set yrange [0:1028]   # Change the y-axis range
set xtics rotate                        # rotate the x-axis lables so they are verticle
set terminal jpeg size 1280, 780 # make a pretty picture for the output
set output 'aug.jpg'                    # name of the output file 
set xdata time                          # set x axis to be time scale
set timefmt "%d-%b-%Y-%H:%M:%S"         # set the input format of the time
set format x "%d/%m"                    # set the output format of the time label
set grid                                # show a grid in the picture
set title "Number of user connections"  # the title of the output plot
set xlabel "Date/Time"                  # label the x axis
plot "aug.tab" using 1:2                # generate the plot using column 1  for x and 2 for y

if you want to plot a single day you want different x-axis labeling, use:
set format x "%d/%m %H:%M"

from within gnuplot to generate the plot from this plot file enter: load 'aug.plot'

You should now get a pretty picture.

Mine look like this:

For the year since the end of July

For Oct to date

For the 8th of Oct

And the 9th - can you spot the DOS attack ?

And this morning

--
edit
I'll be truly surprised if anybody reads this.

Geek post

This is a geek post so don't bother reading any further if your not a code monkey.

We are currently investigating why some software is running slow and are looking at disk IO as a possible problem. Unix can log all sorts of things but you are lucky if they are human readable. This lill' Perl script will grep through all the the io.out logs from iostat into a csv file that you can open in Excel to graph.
Edit the regular expression to get the time - in out case its stamped Eastern Standard Time - EST
Edit the regular expression to find the device you are interested in - in this case d80

--------------------------------
#!/usr/bin/perl
 
#-----
# Perl code to output io files as csv
# usage: $ perl io2csv.pl io.out.* > d80.csv
#
# This little script will turn unix i.o. logs into a csv script
# from whence you can graph them
#-----

print "r/s,w/s,kr/s,kw/s,wait,actv,wsvc_t,asvc_t,%w,%b,device,dateTime\n";
# go through input one line at a time
while(<>)  
{
   # put input into a variable
   my($line) = $_;

   # remove trailing and leading spaces
   chomp($line);  

   # regexp to get the time
   if ($line =~ /EST/) 
   {

      # get the time part of the date time into a variable (split on white
      # space and get 4th element)
      @dateTime = split; 
      $dateTime = $dateTime[3];
   }

   # look for data lines that are related to the mount 
   if ($line =~ /d80/) 
   {

      # make white space separated data into a comma separated string
      $fields = join(",",split); 

      # write output plus date
      print "$fields,$dateTime\n"; 
   }
}

A self indulgent post

Well, it's been a full on few days. Saturday Alex and I moved about a ton of dirt to rebuild the garden that was destroyed by the plumbing works in our unit complex. Sunday morning I spent doing odd jobs including pulling a substantial number of parts out of the innards of a Scandinavian build washing machine to clean the pump. The instructions were in the same pictograph style as an Ikea brochure, possibly written by Fifteen Bobsworth Longfellow *. Needless to say I swore quite a bit but it eventually Yielded to my threats and I managed to emancipate the loose change that had blocked the pump.
Sunday evening we went to see a movie - Så som i himmelen, about a successful international conductor who suddenly interrupts his career and returns alone to his childhood village in Norrland. I'm a total girl in movies and cried just all the way through. Interestingly it took more money in the cinema I went to see it at than Titanic. Proves that I live amongst a bunch of chardonnay drinking intellectual snobs.
On Monday a colleague and I solved a cracker of an IT problem which directly affected the VC of the uni. I celebrated, even tho' it was a school night by getting plastered with Steve (the visiting physicist) and Row (pregnant uni friend) at the Belgian Beer Cafe. Well she didn't have more than a sip but Steve and I got a little silly. Thanks for your company Steve, It is always good to see you.
Tuesday was testing Mondays work and then in the late afternoon ducking home and bringing Annalise and Adrian into uni for show and tell and then up to the library to show Annalise the Tyrannosaurus bones on display. She said "Daddy, the dinosaur has popped and died and only his bones are left". I guess when you are 2 and a half and a bit a popped balloon is the closest you have to a concept of death.
Today, Wednesday; meetings, meetings, gym, meetings and them off indoor climbing with Sean and Andrew.

Then I think that I am going to have a lie down.

---------------
* Fifteen Bobsworth Longfellow was an Adelaide academic who wrote instructions for kit-set model products, mainly balsa wood aircraft and submarines which ran on baking powder. The manual included here was for the assembling of a twenty-five-foot aircraft carrier marketed by Myer stores between 1954 and 1960.

Take the pieces from the package,
Lay them out as per the graph,
Gathering the bits you'll need,
Removing what your shouldn't have.
With the implement provided
Ease the bearings to the left,
Push the little angled mullion
Up into the socket 'F'.
This will free the moulded bracket
Holding back the nylon strand,
Draw the slippery hoop and coupling
Through the right-hand rubber-band.
Put the topside brown side outside,
Push the inside upside down,
Underneath the left-hand wingnut,
Press the folding backward crown.
Overlapping lifting side-flaps
Lower in to fit the screws,
Pack up tools, retire to distance,
Don protective hat, light fuse.


Apologies to John Clarke

They want me to take some leave

Get this - I've been at work 1 week after being away for 24 weeks and they want me to take some leave, aparently I have accumulated 280hrs

Go figure.

Back at work

I'm back at work after 5 months being a stay at home dad.

Free at last, Free at last free at last

Last day for 22 weeks

Just a tad busy

Did I tell you I was about to take 22 weeks parental leave?

Yeah, I'm going to be a stay at home dad looking after Adrian every day and Annalise two days a week. Alex will be going back to work whilst I take 22 weeks from March 6 through until the beginning of Aug.
I will blog, less, but I may not read you as much as my 15 min morning coffee stop when I read you all could be hectic.

I Baked

Made me some ANZAC biscuits (that's cookies for you thar' yanks) for morning tea.

I'm guessing there are not many recipes that one can get on the website for a national war memorial . They were .... hard

Forms

I'm a manager.
I think that means that I'm not supposed to stuff up the little things - like paper work.
On that front I've never had much luck - or rather never had the perseverance and level of care to get the attention to the fickle detail right. Which led to the following email from the Executive Officer in the section of the uni I work in:

"I've had a look at the leave forms and apart from sighing heavily, the alternative would appear to be banning you from ever filling in another one.
However that would just be encouraging you, so can you let me know when you'll be free and I'll come up and visit you in the hope of getting this sorted out nicely?

Many thanks

Head hung quietly in shame I await form filling 101 reeducation... again.