Spatialite and Me

I had the misfortune to need to connect to a Spatialite enhanced SQLite database, and found myself in some sort of titanic struggle. I first tried using JDBC on OSX and got roundly defeated. For round two I tried with Python on Ubuntu.

For what it's worth (and because this should be documented somewhere, for those who follow) I believe that the following are the steps required to get Spatialite up and running in python on Ubuntu 13.04 (Raring Ringtail) and Ubuntu 12.04. I say 'I believe' because I haven't worked with this in any anger yet :(

Thoughts

If you bother to read through the following then you'll see that my problems were largely caused by the PyPi version of pyspatialite. I think that this software is fundamentally broken in several ways:
it assumes a layout and content of the Spatialite site it references, it does not notify the user clearly when those assumptions are broken, and it caches files and does not remove them if an error was encountered.

The net effect is that every time the Spatialite site is changed (what, no stable url's?) the PyPi download is likely to break and will need repair. And innocent users such as myself will waste a lot of time trying to work out what went wrong.

So if installing on Ubuntu simply use:

sudo apt-get install python-pyspatialite

The downside is that this package is only supported on Quantal and Raring Ringtail.

If you need some sort of automated deployment then clearly you are going to do a lot of work if you want to deploy to anything other than Quantal and Ringtail...

Ubuntu 13.04

I started out on Ubuntu 13.04 and decided that I would try to do some sort of self contained install so that I could make use of virtualenv.

Do not do what I did: 
pip install pyspatialite
For if you do you will be met with:
download_file = pattern.findall(download_page)[0]
IndexError: list index out of range

This is because it would appear that the installer is trying to fetch a file from the Internet: a file that does not exist. From my search on the Internet, this seems to be a regular occurrence with the different versions this software, and may be fixed only to reoccur on the next release :(

If you have the misfortune of making this mistake be aware that the installer has already downloaded files, and it will not try to get replacement copies without manual intervention. More on this in the section on installing on Ubuntu 12.04 below. For I think with a little bit of hacking you could fix this.

Be warned: if you get this error and then attempt to install an older version, you will be met with:
error: conflicting types for ‘spatialite_init’

Do not try to clear ~/.pip/cache – because that's not where the files are being cached :( A pip un-install will not work either, as the software was never installed :(

Having run into the above I simply switched to a new VM, discarding my 'broken' one, and in the new VM I ran:
pip install pyspatialite==2.6.2-spatialite.2.4.0-4

But don't you do this yet: first you need to run:
apt-get install python-dev

Or you will get the following error:
fatal error: Python.h: No such file or directory

Wait: don't run that initial install yet! You also need to run:
apt-get install libgoes-dev

Otherwise you will get:
error: geos_c.h: No such file or directory

And be still that twitching finger: Next you need to run:
apt-get install libproj-dev

Otherwise you will get:
error: proj_api.h: No such file or directory

Then still hold off: for next you need to do:
apt-get install libsqlite3-dev

For if you don't you will be met by a:
fatal error: sqlite3.h: No such file or directory

Then (yes, there's more) you need to
cd /usr/lib
sudo ln -s libgeos-3.3.3.so libgeos.so

For if you don't, you will be met by the somewhat mysterious:
/usr/bin/ld: cannot find -lgeos

This last one is happening because the libgeos-3.3.3 package installed with libgeos-dev forgets to create create this needed symlink (!) It used to, I know, because the Internet tells me so.

Then, finally, don't forget to do a:
apt-get install libspatialite-dev

Otherwise you might be met with a:
fatal error: spatialite.h: No such file or directory

Finally, finally you can run:
pip install pyspatialite==2.6.2-spatialite.2.4.0-4

Don't forget to use sudo if not running as root in all of the above…

Having got to this stage, you can then open up a python shell and try to connect to the database. I say 'try' because you are not yet done!

You need to:
gedit /usr/local/lib/python2.7/dis-packages/pyspatialite/dbapi2.py
And change line 50 to be:
version_info = tuple([int(x) if is instance(x, (int, long)) else x for x in version.split(".")])

This is because the version_info contains strings as well as integers, and the original, as written, thus dies at this point:
version_info = tuple([int(x) for x in version.split(".")])

This last step fills me with dread, as who knows how this string is actually used? How might this step bite later on? However, after all of the above, I could now open the two Sqlite databases I have with Spatialite data in python and execute queries against them.

As you can gather, I made each of the mistakes above, and had to resolve where I had gone wrong: a time consuming task. And one that has had me muttering dark words and wondering nasty thoughts about Spatialite.

The alternate process is to simply accept machine wide installation and to do the following:


sudo apt-get install python-pyspatialite

Which gave me my python bindings on more up to date versions of the software without any of the pain I went through doing the pip install.

However, having done all of this I discovered that I actually needed to run Spatialite version 3 on Ubuntu 12.04. So, getting the whole shebang set up on 12.04 became my next task.

Ubuntu 12.04

On 12.04, having been trained by my previous experiences, I simply ran:
sudo apt-get install python-dev
sudo apt-get install libgeos-dev
sudo apt-get install libproj-dev

Then I downloaded pyspatialite 3.0.1 from:
https://pypi.python.org/pypi/pyspatialite/3.0.1
and unpacked it into a directory off of my home directory.

Then I moved to this directory and ran
sudo python setup install
Which crashed and burned with the infamous error message:
download_file = pattern.findall(download_page)[0]
IndexError: list index out of range

I went to the line causing the error (line 97)
and replaced the "pattern.findall(download_page)[0]" with "libspatialite-amalgamation-3.0.1.zip" (the version of the file I wanted to install) I also edited the url in the following line to read: "http://www.gaia-gis.it/gaia-sins/libspatialite-sources/", because as it stood it was not a correct url.
I reran
sudo python setup install
and got the dreaded
error: conflicting types for ‘spatialite_init’
I simply deleted the amalgamation subdirectory into which the installer seems to download stuff into and reran:
sudo python setup install

And this time, it all seemed to work. Sweetness and light, and the trilling of angels ensued. More importantly, I was able to open and access the databases I wanted to work with.

It really shouldn't be this hard, should it? Also knowing all of this, I suspect I could get the pip install that I tried on 13.04 working. But life is too short.

An Introduction to OpenStack

Earlier this month I gave a talk to the December meeting of the LUV, introducing the members of the group to OpenStack.

Unaccustomed as I am to public speaking, I wrote the whole lot out before I prepared the slides.

Which turned out to be a good thing, as we couldn't get the overhead projector to show the slides...

Luckily I had prepared some hand outs, and was able to wing the talk based on the handouts. This is the copy of the talk I prepared before creating the slides. An Introduction to OpenStack

I thoroughly enjoyed both giving the talk and answering the subsequent questions. LUV seems to be a group populated by people both smart and friendly!

Shot!

I thought I'd try my hand at a short story...

I'll never forget my 21^st birthday. A day that was set to be most ordinary turned into the most memorable, and puzzling day of my life so far. For it was the day that I was shot. Let me tell you the story.

I'd been away from home for a few years, doing my national service. Unexpectedly the army sent me home on leave the day before my 21^st birthday.

I thought I should celebrate, but when I phoned around none of my old school friends were in town. So I had a quite celebratory breakfast with my parents, and that was my birthday.

It had been a tough tour, I was probably depressed, so 8:30 that evening I decided that I would simply go to bed.

But I couldn't sleep. I tossed and turned. Sleep just didn't fall. After what seemed like an eternity I reached up, grabbed the dangling Victorian light cord, and switched on the bedroom light. The clock on the wall read 9:30. “Ah, I might as well get dressed and go to the pub” I thought. “I'll find someone to talk to.”

I found the pub unexpectedly crammed, full of people, strangers, shoulder to shoulder, beer to beer.

“What's going on?” I shouted at the closest person to me. “Why are you all here?”

“Conference. University.” he shouted back at me “Wonderful town!”

I waved my thanks at him and elbowed my way through that laughing happy crowd to the bar. The lone barman was struggling with the crowd, racing from customer to pump at the far end of the bar.

The clock behind the bar read 10:20. There'd be plenty of time for a few beers. As I waited for the barman to get to my end of the bar, I pulled a cigarette and out and lit it.

I was tapped on the shoulder: the woman next to me at the bar, blond and vibrant, asked “Can I have a light?”

“Sure” I replied, and held my lighter up to her proffered cigarette. She put her hand on my shoulder to steady herself as she lit her cigarette. My 21^st was suddenly looking up!

“Are you part of the Conference?” I asked.

“No, my husband and I moved here a few months ago” she shouted back.

At that moment there was a disturbance in the crowd to my left.

“You!” a man popped through the crowd, shouting, angry, pointing at me: “You! You're the person who's sleeping with my wife!”

“No!” I shouted back, quizzically. “You're wrong!”

“I'm going to kill you!” As he made the threat, unbelievably, surreally, he pulled a gun out of his jacket and prodded my chest with it.

I felt the woman's hands digging into my shoulder.

“No!” I shouted, now desperate. “You've got this all wrong. It's not what it seems!”

BANG! The bastard had pulled the trigger.

My first reaction was anger. A deep anger at what the stupidity of the whole thing.

But quickly the anger was washed away by a wave of pain. Pain at a scale that I have never felt before or since. It was as though someone had plunged their hand into my chest and pulled a large chunk of it out. Which, in a way, I guess they just had.

And with that wave came powerlessness. I was no longer in control. I was falling, not just physically to the floor, but far worse, the world was falling away from me. Reality was closing down, the light and screams at the end of a tunnel that I was being pulled back into.

The physical world around was gone long before my body hit the floor. All that I had was pain. Excruciating pain. And then that faded away, leaving me as nothing more than a speck of consciousness in a vast empty void.

I tried rationalising.

I'd been shot. I must have died. There was life after death! I was initially triumphant. There is life after death.

But no relative came to take me elsewhere. Just the unrelenting nothingness, that went on and on and on.

I suddenly knew that this must be hell. An eternity of nothingness. I ranted and raged. I didn't deserve this! But no one can hear you scream in a vacuum. Not even yourself.

I calmed down. I tried remembering scenes from my life, remembering people, songs, places. Anything that could occupy me.

Then the insanity hit. If there was nothing but my consciousness, then I could create things. Like hands. I don't know why, but I fixated on hands. I worked really hard on feeling what it would be like to have hands – and I had hands!

I put them together, feeling, treasuring, the sensation of them touching. They hungrily explored each other. They had arms attached. And the arms connected to a body. My body!

But there was a sheet over my body? Why?

I rationalised again. I must be in a hospital!

I lifted the sheet felt underneath, where the wound would be. There was no wound!

But I was blind?

I opened my eyes, and in that infinite darkness, touched my eyeballs. They were there!

I rationalised, yet again. When I fell, I must have hit my head and suffered brain damage. Now, many months later, I had come to, my wounds healed. But my ability to see had been destroyed by the brain damage!

“Nurse” I shouted, “Nurse!” But no one came.

It must be night. It must be night, and the staff were sleeping. In a hospital, there must be an alarm bell, to call the staff.

I felt above my head. And there is was, hanging from a cord!

I pressed it. Light flooded the room. It was my bedroom. The clock on the wall read 10:30. 10 minutes ago I had been in a pub, lighting a cigarette.

You know, there are people who say that if you dream of dying, then you die. I have to disagree with them. For if they are correct then this world we stand in is one I constructed in my death.

Regardless, I have been shot. And yet am here to share the tale.

The mice are driving the experiment

My next speech on. For this one I thought I'd try to give the audience a window into the wild and wacky world of software development, as I see it. And the view isn't that good...

A twig. It all started with a twig. Someone, a long time ago, prodded something, and noticed that it moved. Ever since then we have been poking away at things, and developing increasingly complex machines based on the results of our prodding. That jumbo jet that flies above can be seen as the apex of this twig technology.^[1]

But our world is changing.

Through the wonder of integrated circuits we are now building machines out of the dancing of electrons. There are no moving parts in these machines. We are entering a new age – the silicon age.

Software development stands at the very forefront of this new age.

But if we truly look at the world of software development I fear that it shows we are not doing very well in this new post twig era. There are many issues, but I believe that there are two primary issues causing most of the problems.

But to explain, first you have to understand software.

Software can be seen as machines of the mind. Machines that have been created from words. Small sets of words written down to form tiny languages. With those small languages we instruct the electrons, building at first simple machines. As these tiny languages have such limited vocabularies, we communicate the ideas behind our machines of the mind with others through abstractions and metaphors.

Then we iterate, adding new words to our existing languages, developing new languages, wrapping existing machines to extend their capabilities, building new machines, and so on. We build on our past successes, learning from our mistakes... Of course, with the new languages and new machines come new abstractions and metaphors.

And so the circle of software development goes.

The speed at which we software developers extend and change our machines, languages, abstractions and metaphors mean that half of what a software developer knows today will be obsolete in three years time. This then, is the first issue: the need for continuous learning.^[2]

Very few software companies have learning at their heart. So much so that Starbucks will spend more on training a barrista than the typical software company will spend on training a developer. ^[3] Software companies thus expect their employees to keep abreast of the rapidly changing world through after hours study, or they rely on new hires to bring in the new knowledge they require.

This makes the industry ageist.

Think about it. The more time you spend working on a current language or platform, the less time you have to study and learn the ones that will replace what you currently know. Especially if you have a family.

So the industry is also set against people with normal family lives.

Which makes the industry inherently sexist.

Any women who take time out to have children will find that the longer they leave their return to the industry the more out of touch they will be – and thus less likely to find work.

Google is one of the few companies that I know of that have developed a strategy to help their employees learn: they allow developers 20% of their paid time to work on things that aren't in their job descriptions. To try new things out, to experiment, to learn.^[4]

When I tried to discuss Google's approach with the head of software development at a local company he was outraged – exploding in white hot anger at the thought that his developers would spend one day a week not “working”.

The next issue is subtly related.

People have ideas, and those ideas are turned into theories. But in software, those theories are mostly never tested. If accepted, they simply become dogma, a truth that all are expected to follow, unquestioningly. Those who don't follow are ostracized and ridiculed.

As an example there is a deep seated belief that there is at least a 10 times difference between the productivity of 'good' and 'bad' developers. But the truth is that there are no really valid metrics to measure programmer productivity. People can't even agree on what makes a good or a bad developer. So if developer productivity can't be measured, how can the belief have any basis?

We need look no further than the job adverts for software developers to see these two problems exposed: Employers choose to drill down on technology skills, riddling job adverts with acronyms, seeking years of experience in specific languages, platforms and tools^[5] . The industry is looking for people it doesn't have to train - and applicants have to prove that they don't need the training. Nowadays even people with PhD's can expect to be asked to write a test to prove their competence when they apply for a post.

But here in Australia, for every hundred resume's received by a software company, only one person will be hired. In the United States that figure was two in a hundred.

This is in an industry that claims that that there is a shortage of developers.

Only one in a hundred applicants are accepted.

Isn't this really symptomatic of an industry that is picky beyond belief, doing its level best to only hire the “good” developers that require no further training?

I believe that these two issues expose us as the illogical beings that we really are. Struggling with unfounded beliefs and not able to deal with an the need for continuous learning in our workplaces.

Software development is the experimental frontier of the post twig era. In that experimental frontier I believe that that the mice are driving the experiment.

Footnotes

[1] Douglas Adams was the person who I first heard make this observation. Read the book: it is recommended!

[2] Can anyone say "Innovator's Dilemma"? Perhaps one of the reasons companies claim to look for people who go to user groups and write open source software is that subconsciously we recognize that passion does trump rational decision making?

[3] I believe that McDonald's might also spend more on training their burger flippers than a typical software company will spend on training its developers.

[4] Atlassian also do this: I think that their business focused way of directing this might be a a better way than Google's.

[5] Yes: the more acronyms, years and jargon I see in your job advert, the less I expect that you spend on training your developers. The best job advert for developers I have seen simply read "Wanted: Scala developer". I applied, but didn't make the cut as I was deemed "not to be a good fit for the company culture". I understood that to mean that I would be the only developer on the team with a wife and kids - I was too old :-(

Beware what lies in the sand beneath

Another month, another speech. This one was a fun one to give, as most people in the audience really hadn't heard of toxoplasmosis.

Beware what lies in the sand beneath

There is a parasite that lives amongst us.

A parasite so small that when eaten by you, it will make its way through your gut wall, and then, using your bloodstream as a vehicle it moves to the sites in your body which it will call home.

These sites being your muscles and your brain.

Once the parasite has arrived at its future home it forms cysts inside your cells, hiding in the cysts, safe from your immune system.

It is a parasite that most of us aren't aware of.

Which is odd – because at least one in five of us are infected.

That's just the humans – a whole range of other animals are also subject to attack by this parasite. From mice and rats through to humans.

Not only is it a common parasite – but once infected it is very difficult to clear the infection.

The name of the parasite I am talking about tonight is Toxoplasma Gondii

Until recently it wasn't considered a bad infection to have.

Then it was noticed that toxoplasmosis alters the behaviour of infected rats.

Infected rats started to behave in very un-rat like ways.

They begin to hang out in areas where there are cats – even at times trying to go one on one against the cats, trying to show those cats exactly who is the boss.

Which is not a very bright thing to do, because, well, cats eat rats.

So infected rats, despite their bravery, generally don't have happy endings.

But here's the thought provoking thing: Toxoplasmosis can only breed in the small intestine of a cat.

So it needs to be ingested by a cat to breed. It is critical for the survival of the parasite that it find its way to a cat's gut.

How convenient then, from the parasites point of view, that a rat infected with Toxoplasmosis would change it's behaviour in such a way as to make it more prone to be eaten by cats.

Scientists believe that convenience plays no role in this. Rather, they believe that Toxoplasmosis cruelly and deliberately plays with the brain of its rat hosts to make them go out and get eaten by cats.

Singapore zoo has a its tigers separated from us human viewers by large plate glass windows. From some angles it looks as though nothing separates you from the tigers.

I don't know about you, but in those moments, when it just me and the tiger, my stomach always instinctively tightens in an apprehensive knot.

It is almost as though my subconscious recognises that a few thousand years back, when men were far more puny, and tigers, lion, and other large cats were far more plentiful, that, well, we were the prey.

Let us all do some basic scientific research in the form of a little thought experiment.

Could it be possible that us humans are also susceptible to behavioural changes driven by something so much smaller than one of our cells?

Also, if there are behavioural changes, just what might they be?

Scientists were also intrigued by this line of thought.

Having a fairly large infected population to sample, they found it fairly easy to design experiments and tests that would reveal changes.

And they found them!

Interestingly they found that the changes were gender related

Infected men are far more likely to disregard rules and are more expedient, suspicious, jealous, and dogmatic.

Infected women are more beautiful, warm hearted, outgoing and persistent.

Common to both are impaired motor skills, and slower reactions

So much so that people with toxoplasmosis are six times more likely to have traffic accidents than people without.

Thought provoking. Are these behaviours that increase your chances of hanging out with the cats?

Regardless, the killing tigers have now been replaced by the killing cars.

More disturbingly, there is emerging evidence that some of us might become schizophrenic or neurotic when infected. And now forms of brain cancer are also being linked to infections

How do you get toxoplasmosis?

Easy!

Touching or coming into contact with sand containing infected cat poo.

Eating uncooked, unwashed fruits or vegetables that have been contaminated by manure with infected cat poo.

Eating raw or undercooked meat that's contaminated.

And sometimes, very rarely, by being born with it (a woman who gets a toxoplasmosis infection while pregnant can pass the parasite on to her unborn child).

There is a belief by some that we humans are where we are today because of schizophrenia.

And because of dominant rule breaking people who have pushed the boundaries.

It is a sobering thought that our civilisation might be what it is because of a tiny parasite.

But you know what: looking at the modern world I think that perhaps we can live without this parasite

That the time has come for us to make our own way, brains unaltered, into the future.

So make sure that your children's sand pits are covered when they aren't playing in them.

Keep your cat away from the wildlife.

Do a good job of washing and preparing your food.

Wear gardening gloves.

Such simple steps to stay safe.

And remember, if you get hot under the collar while driving, and want to gun your car from one stop to the next, and those idiot cyclists who just slow you down and you get so tense at the stupidity of others...

Just remember it could be your inner parasite trying to introduce you to your modern tiger.

So take a deep breath, calm down.

Beware what lies in the sand beneath.

I don't understand the Australian IT business

Here in Australia employers are screaming out that they can't get skilled IT employees.

The situation is so bad, apparently, that we need the visa rules changed to allow IT specialists into the country (this link is one example of the articles that flood our papers).

But at the same time, IT companies in Australia will hire only 1 in a 100 applicants!

If you have a shortage of skilled people, then you don't turn your nose up at applicants at this rate. Sorry. You are obviously spoiled for choice and being super, super selective in who you hire. So much so that you are wasting more time in filtering and selecting people than you should be.

This, by the way, is in a country that has as almost standard, a probation period. If the employer doesn't like you in the first few months of your employ then you are toast. No recriminations, no blame. It would be surely be more cost effective to hire people and then shed them if they don't work out?

Do employers really want to open the gates to allow in people who they then aren't going to hire? It makes no sense to me.

I'm guessing that the real reason that we want the visa restrictions relaxed is to drop salaries, in some warped supply/demand way... If true, what does that tell us?

Footnote, 29^th August: Of the 1 in a 100 applicants hired, almost 50% aren't up to the job...

Another month, another speech..

Be Prepared

It has been a long day. The seats on the train are all taken. So you stand, hanging on to a strap as the train hustles you home.

A girl near the end of the carriage unexpectedly jumps up, sobbing, and steps through the inter-leading door between the carriages.

You see her framed by the door window, standing on the exposed, shaking, platform that bridges the two carriages.

Suddenly she picks her leg up, swings herself up and over the guard rails, and falls from sight.

You, and the the people in the carriage that saw this are shocked. What has just happened? Did she really just jump – why was there no sound – are trains really that brutally massive?

All these thoughts race through your mind. But before you can work out what to do you see her head bobbing at the bottom of one of the inter-leading carriage windows.

Relief fills you, and everyone else who saw her vanish. She's alright: she's clinging to the carriage! But she is in a very dangerous spot.

You realise to your horror that all the other people on your carriage who saw this drama play out are now starting to look at their newspapers, to look out the side windows – doing anything they can to exit the stage on which they unintentionally find themselves.

What would you do if that were you in the carriage?

I believe that we must consciously choose to act in the day to day drama's that intersect our lives.

I also know that we choose how we are to act in the drama. Our chosen role can range from deliberate inaction to full participation.

On that train rattling through the night, with the girl clinging to its side, I felt powerless.

She was behaving irrationally: if I went near her what might the result might be? Would she jump – or would she pull me out into the void to be with her?

I couldn't leave her, and I feared getting too close to her? What could I do?

There was another path I could follow. I could contact the driver. Every door on the train has an emergency intercom that allows you to talk directly to the driver.

But the more people there are around an unfolding drama, the harder it is for us to take an action that is different from theirs.

We look to the others around us for leadership – and when none comes we use their lack of action to abrogate our responsibilities.

I ran from door to door on that carriage, desperately looking for an intercom. And found none!

The people on the train were now starting to look at me – I was becoming the focus of their attention – not the girl desperately clinging to the side of the train.

It was just surreally wrong - if that had been your daughter, wife, or best friend of clinging to the side of the train, would you have ignored her to instead watch the strange man who was now running from door to door? I think not.

Yet on that carriage everyone did nothing, complicit in a unison of inaction. And now by focusing on me, made me feel as though I was the one who was strange and different – and that I was in the wrong.

Just then the train slowed and stopped at a station. I escaped their attention by leaping out the train.

I ran up the platform, to the drivers window, knocked on it, and told him of the girl.

Baden Powell, founder of the scout movement, said: “Be prepared”.

By that he meant that you must prepare yourself "by previously thinking out and practicing how to act on any accident or emergency so that you are never taken by surprise".

Later, when I got back on my carriage, after the driver had navigated the girl to the safety of the station platform, I saw that there were indeed intercoms on the carriage: others, standing, had simply been leaning against them, hiding them with their bodies.

Now when I get onto a train I check for the intercoms, making sure I know exactly where to go if something happens.

That preparation has already paid off: just last week another girl collapsed on the train. Whilst an elderly women aided her, I, nervously, because no one else was doing anything, called the driver.

No less than 8 officials met us at Flinders Station and and tended to the girl.

That brave elderly women and I were not punished for stalling that train in rush hour – none of the officials even bothered with the two of us once they were attending to the girl.

We just slipped away with the crowd when we saw the girl responding to their ministrations.

Ratko Mladic has just been extradited to the Hague, to stand trial for the Srebrenica massacre.

A massacre in which over 8000 men died. It is the largest mass murder in Europe since the end of the world war two.

What could tie the terrible deeds of Ratko Mladic to us, who were in that carriage with a girl clinging to its side?

400 heavily armed and highly trained Dutch soldiers were stationed in Srebrenica when the massacre took place. Unbelievably, they turned their backs.

The two events are related by the people who stood by and did nothing.

Evil blossoms when watered by the apathy of those around it.

If we can learn act in the day to day drama's that intersect our lives, we will be more able to act when evil starts to take root.

So be prepared – roll play scenarios in your head, learn from how others have behaved in emergencies.

And remember to choose your role – and act, when you find yourself inadvertently thrust into a drama.

JavaScript, Functional Programming & UML?

I come from a Java background. As I have been learning JavaScript through my naked server experiment I find myself wondering more and more if "traditional" Object Orientation is the right approach to JavaScript. I now agree: JavaScript is the most widely deployed functional language.

When doing Java development we often share our thoughts via simple UML diagrams. They offer a nice visual way to share concepts and discuss ideas. They allow us to abstract the underlying complexity of implementation away. In short, to a Java developer, UML can be jolly useful.

But now I find myself wondering how I can use UML to describe any functional JavaScript that I might write. As a set of utility classes, perhaps? Regardless, it would seem that I am not alone.

In fact the kingdom of nouns extends far further than the shores of Java. It would seem that at some stage the software industry did truly believe the religious  decree that procedural programming was bad...

Mac sales: my curious lack of surprise

I've just seen that at this moment in time Macs are doing rather well: roaring ahead whilst PC sales are declining.

I normally build my own PC's, but recently decided that I want to change this pattern and actually buy one off the shelf.

My principle stumbling block is that I don't see why PC's have to come in big boxes: the last one I built runs dual monitors and fits in a shoe box.

However trying to buy a powerful compact PC seems to be inordinately difficult: the best that I have been able to come up with is either another Shuttle, pre-assembled, or the NRG Frost. Which now sadly seems to have been discontinued. Not helping in my quest is that I want to run some variant of Unix/Linux (Ubuntu preferably) on it, not Windows.

In short, here in Melbourne, Australia, I haven't found anything. Other than perhaps a new iMac. It's compact (beautiful), runs a Unix flavor, and comes with a "free" monitor: all for $1949, about the same as I was quoted for a headless NRG Frost.

The PC world seems to think that bigger = better, and that we all want to run Windows. With this broad stereotyping in play my next PC is most likely to be a Mac. So reports of Apples success don't surprise me in the slightest.

JavaScript notes: AJAX and redirects

More fun and games in my ongoing exploration of the world of JavaScript.

At the moment I am playing with AJAX, and found myself wondering how one would deal with calls from clients whose users were not logged in.

After a bit of experimenting I have found that if you do a plain redirect from the server, the browser happily follows it, but stuffs the result into the response passed to your beautiful AJAX code. So the user is left on the page that ran your AJAX, blissfully unaware of what is going on under the hood. This seems fair: the browser is leaving the decision making up to your AJAX code.

The problem being that if you write a nice filter to redirect all attempts to access protected content to a log in page, and your AJAX code is expecting a nice piece of JSON, there will be tears.

For my naked servlet experiment I have been forced to a slightly hacky solution: I have a created a servlet filter for JSON requests. If it finds that you are not logged in then it wraps the URL of the log in page in a piece of JSON and returns that. If the AJAX handler in the browser finds the redirectTo key in the returned object it promptly redirects the browser to the associated URL and does nothing more.

It would have been nice to handle the whole thing by popping up a log in dialogue (via an iFrame?) and not redirect the user to another page: but I am using Google App Engine as my servlet provider, and it would seem that the terms of service require me not to do this.

This doesn't seem very elegant. I am scratching my head to see if I can come up with a better way, but for the life of me, I can't see it. All hints and tips gratefully accepted!

How to learn a speech?

As a new Toastmaster member I have been troubled by the question of how to learn a speech. All of the other members I have seen talk seem blessed with wonderful memories, able to deliver impressive speeches flawlessly, without recourse to notes, powerpoint bullets, or any other apparent memory aid. 

I last gave memorised speeches at school – and I don't remember being particularly good at them, so I was worried about how I would manage.

But then a slow logical optimism through induction bit. The key word in that last paragraph was “all”. If every speaker was able to memorise their speech so effectively, then surely I should be able to as well? All that I needed to know was their techniques. Simple!

So I asked several of the speakers how they managed to learn their speeches. Dishearteningly, each person I asked gave a slightly different reply. “Repetition”, though, seemed to feature in every reply. They had practised those speeches a lot before actually giving them.

Seeking greater enlightenment, I turned to the Internet. It would seem that people deploy a wide number of techniques to learn speeches. There do seem to be some tried and true methods that shine through, though.

Joshua Foer, in his book, Moonwalking with Einstein, The Art and Science of Remembering Everything, describes one process, called “elaborative encoding”. Using this technique you would convert the paragraphs of your speech into a set of bizarre and engrossing visual images. Then you would order the images in a route through a familiar place, such as your house.

The images and the journey are far easier for you to remember. To you, your speech has just become a journey through Lewis Carroll's looking glass, rather than a pile of hard to remember words. This technique has been tried and tested through time. The Romans knew it as “The Method of Loci”.

Researchers have known since the late 1800's that spaced repetition leads to memorisation. They have shown that when we first learn something it is placed in the brains short term memory store. With time it either forgotten or moved to the brains long term memory store. The key to moving the material to the long term memory store, and to retrieving it from the long term memory store is repetition. Exactly what shone through from my earlier queries!

Distressingly, the timing of the repetition is important. For best effect you have to repeat the learning just as you about to forget it! The time between each “about to forget” moment grows with repetition. This is termed the “spacing effect”. What makes this distressing is that we each appear to have our own optimal intervals of learning. So we have to somehow work out what our own optimal intervals of learning are!

Research also shows that cramming does appear to increase our understanding of a topic – but it really doesn't help us remember what we understood.

All this repetition seems like hard work. But then, know that the harder you work to remember something, the more it will be sealed in your memory. Also note that you can't take current performance as an indicator of your future performance – you might have left that future repetition too late!

One suggestion for easing the hard work is to record back to back copies of your speech. Then simply play them as you fall asleep. On awaking replay the recordings of the speech. And there, voilà! The speech, she is memorised.

For my icebreaker speech I used simple repetition to learn the speech. It worked well for me. In hindsight I need to work on determining my optimal spacing. For my next speech I am going start with the method of loci, and then switch to hard repetition. I might even try using headphones!

My unexpected takeaway from researching how to memorise a speech is that if we manage to become an expert at something, it's not because we remember our lessons or readings on the subject: it's because intense practice in that something keeps our memories fresh.

My sunny optimism...

A speech I recently gave:

I can do anything.

I honestly believe that with a little education, a little time and quite some practice, I can do just about anything a human is capable of.

I am no super man. This is simply an attitude. An attitude that I have eventually realised to be both a blessing and a curse.

A blessing because I have never had to rely on anyone one else when I need to get something done. I research, learn and just do it.

A curse, because I have never had to rely on anyone else when I need to get things done. So I have never had to learn the skill of motivating others – of getting them to help me achieve my goals.

I also have a deep desire to question and understand anything I use and play with.

For example, I can't just get into a car and drive it. I have to know that the wheels are connected to a drive shaft, that that's connected to a gear box, connected to a crank, moved by pistons. Powered by small explosions, created by a spark applied to a petrol and air mix, adjusted by pressure applied to the accelerator.

This need to question and understand is also both a blessing and a curse.

A blessing because if anything goes wrong with the things I use and play with I can fix them.

A curse, because I find it hard to let others fix my broken toys.
“You want to charge how much to repair this? Never mind, I'll do it myself!”

I also spend an inordinate amount of time trying to understand how things work.

These are my dominant characteristics. Oh – I also talk to myself more than I should.

How did I grow to be this person?

I believe that, like an insect trapped in slowly hardening resin, we are largely defined by the family and culture that we grew up in.

Let me describe my formative years and you can get a better understanding of the forces that have shaped me.

I was born in colonial Africa. My father, a structural engineer, was building bridges for the empire!

When I was very young, worried about the changing world around him, my father moved the family to the apparent security of South Africa. In hindsight, I wish he had just kept right on moving!

As I grew my father used me as his cheap labour. Under his direction, I restored cupboards, built sheds. I mixed cement, I laid bricks. I painted, I cleared drains. When I was a teenager I helped to renovate the family home, from the bottom of the foundations to the top of the roof.

My father was a practising Hindu, my mother, a Scottish Presbyterian. They elected to bring me up as a Roman Catholic in a country ruled by Dutch Calvinists.

Once week my father would gather the family in the lounge, and we would discuss religion and philosophy. From this religious cacophony I learned that there was no one truth. That we should question everything. That nothing is as it might seem.

My mother made sure that I had a constant supply of wonderful magazines. "Eagle" - who could forget Dan Dare? “The World of Wonder” “Tell Me Why”, “Look and Learn”, “Speed and Power” - They served as an antidote to the philosophy, bringing certainty to my world, explaining, sometimes in great detail, how things were made, and how they worked. I read my way through all of them, greedily.

It was only natural that at University I received a degree in Physics and Computer Science. In physics we continually strive to understand by building models of the universe, and then try to destroy the models, to prove them wrong. I loved Physics, but wasn't very good at it, so I became a software developer on graduating.

The apartheid regime that I grew up under forced compulsory national service on all young “white” men and so I served two years in the army.
When I idly mentioned this to someone in New Zealand they looked at me, gasped and backed away with "You're a trained killing machine!”

“NO!”I started to tell him. “ That's not me...” when I realised that I may just have missed the point of national service.

During that national service I spent a year patrolling the Namibian bush, keeping it safe. During the day I walked. At night I slept under the stars. With no mattress. No laundry. Carrying all I needed on my back. Occasionally meeting at pre-arraigned points to pick up food and ammunition.

That's were I first started to talk to myself. To give myself company out in that vast, lonely, veldt.

So like this insect trapped, my formative resin has now turned to amber.

I have to question and understand the tools and toys that I use.

With that understanding, given enough time and practice, I believe that I can do almost anything a human is capable of.

And I know that this, my sunny optimism, is both a blessing and a curse.

Scenario Planning and me. A short personal history.

I have been fascinated by scenario planning for quite a few years – actually for most of my adult life. This is because I grew up in apartheid South Africa: and so was briefly exposed to the power of scenario planning. As were most South Africans of my age.

A bit of history might be in order: trying to peer into the future has always been a human fascination. Way back when we used extispicy. We also tried using the position of the stars at birth, the length of creases in a hand, and more recently, how tea leaves fall in the bottom of a tea cup. The list of things humans have done to divine the future is very, very, long.

In the 50's various people tried to put the art of forecasting the future onto a more scientific footing. In the 70's Shell managed to profit handsomely in the oil crunch – and ascribed their success to their scenario planning, built on these initial attempts to forecast the future.

Other companies were dazzled by this, and soon a lot of them were running their own scenario planning groups. Anglo American Corporation, a huge South African mining conglomerate, bought into the whole process in a major way.

Amongst the many scenario's they produced, was a set that predicted two possible futures for South Africa: a “high road” resulting in peace, prosperity and stability, or a “low road”, where the country descended slowly into a spiral of ever increasing conflict and brutality.

So alarmed were they by the prospect of the “low road” that they created a travelling road show, that went from town to town, fronted by the head of their scenario planning division, a Mr. Clem Sunter. Those who attended were given an overview of scenario planning, it's many successes till that point in time, and then walked through each of the two possible futures for South Africa. In excruciating detail.

The torch of newsprint was shone on the road show. It was newsworthy, after all. Editors editorialised, columnists columnised, and opinion pieces were opined. Letters were written, politicians gave speeches. For quite a few months fierce debate raged.

I lived in a small town, and so by the time the road show hit our city the controversy was at its height. The town hall was packed. Mr Sunter didn't disappoint: the show was put together very well, and by the time he was finished I think that most of of the audience felt that the “low road” was a very bad future to look forward to.

As did the apartheid regime – they asked for a private viewing and asked many questions. A few months later they started to unwind the whole apartheid edifice.

Where the two related? I can't say. But from my point of view it would seem that scenario planning had certainly played a key role in shaping the future of South Africa.

So it came as quite some surprise to me when I left South Africa that no one else in the world seemed to know very much about the power of scenario planning. Actually, most people knew nothing of scenario planning!

Every now and then I would turn to the web to see if I could find out more. But what I got was a confusing mass of information that obscured rather than revealed. Being a very busy software developer with a family I never had the time to pull at the end of the information and to find out any more about scenario planning, fascinated though I was.

Till now. I have got my hands on a copy of “Games Foxes Play: Planning for Extraordinary Times” by Chantell Ilbury and one Clem Sunter. A book that claims to transform scenario planning from “an esoteric discipline” to a “practical model” that allows “intense strategic conversation”.

It's been a fun read. I think I'd like to try to work through a very high level scenario planning exercise to get a feel for it. Now I just have to find my conversational partners...

CSS heuristics

As a developer I believe that styling web sites should be done by practiced design guru's. But that us developers should have at the very least a basic understanding of what those design guru's are up to.

Thus I have spent this last week wading through (and playing with) simple CSS experiments. It seems to my uninitiated eyes that the world of CSS is one of craftsmanship - knowledge gathered by bitter experience when bumping into the foibles of the various browsers.

So at the end of the week I have put what I have learned together and come up with the following list of heuristics for me to reach for and to try and follow whenever I have to touch CSS in future. Of course the trick is to keep this initial list up to date as I learn more...

CSS

• Use external style sheets
• Use a reset style sheet
• Combine elements (don't repeat yourself, if at all possible)
• Group your rules by location, ordering your style sheet from generic to specific. E.g.: Put generic classes first, then header, then navigation, etc...
• Try to use composite (shorthand) properties and values where possible.
• Alphabetise properties for easier reading.
• Consider putting the colour scheme used, a version number and your name in a comment at the top of the style sheet.
• Avoid browser specific hacks if at all possible. And if you have them in your CSS files, comment them so people understand why they are there!
• Avoid unneeded selectors – but do consider sand boxing the affect of a selector by making sure it is bounded (i.e.: make sure that the application of a selector is restricted to only a specific area)
• Specify units for non-zero values – don't specify units for zero values
• Minimise the use of HTML element id's
• Refactor often, removing unused rules and properties, and fusing rules where possible
• Try to follow object oriented CSS principles (separate structure and skin, separate container and content).
• For complex CSS consider using XCSS or Sass
• Have (and follow) a naming and coding convention which avoids underscores (for reasons of browser incompatibility). For an example of one see: http://na.isobar.com/standards/#_css
• Keep CSS files under version control (a no brainer for a software developer)
• Consider sizing your text in ems and using Javascript to find the correct starting pixel size. Then your base text should be 1.0 em ( * { font-size: 1.0em })
• Validate your and prettify your CSS.
• Consider using CSS sprites
• Code link pseudo-classes in this order: Link, Visited, Focus, Hover, Active (Leave Visitors Focused on Hoovering Activities)
• During development use separate style sheets with lots of comments. For production, compress, minify and consolidate into one file!
• Avoid using important! (treat its use as in indicator of something being wrong in your styling)
• If designing for both mobile and large screen, consider starting with the mobile design. That way you will be aware of the assets the mobile shouldn't have to load.
• Know that CSS3 "pays off when it comes to production, maintenance and load times"

HTML

• If possible, give every page's body a unique descriptive class: this makes it easy to add page specific styling
• Try not to make class names represent design i.e.: prefer “comment” over “right” - otherwise you are baking design decisions into the HTML
• Try to avoid div and span soup in favour of meaningful HTML elements
• Avoid style attributes in HTML tags
• Use HTML 5's doctype: as it switches all modern browsers into standards mode – even if you aren't using HTML 5
• Validate your HTML!

The naked server

Twitter have abandoned Rails for some parts of their application. I didn't find their description of the move to be fully informative.  E.g.: just what do they mean when they say "we launched a replacement for our Ruby-on-Rails front-end"? Predictably the move got a few comments on the Ruby/Rails newsgroup that I subscribe to. The general consensus seemed to be "so what"? I think we might all be missing the bigger picture. Perhaps this is just a logical step in the progression of web applications? If so, what does it mean for their future? To explain my thoughts, some history:

• Way back when, pages viewed in a browser were fairly simple, being mostly static files that were dished up from directories by a dedicated server application.
• Then we needed to make those static pages more dynamic, and to reflect knowledge related to the identity of the person who was viewing those pages. However, browsers were geared to rendering static pages, so we controlled what we could and put the smarts on the server, building frameworks that allowed us to customize each individual page for the viewer. Our first efforts at these frameworks were quite awful, IMHO, but as time went by we got better at it.
• Now web browsers have an common embedded scripting language (ECMAScript): and can create and change whole pages, dynamically, on the fly.

See the flow here? The complexity started on the server, and then tunneled down those tubes to the browser. Currently it is spread across the browser and the server, with JavaScript frameworks, and server side frameworks being combined to produce an application. Do we still need those server side frameworks to compose and build web pages? Surely we can now start to strip the server side of the equation back? In this vision of the future we would create the bones of the site using HTML and CSS, mix in JavaScript, and serve these as static files to the browser, that then makes the magic happen. The browser fetches any data required from the server in an easy to digest format, such as JSON.  The server loses the complexity that it gained to build dynamic web pages. I'm going to call this vision of the future the naked server.  If correct, I predict two things:

• JavaScript/HTML/CSS is going to grow dramatically in importance (and that seems to be happening).
• node.js is going to become popular. Because if you know JavaScript, and you simply want to fetch some data from the server, or write some back, why not use the same language and, possibly, files, on both sides of the tubes?

Over the next few weeks I am going to see if I can write a simple naked server application. Not being a JavaScript guru yet, I think I might stick to my knitting, and use naked servlets for the server side of the equation. Then I plan to add to my knowledge and to build a node.js cloud based application. This should be fun!

Software craftmanship

Gojko Adzic has taken the bother to go through the Hudson/Jenkins code base and critique it. He doesn't find the code to be pretty, and wonders what impact this might have on software craftsmanship.

I found the comments quite good. I wrote my own lengthy response, which I am going now going to quote, as I want to add a footnote.

"Any long lived piece of code is going to have many people working on it, each with their own unique style and understanding of the code base. 

Every piece of functionality added is going to have it’s own motivating circumstance, so, for example, there might be time or financial limitations driving it that we, the code readers, are not aware of.

It is almost always easy to criticize others code, but without insight as to how that code was created and added to, the criticism is, to me, shallow. It perhaps gives us insight into the critics preferred coding style, and also maybe teaches us a little.

But beyond that, Hudson/Jenkins code works. The Hudson/Jenkins is released often. The Hudson/Jenkins is very popular. I think that is the true test of software. Regardless of our personal feelings on reading the code. 

You ask where does this code leave software craftsmanship? Hudson/Jenkins is the rock on which many continuous integration projects are based, so I guess it seems to show that software craftsmanship is really not as important as releasing early, releasing often, and getting immediate feedback from a wide community of users. That seems to trump code ‘quality’.

But you know what? It does look as though they could do with a little help cleaning that code base up. We should start by contributing some tests…"

The footnote:

I'm with Dan North. "No-one wants your steenking software – they want the capabilities it gives them."

The only craftsmanship the user of an application will experience is via their interface to the software. It is one of the oddities of the software world that we can write a perfectly literate, indeed beautiful, program that is totally unusable.  And as Hudson/Jenkins seems to show, we can write an abomination that is extremely popular, well liked, and very, very important.

At this moment I believe that the software craftsmanship manifesto is by developers, for developers. If it were any other way it would have usability and interaction at its heart, not code.

What software craftsmanship seems to offer is code that more reliable, cheaper to build, and if it passes the first hurdle of customer acceptance, easier to maintain. That's not bad: but I find myself wondering how many of its practices are well founded.

Take the 100 line constructor that Gojko complains about: in Code Complete, Steve McConnel writes "Decades of evidence say that routines of such length (>100 lines) are no more error prone than shorter routines. Let issues such as the routine's cohesion, number of decision points, number of comments needed to explain the routine, and other complexity-related considerations dictate the length of the routine rather than imposing a length restriction per se. That said, if you want to write routines longer than about 200 lines, be careful."

Surely we should forgo our beliefs in favor of research results?

During my career as a software developer I have seen fads come and go. Many seemed to have very little scientific underpinnings, being based more on belief than on reality. I worry that, seductive and appealing though it sounds, software craftsmanship might be in the same boat.

JavaScript notes: an Applet puzzle

This one has driven me a little crazy over the last few hours.

As part of my learning about JavaScript and HTML5 I thought I would try embedding an applet in an HTML5 page. Which I could easily do. Then I made the applet record a sound and then replay it. To do this the applet needs to be signed. So no problem, I used a self signed certificate, loaded my applet and everything just worked. Sweet.

The next thing I then tried to do was to get the JavaScript on the web page to call into the applet.

<form>
 <input type="button" value="record" onClick="document.getElementById('recorderApplet').doRecord();">
</form>

Hacky - In my defense, I was just trying to see if I could get the call to work...

When I pressed the HTML button in my browser:
POW!: java.security.AccessControlException: access denied (javax.sound.sampled.AudioPermission record)

This was a little unexpected - if the applet button was pressed, I recorded. If the HTML button was pressed, then I got a security error.

I Googled the world and found Bug 4406607 and Mozilla bug 60120
But somehow the answers within didn't seem satisfactory.
Then my searches hit gold: Java Access Control Mechanisms (pdf)

A very nice explanation of what was happening, with a simple answer: change my Java method that was being called by the JavaScript to the following:

public String doRecord() {
        return AccessController.doPrivileged(new PrivilegedAction<String>() {
            public String run() {
                replayRecorder.startRecording();
                return "recording started";
            }
        });
    }

Now my applet records when I hit the HTML button. Sweet.

JavaScript notes: a Firebug puzzle

There it stood, written before me on the page "Globals created with var (those created in the program outside of any function) cannot be deleted."¹

Yet in Firebug

var a = 10;
delete a;
console.log("this is a: " + a);

consistently returned:

ReferenceError: a is not defined (Fool!)²

How could this be happening? 

If I reworked the code and ran it as a script in a web page, it gave me "this is a: 10", as expected.

After a bit of research (thank you, Google),  I found the answer. Ironically in a blog entry about another book by the same author that was puzzling me now...

The solution is well worth reading, but I'll see if I can summarize it:

• Variables are in fact properties with a "don't delete" flag set.
• Variables declared in JavaScript code that would otherwise be global, when executed by an eval call are in fact deletable (the flag isn't set)!
• Those deletable variables are added to the calling object's variable object, for want of a better place to put them.

Firebug appears to be executing code entered into the console via an eval call, hence my strange power to be able to delete global variables. No black magic here!

 Of course this answer poses another question: why are global variables that are in code executed in an eval context deletable?

I think that the following code might propose an answer:

(function test() {
   eval('var wtf = "wtf";');
   console.log(wtf);
   console.log(delete wtf);    // true
})();

When you evaluate code you will possibly have variables added to your calling context that you might not want there. By marking them as deletable, you have a chance to clean them up. Now to work out how you can do this automatically...

Footnotes
1 Page 12, JavaScript Patterns by Stoyan Stefanov, September 2010: First Edition.
2 Fool! wasn't really part of the message: that is just how I felt when I saw the error.

What am I doing?

Good question! This entry is not going to be the answer to a deeply philosophical question. It's just a simple description of what's happening my life at the moment.

My current journey was started last year when my wife, Terry, saw an interesting advertisement in her physiotherapy journal. Due to very sad personal circumstances someone was being forced to sell a web site that they owned. We made enquiries and found that not only was the asking price too much for Terry and I to pay, it also seemed, for various reasons, that the data that underpinned the site was not as complete as we would have liked.

As Terry and I explored the opportunity, we talked about what we thought would make a profitable web site. In the midst of our conversations, Terry came up with a what seemed to be a great idea for a web site.

As the year ran out we found that we spent more and more of our time talking about her idea – and so started doing some basic research into the idea. The more research we did, the sounder the idea looked.

So at the end of the year I resigned and came home to work for my wife. On a very unsalaried basis, I must admit. Terry and I calculated that we had enough savings in reserve to keep me at home for just under a year, all going well.

Shortly before I finished up with my work Terry had received some quotes from landscapers to do some cleaning up and rearranging of the garden. The quotes ranged from between 30 to 70 thousand dollars!

We were so confident of Terry's site idea that we decided the first few weeks of my new work would be dedicated to cleaning up the garden.

So first came down the horrible old grey shed in the back garden.

Our grey monster!

There were several trips to the tip!

Was made of more 'stuff' than we thought...

Then in went some retaining walls and a lot of back fill. 

Any wonder I am loosing weight?

Then the builders came and put a new pair of windows into our house.

Our work is just about to start.

We lined and painted the room ourselves. 

Need some lining done?

To me the painting was truly horrible work. It just seemed to mindlessly go on and on without end. I have decided that painting is a special form of torture. But after what seemed like ages the room was finished.

At last!

We were now into the start of March. And although we were far from achieving all we aimed to do in the garden, I really wanted to return to programming. 

Yes. Typical software developer. Underestimated the time it would take.

So we agreed that we would work in the garden and on the house over weekends, as most people do, and I would start to construct Terry's site.

I created some rough paper prototypes, discussed them with my client (Terry), and then based on our discussions built some quick and dirty html mock-ups. We took the mock-ups, played with them, discarded one approach and then settled on a path forward.

I am now working on the technology stack to be used whilst Terry works on creating and building the content.

In creating the content, it would appear that Terry has found the first major wrinkle in our plans. That is that under Australian legislation it would appear we might well have to register the site as a medical device! If true this will throw a whole heap of unexpected expense and overhead our way, which may well render the original idea unviable.

It might well be time for our first course correction!

One that will possibly affect the technology stack I had chosen.

As it looked as though we were going to be creating a web application, I had settled on JavaScript/jQuery and HTML 5 (that degrades gracefully) on the client and Rails as the quick (and dirty) back end. Now if we are going to deliver a site full of content I think we might better be served with Drupal on the server.

As Terry is producing the content, and as we are trying to work out how best to navigate Australian legislation, I am bringing myself up to speed on JavaScript, jQuery and HTML 5. For sadly, as a regular employee working in the Java environment, I had never had the time to truly master JavaScript. I can now see that there is a huge amount that I had missed in the osmotic learning of a salaried environment!