Every year is getting shorter, never seem to find the time
Plans that either come to naught or half a page of scribbled lines

According to the available data, I'm a hard-working and creative person. Yet my productive output tends to hover just above nil. My feelings are generally in line with Roger Water's lyric, quoted above. Why? What causes the gap between work and productivity? But more importantly: can a computer simulation shed light on the underlying problem?

I set about modelling my personal productivity, with javascript and flot.

Here's how the model works:

Every bus trip I work and work and work, always on what is the most pertinent task for the project I am working on.

But as I work, I sometimes think of new projects. I immediately (and at no cost) add those projects to my backlog, then continue with my current task.

As soon as I finish a task, I look over my set of all next actions and choose whichever one feels most interesting at that point in time (modelled as a random selection)

You can play with this simulation yourself:

Creative Dilligence Simulator

The parameters are configurable. By default each project contains 10 tasks (these are mini-projects really). And each time I do a task there is a 25% chance I'll think of a new project.

The results of the simulation are woeful!

Here's an example:

After a year of bus trips, I've done enough hard work that I could've completed 20 projects. Instead I've finished only 2 (emphasized by the green arrow). Re-running it, I sometimes complete as many as 3 or as few as 0. Meanwhile my backlog of work has grown big enough to keep me busy for years to come.

As simplistic as the model is, it already makes me feel uncomfortable with its home truths. I know this feeling all too well. Hard work, but it's like spinning your wheels while stuck in mud. I know that my time is spent always on tasks I consider worthwhile, yet I'm still getting nothing done.

Here's the shocking thing. This low productivity is not caused by all the usual culprits we list when discussing 'procrastination'. This isn't because of low motivation or excessive distraction by non-productive items. This isn't because of fear (the alleged mind killer) -- it's not fear of shipping, fear of criticism, fear of success, or any other paralysing phobia.

Getting 'Things' Done versus Getting 'Projects' Done

This is the difference between utilization and productivity. High utilization can be just busy work.

This is purely organisational pathology, and 100% curable.

The immediate question is what to do about it? Now the modelling becomes fun.

If I can decrease my ability to think of new projects, down to around 10 percent (approximately 1 new project idea for 10 tasks done), then I manage to ship more projects.

At the end of the year I've now completed between 8 and 20 projects. Yay. But now a new problem arises: I frequently run out of work to do! Many bus trips I just sit there, twiddling my thumbs, staring out the window. For me, that's a terrifying prospect, a sort of waking death. But in any case, I think it's outside my control. This continual thinking up of new projects (albeit stupid STUPID projects) is just a fact of life. You might as well ask a cat to stop following a laser pointer.

The trick instead is to come up with a new strategy for picking tasks. When I finish one task and it's time to move onto another, I need to be far more careful about which task I work on next.

So I implemented a strategy called 'stick to a project'. In this strategy, I keep working on my current project until that project is done.

This is the same as Henry Miller's first resolution on his list of writing rules (more such rules here):

1. Work on one thing at a time until finished.

The results are much more satisfying. I still end up with a giant backlog of work -- but the amount of projects that I've shipped is now exactly in line with the effort I've put in.

That's all I needed to get out of the model -- and at this point I can stop modelling and return to doing.

But it's also just the tip of what the model shows you. Different people have different fears and different desires. Depending on your personal preferences different results are optimal. You might be driven by any of these:

• Fear of drowning in work (minimize backlog)
• Fear of running out of work (maximize backlog)
• Love of a big backlog (maximize backlog)
• Hate working on the same thing all the time (continual switching)
• Fear of negative feedback after releasing work (minimize project completion)
• Desire to release work constantly (maximize project completion)

Play with the numbers, you'll see how to get what you want. Or copy the code and change the model to reflect your own behaviour.

That link again:

Creative Dilligence Simulator

Models are not reality

In practice, things get a lot messier. Projects have variable sizes. Projects grow as you work on them. Some projects *should* be abandoned, not stuck to. (If a job's not worth doing, it's not worth doing properly).

The model is very simple but it was complex enough to demonstrate something useful to me, so I'm happy to publish this as is.

Now I must get back to the gormless and unstoppable implementation of whatever addle-brained scheme has floated to the top of my steaming heap of incomplete mini-projects. Wish me well.

Final thoughts from my spirit animal, the artillery man from Jeff Wayan's "War of the World's Musical" —

Journalist: [as narrator] In the cellar was a tunnel scarcely ten yards long, that had taken him a week to dig. I could have dug that much in a day, and I suddenly had my first inkling of the gulf between his dreams and his powers.

Artilleryman: It's doing the working and the thinking that wears a fellow out. I'm ready for a bit of a rest. How about a drink, eh? Nothing but champagne; now I'm the boss!

I put out a new NimbleText a few days ago.

The release notes are here, but do you want the minutely-detailed and wildly self-indulgent story behind the release notes? Of course you do!

Along with a spattering of smaller improvements, the main innovation is that you can now expose your settings (and quickly change them) from the toolbar.

I wanted this for a while, but couldn't find the right way to let the user configure it. Finally, with inspiration from Windows 7, I came up with using a pin/unpin metaphor, inside the Options form, like so:

Coming up with this design was surprisingly difficult. Some of the designs I rejected were:

Some people asked for a "view" menu where you check or uncheck the options you want to have appear on the toolbar. Here's a quick mockup:

This concept was broken for two reasons.

First, it looks like you're turning the option itself on or off -- not turning on whether or not the option is shown in the toolbar.

Second, if you were to click one of those menu items, thus toggling its checked state, this would also have the undesirable side effect of closing the menu. So if you wanted to show or hide three settings you'd need to re-open the menu three times, for a total of six-clicks. Annoying behavior! (Incidentally my NextAction tool suffers this problem, embedding so many settings in deeply-nested context menus that collapse whenever you click them. Overriding this behaviour is problematic!)

So the new plan was to put checkboxes of some sort inside the Options form, adjacent to each option. But take a look at this mockup:

Again it's easy to think that clicking the new checkbox will cause the option to be turned on. I wasn't going to implement this or anything like it. Then I noticed the way application icon pinning works in Windows, from 7 onwards.

It's an elegant solution, maybe the nicest touch in Windows since they shot the dog.

So I had my big-chief-graphic-design-intern (i.e. Me) whip out a dodgy mockup, ran it over to my main opinion-deflection-test-subject (i.e. Rhys) and after a few short bursts of shouting the project was green-lighted and implementation began.

Here's the mockup with which I got approval from the board:

If it seems weird to agonize over a small feature like this, I could point out not only a long history of such behavior from myself, but also I can provide evidence that i'm a bit of a toolbar fetishist, having written 6 articles on the topic previously:

• Rethinking Toolbars in Visual Studio (or any IDE)
• Too Many Damn Arrows in VS.Net!
• Suggested User Experience Improvements for DiffMerge
• Quick Tip: Show Shortcut Keys in Visual Studio
• KeyTraino
• Your next text editor is... MetaNote!

What UX have you agonised over lately?

I had a lengthy article here -- browsers, typewriters, javascript blah blah blah, cultural tolerance to backward incompatability in Mac and Microsoft worlds, blah blah blah -- but really, the Hanselman pretty much beat me to it, and all I've got left is a cartoon.

((As a derivative of a GPL'd work ("Tabs."|"Spaces."|"Both."),) The above image is licensed to you under version 2 of the GNU General Public License.)

Startups love to talk about 'Pivoting' -- those sudden changes in strategy, right angle turns that take you from obscurity to success.

"Burbn" pivoted from a location-checkin app to a stylized photo sharing app and became billion-dollar company, Instagram!

But to find the masters of pivoting, we need look no further than the monty python sketch, the piranha brothers. This is pivoting done *right*!

At the age of fifteen Doug and Dinsdale started attending the Ernest Pythagoras Primary School in Clerkenwell. When the Piranhas left school they were called up but were found by an Army Board to be too unstable even for National Service. Denied the opportunity to use their talents in the service of their country, they began to operate what they called 'The Operation'. They would select a victim and then threaten to beat him up if he paid the so-called protection money.

This must not have been a particularly succesful strategem, for we are about to learn that the Piranha Brothers chose to 'pivot'.

Four months later they started another operation which the called 'The Other Operation'. In this racket they selected another victim and threatened not to beat him up if he didn't pay them.

This strategy also met with limited success and another pivot was in order.

One month later they hit upon 'The Other Other Operation'. In this the victim was threatened that if he didn't pay them, they would beat him up.

This for the Piranha brothers was the turning point.

The scariest code I ever wrote was the dialog in NimbleText that helps you use the command-line.

Much smack has been written in the past about confusing command-line helpers in other apps, so I set out to build this dialog with great trepidation in my heart.

Joseph Cooney has laid into two particular apps, a gui for wget, and a gui for robocopy. Even Jeff Atwood had a stab at wGetGui.

Here's what they looked like:

And here's a typical user response upon first encountering such a command-line helper:

I downloaded both these apps and tried them out.

Okay, the kindest thing you can say is that they are comprehensive, and with their use of tooltip text they do offer a little more help than the screenshots would suggest.

But they still give an immediate slap in the face to the end user. Something I want to steer away from.

So command-line helpers are a challenge. And to increase the pressure a little more: the command-line feature in NimbleText is only unlocked if you buy a license. If I'm expecting this feature to be worth money, then I really have to not screw it up.

What I did.

The first thing was to use descriptive labels, instead of verbatim option names. Instead of having a checkbox named "--rawdata" I'd have a label that said "Raw data". While this is only marginally more readable, it hopefully decreases the effect shown above.

Next I added a textbox at the foot of the form, where the command-line you've created is written, live, so you can see the output of your furious clicking.

There's a button for save, so you can save your command-line straight to a batch file. (It works from powershell too). A copy button, to put the command-line into your clipboard, and an execute button, which launches a cmd.exe process, and tries out the command-line immediately.

Other than that, I just sweated the small stuff. Alignment, spacing, capitalization, tab order, tool tips, everything as consistent as possible.

What I probably failed to do was give The Dialog any breathing space.

(And looking at the screenshot now I see a slight inconsistency with spacing, which should be fixed by the time you look at the application itself.)

Here's what I came up with:

Any suggested improvements? Please send them in.

I tweeted this yesterday, but wanted to discuss it in a little more than 140 chars

"Reading a book" is a classic important but non-urgent task. When your lifestyle lacks any book time, you know you're in the wrong quadrant.

This is a reference to the four quadrants matrix (urgency versus importance) from the book 'First Things First' by Stephen Covey et al.

The idea is that many of the things we do can be ranked as either important or unimportant, and as urgent or non-urgent.

It's a neat and enlightening concept, but there's something utterly impractical about it.

A response from Dan Puzey summed it up well:

The real problem is that "organizing my life into quadrants" always seems a non-important non-urgent task...

Maybe that's why I've always felt uneasy about the four quadrants idea.

Don't spend time categorizing everything into one quadrant or the other. Don't get caught up in grandiose and abstract questions like "Do I have my life values in order? Am I doing first things first every day?"

Just ask yourself the simple, practical question "Have I read any good¹ books lately?"

Your answer sums up a hell of a lot about how you're life is going. If you find you're not reading any good books, then you know right away that your life is out of balance.

Now stop staring at your navel, and go read Slaughterhouse-Five.

In which I follow the advice of Patrick McKenzie to try and get my little software products into the eyeballs of a whole new audience.

So, it was one of those lazy Sunday evenings when a microIsv guy does what he does best: he looks through the Google Analytics of his products, desperately trying to work out why he is not yet a millionaire, desperately trying to find what tiny tweak he can apply that will ensure he has no need to head to work in the morning, or ever again. (This is known as 'Sunday evening blues, microIsv-style')

When I looked at the search traffic for both sites (TimeSnapper and NimbleText), something leapt out at me, the way a tiger in the wilds of India might jump out at a plump looking passerby.

The only search terms people were using to find TimeSnapper were terms like "TimeSnapper", "Time Snapper" or related mis-spellings of the product name.

Noticeably absent from the keyword traffic was every single person in the world who hadn't already heard of the product from some other source. No one looking for "My browser crashed, how do I recover my work?" or "How do I make timesheets easier?" or "How can I understand my own bad habits?" or "Continuous Screenshot Taking" and so on for a million other search terms. (Hint: I just demonstrated the SEO technique of google-bombing oneself ;-) ). So my website -- That Dilligent Little 24 Hours a Day 7 Days a Week Sales Guy, wasn't drumming up one iota of new sales.

And the same for NimbleText. A tiny trickle of people would turn up, but only via search terms like "NimbleText", "Nimble Text" or "World's Simplest Code Generator" (the product's original name) -- and no one else.

So I asked myself, as I sat there on that uneventful Sunday eve: How do I make it happen?

In times like this, I always turn to the writings of Patrick Mckenzie (aka Patio11 on twitter and Hacker News). For SEO he recommends writing 'evergreen' and 'scalable' content.

'Evergreen' content is timeless content: stuff that isn't dependent on today's news cycle or the latest fashion.

'Scalable' content is the sort of content you can write a lot of. The sort of guff that doesn't take a great deal of soul searching.

In relation to NimbleText I easily came up with a basic idea for 'scalable' content generation. Normally, when writing about NimbleText I think about the features, and there's a finite amount I can write. If instead I were to write a short article on every possible specific situation where NimbleText could be used, then you'd be looking at a limitless source of article topics. Think of every type of code it can generate, every example piece of HTML it can produce, every piece of SQL it can concoct, you would be looking at an endless stream of simple, albeit quite repetitive articles. You could churn out such articles at a pretty fast rate. (NimbleText itself could even help with this task.)

Articles such as 'How do I generate insert statements?' may not be the sort of thing that sets the world on fire -- they're never going to attract a viral influx of rabid fans -- but hopefully they'll pander to some fine strange of the long tail of search traffic, and, over time, bring in a trickle of fresh visitors, potential paying customers.

This strategy is a sure winner from an SEO point of view. Wikipedia is essentially nothing but a giant engine built for the creation of Scalable Evergreen content. No wonder it takes first place for just about any search you perform.

So here's the short list of NimbleText-related articles I've written on the bus, since coming up with this strategy:

SQL Master Class (for NimbleText)

• How to Generate Insert Statements
• How to Generate Update Statements
• How to Generate SQL 'IN' Statements
• How to List the Columns of a Table, Quickly!
• Extracting a list of columns from a block of text
• How to Generate Views the easy way

Create HTML Automatically (with NimbleText)

• Generate HTML Tables QUICKLY!
• Generate <OL> Ordered Lists in HTML Quickly
• Generate <UL> Unordered Lists in HTML Lightning fast!
• Generate a list of Definitions in HTML in no time at all!
• Generate a drop down list in HTML (generate a select list)
• Generate a radiobutton list in HTML as fast as Lightning!

It takes less than one bus ride to write such an article, and they're only getting easier. I've got a backlog of thirty such topics and I'm sure with a more concentrated effort I could grow this to many more. Is it worth it? I'm unconvinced, but I'll look at the analytics over time and see what happens.

I've been running this experiment for a few weeks now. Already i've started to see people arrive from new search queries, suited to the articles I've written. The volumes are hardly mega, but the littlest steps bring the most satisfaction.

From this answer at stackoverflow, I read:

I saw this one in a bollywood movie. Our hero was busy romancing with his gf until his friend informs him about upcoming college exams. So, he decides to get examination papers by hacking into his college network. This is how he goes about it:

Enters Lab. Opens up a command prompt window. Types - Hack System

And that's it!!...A window pops up- System Hacked

He gets access to all papers and returns to his gf for a romantic song :)

Mind blown. I just had to try it out:

C:\temp>copy con hack.bat
@echo %* hacked!!!!
^Z
        1 file(s) copied.

C:\temp>hack system
system hacked!!!!

C:\temp>hack internet
internet hacked!!!!

C:\temp>hack FBI
FBI hacked!!!!

Drunk with power, yet trembling in terror; I'm sitting here with the door barricaded, certain the feds are going to burst through the door at any moment.

Do *not* try this at home.

Here's a clever calculator that let's you answer the age-old question: "is this thing worth automating?"

I put this together a few days ago and I just keep needing to use it! Situations keep coming up where I'm gobsmacked to find that our 'gut-feel' about the relative merits of two approaches is just not borne out by the simplest back-of-the-napkin calculation.

The neat thing about this calculator is that it distills the choice down to its most crucial elements, so you can come up with an answer very quickly.

Once you've plugged in some values and gotten your answer, you can easily share it with those chumps in management or with a clever colleague — click the 'Save this result' button, and you'll be given a url that you can send around, preserving all the values you plugged in, allowing others to tinker with your calculation and verify everything for themselves. (Implementing that bit was the funnest of the fun. Remind me to show you the 'GetHashyCode' extension method.)

When you take a moment to play with the figures, there's a bunch of things that leap out at you.

First up — this rather obvious result:

"If you're only going to do it once, it's not worth automating."

That might be quite a shock to some of my automation-happy friends, but I'm afraid the result is unequivocal.

Second: it's amazing how much value you can add by automating something that happens a lot.

Imagine your company has a timesheeting system that takes 10 minutes longer to complete than it should. It's used every week by 20 people, so in the next 2 years it will be filled out approximately 2000 times. You work out a way to save those 10 minutes.... how much effort should you put into making this improvement? Should you bail out if you can't fix it in 1 day? 2 days? 3 days? Here are the figures. It turns out the break even point is 430 hours of work — around 11 weeks! So yes, if it's going to cost you a whole day of work to improve the timesheeting system — go ahead and do it! You'd be insane not to!

Of course, the benefits of automation are more than just the time it can save. When a task becomes free to do it changes the nature of the value proposition. Read about the amazing impact of Jan Ernest Matzelinger — a brilliant automator who revolutionised the shoe industry.

The calculator could be simpler, or it could be more complex.

A simpler version would remove the 'hourly rate' fields — so the answer would be in just hours.

A slightly more complex version would allow there to be a different hourly rate for the person who cleans up when manual work goes wrong. This is realistic. Clean up crews can be expensive. Also the costs of maintaining the automation could be factored in. Cheap automation solutions tend to be very brittle.

Okay — I'm all out of discussion about this little tool. Use it, share it, automate something today.

I can't stop thinking about, wondering about, caring about, reading about the tragic life of Aaron Swartz. There's a lot I want to write. I think I could fill a book just trying to process what it means, what is an appropriate response, what's it all about. But I'm not going to attempt that.

I've been reading Aaron's blog, on and off, for over ten years. Ten years is a long time. And by my own estimates, those particular 10 years were the longest in history.

Long ago I printed out his HOWTO: Be more productive for multiple re-reads and have returned to it many times since.

I wanted to go back, right back, and try to work out the earliest stuff of his that I read. And I wanted to watch the progression of his ideas as they emerged.

From his blog 'raw thought' -- there's a link to 'Older Posts' which takes you to 'the archive' (grouped by theme).

From there is a link to 'Full Archives' which takes you to the reverse-chronological archives.

These stretch back to May 2005 (the oldest entry on that page is about a server crash after which he had to restart his blogging. Under the so called 'Full archives' section there's no link to anything prior to May 2005.

Now I'm certain he was blogging long before that -- I'm certain I was reading his blog long before that.

Is the stuff before that server crash lost? I hoped not, so I set about locating it.

I clearly remember his powerpoint remix (from 2003!) - it got published in a book of Joel Spolsky's - and I soon tracked that down.

Taking a look at the url suggests a numbered blogging system (from Dave Winer's Radio Userland), and from there it's easy to find all of his prior blog entries.

After a bit of binary searching I found what looks like Aaron's first Hello, world, with article id of '81'.

So I wrote a powershell script to download everything (I hardly think aaronsw would object !!) and found that the articles go from number 81 up to 1691, with a few gaps.

Here's the script.

# Downloads aaron's early stuff
# i've done this the hard way because i didn't have time to do it the easy way.

$client = new-object System.Net.WebClient

$nums = 81..1691

#detected up to 1691  (April 26, 2005)
$nums | % {
    $url = [string]::Format( "http://www.aaronsw.com/weblog/{0:000000}",$_)
    $path = join-path $(get-location) ([string]::Format("aaronsw_{0:000000}.html",$_))
    Write-Host "downloading " $url " to " $path
    $client.DownloadFile( $url, $path )
    
    #sleep for 4 seconds before grabbing, to give the server time to exhale.
    Start-Sleep -s 4
}

Then I wrote a script to walk through those files and create an archive page in the same style as Aaron's other archive pages.

It's not pretty code, it got the job done...

dir .\aaronsw_*.html | % {

    #extract the filenumber out of the name... i should've made this easier.
    $num = $_.Name.Split("_")[1].Split(".")[0] 
    
    #calculate the target url for this file
    $url = [string]::Format("http://www.aaronsw.com/weblog/{0}",$num)
    
    #load the file 
    $article = gc $_.Name

    #grab the title
    $titleRegex = [regex]'h1>(.*)</h1>'
    $title = $titleRegex.Match($article).Groups[1].Value
    
    #grab the time
    $timeRegex = [regex]'<p class="posted">posted ([^(]+) \('
    $time = $timeRegex.Match($article).Groups[1].Value
    
    #output the url, title and time, as html
    $item = [string]::Format('<p><a href="{0}">{1}</a> ({2})</p>',$url,$title,$time)
    $item >> archivePreCrash.html
}

So the result is this fairly complete list of pre-server crash articles:

aaronsw archive: early works

Now this takes us up to April 2005. And the post-crash articles start in May 2005, so it probably means that everything's accounted for, except maybe a month's worth of blogging. There are some missing articles within that period, and some lost stuff. I can see that he restored it from the wayback machine where possible, but sometimes there was nothing to grab.

There are a lot of gems in there (and of course a bit of drivel: this starts when he was 15). I was going to pull out a few quotes, but I'd rather let you do that for yourself. He was a thoughtful guy. It'd be great if he was still around.