Stashy is a Really simple Key Value store

In my various websites and web applications, I often need to store some things and later retrieve them.

Mostly for this I use a really simple key value store called "Stashy", that I built, oh, decades ago.

Imagine I have a "plain old class object", and I want to save it. It might be an object called "myNewBlogPost".

I will call:

stashy.Save<BlogPost>(myNewBlogPost.Slug, myNewBlogPost);

Later, if I want to load that blog post, I will need to know the key I used, let's say it was "hello-world":

var myNewBlogPost = stashy.Load<BlogPost>("hello-world");

And there's also a method for loading all objects of a given type:

var myBlogPosts = stashy.LoadAll<BlogPost>();

And for deleting a single object:

stashy.Delete<BlogPost>("hello-world");

And that's it!

It's not distributed. It doesn't have a query language. It doesn't have built in caching semantics. It's not even async.

I tell you what it does have: utility. It's bloody useful! I would definitely not consider it for cases with millions of records, but 1000 or under? It's fine! And one less thing to worry about.

The Stashy class is an implementation of the IStashy interface. This makes it easier to test, and helps me with the dependency injection stuff. Mostly it serves to make sure I'm keeping the contract small.

The IStashy interface is just this:

public interface IStashy<K>
{
    void Save<T>(T t, K id);
    T Load<T>(K id);
    IEnumerable<T> LoadAll<T>();
    void Delete<T>(K id);
    K GetNewId<T>();
}

The type K is the type of key you want to use. I usually use strings as my key. But you can use integers or guids, or anything else, if you like.

As I tweeted recently, The heart and soul of my most used key-value store fits in a single tweet. Easily!

I've used a couple of different implementations of this interface over the years, but currently what it does is serialize the object as JSON and then store it in a file named after the key, in a sub-folder named after the type of the object. (I also have a completely in-memory implementation, for testing etc.)

Here's an implementation I use at the moment in one of my projects. It is not glorious beautiful wonderous code, but it is working code that I just... well I just never have to look at it. But I rely on it all the time:

see gist: FileStashy.cs

It's about 100 lines of code, written years ago, and relies on Newtonsoft.Json to turn objects into Json and vice-versa.

(10 years ago my FileStashy used XML, on Windows... Now it's all Json, and runs anywhere that .net core runs. Maybe one day, one special day, we'll all switch to CSV.)

Sometimes I build other stuff on-top of Stashy, such as indexes, for faster lookups, and "revisions", so I can look at old versions of an object. That was easy enough to write (and very fun of course!) but I wouldn't want to do too much with it, or I'd switch to a real key value store... but then get all the headaches that come from those.

Got any crufty little things you've rolled yourself instead of using something "better"?

 

Tips For Working From Home

Now that everyone else has to work from home let me share my tips from 5 happy years of this way of life:

Get some exercise! Measure your steps and make sure you do a healthy amount every day. Super important. With no incidental walking your fitness can really drop.

Music! Music is crucial for me and maybe for you too. I bought a spotify subscription after a few months, just to get rid of the ads. You need a lot of playlists to fill up a week. Different music is needed at different times of day, and for different moods. Monday mornings and friday afternoons are two extreme opposites, and there many variations in between.

Look after your eyes! Home office is smaller than work office. Make sure your eyes get to look into the middle distance often. If you stare at just monitor/close walls all day, eyes never relax and you go blind! So stare out the window while you're day dreaming.

Plan B. You may need to be prepared in case your home internet connection goes down. (This happened to me yesterday, after a garbage truck took out the NBN for my whole neighborhood.) Make sure you can tether your laptop to your phone, or that you can work fully offline if needed.

The duck. Without a colleague to talk to you may find yourself talking to a rubber duck, explaining all your technical problems. This is a great technique and has been shown to work. Don't let your boss find out though or they'll fire you and keep the duck.

Headset. You'll need a good headset for remote meetings.

some kind of weird helmet that provides oxygen to an office worker

Diet. Don't let high sugar foods enter the house. It's the only way to avoid the temptation and deadly snaccidents.

Environment. Do whatever you can to create a dedicated workspace. It will help you separate work from home and improve your focus. This way too, you can finally get the home office just the way you want it.

finally got the home office just the way i want it

Keyboard, Mouse, Monitor(s). Maybe your setup isn't perfect, but where possible, don't just work directly on your laptop. Use a real keyboard. I think wireless is a waste of time with keyboards, you hardly move it, just used a wired one! And I'm not one of those fetishists who spends all their time and money on them, though I don't wish to shame those who are, well not too much. More monitors are always better, but you may not have the space, the funds, or the opportunity to get that in place right now. I don't have strong opinions on mice either, but with a real keyboard you will also need a real mouse.

Habits. The whole experience hinges on your ability to construct your own habits that create a good life. You can't rely on following the momentum of those around you. You're free to build up or destroy the structure of your day. It doesn't take will power, it just takes sensible routines.

 

Productivity with Powershell: Grab your agenda from outlook as plain text

I use a plain text file to manage my daily todo items. (Gina Trapani championed this idea with the "todo.txt" movement) and I use some little bits of console goodness to make it work my way.


I like to jam with the console cowboys in cyberspace.

At the start of each day I run a powershell script, "today" that does a bunch of calculations, opens up my todo.txt file in notepad++, and puts a bunch of text into my clipboard as a starting point. Then I paste that list at the bottom of the document. I also scroll up to yesterday's list and run a notepad++ macro that "sweeps" any remaining tasks from yesterday into today's list. Then I get to work.

But there is this big problem inside companies called "meetings" and then tend to get in the way of this things called "personal productivity". I work remotely and don't have a lot of meetings, so I don't have the habit of checking my calendar, or living in my calendar, as some people do. Hence I have a bad habit of missing meetings. Bosses don't like this habit.

To alleviate the problem, I've improved my "today" script so that it will grab today's schedule out of my outlook calendar. I've shared the script I wrote for this as a github "gist", here:

Get today's schedule from Outlook, using COM, and format as text for my TODO.txt file

It's not very "elegant" as it is an example of "meware" -- code built for an audience of one.

Heck I'll just post it inline, here's my ugly baby:

# Get a list of meetings occurring today.
function get-meetings() {
	$olFolderCalendar = 9
	$ol = New-Object -ComObject Outlook.Application
	$ns = $ol.GetNamespace('MAPI')
	$Start = (Get-Date).ToShortDateString()
	$End = (Get-Date).ToShortDateString()
	$Filter = "[MessageClass]='IPM.Appointment' AND [Start] > '$Start' AND [End] < '$End'"
	$appointments = $ns.GetDefaultFolder($olFolderCalendar).Items
	$appointments.IncludeRecurrences = $true
	$appointments.Restrict($Filter) |  
	% {
		if ($_.IsRecurring -ne $true) {
			# send the meeting down the pipeline
			$_; 
		} else {
			#"RECURRING... see if it occurs today?"
			try {
				# This will throw an exception if it's not on today. (Note how we combine today's *date* with the start *time* of the meeting)
				$_.GetRecurrencePattern().GetOccurrence( ((Get-Date).ToString("yyyy-MM-dd") + " " + $_.Start.ToString("HH:mm")) )
				# but if it is on today, it will send today's occurrence down the pipeline.
			} 
			catch
			{
				#"Not today"
			}
		}
	} | sort -property Start | % { 
		# split up the names of the attendees to have just 1 firstname/surname and less space.
		$arrr = ($_.RequiredAttendees.split(';') | % { $_.Trim() } | % { $_.split(' ')[1] + ' ' + $_.split(' ')[0] } )
		$attendees = ($arrr -join " ").Replace(", ",",").TrimEnd(',')
		# this is the formatted string that we return, ready for use in 'today'
		("`n`t`t[ ] " + $_.Start.ToString("HH:mm") + " - " + $_.Subject.ToUpper() + " with: " + $attendees )  
	}
}

The resulting text looks like this:

		[ ] 09:00 - STANDUP with: Keena Maloney,Rhea Zamora,Ed Jackson
		[ ] 10:00 - DESIGN REVIEW with: Steph Perkins,Rhea Zamora
		[ ] 14:30 - CUSTOMER COMPLAINTS with: Della Bouchard,Steph Perkins

The way recurring meetings work was the tricky bit. I couldn't find any simple way to query a recurring appointment to ask it "Does it actually occur today?"

As always, I'm very open to any feedback or improvements!

 

2019 By The Numbers

instagram images from 2019

Well it's the roaring 20's at last and here's the blogpost where I look back on the annual output of a machine I call "me".

I spend a lot of time berating myself for the things I didn't do, so it's pretty odd to look back and see that a lot of things did get out the door.

Numbers in parens, e.g. "(3,2,1)", are the figures from last year, the year before that and the year before that.

  • 1 new product launched TimeSnapper for Mac. Jon Schneider did all the work here, but that won't stop me from listing it as my biggest achievement of the year. Top work Jon! (And Atli continues to put in the long hours of keeping TimeSnapper for Windows alive. We're hoping to update the application and the website this year too.)
  • 1 Kanban Simulation game (0): DevShop
  • 0 books published (1,1,0) Joseph and I slowed down work on Evergreen Skills for Developers. The brilliant thing about Evergreen Skills is that it'll still be fresh info even if it takes us 10 year to complete. I compiled 1 complete book in 2019, but haven't launched it due to Amazon "issues", and outlined a few more. See 'how to get nothing done'
  • 13 blog posts written (10, 13, 17) — Most popular thing I published was probably this html quine.
  • 15 new wiki articles completed (14,10,50) — Best new article there is Protective Factors for Mental Health
  • 48 Weekly Project Update emails shared with Richard Mason of Aussie Bushwalking (19,0). These keep the both of us productive all year long.
  • ~200 pictures at insta/secretGeek (~200)
  • 100 Today-I-Learned Entries written (73, 133, 233) — 30142 words (19761, 35289, 42352), 15 new topics (10,21, 44)
  • 389 contributions at Github/secretGeek (156,189,292)
  • 0 talks delivered (0,1,1) phew! 1 conference attended as always: DDD Brisbane.
  • Daily step goal met every day unless gravely ill.
  • Millions of tweets @secretGeek. This seems to be the one that more people saw:

Previously (2018), Previously (2017) previously (2016)

 

5 minutes to import CSV to a database: What do you do?

I challenged my twitter peeps:

Challenge: you have a csv file and you need to import it into a new table in a database. You have 5 mins to get it done. What tools do you use?

@secretgeek

and was overwhelmed with the responses! So many different ways people approach the problem, and so many different tools.

It reminds me of Miles' Law:

Where you stand on an issue depends on where you sit.

  • If you sit in python all day, you'll write a python script.
  • If you sit in powershell all day, you'll use powershell
  • If you sit in SQL Server all day, you might use T-SQL itself to do the work.
  • If you sit in an editor all day, you might use search/replace to transform the data into sql statements.

I put all the solutions I saw into my "today I learneds" site, here's a table of contents for the topic:

In related CSV-news I should mention that I made a git repo: "Awesome CSV" where I collect all the coolest CSV tools, articles and specs from around the planet. It's a happening place for cool people to hang out and learn cool things.

And my own modest tool NimbleText treats everything as delimited text, lets you do powerful things quickly, and is arguably* the greatest invention in the history of the universe.

I also have an Awesome GUID repo, where I store awesome things related to GUIDs. There are surprisingly few things there. Hmmm.


* Whenever someone says 'arguably' it is safe to replace it with 'not'. This is arguably true in every circumstance.

 

Desert Skies - exclusive inside information from the developers

Unless you've been living under a rock like some disgusting sand-sneaking Seeker -- you're aware of the awesome sensation taking the gaming community by storm: Desert Skies.

Here's a video of the game play in case you're not familiar.

I recently sat down for an intense lunch with one of the minds from White Rabbit Games - the creative duo behind this Indy gaming sensation, and I found out some exclusive information.

Although the company is called "White Rabbit Games" the logo is of a cat.

white rabbit games logo... a white cat

SEE!?

What's that all about?

T: we have a fluffy white cat, and it looks like a bunny so we call it "bunny".

Thus the white rabbit is actually a cat.

Woah.

That's enough revelations for one day.

It's a really awesome game. I find it pretty exciting because it's a simple idea that is popular enough that it will get a chance to expand in some really interesting directions. Keep an eye on this!

 

What is NimbleText (a story about cheese)

A good friend asked me the other day, "Wait a second, what's nimble text?" and I stammered out an answer. I'm not good at elevator pitches. Waving my arms around didn't make it any clearer. I tried waving them in bigger circles and pointing as I talked. It still didn't help.

I decided to email him an answer. It turns out I don't have his email address, so I'm writing it here, and I'll transmit the URL some other way. You get to read my reply in public.

What then, is NimbleText?

NimbleText is an adhoc code generator.

For example, say you have a list of "keywords", (he's a digital advertising specialist... it takes all kinds) and you want to do something with each keyword, maybe insert it into a database.

Here's your list of keywords (these are French cheeses)

Abondance
Banon
Beaufort
Bleu
Brie
Brillat-Savarin
Brocciu Cara or Brocciu
Cabécou
Cancoillotte
Camembert de Normandie
Chabichou du Poitou
Chaource
Chevrotin
Comté

...you paste that into NimbleText, and type in a pattern... how about this:

Insert into Keywords(Keyword) Values ('$0')

NimbleText will *magically* create this SQL query for you:

Insert into Keywords(Keyword) Values ('Abondance')
Insert into Keywords(Keyword) Values ('Banon')
Insert into Keywords(Keyword) Values ('Beaufort')
Insert into Keywords(Keyword) Values ('Bleu')
Insert into Keywords(Keyword) Values ('Brie')
Insert into Keywords(Keyword) Values ('Brillat-Savarin')
Insert into Keywords(Keyword) Values ('Brocciu Cara or Brocciu')
Insert into Keywords(Keyword) Values ('Cabécou')
Insert into Keywords(Keyword) Values ('Cancoillotte')
Insert into Keywords(Keyword) Values ('Camembert de Normandie')
Insert into Keywords(Keyword) Values ('Chabichou du Poitou')
Insert into Keywords(Keyword) Values ('Chaource')
Insert into Keywords(Keyword) Values ('Chevrotin')
Insert into Keywords(Keyword) Values ('Comté')

Two seconds later, you run the SQL and insert all the keywords into the database.

You can play with that example here: https://nimbletext.com/Live/398054487/

If the input had more than one column, you would refer to the next column as $1, then $2, etc.

So if the keyword data included, I don't know... some search traffic numbers?...

Abondance,1000
Banon,3000
Beaufort,200
Bleu,12
Brie,6
Brillat-Savarin,90

We could use a pattern like this:

Insert into KeywordResult (Keyword,Result) Values ('$0', $1)

To generate this result:

Insert into KeywordResult (Keyword,Result) Values ('Abondance', 1000)
Insert into KeywordResult (Keyword,Result) Values ('Banon', 3000)
Insert into KeywordResult (Keyword,Result) Values ('Beaufort', 200)
Insert into KeywordResult (Keyword,Result) Values ('Bleu', 12)
Insert into KeywordResult (Keyword,Result) Values ('Brie', 6)
Insert into KeywordResult (Keyword,Result) Values ('Brillat-Savarin', 90)

(See this example here: https://nimbletext.com/Live/284342338/)

But it's not just for generating SQL queries, it's for text in general; and it's not just for keywords and it's not even specific to cheeses. There are many non-cheese uses.

It helps you anytime you want to rearrange some data and do something to every row, to generate a result.

There's an online version, and a downloadable version (with extra features). You can embed javascript inside your patterns to do really tricky formatting stuff. It's versatile.

It's just this thing, you know. For stuff.

 

Secrets of Mastering Excel

Inspired by a comment thread at hacker news I made this mini site:

excel.secretgeek.net

I often want to share Joel Spolsky's famous "You Suck at Excel" video tutorial with "important" business people inside the large enterprise where I spend a lot of time.

But it would be easily misconstrued as trolling if I sent a business customer a URL that literally tells them, right in the heading, that they "suck" at excel.

So, as advised by user TuringTest (in the thread mentioned above) I created a mini site, on its own sub-domain, that is palatable to a business mindset, and brutally overlaid a heading that says "Secrets of Mastering Excel" right over the top of the "You Suck at Excel" heading.

Now I've got something I can recommend to business people without causing disharmony. And I can leave a link to it in my corporate email signature, to nudge others to improve their Excel game.

Ideally the page would detect when the video starts and remove the label, so it doesn't obscure the video. Also, I'd like to write up a set of notes about the talk, but ain't nobody got time for that, so I've linked to the notes from Max Masnick. Perfect is the mortal enemy of done and they are locked in an ultimate death embrace and that is how storm clouds create lightning but I digress.

The whole thing from idea to delivery was about an hour. I was really pleased with the speed on this one.

 

The Market for Ideas (and is contenteditable really terrible?)

Some people invest in gold, others bitcoin. I put my money on ideas. The volatility can be wild.

Take this idea -- the idea that "Contenteditable" attribute is a terrible basis for a Wysiwyg editor, as expressed by this article from medium engineering.

5 years ago, this idea was only worth 3 points and 1 comment.

(I bought all I could! I liquidated all my other ideas so I could go long on this one....)

One year later -- it's worth had grown to 75 points and 9 comments:

(I was tempted to cash out... but some stubborn streak... I held my nerve)

It hit its peak 3 years ago when its worth grew to 155 points and 76 comments

(I sold it all, by god, I cashed it all in, right at the peak! It was glorious!)

Soon after: the idea came crashing down. A year ago it was old news, back down to 3 points and 0 comments.

I guess what I'm saying is... sometimes an idea is before it's time. Not enough people understand the fundamentals to be able to engage with the idea. Then the audience has caught up, and they understand the idea. And then, having grokked it, the idea doesn't captivate them any longer.

A different though related thought is "It's better to be wrong than early" as described at hackernoon, a sentiment that occurs a few times in Ray Dalio's Principles.

contenteditable

I don't really invest in ideas. Not explicitly. (At a deep level, just about everything we do is an investment in an idea.)

p.s. I built something with contenteditable this week... I agree it's terrible. A page that lets you edit its style, without using javascript

 

a html quine

html,

Yesterday I was on a bus! Yes, a bus! I don't get to ride buses so often due to the working from home lyfe, but I took the opportunity to write some simple code (as I used to do on buses in the past... i even gave a talk about it once, when alt.net was a thing.)

And the thing I coded on the bus was a html quine.

Go and see this html quine

It's... different?

html quine.jpg

A quine is a program that produces, as output, its own source code. I've discussed and demonstrated quines over here in the wiki.

And as I say in the html quine link, although the idea had bounced around in my head for a decade, the thing that reminded me to go and do it was seeing this piece of "Code as Art" from Geoff Huntley recently: no yaml. There must be more Code as Art in this world.


See also, lengthy Hacker News discussion