Remove empty lines from a file using Powershell.

I needed to remove the blank lines from a file.

Normally when I need to do this, I use a regular expression in TextPad. (replace "\r\n\r\n" with "\r\n"... and iterate) -- but TextPad wasn't available on this machine, and I couldn't connect to the internet to grab it.

So I fired up PowerShell and messed around with the syntax until it worked.

gc c:\FileWithEmptyLines.txt | where {$_ -ne ""} > c:\FileWithNoEmptyLines.txt

I don't know if that was the prettiest way to do it -- but I got the result I needed ;-)

The nicest thing was that I didn't need to look anything up -- I just tried variations until I got the result I wanted. I didn't remember the syntax of the 'where' statement or the 'not equal to' operator -- I guessed and got them right within one or two guesses. Nice language design, Bruce!

(I didn't even remember the command 'gc' -- but since i wanted the powershell equivalent of the 'type' command, so i entered 'alias type' and found 'Get-Content' is the powershell equivalent of 'Type' which i guessed was also known as just 'gc')

 

A Faster Ruby.

[Devil's Advocate (Dvl) talks to Ruby Advocate (Rby). Inspired by ruby forum discussion, via Scott Hanselman.]

Dvl: Ruby has poor computational performance.

Rby: What? No it doesn't.

Dvl: Yes it does. You ruby people are in denial about the whole topic of Ruby's poor computational speed.

Rby: Denial? What? No we're not! Look over there!

Dvl: Yes you are. Try and say it with me "computational performance."

Rby: Comp... u... tay... What about developer performance, hey? Developer productivity is more important. Elegance, beauty, blah blah, love-ins, simplicity

Dvl: Well if Ruby developers are so damn productive, why can't they write a faster ruby?

Rby (covering ears and shouting): SHUT UP! SHUT UP! SHUT UP!

 

There's Three Ways To Know Anything

Okay, so i've decided there are three ways to know anything. Maybe there's more ways. So, there are at least three ways to know anything. Everything.

Say you want to write really good .Net code: code that fulfills all your needs, does cool stuff, and works reliably. There's three ways.

First, you can read up on the .Net framework, by reading books, by reading other people's code, by reading every article and blog entry you can find -- by reflecting into the IL and reading all of that code too. This is the first way to know everything: learn it all in advance, Just In Case.

Second, you can shun all the books, ignore everyone else's code, concentrate on the language itself, and never use the framework -- always write your own code to achieve whatever needs to be done, from the most basic of first principles. This is the second way to know everything: trust noone, build on first principles, Just Because.

Third, you can run with the fragments of knowledge you've picked up so far, and dive into real world problems. When you hit a barrier -- search for the answer to that specific problem, solve it and move on. Intellisense, Google and Google groups are your friend. This is the third way to know everything: Just In Time.

What do you know? And how?

(p.s. after writing this, i googled for "Just In Case" learning and found that Kathy Sierra covered these ideas much better than me, years and year ago, from a teaching perspective.)

 

Test-Driven Hypocrisy? Who tests the test?

An oft heard mantra in Test-Driven Development is "if it's not tested, it's broken" and I have to admit that this slogan makes me cringe -- and leads to some of my own hidden objections to TDD.

"If it's not tested it's broken" -- okay it's a blatant exaggeration -- yet this seems to be lost on a lot of people. What it really means is something more like:

"If it's not tested with unit tests then it's unlikely to be tested elsewhere and hence we'd be making a fairly safe bet, to assume that it contains bugs."

But somehow this has less punch than the exaggerated mantra

"If tested it ain't, broken it is."

I like exaggeration -- hell, i probably like it a thousand more times than you do -- but i get annoyed when people take exaggerations literally.

For kicks, let's apply the principle literally and see where it gets us:

Say you write some code. Oops. You should've written tests first. Your code's broke.

So now you write code to test your code.

Oops, the code you wrote to test your code is broke. Fool! You didn't write any tests to test the tests that test the code that broke.

Stack overflow. Goodnight.

Discuss.

(First -- thanks to Punky for mentioning this conundrum in the comments to the previous post)

Okay -- wait a moment -- i think i see what went wrong. Let's investigate.

In the following dialogue there are two parts:

Agl: An Agilista.
and Dvl: The Devil's Advocate.

Agl: We don't write tests to tests our unit tests -- so the 'stack overflow' doesn't occur.

Dvl: But that means your tests are broken -- by your own definition.

Agl: No it only means there's a chance that they're broken. So we work to make this chance as small as possible. Firstly, we write very simple unit tests -- the smallest things that can possibly break...

Dvl: Yeh yeh, small unit tests are less likely to contain bugs -- or likely to contain less bugs -- but you need lots of tests, thus lots of code and at least a few bugs. i say again, your unit tests are broke and the whole approach is bunk.

Agl: No, we start off with small unit tests that fail. On purpose -- we write the unit tests before we write the code. The tests fail at first. Then we write the simplest code we can to pass the unit tests. We run it again and now the tests pass. By doing that, we've gotten a hidden benefit -- we've tested our unit tests. And it's at that stage that we sometimes find our unit tests were broken.

Dvl: Nonsense -- just because the tests failed at first and then passed doesn't mean the tests were correct. It might be that there's a bug in both the test and the code being tested. I can easily write up a situation like that.

Agl: True, you can, and it's definitely possible. But it's all about probabilities. We wager that it's unlikely that a pair of bugs together will display that behaviour. And even if they do, we've reduced the likelihood of bugs by an order of magnitude. Certain combinations of bugs will slip through -- but that's a much better situation than the situation we had before, where all bugs slipped through.

Dvl: Weeeel, I'm unconvinced.

Agl: Look, just try it. Plenty of people have tried it and found that their projects are more succesful, with less bugs because of it.

Dvl: It's impossible to know that for certain unless they've performed a repeatable, placebo-controlled double-blinded experiment. And the economics of the situation dictate that no-one would ever pay for that kind of experimentation. If they have, show me the journal and i'll show you the experimental flaws.

Agl: You're just being an a**hat now. Talk to people who've used it and they'll tell you how effective it is.

Dvl: Oh I see, it's like magnets and crystals for healing. Anecdotal evidence a-plenty.

Agl: Then try it for yourself! People who try it seldon turn back.

Dvl: I've heard the same thing about heroine and nicotine. Maybe it creates a psychological dependency. Seems feasible. I wonder if being 'test-infected' would respond to OCD medication?

Agl (closing eyes, covering ears and shouting): SHUT UP! SHUT UP! SHUT UP!

lb: Okay... anyone got a better response?

 

Agile and Test-Driven: A Marriage Made In Hell?

Agile and Test-Driven go together like a horse and carriage... or do they?

Thanks to an excellent series of articles at Raganwald, I'm currently rethinking some of my old prejudices against agile and test-driven practices, to see if i'll start adopting them as a more central part of my style. (Note: I'm not against either of these practices, but i'm certainly not a true believer)

Thus i've been trying to uncover my hidden objections, to see if they can be overcome. But along the way a few boggling conundrums have presented themselves. See what you think.

Problem Number One.

One of the premises of test-driven development is that bugs cost hundreds of times more to fix if caught later. That's cool right?

One of the promises of agile is that when you follow its practices you flatten out the cost of making changes late in the game. That's also pretty cool.

But doesn't this mean that the use of one practice lowers the economic incentive to use the other?

I.e. when using agile practices, the benefits of test-driven development have less economic impact than they would have on a non-agile project. True? Crazy?

Problem Number Two.

One of the mantras of agile is 'YAGNI' -- You Aint Gonna Need It -- meaning, don't waste time writing extra code on the off chance it's needed later.

Test-Driven Development (TDD), on the other, states unequivocally that it's worth writing extra code, lots and lots of extra code, up to five times as much code you'd write otherwise. The reasons for this are: it may stop bugs from escaping your desk -- and it may help out when you need to alter the design later.

Hello? Maybe just maybe YAGNI can be applied to TDD? Or do we pick and choose when YAGNI applies?

So what gives? These seem like obvious reasons why TDD and Agile should be preferred in isolation from each other, not together. Why are they so often co-practiced?

An example where TDD trumps Agile?

If building a Mars Rover, go with TDD, not agile. Because once the Mars Rover has landed on the surface you can't say "okay, let's do another iteration, and really get these features right."

An example where Agile trumps TDD?

On the other hand, if trying to develop software for a vague client, go with agile and forget TDD. There's little value in having a completely bug free system that the client will take one look at and say "y'know, I realise now that what I really want is something else altogether..."

Questions over. Your turn.

[note -- by saying "I'm not against either of these practices, but i'm certainly not a true believer" i realise i'm 'damning them with faint praise'. So be it ;-) ]

 

Essential 'Programming' Skills for Non-Programmers

If you work in IT and you're not a programmer then firstly, welcome... I hope you enjoy the site ;-)

To succeed in IT I think there are still some 'programming' skills that will be worth taking the time to learn.

These are skills that won't die out quickly, and that will help you in general trouble-shooting and comprehension.

Okay -- maybe this is a pompous list... but I welcome any adds, edits, deletes. Here goes.

  • HTML: Dare to 'view source'
  • SQL: Learn to select, from, where, order
  • XML: Learn what makes a document well formed
  • Spreadsheets: Please learn to write formulae in cells
  • Coffee: Now make me a damn coffee you admin nitwit
 

Why Simplicity Pays

For every 25 percent increase in problem complexity, there is a 100 percent increase in solution complexity.

Conversely... for a 20% decrease in complexity -- there is a 50% decrease in solution complexity.

That's powerful stuff.

Fuzzy, dodgy maths, with made up figures, yes...

But POWERFUL fuzzy, dodgy maths, with made up figures!

 

ISV: Independent of what?

The term ISV (Independent Software Vendor) is a funny one -- and it raises the question, independent of what?

I've seen three different answers to this:

  1. Financially independent
  2. Independent of Microsoft (or other large software vendor)
  3. Independent from any particular hardware manufacturer

I suspect the term has its roots in the third meaning -- independence from a hardware vendor -- as it recalls a time when most software was provided as a 'value added' component to the expensive, proprietary computers of the day.

But it's still an ironic and patronising term -- because an ISV is far from Independent.

If we had complete "financial independence" then we'd be able to give our software away, rather than selling it.

An ISV who makes software for Microsoft's operating systems (like, say... us) is naturally very dependent on the ongoing success of Microsoft.

Equally, an ISV who targets a specific platform -- Java, for example -- is dependent on that platform.

So the point is that when you strike out to build an ISV -- you might feel like an explorer, boldly going alone into the wilderness. But really you're a specialist insect, targeting in on a niche software habitat, created in part by large platform vendors (whether they're hardware vendors, operating systems, platforms like Java, ERPs like SAP, or other assorted trouble makers). And you probably also depend on specific 'vertical industries', on current social, economic and technological conditions... all sorts of phenomena which are utterly outside your control.

It's a shaky ecological niche you're headed for. Don't get zealous. Don't get attached, bambi. You can't control the niche you're in much more than an insect can control the elephant he's biting. But before you get stuck on any one elephant, you can be careful about choosing where to sink your teeth.

Bite wisely.

 

TimeSnapper 2.3: Extract Text Using Optical Character Recognition

TimeSnapper 2.3 includes Optical Character Recognitiion

After much effort, we've now released the next incarnation of TimeSnapper -- version 2.3.

(Wondering what TimeSnapper is? Overview here)

The killer feature this time around is Optical Character Recognition. You can now select text straight out of the images you capture.

We've been keen to include this feature since day 1, and finally it's in!

Here's some pictures of it in action...

While viewing a screenshot taken by TimeSnapper -- right click and choose "Extract Text..."

right click and choose "Extract Text..."

The image is quickly analyzed for text. Highlight the words you're interested in...

Highlight the words you are interested in

Press view text... and the extracted words are shown in a "Notepad-like" window.

the extracted words are shown in a "Notepad-like" window

(Or press Copy text to move the text onto the clipboard).

For example, here's the text I extracted in the screenshots above:

Now we're really getting close to one of our original goals:

Never Lose Your Work Again

This feature is already getting a serious workout on my home computer. So many of the programs we use every day do not have any kind of auto-save feature. The Text Extractor in TimeSnapper completely overcomes that problem!

No more slaving over an email for an hour, only to close the browser and lose your work. There's no excuse now.

But also -- we should add that OCR technology is not magic. It doesn't get every letter right, and it can get confused by background images, and other 'artefacts' in the image. If the image you've captured is low fidelity (e.g. jpeg or gif) you're also going to get a lower quality OCR experience. Personally I use png at 100% quality, and I'm happy with the results.

And the credit goes to...

The OCR functionality relies on Microsoft Office Document Imaging -- which is installed with Office 2003 and Office 2007. They in turn licensed their OCR capabilities from TextBridge. So 'Extract Text...' will only work if you've got Office 2003 or Office 2007.

We probably would never have known that these features were available if it weren't for Martin Welker's excellent code project article "OCR with Microsoft Office".

And also, Jon Galloway has been evangelising TimeSnapper, and writing about OCR -- so he gets an honourable mention.

The Unstoppable Feedback Machine That Never Sleeps

Another very helpful factor with this release was the incredible stream of feedback from Adam Cogan. (Adam's a Microsoft Regional director in Australia, and he runs Superior Software for Windows). Adam is a FEEDBACK MACHINE! Nights, weekends, early mornings, emails, IM, phone calls... he's relentless.. unstoppable... i'm just glad he harnesses his energy for good instead of evil.

As a result of his feedback there are numerous improvements to usability, consistency, attention to detail, quality. We haven't yet done everything he's asked for (we're only human) but we're a good part of the way there.

Also, we get a steady stream of emails from people using TimeSnapper who either give us suggestions or make us aware of bugs. In addition to the forum, we're pretty well equipped when it comes to feedback.

But More Feedback is still needed

But of course we still want more feedback. Does the text extraction work for you? Do you need something more from it? Anything that confuses, annoys or delights you with TimeSnapper?

Keep us informed!