The right tools for the job

This post was composed on an iPad. You would think that I would be an early adopter of new technology, of the “camp out in line the night before” variety, and therefore would have had an iPad on Day One. I’ve been burned by that approach before and have learned to let my technical steak rest a bit before carving into it with both halves of my wallet blazing.

The needs and excuses — and missed opportunities — finally got the better of me so I pulled the trigger and made the purchase. White, 32 GB, Wi-Fi only. A dozen apps and iBooks later, I’m now back up to snuff with my geek gadget cred.

Looking back on it, one could assume I’m a true Apple fanboy. I have a G5 iMac, a MacBook Pro, four iPods (a 20GB, two Shuffles, and an iPod Touch), a 3GS iPhone, a recent-model iMac, and now an iPad 2. I like the way AAPL does things and I think they make outstanding products, but ultimately that’s all they are, products, not a religion.

Having said all that, the iPad 2 is far and away the most versatile and well thought out product I have ever used. It’s damn cool, and you can quote me on that.

No drama

Late last week I deployed an upgrade to our intranet that I have been working on for nearly 11 months. It was probably the single largest web development project I have ever completed on my own. A great deal of the changes and improvements were under the hood. I refactored most of the classes that I use to manage data, added a lot of new functionality, and streamlined many common functions and methods used throughout the site.

I even changed the color of the menu bar. Whoohoo!

Considering the size of the upgrade, and the size of the website itself – 1.2 million lines of code – the upgrade went about as good as could possibly be expected. Only one or two minor bugs turned up and they were squashed in less than 30 minutes. Feedback from my users has been very positive, or nonexistent. Let’s consider that last statement for a minute.

If something changes in a major way and no one comments about it, is that a good thing? I believe it is. Most people tend to only speak up and express an opinion when they are unhappy. Otherwise they go about their daily business without comment. It is sort of a “no news is good news” situation.

This upgrade had a two-month alpha testing and beta testing process. It also involved a great deal of user feedback during the design and conceptualization phase. I didn’t just go into my office, close the door, and build it in a vacuum. That’s folly! (Not that I have ever done that before…) Although I did not involve everyone’s opinion on this project, I sought out a representative sample of opinions to cover as many bases as possible while bugging as few users as I could.

Even after the upgrade went live, it continues to be an organic process, continuing to evolve, change, and improve. I have always said that it is easier to edit than to create, and this website is a classic example. People can look at conceptual drawings and mockups all day long and have zero inspiration. But when the site goes live, they are suddenly inspired and say, “Hey, can it do this?” Although all projects have a beginning, middle, and end, I believe development projects such as websites should never truly be considered “done.” They are living, breathing things that should continue to evolve as needs change, situations shift, and opinions vary.

As I consider the feedback on this project, I have adopted a new mantra: “No drama.” Whether I am implementing an IT project or a new website, my goal is to have the deployment occur with as little drama and excitement as possible. Anti-climax is the pinnacle of success.

If Dr. House worked in IT

I enjoy watching the TV show, House. Dr. Gregory House solves medical mysteries with a team of other brilliant doctors. He does so using a process called differential diagnosis, where they outline symptoms and test results and use a process of elimination to narrow down a diagnosis. I laugh a little bit when I watch these shows because they go through the same diagnostic process IT people like me go through when solving technical problems. A recent issue with connectivity where I work illustrates this comparison.

Many of our users access their e-mail through a web interface. It is fast, easy-to-use, has a lot of features, and doesn’t eat up memory by running a separate application on the user’s computer. The only real downside is the user cannot access their e-mail if they do not have an Internet connection. Lately, access to this webmail interface has been slow for some users. Unfortunately, this slowness has not been consistent. As any IT person will tell you, intermittent problems are the most difficult to diagnose. The ideal is a situation where steps 1, 2, and 3 consistently produce the same error condition. This makes diagnostics substantially easier.

I went through a four week process of elimination trying to identify the source of the problem. I implemented my own differential analysis; did it only affect remote users or were local users affected as well? Was the problem only experienced by Windows users or did Mac users experience slowness also? Was the issue associated with a certain time of day or certain day of the week? Did the users choice of web browser matter? We had recently changed the software running on our firewalls so also I spent two weeks experimenting to see if the firewall software was causing the problem.

Through empirical experimentation, I slowly eliminated possible causes until the remaining list was very short. When it comes to diagnosis of IT problems, we often conclude that the source of a problem is likely to be something, but we are very rarely ever certain of it. I had concluded that our slowness issue was most likely a problem with our Internet connection, although I could not be certain of it. I contacted our Internet service provider and let them know what we were experiencing as well as the possible causes I had eliminated from consideration. They ran a test on our Internet connection and saw the same results: inconsistent periods of slowness, almost as if our bandwidth was pausing to take a breath every few seconds. They sent a service tech to our location who installed a new gateway running updated firmware. This solved the problem.

The solution to problems always seems easy after the fact, but reaching that conclusion can often be a long, frustrating experience. Over my career I have learned that a few simple rules can help reach a solution quicker and with more assurance that it is the correct solution (there can often be more than one solution to a given problem, although there is usually one solution that is better than the others). My number one rule is to only try one thing at a time. If you make more than one change and end up solving the problem you won’t know which one was the solution. Also, it is important to identify symptoms that are consistent. If a user performs action X, Y, and Z in that order and always gets the same result, this is more helpful than a process that sometimes works and sometimes does not. Remove from the list things that you can definitively identify as irrelevant. This helps narrow your focus and avoid distraction.

When Dr. House cures a patient, he sometimes brings them to the brink of death before the solution comes to him. It makes good TV but he wouldn’t make a very good doctor in real life. IT is very much the same way. We need to solve problems quickly but in a manner that reduces the impact on users as much as possible without causing harm to the systems we are trying to fix. We also try things that can be reverted if they don’t work out; backups are a great example of this.

There have been a few times in my career where problems are resolved without ever identifying the cause. I may know how to fix something without knowing what is causing the problem in the first place. That is okay, however, because the solution is ultimately more important than knowing the cause (although knowing the cause helps you prevent the problem from occurring in the future). Finally, I have learned that what does not seem rational or likely may still be possible. Sometimes the most bizarre and unlikely scenario you can imagine really is the cause of the problem. That’s another story altogether…

A good craftsman never blames his tools…

But having good tools sure makes the job a lot easier.

I recently upgraded from a 17″ Macbook Pro purchased in late 2006 to a brand new 27″ iMac with the i7 quad-core 2.8 GHz processor, 16 MB of RAM, and a 2 TB hard drive. This computer is beautiful and screaming fast. The main reason for upgrading was I needed more memory and processing power for the types and number of applications I run simultaneously. My Macbook Pro has served me well but my memory was swapping to the hard drive the majority of the time and the processor just couldn’t keep up with all the apps I was running. I maxed it out at 3 MB of RAM (the 2.33 GHz processor couldn’t handle any more than that) and when I’d run Windows 7 or XP in Parallels, it would red-line my memory graph in Activity Monitor.

Where I work I not only develop our web sites, I also provide tech support to our staff which is why I had to run Windows in Parallels. At any given moment I’m doing three things at once, so I need a computer that can keep up. The souped-up iMac is by far the best tool for the job.

Upgrading couldn’t be easier

Mac OS X’s Time Machine backup solution is brilliant. It runs in the background quietly and reliably without slowing down my computer. It is easy to use when restoring files and has saved my bacon several times. The biggest advantage of Time Machine, however, came when I wanted to migrate my files and programs off of my old Macbook Pro onto my new iMac. When the iMac first booted up, it asked if I wanted to restore everything from a Time Machine backup. I plugged in my external Time Machine backup into my iMac’s USB port, selected the drive when it showed up in the list, and let it run for about an hour and a half. When it was done my iMac was set up exactly the same as my Macbook Pro. Everything was there: my applications, my files, even my keychain full of passwords. The folks in Cupertino really have their act together, and it’s things like the Time Machine transfer that really make Macs worth quite a bit more than their price.

Why are man-hole covers round?

How do you kill two birds with one stone?

Throw a rock at the first bird. The dead bird and the rock both fall to the ground. Pick up the rock and throw it at the second bird.

Do computers multiply and divide?

Not really, but they add and subtract multiple times, really fast.

How do you measure the height of a building with a barometer?

Tie the barometer to a string and lower it from the roof down to the ground. Measure the length of the string.

What is the point of all these weird questions?

I’ve been asked questions like these when interviewing for software developer positions, and have asked questions like these when interviewing prospects when I was a hiring manager. After a while I came to the conclusion that these riddle type of questions probably goes back to Dungeons and Dragons days, when players faced unusual challenges that required unorthodox or creative solutions. How do you get past a frictionless room with two large, deep pits in the middle, filled with razor sharp spikes? Flood the room and float across.

The point isn’t to test someone’s knowledge of bird trajectories and the aerodynamic characteristics of pebbles, although knowledge of the air-speed velocity of small, migratory birds can be useful at times. These questions seek to determine someone’s ability to think on their feet, to solve problems they’ve never faced before.

Programmers face situations like this on a daily basis. We may not have to be avian killers, but we run into challenges that are unfamiliar all the time. To be a good programmer you must know more than just code syntax and common algorithms. You need a creative ability to go right by turning left three times. If a file won’t pull down to the client, push it to them from the server. If a regular expression won’t find the string you’re looking for, seek out what the string doesn’t look like instead.

Marketing people and buzzword-happy suits completely ruined the concept of creative problem solving by giving it a catchphrase: thinking outside the box. That’s a shame, because as soon as you give something cool its own catchphrase it quickly ceases being cool. The ability to creatively solve problems, however, is a Jedi power that all great programmers possess, even if it isn’t cool to admit it. Yes, programmers are creative. There, I said it. We are creative.

We just do it with 1′s and 0′s, billions of times per second.

Your virtual estate

If you die, your Will designates how your estate will be distributed. Your home, your car, your money, your investments… all of your assets get allocated to your heirs. But what about your electronic assets? I’m not just talking about your computers but your virtual assets as well.

I have read about families of soldiers killed in Iraq and Afghanistan struggling to obtain access to their e-mail accounts and Facebook pages. Many times the departed have relationships online that their families are completely unaware of. How are these people notified that their friend is no longer with us?

Designating our wishes after we leave these mortal bodies is something most of us take for granted. But we often limit our thinking to only tangible assets. We need to start thinking of our virtual assets as well.

Does that mean we have to put our Gmail password in our will? Not necessarily. However, it’s a good idea to include things like account login information in our estate documents so that those that take care of our assets after we are gone can implement our wishes accordingly.

Like most people, I have several online accounts that I use on a frequent basis. These include online banking, e-mail accounts, blogs, special interest forum accounts, and several others. I also have relationships with people that are purely electronic; I have never met these people in person but maintain relationships with them via chat and e-mail. In the event of my passing, I would want them notified.

When I first began thinking of how to handle my electronic assets, my original idea was to print out login information and other wishes in a document, sealing it in an envelope, and putting that in my safe deposit box alongside my will. Because I change passwords on a frequent basis, however, this approach would become tedious very quickly. I would have to print out the document every time a password changed or I added a new account, I would have to go to the bank, take out the old one and shred it, and replace it with the new one. This would not be a very efficient approach.

A better way is to put all of my login information and wishes into a document and encrypt it with a password. I would write that password down on a piece of paper, seal it in an envelope, and put that in my safe deposit box. Whenever a login password changes or I add a new account or wish to change my wishes, I simply update the document and continue to encrypt it with the same password. It is sort of an electronic version of, “break glass in case of fire.”

[It's a good idea to make backups of that file; computers die, too.]

In the event of my death, my wife would go to our safe deposit box and break the seal on the envelope to get the password. She would decrypt the file, view the document, and follow the directions it contains.

This approach could be used in the workplace as well. Employee passwords and access to sensitive documents could be encrypted using the same approach, to be accessed by their supervisor or other authorized personnel in the event of that persons termination or other departure from the company.

It is rather surprising to consider just how many accounts the average person may have access to without even realizing it. There are probably a dozen different accounts that I use on a daily basis, and perhaps two dozen more that I access infrequently. I remember most usernames and passwords, but for accounts that I seldom access I look them up in a password file that I keep encrypted. I have already been using this approach for many years, the only difference is I would write down the password to that encrypted file and put it in my safe deposit box rather than keeping it only in my head.

Here is a brief list of just some of the types of accounts and other pieces of information a person should have in their encrypted document:

  • e-mail accounts
  • online banking accounts
  • instant messaging accounts
  • computer login passwords
  • vendor accounts, such as Amazon.com or other shopping websites
  • forum and community websites
  • blog accounts

In addition, the e-mail address or other connection information should be listed for every person the deceased wishes to be notified in the event of their death. Often this can be accomplished by simply exporting their address book into a file. It’s a good idea to list the type of relationship you have with that person; friend, coworker, vendor representative, etc.

Finally, it would be helpful to include text that you wish to be posted on your blog or Facebook page in the event of your death. This can be a farewell to friends, a tribute of some kind, or any other message you would want conveyed to those left behind. It would basically be your electronic obituary, in essence. You could even include a video of yourself speaking to the camera if you wanted. Many adults suffering from terminal cancer film themselves talking as if to their small children, to be played to them when the child gets older. There is no reason a person couldn’t do this for their friends as well. It is all up to the individual what they want done. It is also a good idea to specify anything you specifically do not want done.

  • If you use a Mac, follow these instructions from Apple.com to create a securely encrypted ‘disk image’ using OS X’s built-in disk image utility. You can use it to store a plain text file containing your passwords, a document containing your wishes, a PDF copy of your will, or any other document or file you wish to keep secure.
  • If you are a Windows user, there are many third-party encryption tools available, most of which are free. TrueCrypt is probably the best all-around encryption utility available. It is free, open-source, and works on multiple operating system.

Death is not a pleasant subject but it is something we all must face, and unfortunately it usually happens without warning. Planning for what happens to our estate – both tangible and virtual – is very important.

Testing, testing, 1-2-3

After several months of refactoring and upgrading our Intranet, I’m now in the testing phase. This is probably the most tedious — and therefore the most difficult — part of software and web development. I would rather go through the entire codebase and change double-quotes to single-quotes than test it. However, testing is arguably the most important part of the entire development life cycle.

There are a million ways to test a web site, and almost as many aspects of it that need to be tested. One way is to go to a page and test all the different aspects of that page; e.g. broken images, broken links, content spelling and grammar, element formatting, functionality, etc. I don’t subscribe to that approach. Rather, I go over the entire site testing one thing, such as looking for broken images, then go over the entire site looking for something else (broken links, etc.) I can fix all the broken images as I find them, note each broken image on a list and then fix them in a batch, or I can just save that list until the entire testing process is done, then wash, rinse, and repeat from the beginning.

I’ve written about testing before. In many ways, I believe it’s not a good idea for a developer to test their own code. It’s similar to the theory that writers shouldn’t edit or proofread their own text. Since I’m a web development team of one, I don’t have a lot of choice when it comes to handing my work to someone else and saying, “Here, test this.” I have to do it myself. I make efforts to separate the development process from the testing phase by not touching the code when I’m testing. I go through the site and run my tests, making notes as I go, but I never fire up my IDE until testing is done. This makes sure I’m in a testing frame of mind. It helps to decouple my sense of ownership over the code and be more impartial about the testing process. This is very important.

When you are building and maintaining web sites by yourself, you have to pay attention to tricks like these to get the most mileage out of your effort.

Refactoring and seeing the tree’s rings

I’m in the middle of a large effort to both deploy new functionality and refactor old code in the Intranet I built for my employer. I’ve got close to 700,000 lines of code to review and as I get into the older code I feel as if I’m reading the rings of a large tree and getting closer to the center with each day.

It seems every day I learn something new. I can write PHP code, but as any programmer knows there’s always a half-dozen ways to skin the same programmatic cat. Most get the job done but some ways are better than others. As I learn better ways of doing things I implement those methods in new code, but some are worth going back through the old stuff for a retrofit.

As I peruse my old code I remember my rationale for the methods I used. It’s not so much a, “Why did I DO that?” kind of thing, but more of a reminder that sometimes it’s worth taking the time to ask, “Is there a better approach?”

As a general rule I put comments inline whenever something isn’t immediately obvious. This helps tremendously when reviewing code I wrote one or two or three years later. I also put comments next to variables or functions that are defined elsewhere, indicating where to look for the parent code. That has been a big help as well.

As I code in the present, it’s valuable to remember to think of the future. You probably don’t have the programming chops today that you’ll have in a few years, and that’s okay. No one does. Do the best you can with the skills you have (or can look up) today. But code in a way that makes it easy to go back and clean things up when you get the chance.

Codeworks 2010

Yesterday I attended the Portland session of PHP|Architect‘s Codeworks 2010 PHP developers conference. Unlike the previous year’s event which lasted two days, this year’s one day event was a little lighter on actual how-to’s and instead focused more on concepts and inspiration. As Marco Tabini put it, “You should come away from this conference with more questions than you had when you arrived.”

Despite the reduction of how-to substance, I still came away with some valuable insights and — thank you, Marco — inspirations. As was the case when I attended Codeworks 2009 in San Francisco this same time last year, the bulk of what I gained was a needed push toward a more formal development life-cycle. I’m a one-man development shop so I bypass or skip several processes that most multi-member development teams rely on, such as scripted deployment, code analysis, and unit testing.

One of the questions I raised during D. Keith Casey’s session on “Unit Testing Strategies” was, “As any writer knows, it’s not a good idea to proofread or edit their own work. Does the same apply to programmers testing their own code?”

Keith felt the same principle applied. It’s always a good idea to have someone test the code other than the developer that wrote it. However, what about one-man teams like myself, “Test-Driven Development (TDD) works great for that because you write your tests before you write code.” It makes you a tester that writes code rather than a coder that tests.

Compared to the other sessions by Keith Casey and Cal Evans, I felt like Marco’s two sessions, “Learn to Love the Front End” and “Object-Oriented Programming in PHP 5.3″ were more cerebral and theoretical. Marco covered two topics that many people take for granted with an insight that reminded me of a veteran college professor. The depth of Marco’s knowledge and understanding of computer science was apparent. His explanation of the progression from commodity products to the ‘experience’ end of the product spectrum was brilliant (malted barley vs. sharing a pint with friends at your local brewpub). His analogy was even more poignant considering he doesn’t drink.

Cal’s coverage of “API’s from the Ground Up” over-focused on REST-ish ‘verbs’ and response codes — a mere slice of the API pie — yet didn’t even touch on the why of creating an API. He never answered the unspoken question, “Why would I write an API?” It almost felt like he threw together the presentation at the last minute on the topic without really thinking the whole thing through. I understand that these sessions were not supposed to be end-all, beat-all training programs, but if you’re going to give your audience only a thin slice of information on a topic, at least spend 30 seconds telling them where that slice resides within the big picture. I did learn some things about APIs, but wanted more. To his credit, however, Cal has the kind of personality and presentation style that makes it an enjoyable experience. He could read the phone book aloud and make it fun.

Keith Casey’s presentation on scalability was insightful. Unlike Cal’s session on APIs, Keith let the audience know why scalability mattered. He then gave specific areas to look for bottle necks and discussed broad methods — and some specifics — used to correct them. The session was intriguing and stayed true to their goal: motivate us to ask more questions and dig deeper on the subject.

Cal’s second session of the conference, and the last one of the event, “Five Tools Every PHP Developer Should Love” did in fact cover five tools but the usefulness of some is debatable. phing is probably the most useful tool to me, as it helps automate many other tools and tie them together, enabling the development team to create formal deployment processes. That is something I don’t follow, being a lone coder, but I intend to change that.

I suppose that’s the nature of these conferences. Their value can be measured in the worth of a single tidbit or fragment of inspiration. After attending two year’s worth of Codeworks conferences I can wholeheartedly say they are worth the time and expense to attend, even if it’s in a city far away. Not every session’s topic will apply to every attendee, but it’s not feasible to expect it to. Also, the presenters have day jobs writing code and managing teams, so their presentations tend to be more about the content than the polish. I actually think that’s a good thing.

Day Camp 4 Developers

daycamp4developers.com

On Saturday I participated in a webinar called “Day Camp 4 Developers“, hosted by Cal Evans and sponsored by AutoMD, Microsoft, VACO, Ascendant Staffing, EICC, and PHP|Architect. Thank you to the sponsors for making the event possible and to Cal Evans for organizing and hosting it. Great job!

This webinar included presentations on soft skills needed by developers rather than code-centric how-tos. The bang-for-the-buck on this event made it worth a great deal more than the actual cost of $35, and was a great way to spend a few hours on a Saturday.

Presentations included “Open Source Your Career” by Lorna Jane Mitchell, “Technical Writing 101″ by Elizabeth Naramore, “Rocking Your Resume” by Scott Gordon, “Actively Managing Your Career” by Brian Prince, and concluded with “So You Want to be an Architect?” by Josh Holmes.

The event was presented using GoToMeeting’s Webinar capabilities and other than a minor technical glitch during the lunch break it went off without a hitch. What I found fascinating was that every participant, both presenter and attendee, was in a different geographical location. The entire thing was conducted over the Internet and no travel expense or time was involved. I still gained the benefit of the talks without the hassle, time, and cost of traveling to an in-person conference. Granted, I don’t get the benefit of off-line conversations in the hallway and lunch room or bar afterward with the speakers and other attendees, but considering the very low cost it was still a very worthwhile experience.