On the Path

The Power of Accountability

Many years ago when I was getting my red cross first aid certification, I learned that when starting any kind of emergency first aid where additional help is needed, you pick someone out of the crowd, point at them, and say, "YOU!  GO CALL 911!".  The theory goes that by singling someone out, it is more likely to happen.  If instead, you just yelled, "SOMEONE GO CALL 911!", the crowd would continue to gather and watch.

I am reminded of this constantly when working with teams who coordinate most of their interaction in email.

Today we had a question about some important batch processing that was supposed to complete last night.  Someone wondered if the processing completed as expected and sent an email to three people.  The most senior person was out of the office but saw the email and thought, "One of those other two guys will get it". the mid-level person was off on another project for the week, saw the email and thought "I'm out of the loop - one of those other guys will get it".  And the third-level person saw the email and thought "With those two other guys on the list, It's not really my place to respond", and the person needing an answer just got radio silence.

A simple change would have made this go a lot differently - hold one person responsible and ask them.  At the very least, send the message 'to:' one and cc: the others.

This is not an uncommon occurrence - I think of this analogy all of the time; today I decided to write about it.

Success after Failure

Programming is often a love-hate affair. One of the joys and frustrations of software engineering lies in integrating with external systems. There are tremendous upsides:

• you don't have to write or maintain complex code
• ou don't have to keep a service running

But there are also down-sides:

• getting the external system to do exactly what your system needs
• dealing with bugs and problems with the other system

I have recently been working on a project that integrates with QuickBase, Intuit's "in-the-cloud" database with a simple web-based UI for administration and data entry. Recently we experienced some serious downsides (see QuickBase service outages). This is the story of how I came to terms with some of those downsides.

So how do you help protect your app when there are problems with external systems? Defensive coding!

1. Awareness.
   Anticipating failures goes a long way to dealing with their eventuality.

   If your DB goes down, can you continue to limp along using cached data? If your email service becomes blocked, can you re-configure to use another service quickly? If you hosting provider is shut down by law enforcement, can you redeploy to a new machine from another provider?

   You can't solve all problems, but it's vital to know all single points of failure that affect your system.

2. Read the APIs and make sure you handle the exceptions
   The people who write those APIs are kind enough to clue you in to the possibility of things not going the way you expect. There are some exceptions you can handle. Make sure you handle these!

   In the case where you can't do anything to move forward, log the exception and send out appropriate notifications (via popup dialogs, emails, etc.) that let's users and developers know where the failures lie.

3. Try and try again.
   Someone once told me this was the definition of insanity! But experience has taught me otherwise. Some service failures are temporary in nature. I have seen log entries indicate success on that 2nd, 3rd or 4th attempt for network calls. Try using a back-off strategy like that in the rescue_me gem.

4. Point the finger of blame.
   When all else fails, it pays to make it apparent to your clients that you know exactly what is going on, and it's not your wonderful code that has failed them. It still does not help the poor end users, but you may live to code another day.

Facebook Places

You may have heard about Facebook's answer to FourSquare, their recently released Facebook Places. As Facebook says, Places will make it so you "Never miss another chance to connect when you happen to be at the same place at the same time." and allows you to "Easily share where you are, what you're doing and the friends you're with right from your mobile." This new feature lets you check in with friends using GPS from your mobile device to share where you are in real time, and find out which of your friends may be nearby. It also allows you to geotag friends, and allows your friends to geotag you.

Fun?  Yes, probably, but there clearly are security issues involved, so think carefully about whether you really want the world to know where you are. Letting Facebook know that you are miles away from your home could act as an open invitation to the more unscrupulous to pay an unwelcome visit to your home.  Many users of Twitter's similar geotagging feature have also expressed concerns about letting potential stalkers know of their whereabouts. Fortunately, if you favor privacy, you can disable some or all of the geotagging features, and LifeHacker provides some great instructions on how to do this. Now that you can share what you want to share, have fun with it!