On the Path

The Power of Names

I'm working on an interesting project with a few other Sherpas (Arild and Charles) - building an inter-office workflow and legal reporting tool in Ruby on Rails.  Arild and Charles were just having a conversation about a named scope, and what they should call it (a named_scope is a construct in Rails that, basically, returns a collection of objects that match certain criteria).  The purpose of this named scope was to return the number of 'change records' the system had to deal with in a certain time period.

The actual business requirement says "5 minutes"... but being experienced at this kind of thing, we know that when requirements specify absolute values, those are usually the things that change.  Taking it into account now is cheap and easy.  Trying to account for it the week before launch, or when a legal report is overdue, expensive and painful.  So they decided to parameterize it, and they came up with something that looked like:

Record.changed_in_last_x_minutes(5)

They looked at that for a minute and asked me what I thought.  "It's not hateful..." I said, and they both laughed.

It was a diplomatic answer.  We have all spent time in languages and projects where that would have been perfectly acceptable; but in the past few years, developers have spent a lot of time concerned about aesthetics, readability, and maintainability... and this just didn't seem to live up to that expectation.  I was proud of them just for asking - most developers would have just coded it up and moved to the next problem.

"I'd like to have it read like a sentence, and that just doesn't do it for me", I said.

Arild said, "I agree... but the problem is trying to get the number value at the end of the sentence with the 'minutes' in there"...  and both of our eyes lit up.  I think I managed to verbalize it first, but we both had the same flash of insight...  we wanted to pass in a duration, not a number.

Thirty seconds later, and we had an API that looks like:

Record.changed_in_last(5.minutes)

and that api also works now for variations like changed_in_last(2.hours), changed_in_last(47.seconds), changed_in_last(5.days), changed_in_last(2.weeks), and just about every variation you can imagine.

A brief conversation, a small insight, and an additional 30 seconds of code, and we have a fluent interface that can easily implement many requirement variations.  We can also show to a client and ask "is this what you mean?".

It might seem like one small change, but these add up.  Imagine an entire system designed that way - where the code begins not to read so much as code, but more like a formal description with a lot of flexibility...  A system where the developers care what the names of things are and how they read with that attention to detail.

As soon as we had it done, I remembered this story by a friend of mine.  I'm happy to steal his title and write this story as an homage to the point he so elequently made.

Cool kids don't store plain-text

or How I simplified database.yml to avoid storing sensitive passwords in plain-text

Nobody likes putting important passwords in a plain text config file. So what are your options?

1) Use a one-time, throw-away password
I typically do this for most Rails projects that I work on (e.g. user: "acme", pass: "acme")

2) Prompt for the password
Unfortunately there are times when access is tied to a real password. On my current project, the DB password is the same as my email password (served via LDAP).

Prompting in a config file? Yes, this is available by default in Rails, and can be done easily in other Ruby projects. This is all you need to get a command-line password prompt when loading database.yml:

  # Within config/database.yml or your custom config file...
    <%
      # Cool kids don't store plaintext
      require 'highline/import'
      def prompt_for_password
        @passwd ||= ask("DB password: ") { |q| q.echo = false }
      end
    %>
  development:
    host: somehost.example.com
    username: arild.shirazi@example.com
    password: <%= prompt_for_password %>

If you are not using Rails, you will need to evaluate ERB within your config file whenever you load it:

  # If loading your own yaml outside of Rails
  config_erb = ERB.new(IO.read config_file)
  YAML::load config_erb.result

BONUS: When using this method, your checked-in config/database-example.yml could be setup to work for all users out-of-the box. Just add some ERB to determine the correct mysql socket to use (<% system(mysql_config --socket) %>), and create a rake dependency that copies database-example.yml to database.yml when needed and you're good to go.

NOTE: Feel free to roll your own prompts:

  stty -echo; pass = gets.chomp; stty echo

But using the highline library really improves readability. Also, using highline is a great way to promt from inside rake scritps.

CSS Frameworks and CSS Systems

Several years ago I wrote a blog entry entitled YUI Stylesheets are Awesome! I stand by their original awesomeness in saving me from the perils of IE6 hell, but this tradeoff is ugly. In using a grid system with positional CSS, you lose all the semantic meaning CSS can bring. The short term gain, in practice, just led to other long term maintenance problems. There is a better solution, and I want to show it to you.

There are a lot of these frameworks now - YUI, Blueprint, 960gs, and umpteen others that follow the same basic ideas. These CSS Frameworks are all about positioning... splitting the page up into a number of columns and letting you lay out <div>s on the page. As a newcomer to CSS they get you up and running fast, but miss a much bigger picture. In the end, they are nothing but ways of 'tricking' CSS into acting more like tables than semantic markup.

Semantic Markup? You Keep Saying That...

Lets figure that out by looking at the role of the HTML writer.

Imagine I'm writing a website for a new coffee shop. I'm going to design the page for the menu. Like most menus, items are grouped according to types. Maybe "Hot Drinks", "Cold Drinks", "Desserts and Pastries", "Sandwiches", and so on. That HTML writer, using a CSS framework that is all about positioning, might have a page structure that looks like this (any html tags beyond structure are omitted for clarity):

<div class="columns_12" id=menu">
  <div class="columns_6 alpha" id="hot_drinks">
  </div>
  <div class="columns_6 omega" id="cold_drinks">
  </div>
  <div class="columns_10 alpha" id="deserts_n_pastries">
  </div>
  <div class="columns_2 omega" id="dessert_pic_holder">
  </div>
  <div class="columns_2 alpha" id="sandwich_pic_holder">
  </div>
  <div class="columns_10 omega" id="sandwiches">
  </div>
</div>

So we have IDs that represent the real content, but the classes denote something about the positioning within the page. In this case we have a 12 column container named 'menu', and that contains a series of divs that all add up to 12 columns, with 'alpha' indicating it is the first, and 'omega' indicating it is the last. For the most part, minus some bells and whistles, this is how these stylesheet frameworks work.

So why is this a problem?

Imagine the poor html designer when the coffee shop redesigns their menu. At first, some simple positional changes are easy to make... "Move this picture over there... put the hot drinks on the other side..." and so on. But over time, these changes begin to ripple across more and more pages ("Move the logo from the left side to the right side" becomes an edit on a lot of pages). The client asks for things that can't easily fit within the grid system, and that html designer is now left writing CSS that now has to interact with the framework well. Reading the CSS, it never says how to style a MENU.

Enter Semantic CSS - the "CSS Systems" approach.

Why include all that column stuff in your html? After all, we already have these nice IDs that indicate what each section is... furthermore, I might want to use classes to indicate exactly what something is too (I could see a CSS class that wraps every drink as a <div class="hot_drink">, for example, leaving the ID as something that Javascript could uniquely identify the drink by).

Somebody has already explained this better than I can. This presentation by Natalie Downe is well worth the time to view it.

The Best of Both Worlds

I sure like the idea of a semantic CSS system, but shucks - I sure like relying on all that nice CSS that is tested and takes away all the cross-browser pain for me. Is there a way to have my CSS and uh, eat it too? Yes, there is. Take the hour and watch this intro video to Compass, a great CSS editing and composition tool.

The author of Compass has turned many of these grid frameworks into 'mixins' so you can still use the grid system and tested CSS, but you can eaily turn it into a semantic CSS system specific to your project. Want to move around the page contents? The HTML doesn't have to change! VOILA! All the benefits of CSS back in your hands. This can make a BIG difference when the bulk of your html is generated by something like php or Ruby on Rails. Your generated HTML stays the same - no code changes!

A side benefit of using Compass is the use of SASS, a language for writing stylesheets that lets you do all kinds of cool stuff in a syntax a lot nicer than CSS. Again, the screencast shows that.

Puzzle Pieces that Fit Together Nicely

Are you a Rails developer? I knew you were. Guess what? All this fits together with Rails like perfect little puzzle pieces. Inside of erb, the templating mechanism behind Rails' html pages, we can say things like:

<%= div_for(@hot_drink) %>

and get back html that looks like:

<div id="hot_drink_43" class="hot_drink">

Seems kinda trivial, until you realize that now you can have CSS class definitions that match your domain model, which goes a long way towards that "CSS System" ideal. Further, you get a unique ID that represents the object in the div, which you can now use Javascript and the DOM magic of Prototype or JQuery to do Ajaxy-type effects on your page contacts. When your CSS is driven in large part by the display of your domain model, the CSS itself becomes clean, clear, and it is easier to find 'dead' CSS.

Wow.

Rails 2.x Changes to Separator Parameter on render_partial_collection

I recently upgraded a good sized Rails 1.2.6 app to Rails 2.3.4. Peter Marklund's excellent upgrade notes were an incredible help and for the most part the upgrade was fairly straight forward. During the upgrade, I changed many calls of: 

  render_partial_collection

to:

  render :partial=>'', :collection=>''

The one piece of documentation I had a hard time finding was what became of the separator parameter on the now deprecated render_partial_collection. After quite a bit of digging, I found that it's now called :spacer_template. So,

<%= render_partial_collection "category", @post.get_category_hierarchy, 'bar_separater' %>

Became:
<%= render :partial=>"category",
           :collection=>@post.get_category_hierarchy,
           :spacer_template=>'bar_separater' %>


Hope this helps anyone else in the same boat!

How is your journey through life going?

I started using electronic communication earlier than most - I think I sent my first email in 1983, and by 1987 was juggling several email addresses.  I graduated in high school before most people had computers, but by the time I graduated college, America Online was covering the planet with floppy disks...  As a result, I have kept in electronic touch with some college friends for ~18 years, but most high school friends have deteriorated to faded memories.

Thanks to social networks like Facebook, I'm running into more and more friends from high school, as well as acquaintances in college that I could have easily never heard from again.  The contact is nice, but saying "hey - how are you doing?" when referring to the last 20+ years seems... trite.  We need some new societal convention to handle this - something that acknowledges the passage of time that will include the highs of weddings and children, the lows of funerals and lost relationships, and all the little 'life events' in between.  Something along the lines of "How is your journey through life going?", but without sounding like some Jedi philosophy.

Upcoming Public Appearances

Of all the public speaking I do, the NFJS Conference Series has to be some of the most enjoyable. The venues are always nice, the crowds always intelligent and fun to interact with, and the speakers are all social and personable. Between then audiences and and other speakers, I think I learn as much as I teach at these shows. If you haven't been to one of these, you should... They are the gold standard people in the conference world judge against. I have been privileged enough to have been involved in one way or another since 2003.

This year I haven't spoken at nearly as many as I have in the past. Work and home commitments make it hard to take that many weekends. I'm happy to announce that I'll be speaking at the Greater Atlanta Software Symposium, October 23-25, and the Northern Virginia Software Symposium, Nov 6-8. Time permitting, I might be able to speak in another one this year too - we'll see.

I'll be covering some sessions on Agile Estimation Techniques, Career advice for developers promoted into management, and a couple of sessions on project-integrity tools for Java projects (like build tools, test coverage, metrics, and architectural quality).

I hope to see you there!

Installing CentOS 5, ImageMagick, and RMagick, September 2009 Edition

For some reason, this process seems to get more convoluted with time, not less. Before you go through this pain, let me ask you, "are you sure you need to use ImageMagick and RMagick?" There are other options in today's Rails environment.

I wrote about this previously on my older blog. While just going through this pain again, I found several things:

• CentOS has been updated, but still relies on older-than-dirt gems for ImageMagick. Still, there is something to be said for relying on stock gems rather than messing with building and installing yourself.
• The version of RMagick I specifically call out in the old blog post isn't available anymore.
• The newer version of RMagick adds a new default html documentation option you need to turn off.

So sit back, relax, and read my previous blog post, now updated to work as of September, 2009:

I don‘t normally blog about obscure, specific technical topics, mainly because 99% of the time I find a blog entry someplace that has already covered it. This is an exception, as the amount of information out there on this topic is confusing and dated.

Installing ImageMagick and RMagick on CentOS5 seems to be a bit of a dark art, mainly because of the amount of stale information and differences between linux distributions. This entry will eventually be stale as well, so check the date and version numbers – your mileage may vary.

The ImageMagick/RMagick combination is arguably one of the coolest yet painful combinations of ruby/native code out there. There are a lot of attempts to replace, simplify or otherwise solve this pain, but today, rmagic is often a necessary tool in the rubyist‘s toolbox. So here‘s my attempt to lessen that pain.

I'll include details and explanations with each command line. I have you as a captive audience, and the details will give this post more relevance if the version numbers have moved on since this writing. There are only 3 commands, so bear with me.

One more caveat – I‘m taking the short path using the most recent CentOS5 packaged versions. If you need the latest version of ImageMagick, have fun compiling it all yourself. There are plenty of sites out there with directions. For me, one of the reasons I‘m using CentOS5 (with this client) is for the stability and ease of package management.

1. First we install ImageMagick as an rpm using yum. As of this install, the latest CentOS package is 6.2.8.0-4.el5_1.1. This is lagging considerably from the newest version, and that causes some of our issues. Use sudo on these commands as appropriate for your user management style.

   yum install ImageMagick

   This will ask to download and install all kinds of stuff (43 other packages on a stock CentOS server install). Let it install whatever it wants. If the version number of ImageMagick is up to the 6.3.x range by the time you read this, then this post probably isn‘t want you‘re looking for. (although explanations below will still help you). If you are on an x86_64 box, this will also install that rpm file for you.

2. Now install the ImageMagick-devel library. Yes, that could have been done in the same step above, I‘m just trying to be explicit for your benefit.

   yum install ImageMagick-devel

   That wants to install 16 packages on my stock CentOS5 server. This also does the right thing os x86_64.

   So no surprises so far. It is the ruby gem that causes us all the problems...

3. Install the RMagick gem. I‘m about to make this look easy, but it wasn‘t. The first problem is that the latest RMagick gem needs the 6.3.x series of ImageMagick, which isn‘t packaged for CentOS5 yet. So I had to determine the latest version of the RMagick gem that works on the 6.2.x series. As of this writing, that is 1.15.17, and you can find that out at the RMagick gem page.

   But installing that just might fail in a blaze of glory that only a software engineer with a minor in forensic investigation would enjoy. Near as I can tell, the install of the gem would succeed if it weren‘t for some failures building the documentation. These failures vary from machine to machine, as it seems to depend on what fonts you have installed (which might depend on what version of OpenOffice, etc). Solution? We dont need no stinkin' docs! Furthermore, sometime between the last time I did this and now, they have a new option to disable the html documentation. This has to be passed through the gem installer to their script (this the obscure little '--' sitting all by itself in the command.)

   gem install rmagick -v 1.15.17 --no-rdoc --no-ri -- --disable-htmldoc

   a minute later, I get the output:

   Successfully installed rmagick-1.15.17
1 gem installed

By the way, I‘m not writing this from memory. I just installed CentOS 5.3 x86_64 on a stock server at ServerBeach, and these commands are clipped right out of my console window.

Anecdotes on Passenger Memory Usage

We manage a rails site for a client that will go unnamed for now, and for the past year they have experienced about a 23% increase in traffic per month.  The front end is run on a single box; a dual core with 2 gige of ram.  Up until the switch to passenger, we were using a cluster of mongrels.  As the site grew, memory usage (and mongrel restarts with monit) were starting to cause some problems.

So we switched to passenger.  I bet you can tell when by looking at this chart.  This chart represents the last year of memory usage on that box, as sampled every 5 minutes.

Switching to Passenger is nothing new; in fact, if you're still hosting with mongrels, thats getting pretty old-school in the rails community.  I just haven't seen much real-world data with a year of measurements behind it, thought this graph was pretty, and thought I'd share.

Dump IE6?

There's a debate raging on the Internet over the fate of Microsoft's eight-year-old browser, Internet Explorer 6. The essence of that fight is this: IE6 is a terrible, outdated browser; but many corporate and government IT departments still refuse to upgrade. Web developers are fighting to get IE6 dropped from the "supported browsers" list for their applications, and have launched a campaign here to try to convince IT managers of their cause.

The purpose of this post is not to choose a side, but to examine the question: "What should you do?" If you're creating a web application, should you include support for IE6? Or just consign it to the dustbin of history?

First, what makes IE6 so bad? Not all old things are intrinsically bad, after all. However IE6 is truly pretty awful, especially for web developers. It's non-compliant and breaks many standard pages. Here's a short video that explains it from the web developer's and project manager's perspective. The fact is, developing for IE6 DOES take a lot of extra work, since it sticks out like a sore thumb with its display problems (that even later versions of IE don't have). Multiply that extra work a few times: not just for development time, but setting up special development environments (remember, you can't really have more than one version of IE installed at a time without using virtual machines), test environments, and extra testing time.

Of course, maybe your app should run in IE6...maybe. A recent survey found that 70% of users of IE6 are still using it because their corporate IT department won't let them upgrade. So if that's your desired audience, then sure you need to support IE6.

Let's take a look at some numbers. W3schools.com publishes a good set of browser statistics by month, which you can see here. We can see that usage of IE6 reached an all-time high in November of 2003 (71.2%). Wow, that was a long time ago! Since then, the stats have wavered, but numbers for IE6 have generally continued to fall since then. In particular, let's look at the most recent numbers, for August, 2009:

IE6: 13.6%
IE7: 15.1%
IE8: 10.6%
Firefox: 47.4%
Chrome: 7.0%
Safari: 3.3%
Opera: 2.1%

Wow. First of all, the big winner in this list is obviously Firefox. At 47.4%, Firefox has more browser share than all versions of IE combined (39.3%). But even looking at the IE numbers on their own, IE7 has surpassed IE6; and IE8, which was only recently released, is already gaining on both of them. IE7 and IE8 combined account for approximately twice as many site visits as IE6. 

So, we can see that IE6, in general, has a very small market share of browsers in use on the modern web. One could probably safely discard support for IE6 without locking out very many potential users. And if you examine the audience for your application, this may make even more sense. For example, if you're creating a dating site, you probably don't have to worry too much about IE6 - a lot of IE6 users are on large corporate networks where they won't be allowed to access dating sites anyway! Instead, those users will wait until they get home, where they probably have Firefox or IE7 installed. As people upgrade hardware and their operating systems, too, IE6 will become even more marginalized. While it was the default browser installed with Windows XP, anyone running Windows Update regularly will have been automatically upgraded to IE7 by now, if not IE8; Vista did not include IE6, and Windows 7 most certainly will not.

It may be time to "dump IE6" after all.

Securing Client Data with Apple's Disk Utility

Since my last post was about Apple's Disk Utility, I thought I'd share another one.  How to secure your client's data on your laptop.

I'm currently working with a client where I have access to some sensitive school records.  Recently, someone asked me, "So what happens if you lose your laptop?"  I have that covered...  All my client's data is stored on encrypted disk images, and each client has their own. 

Have you ever downloaded a 'disk image' to your mac, double-clicked on it, and it mounts just like you stuck a CD in your computer?  Well, what if you could make those disk images yourself, and what if you could keep them encrypted, so only you could open them?  Sounds like a good idea, but I bet you're thinking it will be painful.  Not so!  Apple's Disk Utility to the rescue. Lets make an encrypted disk image of our own right now, in less time than it takes you to read this post.

In your applications/utilities directory, double-click on the Disk Utility application.  Notice the buttons across the top - one of them says "New Image".  Click it.

You'll get a window that looks like this:

Notice the selections in that window...  I'm saving it in a 'clients' directory, I'm making a new disk the same size as a DVD-R, I'm formatting it as a Mac OS Extended (Journaled) volume, I have 128-bit encryption on, partitioned as a normal dvd, and setting it up as a sparse disk image.  I'll explain those choices below... just follow along for now.

Set your window up with the same settings and click 'create'.  It will ask you for a password.  This is the password for the disk you are creating - you will probably want to un-check the 'remember this password in my keychain' option; if you leave that checked, then someone stealing your laptop won't have to type the password to access the volume.

In less than a minute, you'll have a new disk sitting on your desktop, ready for new files to be copied onto it.  Copy your client's files into that disk.  You can eject the disk when you're done, and get it back at any time by double-clicking on the actual disk image you created.  You'll have to type the password to open the drive up, but once its open, you won't have to type it until you put it away.

Pretty cool, right?  If you are on a Mac, there is no reason*not to protect your client's data.  For those envious windows users reading along, there are several open source and commercial tools that can do basically the same thing.

So lets look at the settings we chose to set this up:

'Save As:' and 'Where:' are your typical options when saving files.  This is the name and location of the disk image you are creating.

'Name:' is the name of the new disk that will appear on the desktop when you open the disk image.

'Size:' is going to be the *maximum* size of this drive.  You can change this later (with some effort), but choose wisely to start.  You can type your own value here, but I like creating images the same size as standard media - If a client ever requests it, I can burn a copy of my working environment right to a CD or DVD and hand it off to them.

'Format:' is the same options from my last post.  You can't go wrong choosing 'Mac OS Exended (Journaled)' when working on a mac.  People who have certain apps or strong opinions might want one of the other options.

'Encryption:' - choose either 128 or 256-bit.  128 bit should be fine unless you work for the NSA.

'Partitions:' - I typically leave this on the setting that is auto-selected when I choose the default 'size:' above.

'Image Format:' should be set to 'sparse disk image'.  This means that, while you have set the maximum disk size above, the actual file will be smaller than that, taking up only the amount of space as the files on it.  If you leave it on the default 'read/write disk image' thats ok too, but realize that your file will be the same size as the maximum size you set above, even when it is completely empty.

I have about a dozen client volumes in my clients directory.  Each one is secure, my client's work is partitioned from each other, I keep my home and documents directories neat and clean, and it doesn't add any real encryption hassle to working with my client's data.  The next time your client asks you, "What do you do to protect the data I give you?", Disk Utility FTW!

Of course, there is more to a complete security policy, but this is a better start than most people are using...