On the Path

Upcoming Public Appearances

Of all the public speaking I do, the NFJS Conference Series has to be some of the most enjoyable. The venues are always nice, the crowds always intelligent and fun to interact with, and the speakers are all social and personable. Between then audiences and and other speakers, I think I learn as much as I teach at these shows. If you haven't been to one of these, you should... They are the gold standard people in the conference world judge against. I have been privileged enough to have been involved in one way or another since 2003.

This year I haven't spoken at nearly as many as I have in the past. Work and home commitments make it hard to take that many weekends. I'm happy to announce that I'll be speaking at the Greater Atlanta Software Symposium, October 23-25, and the Northern Virginia Software Symposium, Nov 6-8. Time permitting, I might be able to speak in another one this year too - we'll see.

I'll be covering some sessions on Agile Estimation Techniques, Career advice for developers promoted into management, and a couple of sessions on project-integrity tools for Java projects (like build tools, test coverage, metrics, and architectural quality).

I hope to see you there!

Installing CentOS 5, ImageMagick, and RMagick, September 2009 Edition

For some reason, this process seems to get more convoluted with time, not less. Before you go through this pain, let me ask you, "are you sure you need to use ImageMagick and RMagick?" There are other options in today's Rails environment.

I wrote about this previously on my older blog. While just going through this pain again, I found several things:

• CentOS has been updated, but still relies on older-than-dirt gems for ImageMagick. Still, there is something to be said for relying on stock gems rather than messing with building and installing yourself.
• The version of RMagick I specifically call out in the old blog post isn't available anymore.
• The newer version of RMagick adds a new default html documentation option you need to turn off.

So sit back, relax, and read my previous blog post, now updated to work as of September, 2009:

I don‘t normally blog about obscure, specific technical topics, mainly because 99% of the time I find a blog entry someplace that has already covered it. This is an exception, as the amount of information out there on this topic is confusing and dated.

Installing ImageMagick and RMagick on CentOS5 seems to be a bit of a dark art, mainly because of the amount of stale information and differences between linux distributions. This entry will eventually be stale as well, so check the date and version numbers – your mileage may vary.

The ImageMagick/RMagick combination is arguably one of the coolest yet painful combinations of ruby/native code out there. There are a lot of attempts to replace, simplify or otherwise solve this pain, but today, rmagic is often a necessary tool in the rubyist‘s toolbox. So here‘s my attempt to lessen that pain.

I'll include details and explanations with each command line. I have you as a captive audience, and the details will give this post more relevance if the version numbers have moved on since this writing. There are only 3 commands, so bear with me.

One more caveat – I‘m taking the short path using the most recent CentOS5 packaged versions. If you need the latest version of ImageMagick, have fun compiling it all yourself. There are plenty of sites out there with directions. For me, one of the reasons I‘m using CentOS5 (with this client) is for the stability and ease of package management.

1. First we install ImageMagick as an rpm using yum. As of this install, the latest CentOS package is 6.2.8.0-4.el5_1.1. This is lagging considerably from the newest version, and that causes some of our issues. Use sudo on these commands as appropriate for your user management style.

   yum install ImageMagick

   This will ask to download and install all kinds of stuff (43 other packages on a stock CentOS server install). Let it install whatever it wants. If the version number of ImageMagick is up to the 6.3.x range by the time you read this, then this post probably isn‘t want you‘re looking for. (although explanations below will still help you). If you are on an x86_64 box, this will also install that rpm file for you.

2. Now install the ImageMagick-devel library. Yes, that could have been done in the same step above, I‘m just trying to be explicit for your benefit.

   yum install ImageMagick-devel

   That wants to install 16 packages on my stock CentOS5 server. This also does the right thing os x86_64.

   So no surprises so far. It is the ruby gem that causes us all the problems...

3. Install the RMagick gem. I‘m about to make this look easy, but it wasn‘t. The first problem is that the latest RMagick gem needs the 6.3.x series of ImageMagick, which isn‘t packaged for CentOS5 yet. So I had to determine the latest version of the RMagick gem that works on the 6.2.x series. As of this writing, that is 1.15.17, and you can find that out at the RMagick gem page.

   But installing that just might fail in a blaze of glory that only a software engineer with a minor in forensic investigation would enjoy. Near as I can tell, the install of the gem would succeed if it weren‘t for some failures building the documentation. These failures vary from machine to machine, as it seems to depend on what fonts you have installed (which might depend on what version of OpenOffice, etc). Solution? We dont need no stinkin' docs! Furthermore, sometime between the last time I did this and now, they have a new option to disable the html documentation. This has to be passed through the gem installer to their script (this the obscure little '--' sitting all by itself in the command.)

   gem install rmagick -v 1.15.17 --no-rdoc --no-ri -- --disable-htmldoc

   a minute later, I get the output:

   Successfully installed rmagick-1.15.17
1 gem installed

By the way, I‘m not writing this from memory. I just installed CentOS 5.3 x86_64 on a stock server at ServerBeach, and these commands are clipped right out of my console window.

Anecdotes on Passenger Memory Usage

We manage a rails site for a client that will go unnamed for now, and for the past year they have experienced about a 23% increase in traffic per month.  The front end is run on a single box; a dual core with 2 gige of ram.  Up until the switch to passenger, we were using a cluster of mongrels.  As the site grew, memory usage (and mongrel restarts with monit) were starting to cause some problems.

So we switched to passenger.  I bet you can tell when by looking at this chart.  This chart represents the last year of memory usage on that box, as sampled every 5 minutes.

Switching to Passenger is nothing new; in fact, if you're still hosting with mongrels, thats getting pretty old-school in the rails community.  I just haven't seen much real-world data with a year of measurements behind it, thought this graph was pretty, and thought I'd share.

Dump IE6?

There's a debate raging on the Internet over the fate of Microsoft's eight-year-old browser, Internet Explorer 6. The essence of that fight is this: IE6 is a terrible, outdated browser; but many corporate and government IT departments still refuse to upgrade. Web developers are fighting to get IE6 dropped from the "supported browsers" list for their applications, and have launched a campaign here to try to convince IT managers of their cause.

The purpose of this post is not to choose a side, but to examine the question: "What should you do?" If you're creating a web application, should you include support for IE6? Or just consign it to the dustbin of history?

First, what makes IE6 so bad? Not all old things are intrinsically bad, after all. However IE6 is truly pretty awful, especially for web developers. It's non-compliant and breaks many standard pages. Here's a short video that explains it from the web developer's and project manager's perspective. The fact is, developing for IE6 DOES take a lot of extra work, since it sticks out like a sore thumb with its display problems (that even later versions of IE don't have). Multiply that extra work a few times: not just for development time, but setting up special development environments (remember, you can't really have more than one version of IE installed at a time without using virtual machines), test environments, and extra testing time.

Of course, maybe your app should run in IE6...maybe. A recent survey found that 70% of users of IE6 are still using it because their corporate IT department won't let them upgrade. So if that's your desired audience, then sure you need to support IE6.

Let's take a look at some numbers. W3schools.com publishes a good set of browser statistics by month, which you can see here. We can see that usage of IE6 reached an all-time high in November of 2003 (71.2%). Wow, that was a long time ago! Since then, the stats have wavered, but numbers for IE6 have generally continued to fall since then. In particular, let's look at the most recent numbers, for August, 2009:

IE6: 13.6%
IE7: 15.1%
IE8: 10.6%
Firefox: 47.4%
Chrome: 7.0%
Safari: 3.3%
Opera: 2.1%

Wow. First of all, the big winner in this list is obviously Firefox. At 47.4%, Firefox has more browser share than all versions of IE combined (39.3%). But even looking at the IE numbers on their own, IE7 has surpassed IE6; and IE8, which was only recently released, is already gaining on both of them. IE7 and IE8 combined account for approximately twice as many site visits as IE6. 

So, we can see that IE6, in general, has a very small market share of browsers in use on the modern web. One could probably safely discard support for IE6 without locking out very many potential users. And if you examine the audience for your application, this may make even more sense. For example, if you're creating a dating site, you probably don't have to worry too much about IE6 - a lot of IE6 users are on large corporate networks where they won't be allowed to access dating sites anyway! Instead, those users will wait until they get home, where they probably have Firefox or IE7 installed. As people upgrade hardware and their operating systems, too, IE6 will become even more marginalized. While it was the default browser installed with Windows XP, anyone running Windows Update regularly will have been automatically upgraded to IE7 by now, if not IE8; Vista did not include IE6, and Windows 7 most certainly will not.

It may be time to "dump IE6" after all.

Securing Client Data with Apple's Disk Utility

Since my last post was about Apple's Disk Utility, I thought I'd share another one.  How to secure your client's data on your laptop.

I'm currently working with a client where I have access to some sensitive school records.  Recently, someone asked me, "So what happens if you lose your laptop?"  I have that covered...  All my client's data is stored on encrypted disk images, and each client has their own. 

Have you ever downloaded a 'disk image' to your mac, double-clicked on it, and it mounts just like you stuck a CD in your computer?  Well, what if you could make those disk images yourself, and what if you could keep them encrypted, so only you could open them?  Sounds like a good idea, but I bet you're thinking it will be painful.  Not so!  Apple's Disk Utility to the rescue. Lets make an encrypted disk image of our own right now, in less time than it takes you to read this post.

In your applications/utilities directory, double-click on the Disk Utility application.  Notice the buttons across the top - one of them says "New Image".  Click it.

You'll get a window that looks like this:

Notice the selections in that window...  I'm saving it in a 'clients' directory, I'm making a new disk the same size as a DVD-R, I'm formatting it as a Mac OS Extended (Journaled) volume, I have 128-bit encryption on, partitioned as a normal dvd, and setting it up as a sparse disk image.  I'll explain those choices below... just follow along for now.

Set your window up with the same settings and click 'create'.  It will ask you for a password.  This is the password for the disk you are creating - you will probably want to un-check the 'remember this password in my keychain' option; if you leave that checked, then someone stealing your laptop won't have to type the password to access the volume.

In less than a minute, you'll have a new disk sitting on your desktop, ready for new files to be copied onto it.  Copy your client's files into that disk.  You can eject the disk when you're done, and get it back at any time by double-clicking on the actual disk image you created.  You'll have to type the password to open the drive up, but once its open, you won't have to type it until you put it away.

Pretty cool, right?  If you are on a Mac, there is no reason*not to protect your client's data.  For those envious windows users reading along, there are several open source and commercial tools that can do basically the same thing.

So lets look at the settings we chose to set this up:

'Save As:' and 'Where:' are your typical options when saving files.  This is the name and location of the disk image you are creating.

'Name:' is the name of the new disk that will appear on the desktop when you open the disk image.

'Size:' is going to be the *maximum* size of this drive.  You can change this later (with some effort), but choose wisely to start.  You can type your own value here, but I like creating images the same size as standard media - If a client ever requests it, I can burn a copy of my working environment right to a CD or DVD and hand it off to them.

'Format:' is the same options from my last post.  You can't go wrong choosing 'Mac OS Exended (Journaled)' when working on a mac.  People who have certain apps or strong opinions might want one of the other options.

'Encryption:' - choose either 128 or 256-bit.  128 bit should be fine unless you work for the NSA.

'Partitions:' - I typically leave this on the setting that is auto-selected when I choose the default 'size:' above.

'Image Format:' should be set to 'sparse disk image'.  This means that, while you have set the maximum disk size above, the actual file will be smaller than that, taking up only the amount of space as the files on it.  If you leave it on the default 'read/write disk image' thats ok too, but realize that your file will be the same size as the maximum size you set above, even when it is completely empty.

I have about a dozen client volumes in my clients directory.  Each one is secure, my client's work is partitioned from each other, I keep my home and documents directories neat and clean, and it doesn't add any real encryption hassle to working with my client's data.  The next time your client asks you, "What do you do to protect the data I give you?", Disk Utility FTW!

Of course, there is more to a complete security policy, but this is a better start than most people are using...

A Useful OS X Hard Drive Tip

I deal with a lot of big files, so I deal with a lot of external storage.  My favorite portable, big hard drive line is the FreeAgent Go from Seagate.   500 gigs, no external power = awesome.  They are becoming available up to 640 gigs, but I haven't played with one of those yet.

When using a drive like this with the Mac though, you can fall into a pretty easy trap.  You can plug it in and it works, but you won't be using a Macintosh file system with it - its formatted for Windows.  99.3% of the time you won't have an issue, but when that .7% bites you, it can result in some wierd problems...  File types going away, files that exist on your main computer, but give you an error when you copy to the external drive, etc.  You're on a Mac, right?  So lets let it run like one.

Lets make sure you really have a problem:

1. Open up the utilities folder under Applications.
2. double-click on the disk utilitity application.
3. Select your hard drive from the list on the left.  Notice how it appears twice, with one indented below the other?  Select the indented one below.
4. If the info at the bottom says "Format: Mac OS Extended (Journaled)", then nevermind - you're fine. (if it says something about case sensitive, journaled, you're ok too...)
5. If it says "Format: MS-DOS (FAT)", "Format: MS-DOS (NTFS)", or some other variation, you want to fix that.

The fix: (CAUTION - only do this on a blank drive - you are going to reformat the drive, so be aware of what you are doing!)

1. you should still be inside Disk Utility, with your MS-DOS formatted drive selected.
2. Select the 'erase' tab in the main body of the window.
3. Select any of the Mac OS options.  The typical option is "Mac OS Extended (Journaled)". If you bought the drive from Apple, that is the way they would ship it.
4. Hit the 'Erase' button.  This should only take a minute to run. (Curious people might select the 'securuty options' button - which is interesting, especially if you are wiping a drive you are about to get rid of, but this is a new drive and those options will only add formatting time.  Potentially turning a 2 minute operation into one lasting several hours).
5. There is no step 5, unless you include quitting Disk Utility.

You're done! Enjoy your new drive.

I'm often amazed that I'll see the same hard drive in the same box with a sticker that says 'Formatted for Mac", with a +$30 price tag.  Save your 30 bucks and format it yourself.