It occurred to me the other day that its been a while since I droned on and on (but not like Graham Chapman) about some obscure, trivial and uninteresting aspect of technology. There has been no recent moaning about my crippling envelope phobia (blue or otherwise). There has been no self-righteous whining about IT vendors proving yet again that Schiller was, if anything, an optimist. There have been no squick-laden moments describing inventive (yet still inappropriate) uses of technology.
In short, it’s about damned time.
In ‘Shorting‘. I described in more detail than was absolutely necessary the parade (well, three) of machines that provided a particular set of services to a particular group of people. We had more than one system that provided these services — in fact, we had several (three or four, more or less), each of which ‘served’ a different ‘constituency.’ Most things that were done to one were done to all. In particular, over the years the hardware was replaced on all of them. ‘Shorting’ described three machines; the system I’m thinking of today also occupied three different machines over the years. This meant… upgrades.
Of course, the ominous sound effects aren’t always necessary. If the designer(s) of whatever it is that you’re upgrading had a smattering of common sense (and maybe the tiniest shred of concern for their users-slash-customers) then the upgrade might be easy. If they didn’t or if there was, say, unwanted ‘assistance‘ then it probably isn’t.
For an application I administered some years ago, for example, upgrades were ‘largely’ pain-free, or at least started that way. Installing the application was done by copying the distribution kit to a location on the system then running the installation script. Upgrades were done by copying the distribution kit to a location on the system then running the installation script. Fixing problems was done by… copying the distribution kit to a location on the system then running the installation script. The installation script and the folks who wrote it were both darned clever.
It was inevitable that this state of affairs wouldn’t last. The fat guys in suits (I blame them for everything; I occasionally dabble in conspiracy theories and this is one of those.) decided that this happy combination of cleverness, efficiency and simplicity definitely Had To Go. Their solution? They added overcomplicated, poorly-performing and buggy customer-control software to their application. Installing and upgrading the application was still pain-free, but any upgrade (heck, any change, no matter how trivial) could cause the customer-control software to poot forth thousands of error messages, could cause complete application failure or even a system crash. This helped drive brisk sales in service contracts so could be considered to be a business success — just not a technical one.
There are, of course, other ways to make upgrades painful. Another application that I administered took the ‘overwhelming obfuscation’ approach to upgrades. To upgrade this application it was necessary to remove all customization that you might have performed, remove all patches that you might have applied, apply a special patch that did the upgrade, then look to see if versions of the patches you had previously applied existed for the new version, acquire and apply them, then reapply your customizations. Someone had to work to make it that annoying. (As you might expect, my money was on fat guys in suits. It always is.)
Oh, and good luck.
But I was talking about a hardware upgrade. In this case (if you care) it was an upgrade from a bar fridge to a Big Purple Box. When you do that, you don’t do it all at once — not if you have a cautious bone anywhere in your body. No, what I did was make a list (it was a long one) of everything that ran on the bar fridge, then install the latest and greatest version of everything on the list onto the Big Purple Box. Some software could be completely installed. This was good; it meant that I could check something off the list. (Making lists shorter is generally a good thing.) Some software, though, couldn’t be installed completely so couldn’t be checked off the list — probably because it used data on the bar fridge, probably data that changed with time and that would have to be copied to the Big Purple Box at The Last Minute. The result of this was that the list got a little shorter and gradually evolved into a list of Last Minute Things To Do.
Eventually, though, there was no more that could be done ‘in advance’, the list was pared down as far as it could go and it was time to schedule an outage to do the last minute stuff and finish the whole upgrade. The Big Purple Box would replace the bar fridge, things would be faster and better, birds would sing, elves would barf nutmeg-scented barf and unicorns would poop glitter. Yay.
This happened at approximately this time of year (mid November) so it was decided that a good time to do this would be between Christmas and New Years. After all, people are fairly dormant then (Too much tryptophan? Perhaps, although I think that’s been debunked.) and wouldn’t mind too much if there was an outage. (And besides, since the university would be closed, if they did mind, there would be no one to complain to. Bonus.) December 27th was chosen as Der Tag.
Now, I mentioned that the university would be closed then. A consequence of this is that someone in a suit decided that it would save some money if building temperatures were lowered while no one was there. (A fair decision, all things considered, as long as they didn’t overdo it.) So on December 27th I showed up, nice and early, with an armload of essential equipment:
- an electric heater because it would be cold in there.
- an extension cord because I didn’t want to plug the heater into the same circuit as anything that I’d be using that day
- a couple of liters of an appropriately caffeinated beverage
- The List
- a stack of Frank Zappa CDs.
The first thing I noticed was that it was cold in there. Really cold. Cold enough to see your breath. Colder than the guy in the suit intended. Cold enough to freeze… damn.
Apparently the regular security sweeps hadn’t noticed this, so I phoned the campus security folks: “It’s cold in here. It’s damned cold in here. Maybe someone should look in to that.” Having done the responsible thing, I set to work. First step? A hefty slug of the appropriately caffeinated beverage. Second step? Consult The List — the list that was made when I was awake and thinking clearly. Third step? Shut down the services on the bar fridge so that no one (well, no one but me) could change the data that I was about to copy elsewhere. (Also, make some system changes to ensure that the services wouldn’t start automatically when the system rebooted.) Just in case that wasn’t enough, change the network address of the bar fridge so that people (except me) wouldn’t even be able to find it. Reboot it to make all the changes happen.
Crash? There shouldn’t have been a crash. That was… in the building. Nearby. Perhaps I should investigate — there is, after all, not supposed to be anyone in here.
I investigated. I found… plumbers. Due to the cold in the building — the cold that no one had noticed EXCEPT ME — some of the pipes had frozen. So far, they had found several frozen (and burst) pipes in radiators. (Well, not radiators, exactly, but I don’t know what their official HVAC name is.) The plumbers were there to locate any and all frozen pipes and repair what they could. (And make loud CRASHes at times that would frighten idiot IT guys, but they didn’t have a work order for that.) I returned to my slightly-less-cold-than-the-rest-of-the-building office and drank more caffeine. Now where was I? Oh yah, dump the password database on the bar fridge and copy it to the Big Purple Box.
Warning? That’s an automated voice bellowing somewhere. That sounds like the burglar alarm in the basement…. Perhaps I should investigate — there is, after all, not supposed to be anyone down there.
I investigated. I found… plumbers. As part of their quest for frozen pipes, they had used their master key to enter the server room without disabling the alarm — because they didn’t have a code to do that. And that meant that the campus security folks should be showing up any…
“What’s going on here? And why is it so cold?”
It took a while but I managed to reduce the level of… excitement and went back to work after begging the plumbers to come see me before opening any more doors with alarm notices on them.
Where was I? Oh yah, the password database. Now the mail spool….
Time passed. I had just started copying everyone’s personal files and mail when it suddenly became noisy. Very noisy. It sounded like the fire alarm. It was the fire alarm. Uh-oh. That meant…
The campus security folks were on the scene in a flash, followed closely by several fire trucks. Before long, a rather… animated looking discussion formed with security guys, firemen and plumbers. I contemplated joining in but in the end decided to just go home. I was out of caffeine anyway.
Upgrades can be hell.