Yesterday night, we did a massive computer system upgrade at work. The upgrade started from 8am in the morning till 2am at night. That is one heck of a long day of work. I put in a 16.5 hour shift. Bleh. I got 4.5 hours of sleep, and got up to go to work again. So, I'm fairly incoherent right now, and I'm in a ranting mood.
It was a pretty crazy upgrade because a lot of stuff wasn't tested completely, but we pushed on anyways. By 11:30pm, it wasn't looking good, and we might have had to undo our changes which would have been a lot of wasted time and effort. However, by 1:30am, I had squashed maybe 20 bugs, and the system was stable. So yay! When I got in this morning, we had about 5 bugs to kill. At least none of the bugs were critical. Given the sheer size of our upgrade, the transition went fairly smoothly.
Which leads me to my rant. One of the managers was complaining that the system would be down for an hour while he was working, and he thought this was an unacceptable disruption in services. He was demanding that we do the system upgrades on the weekend when no one was around. This was kind of annoying because we tried to schedule a weekend upgrade, but we couldn't get our entire team in on a weekend because of scheduling conflicts. On top of that, I think he would be appreciative that we stuck around for 16+ hours for a system upgrade. We don't get paid overtime for this. Heck, even the department's secretaries get paid more than I do, so it's not like there's a lot of financial incentive to bust our backs making sure this works well. (A secretary making more money than a programmer doesn't really seem quite right.) But hey, we did it anyways. The manager kept going on about how at his last organization that he worked for, they had software engineer teams do upgrades all the time on the weekends, and it didn't disrupt anyone. All I have to say is, given an infinite amount of money, resources, and labour, anything is possible. We are a small team in a large department, so we have limits.
My second rant is about a student complaining that our system was down last night. We had a prominent notice on our website for days warning about the system being down for upgrades. The student's advisor caught me this morning and asked if the system was down last night. I thought the advisor was joking because she saw the notice as well. I told her, uhhhhh yeah, it was down from 5pm-2am, as previously announced. She said that the student was complaining that he couldn't apply to jobs because the system was down. Read the notice, and apply earlier. Don't apply at midnight before the job closes!
End rant.
