![]()
Sometime in the small hours of the morning, one of the guys from the other lab comes into the one I was working in and finds that there were no chairs for him to use whilst talking to his group mates, so he decides to sit up on the workbench at the side of the room instead. WHATS THE DEAL WITH DONT PRESS THE RED BUTTON SOFTWAREReminds me of something that happened when I was at University: It was the night before a group project was due to be submitted and the entire second year Computer Science class was camped out in the department's two software labs. As soon as the holidays were over we installed a hard plastic cover with a key for the giant red button. Moral of the story…Don’t Hit The Red Button. With Boss on one side and OpsGuy on the other holding him up he was able to get the entire network back up and happy in less than thirty minutes.Īll in all it took us eight hours to get everything back up and running. He was so intoxicated he could not stay upright. WHATS THE DEAL WITH DONT PRESS THE RED BUTTON MANUALEach medical app had a specific order and manual steps. Network…SAN…Domain controllers…database servers…application servers…web servers. Everything had to come up in a specific order. The guy doing the maintenance on the fire suppression system leaned his ladder against the giant red emergency shut off switch. OpsGuy: I JUST HAD TO GO THE BATHROOM!!! I WAS GONE FOR MAYBE FIVE MINUTES!!!! AND NOW THIS!!!! Some serious crazy eye going on right now. The first thing we notice is an absolutely terrifying silence as we walk into the data center. Walking because none of us should be driving We still have no idea what has happened. Not so excited about the white Christmas now. We manage to make it back to the office slugging through the snow. TheGiantScott: Signals all of us to shut up. Head straight to the data center.Ībsolute silence from around 15 engineers and looks of complete panic. TheGiantScott manages to dial his phone first and puts it on speaker.īoss: Are all of you idiots still at the Irish pub down the street? TheGiantScott: Maybe, what’s going on? Boss: I need all of you to get back here now. Holy shit storm batman we must be up a creek. About two hours and a fair amount of drinks later Seriously how is TheGiantScott still standing? All of our pagers go off at once. We decide we could all use a drink and head to our favorite little Irish Pub down the street. I think they still use them Have a nice holiday.Īll of us are skipping out the door like a bunch of five year olds given a bag of candy. The data center ops guys will be handling it, but pay attention to your pagers please yes we had pagers still. There were many others that did heroic feats this night, however TheGiantScott was my hero and the focus of this story.īoss: Don’t forget the fire suppression people are here doing maintenance on the system. OpsGuy: They Datacenter Operations guy who was supposed to be babysitting the people doing maintenance. ![]() He put up with all of our stupidity and managed to lead us in accomplishing amazing things. Sharp as they came and able to drink enough scotch to put down an elephant. Our boss can see that he’s not going to get anything useful out of any of us and told us we could all go home.Ĭast of characters TheGiantScott: Network Lead. WHATS THE DEAL WITH DONT PRESS THE RED BUTTON WINDOWSThe Unix schleps trying to convert the Windows idiots to the dark side. ![]() Network guys being nice to the server guys. The Engineering floor was all but useless because of the holiday. This was when virtualization was still not trusted for production so we had about 1500 physical servers and everything else that went along with that in our main data center. Being that it was healthcare, applications/servers tended to multiply like bunnies. We had a handful of hospitals and a couple hundred clinics. About ten years ago I worked for a large healthcare company on the Windows Engineering team. I thought I’d share an amusing story from a few years ago with all of you. It’s been a while for me and it’s a slow day at work. ![]()
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |