Disaster recovery test turns into her worst nightmare
By , Tech Update
November 14, 2001
In response to David Berlind's commentary, "How ready is your business for worst-case scenario?" a reader writes:

I recently took over the administration of a system that is paramount to the functioning of a very large bank. After being thrown into a disaster recovery (DR) test mid-way without any information or support, I was horrified to see that upon arriving at my DR location, after 18 straight hours I couldn't get so much as one NT server up and running.

It's like this:

Day Before DR Test:

  1. Pack up truck with all software, backup tapes, even brought a calculator.
  2. Laugh with co-workers about how easy tomorrow will be and how great I'll look.
  3. Go home, have dinner with husband and tell him how easy tomorrow will be.
  4. Talk to boss on phone acting confident and telling him how easy tomorrow will be.

Day of DR Test:

6:30am: Arrive at location to get all the servers up and running before the DR team arrives.

9am: DR team arrives. Progress update -- Beginning to wonder how that IT guy thought I could use my backup tapes to restore my system when I can't even get it to recognize my tape drive.

Noon: DR team goes to lunch. I go outside and smoke 12 cigarettes in 5 minutes, chug another Mountain Dew, and head back inside.

2:30pm: DR team comes back from lunch. Proceeds to spend the next 2 hours grilling me about everything I'm doing as I am literally trying not to cry because I'm realizing that we've been paying 300 bucks a month for offsite tape storage for a system that I couldn't recover with a miracle.

3:30pm: DR team leaves to go to hotel, leaving me alone. I pee for the first time since three this morning.

7pm: DR site rep is looking at his watch and asking when I'll be done. I call the office to warn the managers that if they have to evacuate to load all the servers on a cart and push them out before they leave the building.

8pm: Eat dinner. (Note: dinner was three martinis and six olives.) Catch slight buzz and call husband, crying as I update my resume on my laptop.

8:05pm: Pass out, set alarm and wake up call for 3am.

3am: Wake up, dig through manuals, and search DR websites. Give up.

6am-3pm: Make one last try, praying to God the entire time that somehow the data on my 8mm DAT will magically transform itself to my hard drive.

3pm: Wrap up meeting, great test, learned a lot, come back in two weeks and try again.

3:30pm-6pm: Drive home, eating food purchased every 60 miles at rest stops to drown sorrow.

Yes, this is a true story -- sad but true!

Name withheld