DR1 - What Happened?

Robert · Jan 20, 2005

As some of you may be aware, we had a server problem, actually a big one

For the nerds...

Our main server hard drive decided to die on us along with our DNS server. Under normal circumstances this would not be a problem as we could have just swapped out drives for the 2nd drive that we backup to every 24hrs. If only life was that easy! The 2nd drive that we keep in another data center for safety reasons, decided it wasn't happy mounting and gave us hell for a a few hours.

After the longest period of downtime since 1996 and sleep deprivation...

I'm glad to say, we now have a new main drive that is mirrored to a second drive, plus a new remote back up server. Or in NASA terms, a shared Dell remote back up server with dual Fiber HBAs that connect to an EMC CX700 Clariion SAN with 70TB of raw storage

We still have some areas that are being restored, but all should be 100% within 24hrs.

Thanks for you patience and hopefully we will not have to experience this pain ever again. To all you DR1 junkies that emailed, telephoned and IM'd us, thanks, we really appreciate the support.

andrea9k · Jan 20, 2005

Robert said:
As some of you may be aware, we had a server problem, actually a big one

For the nerds...

Our main server hard drive decided to die on us along with our DNS server. Under normal circumstances this would not be a problem as we could have just swapped out drives for the 2nd drive that we backup to every 24hrs. If only life was that easy! The 2nd drive that we keep in another data center for safety reasons, decided it wasn't happy mounting and gave us hell for a a few hours.

After the longest period of downtime since 1996 and sleep deprivation...

I'm glad to say, we now have a new main drive that is mirrored to a second drive, plus a new remote back up server. Or in NASA terms, a shared Dell remote back up server with dual Fiber HBAs that connect to an EMC CX700 Clariion SAN with 70TB of raw storage

We still have some areas that are being restored, but all should be 100% within 24hrs.

Thanks for you patience and hopefully we will not have to experience this pain ever again. To all you DR1 junkies that emailed, telephoned and IM'd us, thanks, we really appreciate the support.

Does it mean that DR1 would survive an atomic e-bomb?? e-terrorists attacks? e-tsunamis? and e-whatever?

COOL!

Congrats again

Jess

Ps. Now I can go to sleep!!! :bunny:

Keith R · Jan 20, 2005

aw, technology, ain't it wonderful? Need back-ups for the back-ups, and then some.

I can empathize. Just one week after the official launch of my site, right in the middle of a campaign to bring people to the site, my host had a fire in their building and had to evacuate before they could activate their remote back-ups. So just as people were feverishly trying to access my site at my urging, the thing was down for 24 hours.

We missed DR1. Heard from alot of DR1ers in withdrawal. A suggestion: you might want to keep the members list with e-mail addresses on a separate CD, so that if something in the future happens, you can send out an e-mail to all notifying them and the prognosis for return....I was also going to suggest letting all know a non-DR1.com e-mail address for people to contact you and/or Dolores, but then thought maybe during crises you probably would prefer not being flooded with frenzied e-mail's... :laugh:

mkohn · Jan 20, 2005

DR1 drama

We always kept 2 back ups. I was surprised how often they came in handy.
Glad the crisis is over.
mkohn

Lambada · Jan 20, 2005

Did it take out some posts with it? I noticed some which are in the words of Monty Python 'demised, no more, caput,' or whatever it was in dead parrot skit. I ask because strangely enough think I've seen some missing from live parrot thread in Living Forum!

aranoid: Mind you, I guess you don't want to even THINK about parrots after last 24 hours....................

AnnaC · Jan 20, 2005

I'm not a nerd so I don't understand what you posted. All I can say is next time you want to give me the day off you don't need to shut down the whole site.

Glad to be back.

BTW you could have lost all the scores in the arcade you know :laugh:

jruane44 · Jan 20, 2005

I know I don't post that often but I do check in often. I may have turned into a junkie because I kept coming back looking for my fix and the system was down. I was pretty annoyed when I couldn't get my fix. I'm okay now.

PJT · Jan 20, 2005

Rob,

Let's hope after all the effort you put in to get dr1 back on line you have locked yourself in the bedroom, removed all electronic and/or communication devices from the area, and blacked out the window(s) so you may get some deserved rest.

Regards,
PJT

juanita · Jan 20, 2005

Don?t ever do that again!

I had the worst last 2 days. I felt has if I was cut off from the world and was forced to surf the net desperately looking for something interesting to keep me busy. I even started to investigate if it was my office that had blocked the access. Thank god DR1 is back! :classic:

Hillbilly · Jan 20, 2005

I believe in the old addage

Any news os good news for a business.

Which is to say, sure I missed DR1, it's like a second cuppa coffee, sort of comforting, you know? But after I got a flood of emails, and was sitting there cutting and pasting the same message-"Robert told me that the server went down. The site is supposed to be up in a few hours." Or words to that effect- I realized what a huge family we are here. So I grinned and bore it.

Glad that the Dohicky is now reflected in the Whatchamacallit and the Thigamagig is now converted into a He!!ofalota storage space. Them things do get all gombobulated every now and then but a few Goldbergs and you'r outta the dark..

Cordially,

HB

Chris · Jan 20, 2005

Thanks for the 'nerdish' info... That damn second mirrored drive never wants to mount..

Just a question, what are you going to do with 70TB of storage? I had to google it to figure out how much it is really... you know one TB is more than one trillion bytes... 7,000 Gigs? the mind boggles... One can run the whole of the Caribbean plus a few more large African countries on that kind of space..

Anyway, have a good rest! :classic:

A.J. · Jan 20, 2005

What a relief - I spent yesterday coming back to the web site and started to get into panic mode. I decided to take a break and relax my mind thinking tomorrow it will be back tomorrow. As I sat at home last night I was precoocupied with thte fact that maybe tomorrow it would not have been fixed so I spent the night baking chocolate chip cookies lots and lots of them. Thank goodness this morning it was back on now I can go back to feeding my DR1 addiction rather than that baking crap.

Ken · Jan 20, 2005

I was greatly relieved to know it wasn't just me having computer problems.

Robert · Jan 20, 2005

Chris said:
Just a question, what are you going to do with 70TB of storage?

If only it was just for me

It's backup vault as they call it in our offsite data center.

Minni · Jan 20, 2005

Yeahhhhhh

You're finally back!!!! I had DR1 withdrawl. I don't post a lot but I'm always checking in. I was convinced that my computer at work had a bug and wasn't allowing me to get into DR1. I ran all the spy finding stuff, plus the norton and the pest patrol, but no, that wasn't it. I had to resign my self to no DR1 for the day. I was still convinced that it was my computer, but when I got home and got the same I finally got that there was a problem. Anyway I'm extremely happy that you guys are all back. It's funny to see how one becomes sorta of dependent on this website. Best of luck with your new gadgets.

Minni

BigCity27 · Jan 20, 2005

I've been there & know what it feels like. Glad to see that the problem was taken care of. Go take a nap, Robert.

AZB · Jan 20, 2005

My life was ruined yesterday. I felt as if I was deported to jalalabad where no internet exists. I was so bored, I called my long lost friends, dug out my old wives to see if they still remembered me.
I am glad Dr1 is back and now I am happy to be alive again.
AZB

hollywood north · Jan 20, 2005

Ahhh how the minutes dragged yesterday.....
My job is not exactly taxing on the brain-one of the most pleasant diversions is DR1...so glad you are back up and at it.

My personal site was hacked into by some Brazilian neo-political group (can't read Portuguese so who knows what it was about) and yes 2 back ups are the way to go.
HN

AnnaC · Jan 20, 2005

wow that was scary. What just happened? My screen turned green and a message said there was no DR1 site

Hlywud · Jan 20, 2005

Dr1

Glad to see DR1 back on line and I hope all goes well with the portions still not working, esspecially my mail account that Robert so graciously provided for the Media Gallery. I am still waiting for a couple of replies that I know are coming via the DR1 mail account but they can wait, Robert will get things 100% back in order in due time.

Thanks also to AnnaC for posting the DR1 dilema on Deb's Travel Forums, many were emailing and asking, great minds think alike as I went to post Anna's was already posted.

DR1 - What Happened?

Stay Frosty!

New member

"Believe it!"

Bronze

Gold

Gold

Bronze

Silver

Bronze

Moderator

Gold

New member

Platinum

Stay Frosty!

New member

New member

Platinum

New member

Gold

Gold