Hallelujah!

[Jeff]

Man, I thought things were really bad. I thought we were having a big crash again. I got a hold of dallas and they were no help.  I searched google on how to repair an mysql table and found a tutorial, and tried it and IT WORKED!!

Were back after only a short crash!

[LeeB]

That was close. Glad you managed to keep it all together. Thanks Jeff.

[Reddog]

I am sure it wasn't and easy fix. But glad it was fast.

[Mooseherder]

Had a bad feeling as it was happening while I was uploading pics. :(
I'm glad you figured it out and I hope it wern't me. :D

[SwampDonkey]

Yeah, that was the message I was getting. Something to the effect that the database table crashed.

[sbishop]

Jeff, how often are backups done?

Sbishop

[Dave Shepard]

You didn't let Furby near that button again, did you?


Dave

[Jeff]

Jeff, how often are backups done?

Sbishop

They are supposed to be done daily, but unless the tech I was getting nothing from earlier knows even less then nothing, he says he didnt see any back up on the other dirve since last october. >:(

[Jeff]

I just did a manual back up of the database.  Over a half gig of text now that makes up the messages on the forum..

[SwampDonkey]

That's a lot of Dang Text. Concerning the backup, that must have been about the time we came back on line after the crash. Don't they have software the backs stuff up on a set time of day or week? How hard could it be anyway?  I'm not so sure that making contact on a weekend with them is going to get you in contact with the most knowledgeable crowd in the office. The worst always happens on weekends, eh? :

[Jeff]

Yea, it was about that time. I'm just wondering if the daily back up is on tape, and not the extra drive. I'm looking at my order form and it says extra hard drive with daily back up.  Maybe I just assumed the back up was going to the other drive.  I'll be on the phone Monday to find out for sure. Were paying for it now, we dang well better be getting it.

[StorminN]

Jeff,

Maybe I'm not understanding this correctly... but with the price of storage these days (750GB for $150?), it's surprising your provider doesn't have a RAID array on the live server... the array would be two to five disks with the forum data redundantly saved on all of them... and then a scheduled backup to another method, like tape or another disk or RAID array.

I run a bunch of RAID arrays, all of them are two-disk RAID mirrors. The latest one was a simple as two 750GB external firewire drives from Costco ($199 each) so whatever I write to one, writes to both. If one drive dies, the other one has the same data, nothing is lost. The only weakness is that the array is still prone to "pilot error"... if I erase a file, it's erased from both drives at once... so daily / weekly backups to tape or other drive(s) is still a good idea.

-N.

[Jeff]

I dont want anything to do with Raid after talking with Tim from Bio-Mizer. They lost all thier data once. The problem with raid as explained to me is that it creates a real time data copy. If things go wrong with the database server and the tables get corrupted or damaged, the same thing is happening to the real time copies.  In this case, where the database tables began having problems, a raid backup would have been useless.

[logwalker]

I was noticing a lot of lag and delayed page loading the last couple days. Was this related to the problem? Right now it is screamin' fast. Joe

[sbishop]

Jeff, I would talk to them to make sure you are getting what you are paying for. if they say a backup everyday, you should be getting a backup everyday. now having a backup everyday is nice, not tested the backup is no good either. the backups should be tested at least once  week to make sure whats going on tape is able to be restore. Just in case anyone cares i'm a Senior Network Administrator for an investment manager here in NB (no i'm not a logging, welder, builder...etc HAHA). Backups are the most important part of my business because hardrives can die at any time. Using Raid is good too but you still need backups. We just implicated a Disk-to-Disk-Tape backup unit about 6 months ago, talk about some sweet, i don't a backup to disk first, then it duplicates it from disk to tape. I have a month worths of backups on disk ready to be restored within seconds.

All of my servers are setup with 2 disk in a Raid 1 (mirror) for the operating system and 4+ disk in a raid 5 for data plus an online spare in case a drive fails. ok enough techy talk!
;D
Sbishop

[LOGDOG]

Thanks for getting things back up and going Jeff. I was starting to go through withdrawal ... twitchin' , shakin' and everything. Good to have it working again.


LOGDOG

[Warbird]

Ouch.  Glad you got 'er back, Jeff.  Good catch on the back ups.  You know that old saying?  "If ya want something done right..."

[Jeff]

I wish I had a way to run things from here. I mean the servers and everything. I probably don't know half a % of what I would need to know, plus the cable connection aint enough to do it, and is the best available here that I know of.

[Jeff]

I'll be keeping an eye on this, but hopefully, this has been taken care of
*********************************************


Hi Jeff

I spoke to you this morning about the disk to disk backup not working.  I opened a ticket for you on it and one of our unix admins fixed the problem.

Terry


Josh C

Hello,
The permissions on /etc/cron.daily/d2d-backup.sh were incorrect so it was not being executed normally. I have corrected this and started a test run, it should work from this point forward. Full backups are on Sundays, incremental backups on every other day.

best regards,
Josh Craddock
Linux Administrator

  [Edit Ticket Post]   

Terry

Customer bought disk to disk backups October/07
They had a database corruption problem and found that there were no current backups
Customer said the only back up availabe was from Oct 6/2007

[Gary_C]

Did you get a refund?   ;D

[Jeff]

It was a one time charge to set it all up as I bought the extra equipment. I bet I dont.

[SwampDonkey]

I bet I don't.

Me too. But, if ya did, it would be a pleasant surprise. ;D