Site Offline Twice in Two Days

2011 Sept 28 at 12:39 » Tagged as :centos, cloud, xmpppy,

After running smoothly for more than an year, the Site with the LAMP was offline for around twelve hours yesterday, it was caused the by EC2 instance on which the site is hosted going offline  for some unknown reason. It had restarted soon after but all the data in the instance storage had gone missing.  Of course Amazon strongly advices against relying on instance storage, you are supposed to use EBS volumes but lazybones (or cheapskates) like me don't do that all the time.

 

But why was the site offline for so long? Well it took me a few hours to notice the issue and a few hours more before I had a chance to attend to it. Priorities have changed, I know now that a site that gets a few hundred thousand page views a month being offline isn't going to be the end of the world. There was another reason that the restoration took a little bit longer than ideal, the last back up I had was a little bit outdated. I also took the opportunity to switch the OS. Previously the server was running Centos 5.4 , now it's running Amazon Linux. I wonder if I will ever get used to always having to sudo

 

Even then Google cache came to the rescus so I only lost a single blog post  + another one that had been in the draft stage. That's something that I can live with.

 

But hang on a second, twice in two days you said, what happened next? I haven't the foggiest, the site went off line again the today and I had to repeat the whole process all over again. This time the outage was a 'mere' five hours