Site restored following a VMware crash

In the middle of the night last week, Torchwood Archives, LMeve Database, Yearly Stats and Aideron Technologies instance of LMeve vanished from the internet.

But it wasn’t a DDOS attack.

It turns out hypervisors (such as VMware) sometimes crash, just like Windows does.

There was connection to neither the Linux system that powers my websites, nor to the VMware running under the hood. So at first it all looked like a good old fashioned hardware fault. It took a good while to restore, because my server is colocated in a different city, and due to real life I couldn’t just go there and investigate it locally.

Here’s where a remote KVM comes in handy. It’s a special device that acts as keyboard, mouse and monitor, but you can access it remotely by visiting a special web page.
KVM which was connected to my server died a week earlier and was sent to RMA…

With connectivity to Linux and VMware gone, and KVM unavailable, the only way was to have a look locally, by connecting keyboard and monitor directly to the server.

What did the datacenter techie see on my box? Bluescreen I hear you say. Close. But on VMware bluescreens are… Pink. So there was a Pinkscreen waiting there. A simple power restart was enough to restore all the services back online.20121120_124732093_iOS

P.S. My coleague from work says VMware Pinkscreens are not PINK, they are PURPLE.

Why it’s wise to heed warnings in user manuals

I’ve been reading through a manual for an IBM DS 3400 Disk Array, and found a pronounced warning:

never_look_into_fiber

Many people don’t read manuals at all (claiming it’s a waste of time). Even those who do, often don’t treat warnings seriously. The one above is however quite serious. During my SAN training I was told to never look into an operating fiber link. Why? It’s simple. A long-wave fiber interface (which means it’s both for long distance and uses long wavelength) emits an invisible, but nevertheless powerful laser beam. Why it’s invisible? Because the wavelength is in deep infrared 1310-1550nm. Cheap cameras, which don’t have an IR filter (for example mobile phone cameras) can easily detect such IR radiation. This turns mobile phones into cheap, portable devices to check if there is signal on the fiber:

IMG_0782 IMG_0047

Webserver Logs vs Google Analytics

Who should read this post

My fellow EVE bloggers and all other webmasters who care about their site and would like to have more visitors.

Logs vs Reports

Many website admins have to decide which is better for analysing traffic on their website. I was struggling with answering this question as well, but the latest improvements introduced by Google (Real Time (beta) or Intelligence Events) make Analytics a great tool, that no logs or log parsers can compete with.

Raw Apache logs are very useful for most anything – from analyzing the number of “hits” on the website, to debugging errors in the PHP code. Unfortunately, they do not gather as much information as Google does. Analytics on the other hand uses cookies to track users movement around the website and allows to discern returing users from new ones. It also collects information such as screen resolution, flash and java versions, language, website loading time, the domain of the visitor or approximate location based on GeoIP.

Apache LogsI used to favor Apache logs, because with tools such as Webalizer it was possible to see graphs and trends of visits. When skimming through the raw log it is also possible to see the IP address and other corresponding data such as OS and browser versions. Unfortunately, neither raw logs, nor the reports based on them are going to say anything more than that, not to mention that raw logs are not the easiest (or interesting in that matter) things to read.

And there comes Google Analytics. You can’t access the raw data gathered by Google, but Analytics reports are very useful, not just for bloggers, but also for big websites and e-commerce based companies. Moreover, you can create your own custom reports,which can be based on any of the available metrics.

But why am I saying all of this to you?

CCP was here!

Because thanks to Analytics I was able to make some very interesting findings. For one, I know that CCP devs have been visiting my website since I started publishing information about Crucible (so I have promptly added the copyright notices everywhere 😉 ). I know that an Asus EEE Pad Transformer owner from Ontario has visited my website (and I think I know who is that 😉 cheers to Kirith Kodachi).

Visitors world map I know that most EVE Online players come from US, UK Russia and Germany. I know that most of the visits come from Google search. Twitter and various forums only make a fraction of percentage in my incoming traffic.

I know what keywords people used to land on my website and what information to feed them with to achieve more visits. Of course this information/opinions need to have proper quality, because otherwise  these users will not return. Posting accurate information is crucial. When visitors consider your information important and accurate, they are more likely to spread it further, this includes forum links or sharing on Twitter and other social networks.

One day my post about Cockroach ship has received about 150 hits in just one hour. I wondered why people would out of a sudden begin looking for a ship, which is hid deep in the EVE data dump, and is rarely seen by players. So I started investigating. My post was quite high on the Google search for the keywords containing “eve online cockroach”, so the question remained, what is the root cause. I quickly found it was a Facebook post of CCP employee, who linked a killmail of one of the GMs, who lost his Cockroach to a group of players.

Can your webserver logs tell you this much?

I don’t think they can. So now go and connect your blog to Analytics. You will not regret, it’s a very powerful tool.

Paper about virtualization

Paper about virtualization

If you haven’t heard or used virtualization yet, now is a good time: this trend has been growing over the past years. Now most companies are using this technology to cut on IT costs. Fear not, this technology is free and available for everyone. I have written a paper about what virtualization is and how to get involved – feel free to read it here.

I have also plans to write more about each free virtualization product available, so stay tuned.

Become a DBA in just one day

Oracle is the industry standard for databases, no one will argue here. After acquiring Sun Microsystems they have also become one of the leaders as telco-grade systems supplier (HW + OS + DB). Most admins know the basics about how Oracle works, how to start it up, shut it down, and with a bit of googling, how to restore and backup the db.

I had a similar problem today (our DBA is on holiday) and we needed to quickly restore the DB to proper operation.

For those of you looking for a quick cheat-sheet on how to do the basics with Oracle I have prepared a reference page.

Best book for beginner DBA: here.