High Availability or Disaster Recovery

Why you can’t rely on a replica for disaster recovery (courtesy: MySQL Performance Blog)

  • A replication is not a backup. It is aimed to provide High availability (HA) , not disaster recovery (DR).
  • Replication cannot withstand human error like dropping a table in the master database. Hence, it’s imperative to backup your db
  • Oracle supports raw devices so you don’t have to rely on a file system driver for anything! You write the bits out to the device yourself and have complete and total control over that device. InnoDB support it too
  • Use journaling file system (no ext2)

Related post (courtesy: Florian blog)

DRBD doesn’t magically make your app safe

It is a common misconception that DRBD (or any block-level data replication) solution can magically make an application crash-safe that intrinsically isn’t. Baron highlights that misconception in a recent blog post.

I want to reiterate and stress that point here: if your application can’t reliably survive a node crash, it won’t successfully fail over on a replicated (or shared, for that matter) data device. But if it can, and DRBD is replicating synchronously, then DRBD won’t break it. In other words: try pulling the power plug on your machine while your app is running, and power back on. If your application recovers to a consistent state, you’re clear. If it doesn’t, don’t bother adding DRBD until you fix that.

You must fix any layer in your stack that isn’t crash safe, if you even want to start thinking about high availability. ext2, which Baron mentions in his post, isn’t crash safe. MySQL with a database using the MyISAM storage engine isn’t crash safe. KVM with virtual block devices in cache=writeback mode isn’t crash safe. Running on a RAID controller with the write cache enabled when its battery is dead isn’t crash safe.

Thus, if you want high availability, use ext3. Or ext4. Or any journaling file system. Use InnoDB for MySQL. Use cache=none for KVM. And check those batteries. It’s that simple.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s