They say a backup untested is no backup at all. Our server system raid controller decided that enough was enough, and it was time to fully test our backup and recovery protocols.  It further decided that 3 AM was when it had to be done.

Repair, restore, and testing is now complete, and the server is back live.  I need a nap.