Ok – I’ve done a fair bit of work with StorageCraft ShadowProtect over the years – possibly even being one of the first in Australia to test them out. I use ShadowProtect in many of our customer environments, sometimes as the only backup product, other times in conjunction with other products. It really depends on the specific scenarios that we have as to what we use and why.
In my personal SBSfaq.com server environment, more recently I’ve been using the inbuilt SBS 2008 backup, but last week switched over to the ShadowProtect 4.0 release. I should clarify – I installed it but didn’t get a chance to configure it as I left it too late before walking out the door on a business trip. While on the business trip, I forgot that I’d not configured my backups for ShadowProtect 4.0. Therefore my server was not being backed up. Again – had I read my daily SBS 2008 reports, I would have known this. You can’t help some people now can you. 🙁
Ok – so Wednesday around 10:30am, my SBS 2008 server stopped responding to me. No idea why, but given I was heading home that evening, I figured I’ll deal with it then. I got home that evening to find that my server was non responsive. Strange I thought, looks like the only solution is a power cycle it. I power cycled it and then let it reboot. It didn’t quite come up correctly… well it came up with a logon screen, but refused to accept my password. I power cycled it again and tried to get into safe mode – no go – “No logon servers available to service your request” was the error message – or similar to that. Ok – at this point it looked like my server was toast. I could also hear a strange clicking noise coming from the server – a sign that one of my hard drives had failed. Not good at all. My last known good backup was from around a week ago which I’d done using the SBS 2008 backup program. I knew that there was other data and email that had changed in that time and I was not keen to lose that.
So I took my ShadowProtect IT Edition USB stick and plugged it into the failed server – changed the BIOS to boot from removable devices and attempted to boot from the USB stick. “Operating System not found” was my boot message – hmm – not good start. No problems – there’s a tool on the ShadowProtect IT Edition USB stick to create an ISO for just those circumstances. Took the USB stick to my laptop and created the bootable ISO. Yup – boots from that but I need to have the USB stick installed so it can located the licensing files it needs – gotta make sure I’ve not stolen this copy of IT Edition. All cool so far – it’s booted up and now I can see my failed servers hard drives. Ok – time to commence a backup to an attached USB hard drive. Cool – all works well and I get the core of my server down to the USB drive – I’ve got a 500GB mirrored set and a 1.5TB mirrored set – for now – I focus on the 500GB drives as this is where my core OS and email are located. The 1.5TB drives are other things like ISO images etc that is not so critical to me.
I’ve been going to take my SBS 2008 server over to Hyper-V for some time – no time like the present 🙂 I created a fixed size VHD on Hyper-V while the failed server was being backed up, so it was all ready for me to go now. I took the USB hard drive and connected it to the Hyper-V Host. I then used the Disk Management software on Hyper-V to mark the USB Hard drive as offline. Once it’s offline you can add it to the SBS 2008 Virtual Machine as a hard drive. Of course you can’t boot from the IT Edition USB stick, so I booted the Virtual Machine from the CD Image of the IT Edition. As it’s booting up, it looks for the licensing files – naturally it can’t find them – no issues – StorageCraft have thought of this and provided a Licensing Server tool that you can run on one of your machines in the network. The only problem with this idea is that given I am recovering my SBS 2008 server, I don’t have a DHCP server on my network. Therefore I had to reconfigure my Netgear firewall/router to be DHCP so the virtual machine could get an IP and therefore see the licensing server. Oh don’t forget (its written on the screen) that you need to create a firewall exception on the machine hosting the licensing server for port 20248. Make sure you remove the firewall exception later of course. Ok – now we’ve got that all sorted they Hyper-V virtual machine can be rebooted and it now can see the licensing server just fine. The Restore can commence.
I will say here though that this reconfiguration I’ve had to done on my network just to confirm to StorageCrafts licensing requirements is a real pain in the ass. It’s not the type of thing I want to have to do during a disaster recovery scenario. However they’ve been forced to take this path due to people deciding to pirate their wonderful tool. Shame on you shonky resellers. Having said that I hope that StorageCraft have a good think about this fact – it’s not easy to reconfigure your network just to recover a single machine at times.
Ok – the restore is completing now. It took 44mins to restore the C: drive (80GB), and 20mins to restore the E: drive (around 20GB data). I decided not to restore the data partition at the moment – I wanted to get the server up and running. I performed a HIR from within the SP IT Edition environment and then shutdown VM. I disconnected the USB hard drive, and booted back up. Booted it into Safe Mode first up so I can fix an IP address on the SBS 2008 servers NIC and then rebooted. Woohoo – I’m back on the air. Most people would stop there, but you need to re-run the Connect To Internet wizard from the SBS 2008 console. You need to do this as the SBS 2008 server stamps certain information into the registry based on this wizard. Now we’re cooking – my email is flowing in again. Given it was a Saturday, I left it running for a few hours before shutting it down and restoring the F: data drive (this was around 250GB of data). This took 3.5 hours to restore. I’ve yet to restore the 1.5TB drive – will leave that for later and restore it to an alternate VHD in a seperate VM and then later move the VHD over to the SBS 2008 server.
Ok – so what’s the lessons here…
1. Restoration time too the better part of a day all up. In a real business DR scenario this is pretty good – but it could have been reduced dramatically if I’d been using a feature called Head Start Restore from ShadowProtect. This feature is very cool and keeps a “near ready” image of your working server in your chosen format – ie – I could have had it have a VHD standing by ready to be mounted.
2. Be prepared to reconfigure your network if needed. I’m not real happy about having to do this just to conform to the licensing requirements. It’s not something I was expecting, and there’s a few other things that I found on the way that I’ve reported to StorageCraft as bugs/suggested improvements – but during a DR – you need to do the minimum needed to get the systems running.
All in all a great result – woohoo for ShadowProtect IT Edition… but I could have improved my DR had i been running the ShadowProtect SBS Edition BEFORE the disaster occurred. I’ll blog separately about the reason for the DR which in itself is very worrying due to the funky problem with the hardware (nothing related to ShadowProtect at all).