WD Sentinel DX 4000 – Initializing

Yesterday, my WD Sentinel DX 4000 appeared to have locked up – it had stopped responding to the front panel buttons. I wondered why it had done this and then I found I had inadvertently taken it off the UPS when I re-cabled some things the other day and I suspect we had a power fluctuation which caused it to glitch. Anyway – the only way to resolve this was to hard power it. Now this is something that you normally SHOULD NOT DO to any computer, least of all a server like this. But it was my only choice.

After the reboot, I found that the front panel had advised that it was Initializing…. now at first I panicked and thought that it was erasing and initialising everything on the server. I was however able to RDP to the server and verify that my data was there. I left it and after 8 hours, it was up to 5% done. I also noticed that anything I tried to do via RDP on the DX 4000 was very slow as well. I figured that the initialisation process was very disk intensive, so I left it at that. I’d heard of this before and wondered what it all meant. I sent an email off to some people I know at WD to get some answers. Given it was late at night, and me being impatient, I later decided to reboot the server and found that it seemed to be far more responsive, however the Initializing process continued at it’s normal rate. Given my data was not at risk, and that performance seemed to be fine, this didn’t bother me.

This morning, my status was as below on the front panel.

You can see via the Dashboard, that it’s showing much the same thing – all disks are normal and my data is intact.

Ok – so the official explanation of what is going on here.

This is normal behaviour for the WD Sentinel DX 4000 under the following circumstances:

• Normally a RAID system is not initialized and instead relies on the fact that every write to the array updates the parity. This is true until an unexpected power cycle occurs. The parity status may have been compromised and therefore the entire RAID system needs to be initialized.
• Likewise the first time WD Sentinel unit encounters an unclean shutdown, it goes into RAID initialization mode. This is a slow and long running operation, even more so on the larger capacity units e.g. 12 TB. WD Sentinel’s design ensures that even during the RAID initialization or RAID verification process, the system and access to the data is maintained.
• As you would expect, the RAID initialization process is quite resource intensive, and further to maintain enough bandwidth to sustain access to server and data, the bandwidth for RAID initialization is conceptually throttled. Our tests have indicated that at 20MB/s a 12 TB system could take up to 7 days to complete the initialization.
• Any subsequent abrupt power loss would result in RAID verification cycle instead of RAID initialization, which is a relatively much faster process.

I know that WD are aware of the issue and they are looking into ways to better handle this in advance of any unclean shutdown. They have assured me that the chances of data loss are quite minimal provided I allow the initialization process to complete – which of course I am doing.

Oh – I forgot to mention – I’ve moved this back over to the UPS so that it won’t experience any unplanned failures again.