Microsoft didn’t kill SBSfaq.com

You may have read my post from yesterday about how SBSfaq.com was dead in the water for 4-5 hours yesterday.  I was instantly suspicious of Microsoft and their patches as being the root cause of the problem.  I mean it had to be Microsofts fault because patches had just been applied to the box and it had been rebooted right!!!  WRONG.

It just goes to show that you can’t always jump to conclusions.  I find often when troubleshooting a problem with others, that they get cemented into a one track thought process too early on in the investigation.  once they have an idea in mind, the close their thoughts to anything else.  They often throw all other logic out the door in the thought that they are on the right path.  This is in fact VERY dangerous. 

When troubleshooting a problem – ALWAYS remember the basics…

1. When did it last work? For me – it was last working before the reboot – NOT before the Microsoft patches – and that’s a fine line but you need to be clear about it.  The server WAS working fine up until the last reboot.  The reboot was a side effect of the Microsoft patches.
2. What has changed? For me the things I KNEW had changed was, the Microsoft patches had been applied, a reboot had occurred and the website was not working.
3. What do the log files say? I firstly reviewed Windows System log for any clues there – and there were none.  I then reviewed the Windows Application log – there were a few IIS related errors but nothing seemed to be “serious” enough to cause the site to be down.
4. What is still working?  Break the problem down into smaller chunks to prove / disprove things.  For me – I have a few websites on this server – some are straight HTML sites and others are PHP sites.  Placed a very basic “Hello world” HTML file inside SBSfaq.com and tested it – it worked just fine.  That suggested to me that the issue was then related not to IIS as all other IIS sites on the server were working, but to something else linked to SBSfaq.com.  And that lead me to look at PHP.  I placed a basic PHP file in the SBSfaq.com and that worked too.  I then decided I needed to look further into PHP and went searching for it’s error logs.  I found they were located in the c:\windows\temp folder and that led me to finding the huge number of session files.  I pondered if this was part of the problem and decided to delete them as based on my digging on the web it was not going to hurt it.

Anyway – that’s how my brain worked through this problem – I hope it helps someone else think through their problems when they strike an issue.  Thanks go out to Radek at VMVault – he restored my server to their DR environment that allowed me to later review this problem and prove it was not the Microsoft patches that caused it – the backup was taken BEFORE the patches were applied – one more reason to do good backups.