Exchange 2007 SP1 fixes lots more than is documented

I’ve been wrestling with Exchange 2007 on my one of my larger sites now for the last 7 weeks. I’ve had the case with MS PSS here in Australia for that long too and they have not been able to make ANY progress on it at all. When I first lodged the case, Exchange 2007 SP1 was in beta – and at that time I asked if the issues we were experiencing were resolved by it at all – apparently they could find no such solution to our problem in the notes for SP1 – therefore they did not (and as at last Friday – a week and a bit AFTER SP1 was released) suggest or recommend we install it. Their words were “you can try it if want but you don’t have to and we don’t know of anything that will fix it”.

Despite all this – the customer is getting increasingly frustrated at the lack of progress with this issue – we have put a workaround in place using Virtualised Exchange 2003 servers on each of the remote sites, but that was a band aid solution. I decided that I had to try as I know from experience that not all the fixes documented in a service pack actually fix things the way the software vendor intended them to. So I applied Exchange 2007 SP1 to the sites and it worked!!! I got a few funny error messages along the way and I’ve detailed them below in case you strike them yourself.

Remember – when your working with a software vendors support team – don’t assume that they know it all. Ask intelligent questions and always question what they ask you to do so that you know where you are heading with the fault investigation. If they don’t have a plan of attack with the next few steps thought out then ask for another engineer.

Issues we had that were directly fixed with Exchange 2007 SP1 included;

1. Mail was backing up in mail queues on the remote site servers and no mail flow was possible between the remote servers (although strangely mail would flow from the remote sites OUT via the Head Office Exchange 2007 server to the net, but not the other way).

In the mail queue viewer we got the following messages depending on if we were looking at the headoffice or the remote servers mail queues;

421 4.4.2 Connection Dropped – this was seen when the HO server was talking to some remote sites

451 5.7.3 Cannot Achieve Exchange Server Authentication – this was seen when the HO server was talking to ONE remote site

The SMTP troubleshooter when run on all remote servers gave even more cryptic messages including;

530 5.7.1 Client was not authenticated – this was seen when the remote sites were talking to ALL other remote sites (except ONE)

Remote server servername.domain.local failed the mail acceptance test. BDAT command: Respond = Remote socket is not available. Check for firewalls and applications that can possibly block the BDAT command. – This was seen when the both the head office site and remote sites were talking to ONE remote site

We applied Exchange 2007 SP1 to all sites and mail flowed again without a problem.

2. The second issue we had on ONE server only (and this may be a side effect of the investigations done to date) was that whenever we tried to assign a certificate to the SMTP service we got the following messages;

And further more when we went to install Exchange 2007 SP1 we got this message;

Hub Transport Role

Failed

Error:

Active Directory operation failed on servername.domain.local. This error is not retriable. Additional information: Insufficient access rights to perform the operation.

Active directory response: 00002098: SecErr: DSID-03150A45, problem 4003 (INSUFF_ACCESS_RIGHTS), data 0

The user has insufficient access rights.

We resolved this issue by deleting cached credentials (go figure) using the following command;

Start > Run > “control keymgr.dll” and then delete all entries that were there

We then reapplied Exchange 2007 SP1 and it worked fine. This site was the one that gave funky authentication messages and the BDAT message listed above – I don’t know how the cached password could have caused this issue or even if it was the fault – but I mention it for sake of completeness.

3. The last issue was that when applying Exchange 2007 SP1 to a another server, it stopped part way through with an error indicating that the Application log file was full. We cleared the application log and restarted the SP install – but it then failed with an error indicating that IIS was not running or installed or disabled.

I looked and found that the following services were stopped and set to disabled IIS Admin Service, HTTP SSL Service and World Wide Web Publishing Service. I set them to automatic and started them and then restarted the SP1 install – all went through without an issue.

Hopefully my experiences will save you from a similar fate – good luck 🙂