Unable to deploy NSX VIBs after updating certificates in vSphere 6.0

I ran into an issue the other day while helping a customer install the NSX agents on his ESXi hosts. Whenver we tried to deploy the agents to the host, we would get a generic error in the vSphere Web Client. To troubleshoot the issue, we first navigated to Home -> Administration -> vCenter Server Extensions and double clicked vSphere ESX Agent Manager.
 

 
By clicking the Manage tab we saw no ESX Agencies / an error similar to:
 
“Error while creating eam agency for deployment”
 
In the NSX Manager log you see the following:
 

show manager log
2015-05-29 17:23:42.120 GMT ERROR taskScheduler-15 InstallTask:190 - error while creating eam agency for deployment 
com.vmware.vim.binding.eam.fault.NoConnectionToVCenter:
inherited from com.vmware.vim.binding.eam.fault.EamRuntimeFault:
inherited from com.vmware.vim.binding.eam.fault.NoConnectionToVCenter

 
This pointed to a problem with the ESX Agent Manager service on vCenter Server. After checking the EAM logs under we were able to find the following error message.
 
Note: The EAM logs are located here:

Appliance: /var/log/vmware/eam/eam.log 
Windows: C:\ProgramData\VMware\vCenterServer\logs\eam\eam.log

eam.log
Connecting to vCenter as com.vmware.vim.eam extension
Connecting to https://:8089/sdk/vimService via vCenter proxy http://localhost:80
HealtStatus request's token subject name: machine-7502fb4c-3521-48c7-93ed-3d1865e0fff1, subject domain: vsphere.local
Failed to login to vCenter as extension. vCenter has probably not loaded the EAM extension.xml yet.: Cannot complete login due to an incorrect user name or password.
 |  WARN | eam-0 | VcListener.java | 134 | Trying to recover from error
(vim.fault.InvalidLogin) {
  faultCause = null,
  faultMessage = null
}
 at sun.reflect.GeneratedConstructorAccessor82.newInstance(Unknown Source)
 at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
 at java.lang.reflect.Constructor.newInstance(Unknown Source)
 at java.lang.Class.newInstance(Unknown Source)
 at com.vmware.vim.vmomi.core.types.impl.ComplexTypeImpl.newInstance(ComplexTypeImpl.java:173)
 at com.vmware.vim.vmomi.core.types.impl.DefaultDataObjectFactory.newDataObject(DefaultDataObjectFactory.java:26)
 at com.vmware.vim.vmomi.core.soap.impl.unmarshaller.ComplexStackContext.(ComplexStackContext.java:31)
 at com.vmware.vim.vmomi.core.soap.impl.unmarshaller.UnmarshallerImpl$UnmarshallSoapFaultContext.parse(UnmarshallerImpl.java:141)
 at com.vmware.vim.vmomi.core.soap.impl.unmarshaller.UnmarshallerImpl$UnmarshallSoapFaultContext.unmarshall(UnmarshallerImpl.java:102)
 at com.vmware.vim.vmomi.core.soap.impl.unmarshaller.UnmarshallerImpl.unmarshalSoapFault(UnmarshallerImpl.java:89)
 at com.vmware.vim.vmomi.core.soap.impl.unmarshaller.UnmarshallerImpl.unmarshalSoapFault(UnmarshallerImpl.java:84)
 at com.vmware.vim.vmomi.client.common.impl.SoapFaultStackContext.setValue(SoapFaultStackContext.java:41)
 at com.vmware.vim.vmomi.client.common.impl.ResponseUnmarshaller.unmarshal(ResponseUnmarshaller.java:112)
 at com.vmware.vim.vmomi.client.common.impl.ResponseImpl.unmarshalResponse(ResponseImpl.java:273)
 at com.vmware.vim.vmomi.client.common.impl.ResponseImpl.setResponse(ResponseImpl.java:230)
 at com.vmware.vim.vmomi.client.http.impl.HttpExchangeBase.parseResponse(HttpExchangeBase.java:144)
 at com.vmware.vim.vmomi.client.http.impl.HttpExchange.run(HttpExchange.java:51)
 at com.vmware.vim.vmomi.client.http.impl.HttpProtocolBindingBase.executeRunnable(HttpProtocolBindingBase.java:186)
 at com.vmware.vim.vmomi.client.http.impl.HttpProtocolBindingImpl.send(HttpProtocolBindingImpl.java:77)
 at com.vmware.vim.vmomi.client.common.impl.MethodInvocationHandlerImpl$CallExecutor.sendCall(MethodInvocationHandlerImpl.java:581)
 at com.vmware.vim.vmomi.client.common.impl.MethodInvocationHandlerImpl$CallExecutor.executeCall(MethodInvocationHandlerImpl.java:562)
 at com.vmware.vim.vmomi.client.common.impl.MethodInvocationHandlerImpl.completeCall(MethodInvocationHandlerImpl.java:348)
 at com.vmware.vim.vmomi.client.common.impl.MethodInvocationHandlerImpl.invokeOperation(MethodInvocationHandlerImpl.java:308)
 at com.vmware.vim.vmomi.client.common.impl.MethodInvocationHandlerImpl.invoke(MethodInvocationHandlerImpl.java:182)
 at com.sun.proxy.$Proxy48.loginExtensionByCertificate(Unknown Source)
 at com.vmware.eam.vc.VcConnection.connectEam(VcConnection.java:171)
 at com.vmware.eam.vc.VcListener.login(VcListener.java:149)
 at com.vmware.eam.vc.VcListener.main(VcListener.java:129)
 at com.vmware.eam.vc.VcListener.call(VcListener.java:111)
 at com.vmware.eam.vc.VcListener.call(VcListener.java:60)
 at com.vmware.eam.async.impl.AuditedJob.call(AuditedJob.java:35)
 at com.vmware.eam.async.impl.FutureRunnable.run(FutureRunnable.java:52)
 at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
 at java.lang.Thread.run(Unknown Source)
VcListener.java | 121 | Retrying in 10

 
The only thing we did to induce this error message was to change the vCenter Server certificates. It appears that once you do this, the EAM service is no longer able to communicate properly with vCenter Server. To resolve this issue, you can follow the steps below.
 

Windows

 
Open a command prompt as administrator and run the following commands. The first two will retrieve the vpxd-extension solution user cert and key; the last commands will update the EAM certificate with vCenter Server.

# "%VMWARE_CIS_HOME%"\vmafdd\vecs-cli entry getcert --store vpxd-extension --alias vpxd-extension --output c:\certificates\vpxd-extension.crt

# "%VMWARE_CIS_HOME%"\vmafdd\vecs-cli entry getkey --store vpxd-extension --alias vpxd-extension --output c:\certificates\vpxd-extension.key

# cd C:\Program Files\VMware\vCenter Server\vpxd\scripts\

# "%VMWARE_PYTHON_BIN%" updateExtensionCertInVC.py -e com.vmware.vim.eam -c C:\Certificates\vpxd-extension.crt -k C:\Certificates\vpxd-extension.key -s localhost -u Administrator@vsphere.local

 
Provide your administrator@vsphere.local password when prompted.
 

Appliance

 
Log into the vCenter Server appliance via SSH and run the following commands.
 

# shell.set --enabled true

# mkdir /certificate

# /usr/lib/vmware-vmafd/bin/vecs-cli entry getcert --store vpxd-extension --alias vpxd-extension --output /certificate/vpxd-extension.crt

# /usr/lib/vmware-vmafd/bin/vecs-cli entry getkey --store vpxd-extension --alias vpxd-extension --output /certificate/vpxd-extension.key

 
Provide your administrator@vsphere.local password when prompted.