Search This Blog

Monday, May 16, 2011

Verifying and reinstalling the correct version of the vCenter Server agents

Verifying and reinstalling the correct version of the vCenter Server agents

Symptoms

  • Configuring VMware High Availability (HA) fails
  • If you try to configure VMware HA, you see the error:

    Could not Enable aam firewall ruleset :vim.fault.HostConfigFault
     
  • You cannot connect an ESX host to VirtualCenter/vCenter Server
  • If you try to connect an ESX host to vCenter Server, you see the error:
    unable to access the specified host, either it doesn't exist, the server software is not responding, or there is a network problem
     
  • The hostd.log file contains one or more of the following entries:
    [2008-05-26 10:45:51.463 'Vmomi' 15752112 info] Throw vim.fault.AlreadyExists
    [2008-05-26 10:45:51.463 'Vmomi' 15752112 info] Result:
    (vim.fault.AlreadyExists) {
       name = "vpxuser"
       msg = ""
    }
2011-05-11 17:55:31.745 F5769B90 info 'Vmomi' opID=EE62B2D1-0000184B-ec-ab] Act
ivation [N5Vmomi10ActivationE:0x5bd7b5d8] : Invoke done [enableRuleset] on [vim.
host.FirewallSystem:firewallSystem]
[2011-05-11 17:55:31.745 F5769B90 verbose 'Vmomi' opID=EE62B2D1-0000184B-ec-ab]Arg id:"aam"[2011-05-11 17:55:31.745 F5769B90 info 'Vmomi' opID=EE62B2D1-0000184B-ec-ab] Throw vim.fault.NotFound[2011-05-11 17:55:31.745 F5769B90 info 'Vmomi' opID=EE62B2D1-0000184B-ec-ab] Result:(vim.fault.NotFound) {   dynamicType = <unset>,   faultCause = (vmodl.MethodFault) null,   msg = "",}

Purpose

This article helps you determine the version of the vCenter Server agent that is installed on an ESX host and provides steps to reinstall the VMware HA and vpxa agents if the version is incorrect.

Resolution

Note: Proceed directly to Reinstalling the agents on ESX if you see this error when you try to connect an ESX host to vCenter Server: unable to access the specified host, either it doesn't exist, the server software is not responding, or there is a network problem

Verifying that the correct version of vCenter Server agents are installed on ESX

To verify that the correct version of vCenter Server is installed:
  1. To determine the version and build number of your vCenter Server installation, click Help > About from within vCenter Server.
  2. Right-click the ESX host and click Disconnect to disconnect the ESX host from vCenter Server.
  3. Log in as root to the ESX host using an SSH client.
  4. Run this command to determine the version of the vCenter Server agent (vpxa) that is installed:

    [root@esx-server /]# rpm -qa | grep vpxa

    The output appears similar to:

    VMware-vpxa-2.5.0-64192
  5. Ensure that the output matches the information from step 1. If the versions do not match, reinstall the agents.

Reinstalling the agents on ESX

  1. Disconnect the ESX host in vCenter Server. Right-click the host and click Disconnect.
  2. Log in as root to the ESX host using an SSH client.
  3. Paste this string of commands into the SSH session:
    service mgmt-vmware stop && service vmware-vpxa stop && service vmware-vmkauthd stop && service xinetd restart && rpm -qa | grep -i vpxa | awk '{print $1}' | xargs rpm -ef $1 && userdel vpxuser && rpm -qa | grep -i aam | awk '{print $1}' | xargs rpm -ef $1 && service mgmt-vmware start && service vmware-vmkauthd start

    Note: This is a concatenated list of commands strung together with the "&&" operator. This is useful in stringing together multiple commands as the next command does not start until the previous one has completed successfully.

    Here is a description of the commands:
    • The service mgmt-vmware stop && service vmware-vpxa stop && service vmware-vmkauthd stop && service xinetd restart command stops the management services for the service console.
    • The rpm -qa | grep -i vpxa | awk '{print $1}' | xargs rpm -ef $1 command removes the vpxa agent.
    • The userdel vpxuser command removes the vpxa user that gets installed once you connect the host to vCenter Server.
    • The rpm -qa | grep -i aam | awk '{print $1}' | xargs rpm -ef $1 command removes the AAM RPM (HA) which gets reinstalled when you configure HA.
    • The service mgmt-vmware start && service vmware-vmkauthd start command following restarts the services.
    • The vmware-cmd -l command lets you know when hostd is up and running so you can reattach it to vCenter Server.

      Note: When you see a list of virtual machines output, the command sequence is complete.

  4. Log in to the vCenter Server using VMware Infrastructure/vSphere Client.
  5. Choose the ESX host from the Inventory.
  6. Right-click on the ESX host and click Connect. This step initiates a re-install of the agent. 

Reinstalling the VMware HA components on ESX

To reinstall the VMware HA components:
  1. Remove the ESX host from the vCenter Server Inventory.
  2. Log in as root to the ESX service console.
  3. Run the command:

     rpm -qa | grep -i aam
    .
    This returns two packages that are named similar to:
    • VMware-aam-haa-#.#.#-#
    • VMware-aam-vcint-#.#.#-#
       
  4. Run these commands to remove the packages returned by Step 3:
    rpm -e
    VMware-aam-vcint-#.#.#-#
    rpm -e VMware-aam-haa-#.#.#-#
    Note: Ensure to remove the VMware-aam-vcint-#.#.#-# package first.
  5. Run the command:

    rpm -qa | grep -i vpxa
    A package named VMware-vpxa-#.#.#-##### is returned.
     
  6. Run this command to remove the package returned by Step 6:

    rpm -e VMware-vpxa-#.#.#-#####
  7. Test adding the host to the newly created cluster to see if this has resolved the issue. 

Verifying that the correct version of vCenter Server agents are installed on ESXi

  1. To determine the version and build number of your vCenter Server installation, click Help > About from within vCenter Server.
  2. Right-click on the ESXi host and click Disconnect to disconnect the ESXi host from vCenter Server.
  3. Log in to ESXi Troubleshooting Mode.  For more information, see Tech Support Mode for Emergency Support (1003677).
  4. Run this command to determine the version of the vCenter Server agent (vpxa) that is installed:
    [root@esx-server /]# cat `ls -t /var/log/vmware/vpx/vpxa*` | head -1

    The output appears similar to:
    Log for VMware VirtualCenter Agent, pid=1976, version=2.5.0, build=build-147633, option=Release, section=394
     
  5. Ensure that the output matches the information from step 1. If the versions do not match, reinstall the agents.

Reinstalling the ESX host management agents and HA agents on ESXi

  1. Right-click on the ESXi host and click Disconnect. This disconnects the ESXi host from the vCenter Server Inventory.
  2. Log in to ESXi Troubleshooting Mode. For more information, see Tech Support Mode for Emergency Support (1003677).
  3. Uninstall the vpxa and aam components from the VMware ESXi host using these commands:
    /opt/vmware/uninstallers/VMware-vpxa-uninstall.sh

    /opt/vmware/uninstallers/VMware-aam-ha-uninstall.sh


    Note: Run the cat /etc/shadow command to verify if the vpxuser has been removed by the uninstall script. If the vpxuser has not been removed, run userdel vpxuser.
     
  4. Right-click on the ESXi host and click Connect. This initiates a re-install of the vpxa agent. 
  5. Add the host to the appropriate cluster. This initiates a re-install of the aam agent.
If the agent fails to automatically install when connecting the ESX host in the vCenter Server:
  1. Verify that /tmp/vmware-root exists on the ESX host. vCenter stages the vpxa RPM in this folder. To add this directory, execute:

    mkdir -p /tmp/vmware-root 
  2. For more information on investigating disk space problems on ESX hosts, see Investigating disk space on an ESX host (1003564).

No comments:

Post a Comment