Tag Archives: ESXi

vSAN- Controller Driver is (not) VMware Certified

In the process of upgrading a vSAN ReadyNode cluster from ESXi 6.5 to 6.7 a warning appeared in the vSAN Health check. The first host in the cluster had gone through the upgrade and was now showing the warning “Controller driver is VMware certified” (Note 1 in the image below, click on it for a larger view). The Dell HBA330 card was using an older version of the driver (2 in the image below) than recommended (3).

image

All workloads were still online, but running VMware Update Manager (VUM) did not clear this warning. Looking in the VUM patch listing showed the driver for ESXi 6.5 (4) but not the version recommended for 6.7.

image

Solution

It was necessary to manually load these replacement drivers in. A quick google showed they could be sourced from VMware’s download site. Extract the ZIP file from the download and then use the “Upload from File” option in VUM (5) to upload the ZIP file which was inside (in this case “VMW-ESX-6.7.0-lsi_msgpt3-17.00.01.00-offline_bundle-9702440.zip“). The new driver should then appear in the list (6) and will automatically be added to the “Non-Critical Host Patches” baseline (7). Final remediation is now just a case of applying that updating baseline to the host.

SNAGHTML3018550

In this particular instance the hosts were Dell PowerEdge R630 vSAN ReadyNodes with the HBA330 SAS HBA Controller option but the principles outlined in this post should apply to other configurations with the same symptoms.

vSphere 6.0- time to upgrade

vCenter-logoIf you’re running VMware vCenter and ESXi 6.0 it’s time to start planning to upgrade as General Support ends on 12 March 2020- one year from now and five years from it’s release. Thankfully the upgrade from 6.0 to 6.5 or 6.7 is usually quite straightforward, and VMware have put a lot of work into streamlining this process.

Looking at the Product Lifecycle Matrix other notable products in the VMware stable worth keeping an eye on include NSX for vSphere (NSXv) 6.2, Site Recovery Manager (SRM) 6.0 and 6.1, and vSAN 6.0-6.2.

Powered Off VM cannot be Powered On

Symptoms

A powered off VM on ESXi 6.5 will not power on and returns the error “Failed to power on virtual machine…. The attempted operation cannot be performed in the current state (Powered off)”.

(i.e. the VM cannot be powered on BECAUSE it is not powered on!)

2019-02-19 (12)

Prior to being powered down the VM properties had just been modified. In this particular case it was immediately following a manual Ubuntu install and the install DVD (from a datastore ISO) was disconnected and the CD Drive switch to” “Host Device”. These operations were performed from the ESXi Web interface.

Repeated attempts to start the VM all fail the same way.

Solution

Unregister the VM, then locate the vmx file in the datastore and re-register it. The VM should now power on.

image

ESXi UNMAP not working on Replicated EqualLogic Volume

Symptoms

  • The VMware vSphere ESXi UNMAP command doesn’t release space on some or all volumes on a Dell EqualLogic SAN array running v8 firmware (may apply to other versions too). Using the following command in an SSH session to a 6.0u2 host (again, will apply to other versions):
    esxcli storage vmfs unmap –l MYVOLUMENAME
  • The volumes are VMFS5 (and always have been- they haven’t been upgraded from VMFS3).
  • Replication is enabled for the volumes that won’t rethin.

image

Cause

UNMAP doesn’t work on the EqualLogic when Replication is enabled. It doesn’t return an error to the SSH session, and the temporary rethinning file is still created, but the disk is not thinned.

Solution

Disable replication on the volume, re-thin the volume using the UNMAP command, then re-configure replication. Unfortunately this means the entire volume must be re-copied to the replication partner and this may impact bandwidth usage and replication schedules on larger volumes.

image

The All New vSphere 6.5

vmworldHighlights

  1. A new version of VMware vSphere, 6.5, will be released shortly
  2. Migration/Upgrade tools from previous versions (including Windows vCenter) to new VCSA.
  3. VCSA Native High Availability
  4. VCSA Integrated VMware Update Manager
  5. Native vCenter Backup and Restore
  6. Improved Appliance Management
  7. vSphere Clients
  8. Encryption

New vSphere coming soon

VMware has bucked the trend in versioning adopted by other major software companies and decided not to call it’s new vSphere version “10” and opted for the more traditional “vSphere 6.5” to succeed version 6.0 which was originally released back in March 2015. Announced at VMworld Europe 2016 with GA to follow, vSphere 6.5 is a continuation of the product which forms the core of the Software Defined Datacentre chunk of VMware’s “Any Cloud” Cross-Cloud Architecture portfolio. A lot of work has been put into making the experience of installing and operating a vSphere virtualised environment easier; Ignoring any improvements under the hood, and just looking at what’s on the surface there’s a whole bunch of features designed to make life run smoother for the IT Professional, some of which are highlighted in this post.

The new vCenter Server Appliance is a core part to this simplicity, and VMware have answered the requirements of anyone currently sticking to the Windows-based vCenter. If you can get more features and more reliability for less cost and less effort then it’s definitely the way forwards in my opinion. Some of the features discussed here- notably Native HA and Backup/Restore- will only be available in the appliance version of vCenter.

VCSA Upgrade and Migration

 

image Again out to both simplify the life of IT Professionals and encourage vCenter Appliance adoption, VMware has put a lot of effort into creating straightforward, and comprehensive, upgrade and migration tools. As more and more operations and data are handled by vCenter it becomes more and more important that the system can be smoothly navigated from version to version with minimal human effort.

Migrations are possible from Windows vCenters running version 5.5 or 6.0, and both the embedded and external database topologies are supported. Additionally, the new vCenter will assume the identity of the old Windows vCenter so any external interfaces, scripts, and automation should continue to work post-migration.

VCSA Native High Availability

VCSA 6.5 offers a built-in high availability deployment taking away the need for any 3rd party clustering or database solutions. The appliance deploys as an active/passive pair (plus witness) which automatically sets up replication of the integrated database and required vCenter files. The basic setup option also places these nodes intelligently using DRS and SDRS technology and automatically creates the necessary affinity rules and private IP comms, keeping everything simple. For infrastructures with unique and challenging topologies, there’s still an advanced workflow that can be used.

image

Integrated VMware Update Manager

Prior to 6.5 using VUM to manage the patching of a vSphere infrastructure based on the vCenter Appliance has been, how can we put it?, “annoying”. After deploying the slick appliance it was then necessary to spin up (and license) a separate Windows VM just to handle the update system. This requirement has been removed in the new version- VUM is now integrated into the VCSA, enabled by default, and shares the same database instance. The new VUM integration also leverages the VCSA High Availability and Backup functionality.

Native vCenter Backup and Restore

Also new to the vCenter Server Appliance is integrated backup and restore functionality. A great step forward in the simplification of deploying a system this provides a built in solution to backup vCenter to an external location (SCP, SFTP, HTTPS locations for example) and then be able to recover by deploying a clean OVA and choosing the Restore option. image

 

Improved Appliance Management and Monitoring

The vCenter Server Appliance Management Interface- VAMI – has also had a makeover, with many features being added. The 6.0 version had an interface limited to changing IP and NTP settings, rebooting the appliance, and little else. 6.5 adds in built in monitoring of Network, CPU, Memory and the vPostgres database. There is also the option to configure Syslog for deeper external monitoring of the vCenter infrastructure- this allows fully verbose logs to be kept for auditing and troubleshooting processes.

image

vCenter Server Appliance 6.0 Management Interface

image

vCenter Server Appliance 6.5 Management Interface

vSphere Client(s)

Work continues to focus on delivering a fully functioned HTML5 client, but in the interim vCenter 6.5 will come shipped with a new (limited) HTML5 based “vSphere Client”- evolved from the current fling – as well as an improved flash based “vSphere Web Client”. Expect the “vSphere Client” to see continuous improvement and feature addition through the lifetime of the platform –driven through the Fling programme.

Encryption

As with the other topics here encryption in the new vSphere could easily be a post in itself (or a whole series), but to summarise the new features in this area, vSphere is now offering built-in VM encryption. The encryption happens between the VM and the storage so is invisible to the guest.

Local keys are generated within vSphere, and encrypted using keys held in an external (third-party) KMS- this would usually be managed by the IT Security team. Back in vCenter encryption is implemented through Storage Policies, so a VM can be encrypted simply by assigning the correct policy to it. Through the GUI (or API/PowerCLI) it’s possible to set  encryption covering  the Disks, the VMX/Swap files, or the whole lot on a per-VM basis. Through the API/PowerCLI it’s also possible to arrange encryption on a per-VHD level, potentially encrypting different disks on a VM with different keys.

VSAN encryption is on the way- there’s currently an ongoing beta – but will not be available in the 6.5 release. Based on the recent cadence I’d expect to see something in Spring 2017, but that’s just my speculation.

Summary

In summary, there’s lots to look for in the new vSphere release and in particular the vCenter Server Applicance. This week’s VMworld should reveal a lot more in depth into these advances.