The Home Lab

Automated Deployment in the HomeLab- Part 1

I’m commencing a project with my HomeLab- I’m going to build a system whereby I can produce custom mini-lab environments by means of a script. There are off the shelf solutions to do this (see AutoLab as an example) but if I build this myself I get something tailored exactly to my needs (and available resources) and hopefully learn something along the way- which is what the HomeLab is all about really. This is the first post in what should develop into a series showing how I work through the process to create my automation system.

 

The Aims

a.k.a. what I want to achieve

  • ExampleMiniLabThe ability to run a script to deploy a predefined lab environment. For example running “Build-Project-Lab-One.ps1” makes 3 Windows Server VMs, connected on a private switch, with one running AD/DNS/DHCP roles, one acting as a Gateway, and one ready for whatever experiment I throw at it
  • The ability to quickly and easily modify a copy of that script to produce a lab with a different configuration. Then I can have a script that builds me a WDS platform, or another one that produces a SCOM test environment. I can use this library to quickly rebuild, or build a copy of, and of my environments within the HomeLab
  • This script should also create a second script for decommissioning  /destroying the lab environment when I’ve finished.
  • Whilst perhaps not meeting full “production” standards, the scripts should be at least in a state whereby I can post them online and not have to hide in a cave for the next decade whilst they get laughed at.

 

The Resources

a.k.a. what I have to play with

  • One Intel NUC host running vSphere ESXi 6 providing some compute, memory, storageIntel NUC
  • One VMUG Advantage Subscription complete with VMware EVAL Experience licensing- this provides VMware vCenter amongst other things.
  • One Microsoft DreamSpark Subscription and Microsoft Evaluation Licensing (see “Microsoft Licensing” on the Open Homelab Project for details on how to get these)
  • Me with my knowledge of Windows, vSphere, PowerShell, PowerCLI, and how to Google for stuff.
  • The community who not only kindly put content up on the internet for me to Google for but also are there for me to tweet, slack, and (shock, horror) talk to when I encounter problems or lose direction.

 

The Plan

a.k.a. How I’m hoping to achieve those aims with those resources.

To do all this I’m starting out by preparing a vSphere template of Windows Server 2012R2. I can deploy this- with customisations- using PowerCLI to form the building blocks of the lab environment. Once I have Windows VMs deployed I need to be able to configure them- this is where PowerShell remoting will come in handy- I can deploy roles and features and do some basic configuration. I’ll put together a PowerShell function to do all that. This function can then be re-used in the script to deploy multiple VMs with different configurations. For example:

CreateVM "Server1" $TemplateName $CustomizationSpec "Web-Server"
CreateVM "Server2" $TemplateName $CustomizationSpec "WDS"

I’ll use PowerCLI to deploy a private network within the Hypervisor and connect the VMs to it. This method will also be used to configure the connections to the gateway -one NIC pointing at the private switch and one at the internet-facing vSwitch already in place.

Some more in-depth PowerShell (possibly also arranged into reusable functions) to do the in-depth configuration of the roles. For example, when the script completes I want the Active Directory to be up and running, the Gateway providing an internet connection to the VMs, the VMs getting IP addresses from the lab DHCP and domain-joined. Basically I want to be able to run the script, make a brew, and come back and find a fully configured system ready to go.

 

Coming Soon- Part 2, full of scripting goodness.


Advert:

vSphere HTML5 Web Client

HTML5 Fling iconWith ESXi 6.0 update 2 we were treated to a HTML-based web host client. This meant we were no longer confined to using the Windows (“C++”) client for host management, however it was still necessary to use the Flash-based web client for managing a vCenter infrastructure. The vSphere HTML5 Web Client is the solution to this- providing a HTML5, standards-based, web front end to a VMware virtual infrastructure. The product is currently in the “Fling” stage- so it’s out there to try but not quite finished to “production” standards yet.

vSphere Web Client

vSphere Web Client Fling running in Chrome on Windows Desktop

In my Homelab environment, installation was simple- I downloaded and deployed the OVA from the Flings website and then followed the comprehensive instructions on the vSphere Blog. Within a few minutes I had the client up and running and it sits alongside the existing host client and flash-based vCenter client (via VCSA) so all my existing management interfaces are preserved.

vSphere Web Client

vSphere Web Client Fling running in Edge on Windows Mobile

So, what does this give me? Well, apart from giving an insight into the future direction of the VMware hypervisor management console, it allows me to do basic vCenter management tasks from my mobile phone or non-flash compatible portable devices (although there’s a little way to go to make the app really usable on my mobile at least- see screenshot). I’ve blogged before about using the Host client on the Xbox One, and now it’s possible to manage some vCenter functions from the console too.

With the news that the “fat” C++ based Windows client will be depreciated in the next release of vSphere having started being phased out in version 5.5, I’m looking forward to seeing where this goes.

.

.

Timeout Value on vSphere Host Client

If you’re running the HTML ESXi Host Client that comes with version 6.0 Update 2, you may find your screen reverting to the login prompt after only a short period of inactivity. This is because there’s an Application Timeout setting that defaults to 15 minutes.

Application Timeout

Application Timeout setting (Click for full size image)


To change this setting login to the host client and locate the user@host drop down menu in the top right hand corner, select “Client Settings” and then “Application timeout”. The timeout can be set to 15, 30, 60, or 120 minutes, or disabled completely.

In an important production environment it’s obviously more secure to keep this enabled as it will prevent you accidentally leaving yourself logged into the client and open to malicious passers wreaking havoc on your infrastructure. However, in a test environment, or your Home Lab, it’s probably quite safe to lengthen or disable this timeout assuming that you’re not concerned about small family members or pets getting hold of the mouse and playing with your VMs.

The Open HomeLab Project

Open Homelab ProjectIf you run a HomeLab, or have ever considered setting one up, you should check out the new Open HomeLab Project. This is a new wiki-based initiative to collect together information on the Home Lab ecosystem for newbies and hardened enthusiasts alike.

The Project was kicked off by Alex Galbraith following the London VMUG event in April and in the build up to today’s public launch a team has been busy collating together plenty of information on the uses, costs, hardware, and software of a HomeLab. If you’re learning about IT infrastructure and want somewhere to experiment, or if you just feel the need for a semi-enterprise IT environment in your house or garage, then there’s something here for you and the project would love to have input on what you’re up to.

I’m a relative newcomer to the “proper” HomeLab scene, having started in earnest back in January (check out my HomeLab series here), but found myself as one of the founding members of the project and have already been able to contribute some content from my experiences. This is proof that this can be a real community driven resource- you don’t need to be a multi-qualified VCDX, MCP wielding enterprise architect to be able to participate and share your knowledge and experience with others.

The site is now in a public Alpha, so here’s your chance to get in early. Go and have a look, peruse, and add content: http://openhomelab.org/

Top vBlog 2016 Awards

Voting is now open for the Top vBlog 2016 Awards, and I am pleased to say that this blog- IT Should Just Work – is in the running for the first time. It’s an honour to be just in the list with all these prestigious names, so please go and check it out (and if you are feeling very kind give me a vote!). Full details at vSphere Land here, and the voting form is here.

Top vBlog 2016

Top vBlog 2016

CloudCred- Gamification of VMware Learning

Recently I’ve been spending some time looking at CloudCredibility, a VMware website which provides some direction for learning about their ecosystem and technology.

Whilst studying for a specific certification it’s easy (easier?) to keep on track and pick off objectives, general technology learning is harder to keep in focus. This is where CloudCred comes in. It provides tasks which earn you badges and points, and points mean prizes- not just a PNG badge, but (if you stick at it) real world physical things as well like clothing, gadgets, and event tickets.

The tasks vary from “read this blog” to “complete this Hands-On Lab” right through to “get this qualification”, and generally the ones’ that will take more time will accrue you more points. There are categories, so if you want to learn about NSX for example you can work through the list relating to that particular product.

Personally, I’ve found this semi-directed learning to be quite useful. It’s a bit more structured that reading random blogs or watching the #vBrownBag or VMworld videos with the most hits, and the offer of a freebie always encourages me to not pick up the game controller and do just one more task instead. The chance to do team events (either with people you know or with the random team you are assigned to as a newbie) encourages this as well. Continue reading

“Error Fetching Datastores” when installing VCSA

Symptoms
When installing VCSA onto an existing vCenter environment (as a VM, not as an in-place upgrade of that environment), on Step-9 “Select Datastore” the installer shows the message “Error Fetching Datastores” and no datastores are listed so installation cannot continue.

Error Fetching Datastores

Error Fetching Datastores

Workaround
Go back through the installer and select an ESXi host rather than the vCenter server in Step 2 “Connect to target server”. Remember to use the host credentials which may differ from the vCenter credentials. The Datastores are then discovered correctly and installation can continue.

VMUG Banner

London VMUG April 2016 (Part 2)

Last week was the second London VMware User Group of 2016– and I came away with so many notes I couldn’t justify just squeezing them all into one post. So, check out Part 1 here if you haven’t already and what to know what happened in the morning.

From the agenda I chose to start my afternoon with a look at VMware Virtual Volumes (VVOLs) and how they fit with the offerings from Nimble Storage. Nick Dyer ran through the company background and an overview of their products. One of these was InfoSight Predictive Analytics, this cloud based analytics solution collects data from across Nimble’s entire install base which not only allows them to use your own data to help spot problems and improve performance with your own infrastructure but also the other customers’.

On the VMware front he discussed how VVols abstracts away from the datastore model, allowing the storage vendor to present their capabilities directly up to the virtualisation level using VASA 2.0 technology. A VM is assigned a storage policy and therefore can be hosted on the most appropriate storage at a very granular level- this can affect not only the performance tier the data is placed on but also things like the volume encryption requirements.

Open Homelab Project

Coming Soon…..

After the afternoon break in the (Not Quite on the) Thames Suite we had a round(ish)table discussion about HomeLabs chaired by Alex Galbraith. I’ve mentioned in other posts that the London VMUG community set me on my HomeLab path, and this was a great chance to see what other people were doing and to exchange ideas. There’s a whole range of systems out there, from things like my small NUC-based deployment right through to people with racks of blades in their garden sheds. There are also those using or looking at hybrid or full cloud solutions to exchange that Capex for Opex, although if you turn that under-bed datacentre off you’ll need to heat your house more conventionally in winter . I thoroughly enjoyed this session and it already looks like it will lead to further community discussion and resources – keep your eyes peeled on #OpenHomeLab for more.

My last session of the day was with Neil Andrew from VMware who talked about vRealize Business. A mention of some of the features that morning had piqued my interest, and it’s was great to have this timely opportunity to delve deeper into the technology on offer.

image

vRealize Business can help answer those questions such as “How much is a VM costing me?” and “Is it cheaper to relocate this to a cloud provider?”. Although real Chargeback is potentially tricky to implement fully in many organisations, I could see how the idea of Showback is quite achievable and could not only help plan (re)deployments of existing services but also be a valuable tool in planning the budgets for new ones. The customer (internal or external) can be given a real cost of the infrastructure they are requesting, rather than a piecemeal hardware/ licenses/ manpower/ environment/ etc. cost usually calculated on a difficult to maintain spreadsheet.

After the closing statements, and the prize giveaways (including a rather awesome prize of an Anki Overdrive kit from PernixData) we adjourned to the 10-Zig sponsored vBeers where the technical discussions continued.

There’s another London VMUG in a couple of months, and dates for 2017 are already booked in. See you there?

London VMUG dates

Upcoming London VMUG dates

VMUG Banner

London VMUG April 2016 (Part 1)

Thursday saw another trip to TechUK for the second London VMware Users Group of the year. It was, yet again, a very educational and entertaining community driven event with sessions covering a whole range of topics from enterprise scale implementations of hardware and software right down to the nitty-gritty of running your own HomeLab.

After the initial welcomes and introductions, the day kicked of with a session from Luca Dell’Oca of Veeam. Entitled “Veeam Backup and Replication: Worst Practices”, this was an often humorous look at what not to do when planning and implementing a backup regime. He covered many of the common mistakes when backing up a virtual environment, and although some parts were product specific (for example how the Veeam scheduler prior to v9 was not optimised for the hundreds of jobs a per-VM backup strategy would involve) there was a lot of generic advice. Highlights for me included

  • Remember that in-guest iSCSI connections will not be touched by VMware snapshots so don’t forget about them when backing up the VMs.
  • Use monitoring tools- in particular keep a close eye on storage space used by snapshots and performance. vCenter performance is critical and can often be a bottleneck.
  • When are other infrastructure tasks scheduled? Don’t run backups when your vCenter SQL maintenance plan is running for example.
  • Find the balance between one backup job per VM and one backup job for all VMs. Group VMs together based on backup policy.
  • Don’t be a cheapskate! Fast, Good, and Cheap is not possible.
  • It’s not necessary to fiddle with every advanced setting. Don’t change what you don’t understand!

Next on the agenda was the Plenary Keynote, where Simon Richardson of VMware went through the new developments in the Software Defined Datacentre portfolio. He pointed out that the only constant is change, and that means there’s always a lot to catch up on.

Simon discussed using the Hyperconverged SDDC as the best of both the traditional DC world and the Google/Facebook/Amazon “extreme” style of datacentre. Applying the analogy of a water company to IT he suggested we want to be in the position where we can turn on a tap and compute comes out.


There’s new features in the vRealize suite as well, vROps producing intelligent workload balancing and Log Insight for diving into the cause of a fault or performance issue.

vRealize Business was a new component for me (more on that later in the day) and I can definitely see the use of being able to show (or chargeback) the actual cost of a VM- taking into account all the factors- environment, hardware, software, support, manpower, and so on. Although there was little OpenStack adoption in the audience, Simon also touched on VIO- VMware Integrated OpenStack.

After a quick break in the Thames Suite (named in honour the breakroom in the previous London VMUG venue 🙂 ) I caught up with VSAN in more depth in a talk by Simon Todd (and earned some Cloud Credibility points along the way). VMware’s Virtual SAN is getting wider and wider adoption, even branching outside of the traditional Datacentre environment into Offshore Oil Rigs and even Submarines. This is more than just Business critical this really is Mission critical. The ability to deploy (and scale) large arrays quickly is a key selling point, multipetabyte arrays spun up in a matter of minutes and customers who had cut both costs and deployment times.

@MrVirtualSAN at @LonVMUG

What was my vmdk on VSAN up to at 4am last night? A user complained their service was slow.

Moving a bit more into the technical detail- Simon explained how VSAN is an object store with each VMDK, each file, becoming an object. This allows for per-object policy, so performance, IOPS limits, and resilience can be set, and then monitored, at the VMDK level. When pushing for that ultimate performance, this granularity could be of great benefit.

That brought us to the halfway point in the day’s proceedings, where there was lunch an and opportunity to catch up with the events sponsors- PernixData, Veeam, and Nimble Storage.

London VMUG dates

Upcoming London VMUG dates

Edit- Part 2 can be found here.