My 3 Big VMworld Goals from 2014

James Green · Aug 29, 2015 ·

Last year at VMworld 2014, I had some fantastic experiences and met heaps of great people (some of them are the kind of people who talk funny and use words like ‘heaps!’). I found myself challenged by certain conversations, and motivated, inspired, and excited by others. During the course of the show last year, at some specific point and more or less unintentionally, I found myself setting some goals based on what I was observing and hearing. I said to myself – “By this time next year, I will accomplish X, Y, and Z.” This post may be my first non-technical article on my blog, so I’m excited to break into that space and share two things: the three goals I set for myself and what my experience was in my pursuit of them.

Perhaps not surprisingly, these goals I set myself came about from things I realized in 1:1 conversation, or in a 1:few setting. Large groups are so fun, but I also love the depth that an intimate conversation setting can facilitate. I spent the day of registration wandering around with my friends Eric Wright and Angelo Luciani, and Eric and I were in the same hotel a 20 minute walk from the conference center, so we walked alone back to the hotel a time or two. They had no idea about this, and the only thing I’ve ever said about it was a hint I dropped a moonth or two ago. But it was with them that either a conversation or an experience caused me to set each one of these three goals. I’m on a plane now headed to VMworld 2015 and I’m so excited to land at SFO and wander around downtown San Francisco with those guys (and others) again, and who knows – maybe set some goals for next year! So, about the goals…

Backing up vPostgres Database

James Green · Aug 14, 2015 ·

Something I have learned over my career as a consultant is to back up everything before performing an upgrade, and then back it up again another way 🙂 There’s nothing worse than an upgrade going sideways and having no way to back out. Also, if you’re doing any sort of work in an enterprise setting, furnishing a back-out plan before you begin may well be a requirement. This is to ward off extended downtime and potential financial loss as a result. As deploying the vCenter Server Appliance (or whatever we’re calling it now…) becomes the de facto method of deploying vCenter, a Windows admin can sometimes no longer pop into SSMS and fire off a quick backup of the database. Especially as of version 6.0, the embedded vPostgres database is capable of handling quite a large environment, so more folks are opting to use it. As such, I’m going to go over how to grab a backup of it. It’s good practice to do this regularly if you aren’t getting a backup of it another way, and I also recommend doing this before any sort of upgrade (as I’m doing now). Here we go!

Quick Tip: Cannot delete ISO on VMware datastore

James Green · Aug 12, 2015 ·

There’s a VMware KB article that talks about trying to delete a file from a VMFS datastore, only to find that it’s locked by a particular ESXi host. The article walks through identifying which host as the file locked via a message in the vmkernel log file. By parsing the log message, one can deduce the MAC address of the host locking the file. From there the lock can be removed and the file deleted. The message in the logs looks something like this:

cpu6:5455056)FS3: 1393: vol ‘datastore_name‘, lock at 173475840: [Req mode: 1] Not free:

cpu6:5455056)[type 10c00004 offset 173475840 v 158, hb offset 3338752gen 4307, mode 1, owner 51b23791-f35c96e9-44ed-3c4a92f61680 mtime 2278287

cpu6:5455056)Res3: 5696: Rank violation threshold reached: cid 0xc1d0000c, resType 3, cnum 1410

The bolded section is a MAC address and will help chase down the host in question. But there’s something I’ve run into a number of times when trying to get a file deleted that I thought I’d share. When trying to delete an ISO (specifically an ISO) from a datastore, you’ll get the same error message about the file being locked. You search for the error you’re having and the KB article I referenced turns up. So you go look at vmkernel.log and find this:

2015-08-12T13:19:21.219Z cpu4:3476013)[type 10c00001 offset 201531392 v 18641, hb offset 3956736

gen 1051, mode 2, owner 00000000-00000000-0000-000000000000 mtime 44516

Now, what in the world does one do when the MAC address portion is empty? Well, after banging my head against my desk for about 5 minutes more times than I care to admit, I usually figure it out. I hope that after posting this I’ll remember that this means the ISO is mounted to a VM somewhere! Duh. If I can get this post to rank well in search engine results, maybe I’ll save someone from 4 minutes of those 5 minutes of head/desk banging agony. 🙂

PowerCLI – Configure Syslog for All Hosts

James Green · Aug 10, 2015 ·

Here’s a quick bit of PowerCLI to configure syslog server on all hosts, place each hosts logs in a unique directory, and then enable the firewall exception to allow it to be used. This will do every host in the vCenter you’re connected to. I highly recommend that anyone managing a vSphere environment set up a syslog destination for ESXi. There’s nothing more frustrating than attempting root-cause analysis on a failure when logs aren’t persistent and in a central location.

Cheers!

[code language=”ps”] #Get all ESXi hosts
$hosts = Get-VMHost

#Update Syslog configuration
$hosts | Set-VMHostAdvancedConfiguration Syslog.global.logHost -Value 0.0.0.0
$hosts | Set-VMHostAdvancedConfiguration Syslog.global.logDirUnique -Value True

#Enable firewall exception
$hosts | Get-VMHostFirewallException | where {$_.Name.StartsWith(‘syslog’)} |
Set-VMHostFirewallException -Enabled $true
[/code]

Vagrant Series » Multi-Machine Environments (Part 6)

James Green · Aug 7, 2015 ·

Vagrant is neat if you need a whip up a machine to quickly test a script or check compatibility on an operating system that is different than what you normally run. And the value of that isn’t to be minimized. But some of the real power of Vagrant happens when you start to stand up full environments (think 3-tier applications or large scale web server farms) in a repeatable, effortless way. In Hashicorp Land, this is specifically referred to as a “multi-machine” environment. Any Vagrantfile that defines and controls multiple guest machines qualifies for this descriptor.

Defining Multi-Machine Environments

Building a Vagrantfile that will configure a multi-machine environment is as simple as defining multiple VMs with the parameter config.vm.define. In the below example nabbed from Hashicorp documentation, one case see that an Apache front end and MySQL back end are both defined.

Vagrant.configure("2") do |config|
  config.vm.provision "shell", inline: "echo Hello"

  config.vm.define "web" do |web|
    web.vm.box = "apache"
  end
  config.vm.define "db" do |db|
    db.vm.box = "mysql"
  end
end

Once the VMs are defined, configuration takes place by using the inner variable with the same name. In the example, web.anything and db.anything will apply configurations specifically to that machine. Global configurations can still be defined and will apply to all VMs. When configuring a multi-machine environment, it’s important to consider the load order of a Vagrantfile. I covered this in Part 3 of this series, but just to quickly recap, the load order is:

Vagrantfile packaged with the box that is to be used for a given machine.
Vagrantfile in your Vagrant home directory (defaults to ~/.vagrant.d). This lets you specify some defaults for your system user.
Vagrantfile from the project directory. This is the Vagrantfile that you’ll be modifying most of the time.
Multi-machine overrides if any.
Provider-specific overrides, if any.

As the Vagrantfile grows and a large number of machines are defined, it can become a bit of a mess. Scott Lowe wrote about an idea he found over on this blog that breaks all the configuration data out into a separate YAML file. You could also use JSON or XML if you’re more comfortable with that, as in this example.

Accessing Multi-Machine Environments

When using a multi-machine environment, some vagrant commands need to become more specific. If you issue vagrant sshin a multi-machine environment, which machine should be connected to? The commands in a multi-machine environment that only make sense for one machine need to have the machine specified in the command. So in the case of SSH, the command would be vagrant ssh web. In the case of a command like reload, you could issue vagrant reload and reload the whole environment, or you could specify one VM, as in vagrant reload db. Since sometimes one needs to be specific but there’s also a large number of VM’s, the Vagrantfile will also parse regex when it identifies a regular expression for the name, such as /web[A-Za-z0-9]/.