i upgraded vcenter 4.1 to 5.1 a few nights ago. the whole process took about 3.5 hours for me because of a couple of hiccups during the install and an unexpected surprise after the install finished.

i did all of the SSO database prep work previously so i didn’t have to deal with that at install time. the SSO component is installed first. it installed fine with no complaints besides saying it couldn’t detect the AD domain settings (even though i specifically logged in as a domain user for this). i saw online that other people have experienced this and noted in the vmware docs that i could just add the domain settings later. so, the SSO part was not a concern to me so far. i should note that the SSO admin account/password (admin@System-Domain) is very important in this release so be sure to commit it to memory or notate it somewhere.

next you install the inventory service. i didn’t experience any problems here.

and lastly, the vcenter component is installed. here’s where i experienced a couple of problems. the vcenter 5.1 installer seems much pickier than previous vcenter installers…

problem #1
in the midst of the vcenter install, a dialog window popped up saying that SSL certificate verification for hosts was disabled and the installer could not proceed further without this feature being enabled. unfortunately, the only way you can enable this setting is to cancel the install, restart the vcenter service, and then login with a vsphere client and enable the setting. i manually verified each host in hopes of avoiding further issues.
http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2035594

problem #2
another SSL certificate issue. this time the installer pointed out that the vcenter’s own SSL certificate was expired. something like this had never ever even crossed my mind before. it had probably been expired for years and never caused an issue. so i’m thinking at this point, okay installer, generate a new cert then… after all it’s just a self-signed cert, so what’s the big deal? well it’s not quite that convenient of a process. and during moments like this it’s a godsend to have a linux machine within reach. so i used openssl on linux to generate a new cert. i suppose you could install openssl on windows, but i have no idea if that’s a hassle or not. in linux, openssl is pretty much just always there. the good news is that this new cert should last for 10 years.
http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1009092

after fixing the above 2 problems the upgrade eventually finished. during the install process i was warned that certain pre-existing administrators would be losing access. the installer indicated that the accounts affected would be notated in 2 text files. i could not locate these text files with a file search for some reason. i recalled vaguely reading about something like this in the docs, but had thought that accounts that had local administrator access would be either unaffected or auto-readded. well…i definitely thought wrong, but i’ll get to that later…

immediately after the upgrade finished i launched the vsphere client on my workstation and proceeded to upgrade the client. i could not login with my AD credentials with the new 5.1 client. i did a quick google for something like “can’t login after vcenter 5.1 upgrade” and saw a page or two of related blog posts. so i immediately thought oh great it’s going to be one of those type of things…

common sense told me that it had to be related to the new SSO stuff. and i knew from earlier that SSO was unaware of my AD domain. so after some quick research i found out that SSO settings can only be configured from the vsphere web client. i logged in with the admin@System-Domain account. then i went to Administration->Sign-On and Discovery-Configuration->Identity Sources.

the first thing you’re going to want to do is configure your AD domain as a identity source. here’s an example:
Type: Active Directory
Name: (something meaningful to you that describes your domain)
Primary server URL: ldaps://dc.mydomain.com:636 (LDAPS isn’t mandatory)
Base DN for users/groups: (if you leave this blank it will auto-populate with the root of your domain)
Domain name: mydomain.com
Domain alias: MYDOMAIN (i originally left this blank because it’s listed as optional in the docs. you won’t want to leave this blank…i’ll go into this later)
Authentication type: Password (i created an AD user with read-only access to the domain to be used for this)

then i tested the connection and added this identity source to the list of default domains. also i moved it to the top of the list. i took a minute to go to “SSO Users and Groups” and added the Domain Admins group to the SSO Administrators group (named __Administrators__) as well. a final note, if you mess up anything when setting up the identity source, you can’t easily edit it and fix your mistake. a lot of times you’ll need to delete it and recreate it from scratch.
http://pubs.vmware.com/vsphere-51/index.jsp?topic=%2Fcom.vmware.vsphere.security.doc%2FGUID-B23B1360-8838-4FF2-B074-71643C4CB040.html
http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2035758

i was ready to test my AD login again with the vsphere client at this point. in the mean time, i also had the vsphere client upgrading on the vcenter server. this time around i determined the “mydomain\username” format was no longer being accepted, but “username@mydomain.com” was. “mydomain\username” and the Use Windows session credentials options would both return incorrect username or password. when using “username@mydomain.com” i would get a different error. something along the lines of “user does not have permissions to vcenter server”. so i was getting closer…

however, at this point it was getting to be late at night and i was running out of ideas. i think i’ve gotten to a point in my life where staying really late at work is no longer cute. i prefer to be at home and comfortable now and try to avoid excessive late work hours if i can. so i dialed up vmware support to expedite the situation. support quickly determined what was going on. this was the surprise i didn’t count on… either it was it unclear in the vmware docs or i somehow missed it entirely. so what happens after a vcenter 5.1 upgrade is that all vcenter user permissions are completely wiped out except for the lone vcenter administrator account (local administrator on the vcenter server). you have to login to the vsphere client on the vcenter server with this account and then reset all the user permissions to vcenter from the vsphere client. i’m fairly certain i had to specify the Adminstrator login in this format “<your vcenter server name>\Administrator”. so the immediate problem was quickly solved. and i was really glad i cleaned up the vcenter access groups in AD a few months prior. if someone potentially had a ton of VMs with individual user access set on them…then they’d have a really long night ahead of them.

i wasn’t quite ready to end the support call though. as i mentioned above the Use Windows session credentials option was still not functioning with the vsphere client. support eventually determined it was because i did not specify the domain alias in the SSO identity source for the AD domain. the reason i did not do this was because it was listed as optional in the vmware documentation.

some useful install visuals here:
http://www.virtuallyimpossible.co.uk/updating-vmware-vcenter-from-5-0-to-5-1/

update 4/9/2013:
forgot to notate that the port for the vsphere web client is 9443…therefore https://vcenter.mydomain.com:9443.

Comments are closed.