Thenewlucky

What's in the Release Notes

What's New

NSX vSphere 6.1.2 updates NSX Edge and NSX controllers to OpenSSL 101j and addresses several other CVEs. This release includes an updated API call to address the POODLE vulnerability. Using this API call, you can disable SSLv3 on specific NSX Edges in your environment. For more information, see Resolved Issues.
To maintain API compatibility, SSLv3 is not disabled on NSX Manager. Administrators may choose to explicitly disable SSLv3 in their browser as a precaution.
You can upgrade to this release from NSX vSphere 6.0.x and 6.1.x. If you are currently using vCloud Networking and Security, you must first upgrade to NSX vSphere 6.1.1, and then upgrade to NSX vSphere 6.1.2.

System Requirements and Installation

VMware Product Interoperability Matrix provides details about the compatibility of current and previous versions of VMware products and components, such as VMware vCenter Server.

If you are upgrading from NSX version 6.1.1, or do not have Data Security in your environment, see the NSX Installation and Upgrade Guide for instructions on upgrading to this release.

If you are upgrading from a release prior to NSX 6.1.1 and have Data Security in your environment, follow the steps below to upgrade to this release:

1. Un-install Data Security from all the clusters that have the service installed.
2. Upgrade NSX Manager to version 6.1.2. See the NSX Installation and Upgrade Guide for instructions.
3. Install or upgrade Guest Introspection and other services on appropriate clusters.
4. Install Data Security on appropriate clusters.
5. Upgrade the remaining components. See the NSX Installation and Upgrade Guide for instructions.

Resolved Issues

The following issues have been resolved in the 6.1.2 release.

• Disable SSLv3 Support on NSX Edge
  You can use the sslVersionList parameter in the following API call to white list an SSL version on NSX Edge.
  PUT https://<nsxmgr-ip>/api/4.0/edges/<edge-id>/sslvpn/config/server/
  Request Body:
  <serverSettings>
<serverAddresses>
<ipAddress>10.112.243.109</ipAddress>
</serverAddresses>
<port>443</port>
<sslVersionList>
<version>SSLv3</version>
<version>TLSv1</version>
<version>TLSv1_2</version>
<version>TLSv1_1</version>
</sslVersionList>
<cipherList>
<cipher>RC4-MD5</cipher>
<cipher>AES128-SHA</cipher>
<cipher>AES256-SHA</cipher>
<cipherDES-CBC3-SHA</cipher>
</cipherList>
</serverSettings>
  If the sslVersionList parameter is blank, all SSL versions in the above example are white listed. (This issue was resolved in NSX 6.1.2.)
• vNICs get ejected because of insufficient ESXi heap memory
  Based on the number of filters, firewall rules, and grouping constructs on an ESXi host, the allocated heap memory on the host may be exceeded. This results in the vNICs being disconnected.
  The ESXi heap memory has been increased to 1.5 GB in this release. This issue was resolved in NSX 6.1.2.
  
• When objects used in firewall rules are renamed, the new name is not reflected on the Firewall table. (This issue was resolved in NSX 6.1.2.)
• Request to add support for NTLM authentication in NSX Edge Load Balancer. (This issue was resolved in NSX 6.1.2.)
• NSX vSphere CPU licenses are displayed as VM licenses
  NSX vSphere CPU entitlements are displayed as VM entitlements in the vSphere Licensing tab. For example, if a customer has licenses for 100 CPUs, the UI displays 100 VMs. (This issue was resolved in NSX 6.1.2.)
• Implicit deny rule for BGP filters created on Edge Services Gateway but not on Logical Router
  When a BGP outbound neighbor filter is configured on an Edge Services Gateway, only prefixes with explicit accept policy are advertised. Hence an implicit deny rule is created automatically. On a Logical Router, all prefixes are advertised unless explicitly blocked. (This issue was resolved in NSX 6.1.2.)

Known Issues

Known issues are grouped as follows:

Installation and Upgrade Issues

NSX Edge upgrade fails if L2 VPN is enabled on the Edge
L2 VPN configuration update from 5.x or 6.x to 6.1 is not supported. Hence, Edge upgrade fails if it has L2 VPN configured on it.
Workaround: Delete L2 VPN configuration before upgrading NSX Edge. After the upgrade, re-configure L2 VPN.

SSL VPN does not send upgrade notification to remote client
SSL VPN gateway does not send an upgrade notification to the user. The administrator has to manually communicate that the SSL VPN gateway (server) is updated to the remote user and that they must update their clients.

After upgrading NSX from version 6.0 to 6.0.x or 6.1, NSX Edges are not listed on the UI
When you upgrade from NSX 6.0 to NSX 6.0.x or 6.1, the vSphere Web Client plug-in may not upgrade correctly. This may result in UI display issues such as missing NSX Edges.
This issue is not seen if you are upgrading from NSX 6.0.1 or later.
Workaround: Follow the steps below.

1. On the vSphere Server, navigate to the following location:
   /var/lib/vmware/vsphere-client/vc-packages/vsphere-client-serenity
2. Delete the following folders:
   com.vmware.vShieldManager-6.0.x.1546773
com.vmware.vShieldManager-6.0.1378053
3. Restart the vSphere Web Client service.

vSphere Distributed Switch MTU does not get updated
If you specify an MTU value lower than the MTU of the vSphere distributed switch when preparing a cluster, the vSphere Distributed Switch does not get updated to this value. This is to ensure that existing traffic with the higher frame size isn't unintentionally dropped.
Workaround: Ensure that the MTU you specify when preparing the cluster is higher than or matches the current MTU of the vSphere distributed switch. The minimum required MTU for VXLAN is 1550.

If all clusters in your environment are not prepared, the Upgrade message for Distributed Firewall does not appear on the Host Preparation tab of Installation page
When you prepare clusters for network virtualization, Distributed Firewall is enabled on those clusters. If all clusters in your environment are not prepared, the upgrade message for Distributed Firewall does not appear on the Host Preparation tab.
Workaround: Use the following REST call to upgrade Distributed Firewall:
PUT https://vsm-ip/api/4.0/firewall/globalroot-0/state

Service groups are expanded in Edge Firewall table during upgrade from vCloud Networking and Security 5.5 to NSX
User created service groups are expanded in the Edge Firewall table during upgrade - i.e., the Service column in the firewall table displays all services within the service group. If the service group is modified after the upgrade to add or remove services, these changes are not reflected in the firewall table.
Workaround: Follow the steps below:

1. Re-create the service groups in the Grouping Objects tab after upgrade.
2. Edit the Service column for the affected firewall rules and point to the appropriate service groups.

Guest Introspection installation fails with error
When installing Guest Introspection on a cluster, the install fails with the following error:
Invalid format for VIB Module
Workaround: In the vCenter Web Client, navigate to vCenter Home > Hosts and Clusters and reboot the hosts that display Reboot Required.

Service virtual machine deployed using the Service Deployments tab on the Installation page does not get powered on
Workaround: Follow the steps below.

1. Manually remove the service virtual machine from the ESX Agents resource pool in the cluster.
2. Click Networking and Security and then click Installation.
3. Click the Service Deployments tab.
4. Select the appropriate service and click the Resolve icon.
   The service virtual machine is redeployed.

If a service profile created in 6.0.x is bound to both security group and distributed portgroup or logical switch, Service Composer firewall rules are out of sync after upgrade to NSX 6.1
If a service profile binding was done to both security group and distributed portgroup or logical switch in 6.0.x, Service Composer rules are out of sync after the upgrade to 6.1. Rules cannot be published from the Service composer UI.
Workaround: Follow the steps below.

1. Unbind the service profile from the distributed portgroup or logical switch through the Service definition UI.
2. Create a new security group with the required distributed portgroup or logical switch as a member of that security group.
3. Bind the service profile to the new security group through the Service Definition UI.
4. Synchronize the firewall rules through the Service Composer UI.

General Issues

Simultaneous deployment of a high number of virtual machines results in a network adapter connection failure
This issue occurs because HA aborts several failover attempts for virtual machines after deployment and no dvPort data is loaded. This results in the affected virtual machines being marked to start with their network adapters disconnected. Virtual machines were left unprotected by HA because the flag cleanPowerOff is incorrectly set to true before the virtual machine configuration is fully loaded. This can happen when the hostd process is under a lot of load.
Workaround: This issue is resolved in ESXi 5.5 Update 2. If you are using a different ESXi version, follow one of the options below:

• Restart the vpxa process on the ESXi host:
1. Connect to the affected ESXi host with an SSH session. For more information, see Using ESXi Shell in ESXi 5.x (2004746).
2. Run this command to restart the vpxa process:
   /etc/init.d/vpxa restart
• Change the portgroup for the virtual network adapter, then change it back to the original one:
1. Connect to the affected host with the vSphere Client.
2. Right-click the affected virtual machine and click Edit Settings.
3. Select the Network Adapter from the left pane.
4. Change the network label form the dropdown menu under Network Connection to another portgroup.
5. Click OK.
6. Change the portgroup back to the original portgroup.
7. Select Connected and Connect at power on.
8. Click OK.

NSX Manager Issues

Cannot remove and re-add a host to a cluster protected by Guest Introspection and third-party security solutions
If you remove a host from a cluster protected by Guest Introspection and third-party security solutions by disconnecting it and then removing it from vCenter Server, you may experience problems if you try to re-add the same host to the same cluster.
Workaround: To remove a host from a protected cluster, first put the host in maintenance mode. Next, move the host into an unprotected cluster or outside all clusters and then disconnect and remove the host.

vMotion of NSX Manager may display error Virtual ethernet card Network adapter 1 is not supported
You can ignore this error. Networking will work correctly after vMotion.

Cannot delete 3rd party services after restoring NSX Manager backup
Deployments of 3rd party service(s) can be deleted from vSphere Web Client only if the restored state of NSX Manager contains the 3rd party service registration.
Workaround: Take a backup of NSX Manager database after all 3rd party services have been registered.

NSX Edge Issues

OSPF protocol does not work on sub interfaces
When OSPF is enabled on a sub interface, the adjacency sometimes goes into two way mode instead of going into full.
Workaround: None.

SSL VPN Mac client for OS X Yosemite displays certificate authentication error
Since Yosemite does not use /Library/StartupItems/ as a startup item, the VMware startup script is not executed when the machine boots up.
Workaround: Follow the steps below:

1. Run the following command to set kext-dev-mode
   sudo nvram boot-args='kext-dev-mode=1'
2. Reboot the machine.
3. Install the client.

Enabling Equal-Cost Multi-Path routing (ECMP) on a Logical Router disables firewall on the router control virtual machine
Workaround: None.

Dynamic routing protocols not supported on sub interfaces
Workaround: None.

Adding a route that is learned through protocol as connected results in the local Forwarding Information Base (FIB) table showing both connected and dynamically learned routes
If you add a route already learned through protocol as connected, the local FIB shows both connected and dynamically learned routes. The dynamically learned route is shown as preferred over the route directly connected.
Workaround: Withdraw the learned route from the route advertisement so that it gets deleted from the FIB table and configure only the connected route.

If an NSX Edge virtual machine with one sub interface backed by a logical switch is deleted through the vCenter Web Client user interface, data path may not work for a new virtual machine that connects to the same port
When the Edge virtual machine is deleted through the vCenter Web Client user interface (and not from NSX Manager), the vxlan trunk configured on dvPort over opaque channel does not get reset. This is because trunk configuration is managed by NSX Manager.
Workaround: Manually delete the vxlan trunk configuration by following the steps below:

1. Navigate to the vCenter Managed Object Browser by typing the following in a browser window:
   https://vc-ip/mob?vmodl=1
2. Click Content.
3. Retrieve the dvsUuid value by following the steps below.
1. Click the rootFolder link (for example, group-d1(Datacenters)).
2. Click the data center name link (for example, datacenter-1).
3. Click the networkFolder link (for example, group-n6).
4. Click the DVS name link (for example, dvs-1)
5. Copy the value of uuid.
4. Click DVSManager and then click updateOpaqueDataEx.
5. In selectionSet, add the following XML
   <selectionSet xsi:type='DVPortSelection'>
<dvsUuid>value</dvsUuid>
<portKey>value</portKeyv <!--port number of the DVPG where trunk vnic got connected-->
</selectionSet>
6. In opaqueDataSpec, add the following XML
   <opaqueDataSpec>
<operation>remove</operation>
<opaqueData>
<key>com.vmware.net.vxlan.trunkcfg</key>
<opaqueData></opaqueData>
</opaqueData>
</opaqueDataSpec>
7. Set isRuntime to false.
8. Click Invoke Method.
9. Repeat steps 5 through 8 for each trunk port configured on the deleted Edge virtual machine.

When Default Originate is enabled, BGP filter for deny default route does not get applied
When BGP Default Originate is enabled on an NSX Edge, a default route gets advertised to all BGP neighbors unconditionally. If you do not want a BGP neighbor to install the default route advertised by this BGP speaker, you must configure an inbound policy on that BGP neighbor to reject the default route.
Workaround: Configure an inbound policy on the appropriate BGP neighbor to reject the default route.

Cannot add non-ASCII characters in bridge or tenant name for Logical Router
NSX controller APIs do not support non-ASCII characters.
Workaround: Use ASCII characters in bridge and tenant names. You can then edit the names to include non-ASCII characters.

SNAT and Load Balancer (with L4 SNAT) configured on a sub interface do not work
SNAT rule configuration passes on NSX Edge but the data path for the rule does not work due to RP Filter checks.
Workaround: Contact VMware Support for help in relaxing the RP filter check on NSX Edge.

When egress optimization is enabled for L2 VPN, load balancer with pool members stretched across site are shown as down
With egress optimization, both L2 VPN client and server have the same internal IP address. Because of this, any packet from a pool member to the load balancer does not reach NSX Edge.
Workaround: Do one of the following.

• Disable egress optimization.
• Assign an IP address for load balancer that is different from the egress-optimized IP address.

Static routes do not get pushed to hosts when a next hop address is not specified
The UI allows you to create a static route on an NSX Edge device without specifying a next hop address. If you do not specify a next hop address, the static route does not get pushed to hosts.
Workaround: Always specify a next hop address for static routes.

Cannot configure NSX firewall using security groups or other grouping objects defined at global scope
Administrator users defined at the NSX Edge scope cannot access objects defined at the global scope. For example, if user abc is defined at Edge scope and security group sg-1 is defined at global scope, then abc will not be able to use sg-1 in firewall configuration on the NSX Edge.
Workaround: The administrator must use grouping objects defined at NSX Edge scope only, or must create a copy of the global scope objects at the NSX Edge scope.

Logical Router LIF routes are advertised by upstream Edge Services Gateway even if Logical Router OSPF is disabled
Upstream Edge Services Gateway will continue to advertise OSPF external LSAs learned from Logical Router connected interfaces even when Logical Router OSPF is disabled.
Workaround: Disable redistribution of connected routes into OSPF manually and publish before disabling OSPF protocol. This ensures that routes are properly withdrawn.

When HA is enabled on Edge Services Gateway, OSPF hello and dead interval configured to values other than 30 seconds and 120 seconds respectively can cause some traffic loss during failover
When the primary NSX Edge fails with OSPF running and HA enabled, the time required for standby to take over exceeds the graceful restart timeout and results in OSPF neighbors removing learned routes from their Forwarding Information Base (FIB) table. This results in dataplane outage until OSPF re converges.
Workaround: Set the default hello and dead interval timeouts on all neighboring routers to 30 seconds for hello interval and 120 seconds for dead interval. This enables graceful failover without traffic loss.

The UI allows you to add multiple IP addresses to a Logical Router interface though it is not supported
This release does not support multiple IP addresses for a logical router interface.
Workaround: None.

Error displayed when modifying login or logoff scripts
When you modify a login or logoff script, the following error is displayed:
ObjectNotFoundException: core-services:202: The requested object : logon1.logoff could not be found. Object identifiers are case sensitive..
Workaround: Delete the existing script and a new one with modified parameters.

SSL VPN does not support Certificate Revocation Lists (CRL)
You can add a CRL to an NSX Edge, but this CRL is not consumed by SSL VPN.
Workaround: CRL is not supported, but you can enable user authentication with client certificate authentication.

Must use IP address, not hostname, to add an external authentication server to SSL VPN-Plus
You cannot use the FQDN or hostname of the external authentication server.
Workaround: You must use the IP address of the external authentication server.

Firewall Issues

Virtual machines experience a network interruption of up to 30 seconds after being vMotioned from one ESXi host to another when Distributed Firewall rules use security groups in the Source and /or Destination fields
When a virtual machine is vMotioned to another host, firewall rules and containers are sent over to the destination host so that existing sessions continue to work. As part of the vMotion activity, vCenter sends an empty IP address for the virtual machine to NSX Manager, which clears the virtual machine's IP address from the firewall containers. vCenter then relearns the IP address and sends another notification to NSX Manager, and the firewall container is updated with the correct IP address. The period between clearing and refilling the IP address causes an invalid policy to be applied on the virtual machine on the destination host, resulting in the connection drop.
This issue occurs only on virtual machines multiple vNICs running VMware NSX for vSphere 6.1 or 6.1.1 Distributed Firewall and vCenter Server versions 5.5 Update 1, 5.5 Update 2, or 5.5 Update 2b.
Workaround: Define static IP sets and use them in firewall rules. These rules are unaffected by vMotion activity, and there is no traffic interruption.

If you delete the firewall configuration using a REST API call, you cannot load and publish saved configurations
When you delete the firewall configuration, a new default section is created with a new section ID. When you load a saved draft (that has the same section name but an older section ID), section names conflict and display the following error:
Duplicate key value violates unique constraint firewall_section_name_key
Workaround: Do one of the following:

• Rename the current default firewall section after loading a saved configuration.
• Rename the default section on a loaded saved configuration before publishing it.

When IPFIX configuration is enabled for Distributed Firewall, firewall ports in the ESXi management interface for NetFlow for vDS or SNMP may be removed
When a collector IP and port is defined for IPFIX, the firewall for ESXi management interface is opened up in the outbound direction for the specified UDP collector ports. This operation may remove the dynamic ruleset configuration on ESXi management interface firewall for the following services if they were previously configured on the ESXi host:

• Netflow collector port configuration on vDS
• SNMP target port configuration

Logical Switch Issues

Creating a large number of logical switches with high concurrency using an API call may result in some failures
This issue may occur if you create a large number of logical switches using the following API call:
POST https://<nsxmgr-ip>/api/2.0/vdn/scopes/scopeID/virtualwires
Some logical switches may not be created.
Workaround: Re-run the API call.

Service Deployment Issues

Old service virtual machines not functioning
Old service virtual machines that were left behind on hosts during host removal from the vCenter Server remain disconnected and are unable to function when the host is added back to the same vCenter Server.
Workaround: Follow the steps below

1. Move the host from the protected cluster to either an unprotected cluster or outside all clusters. This will uninstall the service virtual machines from the host.
2. Remove the host from the vCenter Server.