vLCM versus VxRAIL

Q: with the introduction of vLCM with readynodes, what would the value out of selling VxRail ?

A: Though vLCM provides firmware support, it doesn’t provide pre-validated and pre-integrated bundles, which VxRail does. Our customers have told us that this is a big pain point. VxRail images are getting customer from one valid image/driver/firmware state to another valid state.

VxRail provides other value such as enhanced phone home support and additional automation such as auto-buildout of clusters.

vSAN/DRS awareness to be introduced in vSAN/vSphere 7.0! (NOT RELEASE in 7.0)

Originally piblished here ... 

http://www.yellow-bricks.com/2020/03/19/vsan-drs-awareness-to-be-introduced-in-vsan-vsphere-7-0/

It was briefly mentioned here, but I figured I would elaborate on this new cool feature for vSAN Stretched Clusters which is DRS Awareness of vSAN Stretched Clusters. So what does this mean? Well, it is fairly straight forward. DRS will take vSAN resync traffic into consideration when the DRS algorithm runs. I can probably explain best by talking through a scenario:

• vSAN Stretched Cluster environment with 4 hosts and a witness
• VMs running in Preferred and in Secondary
• VMs configured with "should rules" to stay within their fault domain
• ISL between "data locations" is impacted
• HA has restarted the VMs of the secondary site in the preferred site
• ISL is now restored

What would happen without DRS awareness of vSAN stretched clusters is that DRS would automatically migrate VMs back to the Secondary site as soon as it becomes available. DRS runs every minute in vSphere 7.0 so it is very likely that vSAN is still resyncing data. The problem with this is two-fold:

1. The vMotion process will slow down the resync of data temporarily
2. Blocks which have not been resynced and are being read by the VM will need to be fetched from the remote location

As you can imagine this is an undesired situation. As such in vSphere / vSAN 7.0 a whole new level of integration is introduced between DRS and vSAN. Now DRS will be aware of what is happening on the vSAN layer. If vSAN is syncing a particular component of a virtual machine, then DRS will not move the VM back! It will wait until the resync has completed and then move the VM back. This ensures that the migration won't conflict with the resync, and of course that when the VM is migrated that it will have "site read locality".

It is a feature our team had been asking for and which was tested within VMware Cloud on AWS, and I am happy to see it made it into the "regular" vSphere release.

vSphere Support for Intel Optane Persistent Memory

VMware and Intel are working closely to develop the market and use-cases for Intel’s Optane Persistent Memory (PMEM).

This technology is available in two modes: 

• App-direct mode (AD in short, also known as Persistent Memory): vSphere 6.7 U3 enables Intel®Optane™ DC Persistent Memory in “App-Direct” mode. You can take advantage of the large capacity, affordability and persistence benefits offered in this mode and deploy in production any supported 3rd party application without any restriction with full VMware support. VMware encourages its customers to leverage this technology in “App-Direct” mode. For more information on the App-Direct mode performance benefits in virtualization environment, please refer to PMEM App-Direct WP.
• Memory-Mode (MM): vSphere 6.7 Update 3 enable Intel® Optane™ DC Persistent Memory in“Memory” mode. vSphere usage of Intel® Optane™ DC Persistent Memory in “Memory mode”can offer increased memory capacity and TCO improvements for relevant workloads. Initially, VMware will support “Memory” mode for appropriate use-cases in production deployments (refer toPMEM memory-mode WP); such a deployment should go through RPQ process to secure VMware support.

Specific vSphere and VSAN support statement for this technology is available in this KB articlevSphere Support for Intel's Optane DC Persistent Memory (PMEM) (67645). Please note the recommended version to use is vSphere is 6.7u3.

If customers are using this technology in App-Direct Mode, there is no explicit approval needed. VMware support this technology on certified hardware. You can find the list of certified hardwarehere.

If customers are using this technology in Memory-Mode, customer need to procure an RPQ approval from VMware. 

• It is important to highlight that VMware is supporting the Intel Optane Persistent Memory in memory mode and committed to develop the use-cases and market in close collaboration with Intel
• As this technology is new and runs at slower speed than DDR memory, we want to educate the market and develop the right use-cases and expectation. Due to this reason, VMware wants to work closely with early customers and help them succeed
• To address the above requirement, VMware is leveraging the existing RPQ process for early customers, VMware representative need to file the RPQ for interested customers. VMware request specific information from the customer environment. You can find the detailed information about filing the customer RPQ for Intel Optane Persistent Memory-mode here
• It is important to note that RPQ process is only for early customers. Once we develop the early success stories and use-case, VMware has all the intent to remove the RPQ requirements and make this technology as generally supported. 

Please note, VMware is committed to support Intel Optane Persistent Memory in both modes “App-Direct” and “Memory-mode”. If you have any question, please feel free to reach out to @Sudhanshu Jain

Sudhanshu (Suds) Jain

Product Management – Cloud Infrastructure

3401 Hillview Avenue, Palo Alto, CA 94304

Office: 650.427.7672 | Mobile: 408.393.7668

MobaXterm Xserver with SSH, telnet, RDP, VNC and X11 - Features

https://mobaxterm.mobatek.net/features.html

Domaci automatiazce SONOFF

https://sonoff.tech/

Routing - Administrative distance

Administrative distance

https://en.wikipedia.org/wiki/Administrative_distance

External BGP - 20
Internal BGP - 200

Host cannot communicate with one or more other nodes in the vSAN enabled cluster

Ping between nodes was working so it was not a physical network issue. This is the lab environment so all services (mgmt, vMotion, vSAN) are enabled on single VMKNI (vmknic0).

So what's the problem?

I did some google searching and found that some people were experiencing problems with vSAN unicast agents.

Here is the command to list of unicast agents on vSAN node

esxcli vsan cluster unicastagent list

Grrrr. The list is empty!!!! On all ESXi hosts in my 3 nodes vSAN cluster.

Let's try to configure it manually.

Each vSAN node should have a connection to agents on other vSAN nodes in the cluster.

For example, one vSAN node from 4-node vSAN Cluster should have 3 connections

 [root@n-esx04:~] esxcli vsan cluster unicastagent list  
 NodeUuid               IsWitness Supports Unicast IP Address    Port Iface Name Cert Thumbprint  
 ------------------------------------ --------- ---------------- -------------- ----- ---------- -----------------------------------------------------------  
 5e3ec640-c033-7c7d-888f-00505692f54d     0       true 192.168.11.105 12321       18:F3:B7:9F:66:C4:C4:3E:0F:7D:69:BB:55:92:BC:A3:AC:E4:DD:5F  
 5df792b0-f49f-6d76-45af-005056a89963     0       true 192.168.11.107 12321       20:4C:C1:48:F5:2D:04:16:55:F1:D3:F1:4C:26:B5:C4:23:E5:B4:12  
 5e3e467a-1c1b-f803-3d0f-00505692ddc7     0       true 192.168.11.106 12321       53:99:00:B8:9D:1A:97:42:C0:10:C0:AF:8C:AD:91:59:22:8E:C9:79  

We need the get local UUID of the cluster node.

 [root@n-esx08:~] esxcli vsan cluster get  
 Cluster Information  
   Enabled: true  
   Current Local Time: 2020-02-11T08:32:55Z  
   Local Node UUID: 5df792b0-f49f-6d76-45af-005056a89963  
   Local Node Type: NORMAL  
   Local Node State: MASTER  
   Local Node Health State: HEALTHY  
   Sub-Cluster Master UUID: 5df792b0-f49f-6d76-45af-005056a89963  
   Sub-Cluster Backup UUID:  
   Sub-Cluster UUID: 52c99c6b-6b7a-3e67-4430-4c0aeb96f3f4  
   Sub-Cluster Membership Entry Revision: 0  
   Sub-Cluster Member Count: 1  
   Sub-Cluster Member UUIDs: 5df792b0-f49f-6d76-45af-005056a89963  
   Sub-Cluster Member HostNames: n-esx08.home.uw.cz  
   Sub-Cluster Membership UUID: f8d4415e-aca5-a597-636d-005056997c1d  
   Unicast Mode Enabled: true  
   Maintenance Mode State: ON  
   Config Generation: 7ef88f9d-a402-48e3-8d3f-2c33f951fce1 6 2020-02-10T21:58:16.349  

So here are my nodes
n-esx08 - 192.168.11.108 - 5df792b0-f49f-6d76-45af-005056a89963
n-esx09 - 192.168.11.109 - 5df792b0-f49f-6d76-45af-005056a89963
n-esx10 - 192.168.11.110 - 5df792b0-f49f-6d76-45af-005056a89963

And the problem is clear. All vSAN nodes have the same UUID.
Why?  Let's check ESXi system UUIDs on each ESXi host.

 [root@n-esx08:~] esxcli system uuid get  
 5df792b0-f49f-6d76-45af-005056a89963  
 [root@n-esx08:~]  

 [root@n-esx09:~] esxcli system uuid get  
 5df792b0-f49f-6d76-45af-005056a89963  
 [root@n-esx09:~]  

 [root@n-esx10:~] esxcli system uuid get  
 5df792b0-f49f-6d76-45af-005056a89963  
 [root@n-esx10:~]  

So the root cause is obvious. I use nested ESXi to test vSAN and I forgot to regenerate system UUID after the clone. The solution is easy. Just delete UUID from /etc/vmware/esx.conf and restart ESXi hosts.

ESXi system UUID in /etc/vmware/esx.conf

You can do it from command line as well

sed -i 's/system\/uuid.*//' /etc/vmware/esx.conf
reboot

So we have identified the problem and we are done. After ESXi hosts restart vSAN Cluster Nodes UUIDs are changed automatically and vSAN unicastagents are automatically configured on vSAN nodes as well.

However, if you are interested in how to manually add a connection to a unicast agent on a particular node, you would execute the following command

esxcli vsan cluster unicastagent add –a <ip address unicast agent> –U <supports unicast> –u <Local UUID> -t < type>

Anyway, such a manual configuration should not be necessary and you should do it only when instructed by VMware support.

Hope this helps someone else in VMware community.