Pages

Monday, February 10, 2020

Host cannot communicate with one or more other nodes in the vSAN enabled cluster

Ping between nodes was working so it was not a physical network issue. This is the lab environment so all services (mgmt, vMotion, vSAN) are enabled on single VMKNI (vmknic0).

So what's the problem?

I did some google searching and found that some people were experiencing problems with vSAN unicast agents.

Here is the command to list of unicast agents on vSAN node

esxcli vsan cluster unicastagent list

Grrrr. The list is empty!!!! On all ESXi hosts in my 3 nodes vSAN cluster.

Let's try to configure it manually.

Each vSAN node should have a connection to agents on other vSAN nodes in the cluster.

For example, one vSAN node from 4-node vSAN Cluster should have 3 connections

 [root@n-esx04:~] esxcli vsan cluster unicastagent list  
 NodeUuid               IsWitness Supports Unicast IP Address    Port Iface Name Cert Thumbprint  
 ------------------------------------ --------- ---------------- -------------- ----- ---------- -----------------------------------------------------------  
 5e3ec640-c033-7c7d-888f-00505692f54d     0       true 192.168.11.105 12321       18:F3:B7:9F:66:C4:C4:3E:0F:7D:69:BB:55:92:BC:A3:AC:E4:DD:5F  
 5df792b0-f49f-6d76-45af-005056a89963     0       true 192.168.11.107 12321       20:4C:C1:48:F5:2D:04:16:55:F1:D3:F1:4C:26:B5:C4:23:E5:B4:12  
 5e3e467a-1c1b-f803-3d0f-00505692ddc7     0       true 192.168.11.106 12321       53:99:00:B8:9D:1A:97:42:C0:10:C0:AF:8C:AD:91:59:22:8E:C9:79  

We need the get local UUID of the cluster node.

 [root@n-esx08:~] esxcli vsan cluster get  
 Cluster Information  
   Enabled: true  
   Current Local Time: 2020-02-11T08:32:55Z  
   Local Node UUID: 5df792b0-f49f-6d76-45af-005056a89963  
   Local Node Type: NORMAL  
   Local Node State: MASTER  
   Local Node Health State: HEALTHY  
   Sub-Cluster Master UUID: 5df792b0-f49f-6d76-45af-005056a89963  
   Sub-Cluster Backup UUID:  
   Sub-Cluster UUID: 52c99c6b-6b7a-3e67-4430-4c0aeb96f3f4  
   Sub-Cluster Membership Entry Revision: 0  
   Sub-Cluster Member Count: 1  
   Sub-Cluster Member UUIDs: 5df792b0-f49f-6d76-45af-005056a89963  
   Sub-Cluster Member HostNames: n-esx08.home.uw.cz  
   Sub-Cluster Membership UUID: f8d4415e-aca5-a597-636d-005056997c1d  
   Unicast Mode Enabled: true  
   Maintenance Mode State: ON  
   Config Generation: 7ef88f9d-a402-48e3-8d3f-2c33f951fce1 6 2020-02-10T21:58:16.349  

So here are my nodes
n-esx08 - 192.168.11.108 - 5df792b0-f49f-6d76-45af-005056a89963
n-esx09 - 192.168.11.109 - 5df792b0-f49f-6d76-45af-005056a89963
n-esx10 - 192.168.11.110 - 5df792b0-f49f-6d76-45af-005056a89963

And the problem is clear. All vSAN nodes have the same UUID.
Why?  Let's check ESXi system UUIDs on each ESXi host.

 [root@n-esx08:~] esxcli system uuid get  
 5df792b0-f49f-6d76-45af-005056a89963  
 [root@n-esx08:~]  

 [root@n-esx09:~] esxcli system uuid get  
 5df792b0-f49f-6d76-45af-005056a89963  
 [root@n-esx09:~]  

 [root@n-esx10:~] esxcli system uuid get  
 5df792b0-f49f-6d76-45af-005056a89963  
 [root@n-esx10:~]  

So the root cause is obvious. I use nested ESXi to test vSAN and I forgot to regenerate system UUID after the clone. The solution is easy. Just delete UUID from /etc/vmware/esx.conf and restart ESXi hosts.

ESXi system UUID in /etc/vmware/esx.conf

You can do it from command line as well

sed -i 's/system\/uuid.*//' /etc/vmware/esx.conf
reboot

So we have identified the problem and we are done. After ESXi hosts restart vSAN Cluster Nodes UUIDs are changed automatically and vSAN unicastagents are automatically configured on vSAN nodes as well.

However, if you are interested in how to manually add a connection to a unicast agent on a particular node, you would execute the following command

esxcli vsan cluster unicastagent add –a <ip address unicast agent> –U <supports unicast> –u <Local UUID> -t < type>

Anyway, such a manual configuration should not be necessary and you should do it only when instructed by VMware support.

Hope this helps someone else in VMware community.

Saturday, February 1, 2020

macOS – SSH Error ‘No Matching Exchange Method Found’

Davids-MacBook-Pro:~ dpasek$ ssh admin@192.168.4.253
Unable to negotiate with 192.168.4.253 port 22: no matching key exchange method found. Their offer: diffie-hellman-group1-sha1,

Solution ...
https://www.petenetlive.com/KB/Article/0001245

Solution

This is not Apple’s fault, it’s OpenSSH version 7. SHA1 is weak, so support for it has been removed. Which is fine, but all my clients Cisco Firewalls/Routers/Switches are probably all using  RSA/SHA1. So until they re all updated I’m going to need to re-enable SHA1.
Open a terminal windows and execute the following;
sudo nano /etc/ssh/ssh_config
ENTER YOUR PASSWORD
Locate the line ‘ #   MACs hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-ripemd160′ and remove the Hash/Pound sight from the beginning.
Locate the line ‘ #   Ciphers aes128-ctr,aes192-ctr,aes256-ctr,aes128-cbc,3des-cbc’ and remove the Hash/Pound sight from the beginning.
Then paste the following on the end;
HostkeyAlgorithms ssh-dss,ssh-rsa
KexAlgorithms +diffie-hellman-group1-sha1

Thursday, January 30, 2020

VMware Cloud Foundation - Overview

Overview

  1. VCF lze postavit ve dvou deployment modelech
    1. Standardní – to je jedna Management domain (minimum 4 hosty, maximum 64) + až 15 oddělených Workload domains (minimum 3 hosty per pro každou, maximum 64).
V tomto modelu je tedy potřeba mimimálně 7 hostů (4 + 3), oddělení workloadu ja na úrovni samotných hostů.
    1. Konsolidovaný -  Management a Workload domains je společná, minimální počet jsou 4 hosty a lze rozšiřovat až na 64 hostů.
Oddělení workloadu je v tomto modelu na úrovni Resource pool-ů.
Tento model nepodporuje automatický deployment VMware Enterprise PKS a VMware Horizon.
V budoucnu lze přejít z konsolidovaného modelu na standardní.

  1. vCenter
    1. Licence vCenter není součástí balíku VCF a je potřeba zajistit samostatně
    2. Pro VCF je potřeba jedna licence vCenter (platí pro oba deployment modely).
V případě Standardního modelu je součástí každé Workload domain vlastní instance vCenter serveru (ty se ale nelicencují, opět stačí jedna licence).
    1. Není podporováno použití existujícího/externího vCenter serveru.

  1. Hardware
    1. Management domain případně Konsolidovaný model musí mít pod sebou vSAN. Ve Standardním modelu jsou pro Workload domény podporovány i externí storage (NFS, FC SAN).
    2. Do verze VCF 3.9.1 bylo možné pro komunikaci mezi hosty (management, vSAN, vMotion) vyžít pouze dvě fyzické NIC, od verze 3.9.1 je možné použít až 4 pNICs, pokud je použito NSX-V nebo až 6 pNICs, pokud je použito NSX-T. 

  1. Networking
    1. Management domain a obecně konsolidovaný model podporuje pouze NSX-V
    2. Do Workload domain lze zvolit zda bude použito NSX-V nebo NSX-T
    3. VCF od verze 4.0 by měl podporovat již pouze NSX-T (Přechod z NSX-V by měl být vyřešen při upgrade)

Sunday, January 26, 2020

Windows 10 - ICMP Allow incoming V4 echo request

Command

netsh advfirewall firewall add rule name="ICMP Allow incoming V4 echo request" protocol=icmpv4:8,any dir=in action=allow

Source: https://www.howtogeek.com/howto/windows-vista/allow-pings-icmp-echo-request-through-your-windows-vista-firewall/

Alternativ way over GUI

Windows Firewall
Search for Windows Firewall, and click to open it.
Click Advanced Settings on the left.
From the left pane of the resulting window, click Inbound Rules.
In the right pane, find the rules titled File and Printer Sharing (Echo Request - ICMPv4-In).
Right-click each rule and choose Enable Rule.

Source: https://kb.iu.edu/d/aopy