Pages

Tuesday, June 21, 2016

problem s DVS po export/import DVS do noveho vCentra - padajici VPXD proces

Hi.

The vpx process was crashing with the following error in /storage/log/vmware/vpx/vpxd.log
mem> 2016-06-13T09:05:57.713Z [7F4B0DC39700 error 'commonvpxCommon' opID=49A39FF7-0000006E-b6] [Vpxd_HandleVmRootError] Received unrecoverable VmRootError. Generating minidump ...
mem> 2016-06-13T09:05:57.713Z [7F4B0DC39700 error 'Default' opID=49A39FF7-0000006E-b6] An unrecoverable problem has occurred, stopping the VMware VirtualCenter service. Error: Error[VdbODBCE
rror] (-1) "ODBC error: (23505) - ERROR: duplicate key value violates unique constraint "pk_vpx_dvport_membership"
mem> --> Key (dvs_id, dvport_key)=(354, 152) already exists.;
mem> --> Error while executing the query" is returned when executing SQL statement "INSERT INTO VPX_DVPORT_MEMBERSHIP (DVS_ID, DVPORT_KEY, DVPORTGROUP_ID, HOST_ID, LAG_KEY) VALUES (?, ?, ?,
?, ?)"
mem> 2016-06-13T09:05:57.713Z [7F4B0DC39700 verbose 'commonvpxCommon' opID=49A39FF7-0000006E-b6] Backtrace:
mem> -->
mem> 2016-06-13T09:05:57.728Z [7F4B0DC39700 panic 'Default' opID=49A39FF7-0000006E-b6] (Log recursion level 2) Unrecoverable VmRootError. Panic!

Solution:
http://www.hivmr.com/db/skz3713kkp8kssmszcj8jsxckds8fczj   how to log into PostgreSQL DB on VCSA
/opt/vmware/vpostgres/current/bin/psql -d VCDB -U postgres

https://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&externalId=2092070&sliceId=2&docTypeID=DT_KB_1_1&dialogID=120672744&stateId=1%200%20120684601           how to change values in PostgreSQL DB, but error messages are different
UPDATE VPX_DVS SET PORT_COUNTER=((SELECT MAX(CAST(DVPORT_KEY AS INT)) FROM VPX_DVPORT_MEMBERSHIP WHERE DVS_ID='DVS_ID')+1) WHERE ID='DVS_ID';
Repeat for every DVS, with different DVS_ID.

http://vninja.net/virtualization/vpostgres-database-backup-vcsa-5-5/     how to backup PostgreSQL DB on VCSA
/opt/vmware/vpostgres/1.0/bin/pg_dump EMB_DB_INSTANCE -U EMB_DB_USER -Fp -c > VCDBBackupFile


After import of DVS settings (only thing that comes to mind that could have caused this, since the VCSA is a new installation, but DVS were imported from previous vCenter instance), the values in port_counter column in table VPX_DVS were zero instead of the correct value.
When creating a new port (e.g. when creating a new portgroup with static binding), vpxd picks too-small port numbers for new ports. Those number are already in use in VPX_DVPORT_MEMBERSHIP and cause primary key violation.

Wednesday, June 8, 2016

vSphere 6 SSL Certificates


INTRODUCION VIDEO

Certificate Management Overview

Certificate management CLIs
Perform all certificate management tasks with dir-cli, certool, and vecs-cli.

VCSA files and tools
Template file for a CSR request is at /usr/lib/vmware-vmca/share/config/certool.cfg

VMCA – VMware Certificate Authority
VMCA Certificate Manager -  /usr/lib/vmware-vmca/bin/certificate-manager
VMCA Certificate Tool - /usr/lib/vmware-vmca/bin/certool

VECS - VMware Endpoint Certificate Store
VECS CLI - /usr/lib/vmware-vmafd/bin/vecs-cli 

Other resources:
vSphere 6 SSL certificate Replacement / Implementation using the Certificate-Manager automation tool

Understanding and using vSphere 6.0 Certificate Manager (2097936)

Using vecs-cli to manage VMware Certificate Endpoint Store (VECS) instances

Monday, May 23, 2016

VMware Snapshot improvements in vSphere 6

A snapshot removal can stop a virtual machine for long time (1002836)
KB je aplikovatelne jak na vSphere 5.x tak i 6.x. Cili jak jsem rikal na webexu, u vSphere 6.x byla consolidace snapshotu vylepsena o pouziti mirror driver technologii (stejna se pouziva napriklad u storage vMotion) – zdokumentovana a vysvetlena je treba zde: http://www.yellow-bricks.com/2011/07/14/vsphere-5-0-storage-vmotion-and-the-mirror-driver/

Porovnani consolidace snapshotu na vSphere 5.x a vSphere 6.x je zde:

(To co na vsphere 5.x trvalo 12.6 sekund trvalo na vSphere 6.x 1 sekundu – tedy mnohonasobne vylepseni. Jak jsem vsak rikal, zalezi to hodne na use-case, aplikaci atp. Urcite bych nespolehal na to, ze vSphere 6.x zcela vyresi problem.)

Sunday, May 15, 2016

RENAMING A WINDOWS DOMAIN WITH RENDOM

http://www.bauer-power.net/2011/05/renaming-windows-domain-with-rendom.html

Monday, April 25, 2016

NSX multiple VTEPs load balancing

From: Kevin Barrass <kbarrass@vmware.com>
Date: Mittwoch, 24. Februar 2016 10:31
To: Yves Fauser <yfauser@vmware.com>, David Pasek <dpasek@vmware.com>, Emanuele Mazza <emazza@vmware.com>, Dimitri Desmidt <ddesmidt@vmware.com>
Subject: Re: NSX Question

Hi Yves, David.

Short answer is yes VM’s on the same logical switch on a host with two VTEP’s will most likely be balanced across those two VTEPs. Longer answer below :)

When you configure load balance SRCID or SRC MAC. We map dvPorts VM’s are attached to by either dvPort ID or MAC address of VM to one of the dvUplinks on the dvSwitch.
We also statically map each VTEP vmkernel interface to a dvUplink on the dvSwitch. This results in an approximate even split of VM’s across both uplinks.

Each VM will then be encapsulated in VXLAN by the IOChain of the dvUplink the VM is mapped to, the SRC IP of that encapsulation will be the VTEP vmkernel IP address that is mapped to the same dvUplink.
The Local Control Plane (netcpa) will report up to the central control plane this VM dvPort (MAC address) to VTEP mapping.

If a dvUplink on the dvSwitch should fail, all VM’s that were mapped to that dvUplink and the associated VTEP vmkernel interface will be re-mapped to one of the remaining dvUplinks. Also the Local Control Plane agent will report this re-mapping up to the central control plane.

You can view this mapping on the ESXi dataplane from using either esxtop or esxcli as well as on the central control plane as below,:

ESXTOP

 9:12:12am up 13 days 17:46, 492 worlds, 3 VMs, 4 vCPUs; CPU load average: 0.03, 0.03, 0.03

   PORT-ID              USED-BY  TEAM-PNIC DNAME              PKTTX/s  MbTX/s    PKTRX/s  MbRX/s %DRPTX %DRPRX 
  33554433           Management        n/a vSwitch0              0.00    0.00       0.00    0.00   0.00   0.00 
  33554434               vmnic0          - vSwitch0             76.29    0.19      53.41    0.06   0.00   0.00 
  33554435     Shadow of vmnic0        n/a vSwitch0              0.00    0.00       0.00    0.00   0.00   0.00 
  33554436                 vmk0     vmnic0 vSwitch0             76.29    0.19       0.00    0.00   0.00   0.00 
  50331649           Management        n/a DvsPortset-0          0.00    0.00       0.00    0.00   0.00   0.00 
  50331650               vmnic3          - DvsPortset-0          0.00    0.00       0.00    0.00   0.00   0.00 
  50331651     Shadow of vmnic3        n/a DvsPortset-0          0.00    0.00       0.00    0.00   0.00   0.00 
  50331652               vmnic2          - DvsPortset-0          0.00    0.00       0.00    0.00   0.00   0.00 
  50331653     Shadow of vmnic2        n/a DvsPortset-0          0.00    0.00       0.00    0.00   0.00   0.00 
  50331654                 vmk1     vmnic3 DvsPortset-0          0.00    0.00       0.00    0.00   0.00   0.00 
  50331655          vdr-vdrPort     vmnic3 DvsPortset-0          0.00    0.00       0.00    0.00   0.00   0.00 
  50331660 52305:Palo Alto Netw     vmnic2 DvsPortset-0          0.00    0.00       0.00    0.00   0.00   0.00 
  50331661 37661:Dom-Ubuntu01.e     vmnic3 DvsPortset-0          0.00    0.00       0.00    0.00   0.00   0.00 
  50331662 41267:Dom-Windows02.     vmnic3 DvsPortset-0          0.00    0.00       0.00    0.00   0.00   0.00 
  67108865           Management        n/a DvsPortset-1          0.00    0.00       0.00    0.00   0.00   0.00 
  67108866               vmnic1          - DvsPortset-1          0.00    0.00     129.70    0.25   0.00   0.00 
  67108867     Shadow of vmnic1        n/a DvsPortset-1          0.00    0.00       0.00    0.00   0.00   0.00 


ESXCLI

To view VM to dvUplink mapping

~ # esxcli network vm list
World ID  Name                         Num Ports  Networks       
--------  ---------------------------  ---------  ---------------
   37661  Dom-Ubuntu01                         1  dvportgroup-540
   41267  Dom-Windows02                        1  dvportgroup-585
   52305  Palo_Alto_Networks_NGFW_(1)          1  dvportgroup-520

~ # esxcli network vm port list -w 37661
   Port ID: 50331661
   vSwitch: DSwitch-Res01
   Portgroup: dvportgroup-540
   DVPort ID: 222
   MAC Address: 00:50:56:87:6f:a9
   IP Address: 0.0.0.0
   Team Uplink: vmnic3
   Uplink Port ID: 50331650
   Active Filters: dvfilter-generic-vmware-swsec, vmware-sfw, serviceinstance-1

To View VM to vmkernel VTEP mapping

~ # esxcli network  vswitch dvs vmware vxlan network port list --vds-name=DSwitch-Res01 --vxlan-id=5007
Switch Port ID  VDS Port ID  VMKNIC ID
--------------  -----------  ---------
      50331655  vdrPort              0
      50331661  222                  0
~ # 

~ # esxcli network  vswitch dvs vmware vxlan  vmknic list --vds-name=DSwitch-Res01
Vmknic Name  Switch Port ID  VDS Port ID  Endpoint ID  VLAN ID  IP          Netmask        IP Acquire Timeout  Multicast Group Count  Segment ID
-----------  --------------  -----------  -----------  -------  ----------  -------------  ------------------  ---------------------  ----------
vmk1               50331654  20                     0       10  172.16.1.4  255.255.255.0                   0                      0  172.16.1.0
~ # 


Central Control Plane

htb-1n-eng-dhcp10 # show control-cluster logical-switches mac-table 5007
VNI      MAC               VTEP-IP         Connection-ID
5007     00:50:56:87:6f:a9 172.16.1.4      51           
5007     00:50:56:87:92:b1 172.16.1.5      52           
5007     00:50:56:87:6e:32 172.16.1.3      45   


Please don’t hesitate to contact me if you have anymore questions.

Kind Regards
Kev
Kevin Barrass – VCDX#191
Senior NSX Solutions Architect
Network and Security Business Unit 
+44 (0)7825 034393


From: Yves Fauser <yfauser@vmware.com>
Date: Wednesday, 24 February 2016 08:59
To: David Pasek <dpasek@vmware.com>, Emanuele Mazza <emazza@vmware.com>, Dimitri Desmidt <ddesmidt@vmware.com>, Kevin Barrass <kbarrass@vmware.com>
Subject: Re: NSX Question

{Adding Emanuele, Dimitri and Kev}

Hi David,

I must admit that I don’t know the logic of how VM traffic gets placed onto the different VTEPs in a setup where multiple VTEPs are deployed per ESXi Host.
So I can’t tell you if we pin whole logical switches or individual VMs to the different VTEPs.

I’m sure one of our colleagues I added knows this and will educate me and yourself on it ;-)

Cheers,
Yves

Yves Fauser
Senior Solutions Architect
Mobile: +49 172 254 7415


From: David Pasek <dpasek@vmware.com>
Date: Dienstag, 23. Februar 2016 12:17
To: Yves Fauser <yfauser@vmware.com>
Subject: NSX Question

Hi Yves.

I have a simple question but so far I have get several contradictory answers from different VMware’s NSX experts.

Let’s assume I have NSX with multiple VTEPs (2) per ESXi host. 

Is there a chance that each VM connected to the same logical switch will be load balanced across these two VTEPs?

Of course, it depends on hash algorithm result, but let’s assume hash result is unique for each VM.

Thanks in advance.
—

David Pasek, Senior Technical Account Manager

Wednesday, April 20, 2016

VM samples and VMX core dump on a running VM

A virtual machine is composed of several processes or userworlds that run in the VMkernel. Combined, the processes collectively make up a group. The following is a summary of components of a virtual machine:
  • Virtual Machine Executable (VMX) process - A process that runs in the VMkernel that is responsible for handling I/O to devices that are not critical to performance. The VMX is also responsible for communicating with user interfaces, snapshot managers, and remote console.
  • Virtual Machine Monitor (VMM) process - A process that runs in the VMkernel that is responsible for virtualizing the guest OS instructions, and manages memory. The VMM passes storage and network I/O requests to the VMkernel, and passes all other requests to the VMX process. There is a VMM for each virtual CPU assigned to a virtual machine.
  • Mouse Keyboard Screen (MKS) process - A process that is responsible for rendering the guest video and handling guest operating system user input.
*************************************

To set vmsamples up on a running VM, use "vmdumper -l" to list world IDs of running VMs and then "vmdumper <worldid> samples_on" to turn it on for that VM. This will last until the VM is power cycled/powered off or until a samples_off command is run.


Turn VM sample on

#vmdumper <worldid>samples_on

VMM core dump ...
vmdumper <worldid> unsync
vmdumper <worldid> sync

VMX core dump ...
vmdumper <worldid> vmx
vmdumper <worldid> vmx_force

VMkernel core dump ...
You can collect the live dump without crashing the box using -
   localcli --plugin-dir /usr/lib/vmware/esxcli/int/ debug livedump perform

Turn VM sample off
#vmdumper <worldid>samples_off

Refer https://wiki.eng.vmware.com/DebuggingEsx#Creating_a_Live_Crash_Dump_for_ESXi

***************************************
Additional info:
Interpreting virtual machine monitor and executable failures (1019471)

Tags:
esxi, vmkernel, debug, coredump