M1000e CMC and IDRAC on separate subnets via CLI

Here is a script to set the VLAN IDs and IP addresses on elements of the chassis via CLI (when you want different VLANs, that is).

(Caveat: I haven't found a successfully tested method of setting server iDRAC IP addresses via CLI, although the VLAN takes.)

# Configuring CMC/Chassis management elements with separate VLANs via CLI

#

# Assign CMC IP, place on VLAN 5 with no dot1p priority

#

setniccfg -s 10.10.5.2 255.255.255.0 10.10.5.1

setniccfg -v 5 0

#

# (Alternative syntax)

# config -g cfgLanNetworking -o cfgNicVLanEnable 1

# config -g cfgLanNetworking -o cfgNicVlanID 5

#

# Place iDRACs on VLAN 10, dot1p priority 0

#

setniccfg -m server-1 -v 10 0

setniccfg -m server-2 -v 10 0

setniccfg -m server-3 -v 10 0

setniccfg -m server-4 -v 10 0

setniccfg -m server-5 -v 10 0

setniccfg -m server-6 -v 10 0

setniccfg -m server-7 -v 10 0

setniccfg -m server-8 -v 10 0

setniccfg -m server-9 -v 10 0

setniccfg -m server-10 -v 10 0

setniccfg -m server-11 -v 10 0

setniccfg -m server-12 -v 10 0

setniccfg -m server-13 -v 10 0

setniccfg -m server-14 -v 10 0

setniccfg -m server-15 -v 10 0

setniccfg -m server-16 -v 10 0

#

# Place switch management interfaces on VLAN 15, dot1p priority 0

#

setniccfg -m switch-1 -v 15 0

setniccfg -m switch-2 -v 15 0

setniccfg -m switch-3 -v 15 0

setniccfg -m switch-4 -v 15 0

setniccfg -m switch-5 -v 15 0

setniccfg -m switch-6 -v 15 0

#

# Assign switch management static IP addresses

#

setniccfg -m switch-1 -s 10.10.15.3 255.255.255.0 10.10.15.1

setniccfg -m switch-2 -s 10.10.15.4 255.255.255.0 10.10.15.1

setniccfg -m switch-3 -s 10.10.15.5 255.255.255.0 10.10.15.1

setniccfg -m switch-4 -s 10.10.15.6 255.255.255.0 10.10.15.1

setniccfg -m switch-5 -s 10.10.15.7 255.255.255.0 10.10.15.1

setniccfg -m switch-6 -s 10.10.15.8 255.255.255.0 10.10.15.1

#

# Clear VLAN settings for any interfaces

#

setniccfg -v

setniccfg -m server-1 -v

setniccfg -m switch-3 -v

#

#

#

Routed VLT vs VRRP

There were a lot of queries on how R-VLT can be used in place of VRRP and what are the advantages and disadvantages. Further to this I had a call with some of you and provided answers to all these queries. I am summarizing the details here. If there are further questions please email me.

R-VLT or peer-routing (as it is called in configuration guides) provide the ability to do the default gateway functionality similar to VRRP. Apart from that, one can run any IPv4 / IPv6 routing protocols in a VLT based core with R-VLT turned on. Please note this is a unique capability among vendors that offer MLAG type solutions.  For more information on benefits of R-VLT, please refer to ‘VLT Overview 2.0’ AND ‘VLT reference architecture 2.0” in sales edge.

Ø  R-VLT is recommended where larger # of L3 VLANs are required. VRRP has a limitation of 255 L3 VLANs, so if the customer requires more than 255 L3 VLANS please recommend R-VLT. Today we support 512 L3 VLANs in R-VLT and we are going to increase this number to 2,500 L3 VLANs in OS 9.7 (Klamath) release

Ø  There was a limitation when a peer-node goes down the other peer was not responding to ARP requests sent to default gateway address. This was fixed when “Proxy ARP” functionality for peer-VLT nodes was introduced in OS 9.3

Ø  VRRP-v4 and R-VLT can co-exist together. Certain set of VLANs can be using VRRP and other set can be using R-VLT. This is required during migration from VRRP to R-VLT and the customer is not willing to convert all VLANs to R-VLT in one shot

Ø  Caveat with R-VLT: In a VLT setup between switches A and B, where A’s IP is configured as default gateway, when both the nodes go down and only B comes up, the gateway functionality will not be available. R-VLT requires a handshake between A and B at least once. Hence in such cases the network will not have a gateway until the other node comes up. 

o    This is a corner case that can happen when both the nodes go down and only one comes up (power outage with other issues) or when software is upgraded on both VLT node (upgrade done one after other). The former will have other network issues like convergence etc. as there is a DC wide power outage and the later will likely be done during a maintenance window.

o    We are exploring ways to solve this issue in the future OS 9.x release, but there are no simpler ways of solving this issue. So unlikely to get fixed in the near future. 

For all the functionalities introduced in VLT since OS 9.2, please refer to corresponding DNL slides available on DNL playback site. For any other queries please reach out to me or ASK-NETWORKING-PLM alias.

Thanks & regards,

Shankar Vasudevan

-------------------------------------------------------

Product Manager | Dell Networking | Enterprise Solutions Group

office: +91-44-3920-8451, mobile: +91-9500018850

Chennai, India

Compellent Serial Console over iDRAC

To get it working a few steps have to be taken on both Controllers:

1.       Configure iDRAC

a.       Go to Network->Serial

b.      Set IPMI’s Baud Rate to 115.2 kbps (Compellent Serial Port Baud Rate)

c.       Apply Settings

2.       During boot enter the Controller’s BIOS

a.       Go to “Serial Communication”

b.      Switch from “Off” to “On without Redirection”

c.       Switch Port Configuration from “Serial Device1=COM1;Serial Device2=COM2” to “Serial Device1=COM2;Serial Device2=COM1”

d.      Save Settings and Reboot Controller

After these steps the Compellent’s serial console is available via iDRAC:

Login to iDRAC using SSH and type “connect” at the prompt. After that the SSH session shows the serial console as if directly connected to the system’s serial port.

Force10 IOM VLT

Stack-unit 0 iom-mode vlt uplink-speed 40

EMC VNX SRM

Implementation Plan
http://www.cosonok.com/2012/11/implementing-vmware-site-recovery.html

vSphere Advanced Settings

ESX Host Advanced Settings

ESX Advanced parameter	Default value	Changed Value	Justification
Syslog.global.logHost	empty	Syslog servers See. Table 122 SYSLOG Servers	Centralized syslog for troubleshooting and security audits.
Syslog.global.logDirUnique	false	true	Creates unique subdirectories in shared datastore scratch location.
Syslog.global.logDir		[TEMPLATES-01] /scratch/log/	We use SD cards in ESX hosts where ramdisk is used for logs and core dumps. This setting instructs ESXi to use shared datastore instead local ramdisk for scratch location.
UserVars.ESXiShellInteractiveTimeOut	0	1800	ESXi Shell (SSH, concole) log out time-out value in seconds. Changed value 1800 seconds (30 min) increase security.
UserVars.SuppressShellWarning	0	1	Disables warning message that SSH is enabled.
Config.HostAgent.plugins.hostsvc.esxAdminsGroup	ESX Admins	PPOD-TEC-NG-Admins PPOD-TEC-CH-Admins	We have two AD groups of ESX Admins managing pPODs in different datacenters.
VMkernel.Boot.terminateVMOnPDL	no	yes	Terminates VMs in case LUN device is permanently lost.
Disk.AutoremoveOnPDL	enabled	disabled	Don't remove datastores in PDL automatically.

vSphere HA Advanced Settings

HA Cluster Advanced parameter	Default value	Changed Value	Justification
das.vmcpuminmhz	32MHz	570MHz	Defines the default CPU resource value assigned to a virtual machine if its CPU reservation is not specified or zero. This is used for the Host Failures Cluster Tolerates admission control policy. Default min reservation 570MHz per VM solves vCloud Director CPU Max OverBooking Ratio. Single ESX host can serve 29GHz. 50 * 570MHz = 28.5GHz
das.maskCleanShutdownEnabled	false	true	This is an accompanying configuration that helps vSphere HA distinguish between VMs that were once powered on and should be restarted versus VMs that were already powered off when a PDL occurred therefore these are VMs that don’t need to be and more importantly probably should not be restarted

ESX disk scheduler vDisk behavior

vDisk IOPS limit = 400 IOPS

IO Block Size	# of  threads	vDisk  IOPS  limit	vDisk IOPS achieved
512B	1	400	400
512B	4	400	400
512B	8	400	400
512B	16	400	400
2kB	1	400	400
2kB	4	400	400
2kB	8	400	400
2kB	16	400	400
4kB	1	400	300
4kB	4	400	400
4kB	8	400	400
4kB	16	400	400
16kB	1	400	240
16kB	4	400	400
16kB	8	400	400
16kB	16	400	400
32kB	1	400	200
32kB	4	400	400
32kB	8	400	400
32kB	16	400	400
64kB	1	400	150
64kB	4	400	320
64kB	8	400	400
64kB	16	400	400
256kB	1	400	115
256kB	4	400	185
256kB	8	400	240
256kB	16	400	380
256kB	32	400	380
1MB	1	400	80
1MB	8	400	180
1MB	16	400	180
1MB	24	400	180
1MB	32	400	180