DavidRing.ie

For VxRail 4.7 please see earlier post:

VxRail 4.7 – Install Notes

Note: this is an example for reference only please use the VxRail installation procedures provided by Dell EMC.

Prerequisites for installation include:

• Create all relevant ESXi, VC and VxRAIL manager DNS entries on AD/DNS server.
• Verify the customer DNS server can resolve the following components, test both forward and reverse lookups using nslookup:
  • ESXi Mgmt
  • VxRail Manager
  • vCenter
  • SRS Gateway
• Ping all the above components in order to rule out any duplicate IP address conflicts.
• Verify NTP is working correctly, the following are some useful links: https://kb.vmware.com/s/article/1035833https://kb.vmware.com/s/article/1003736
  • Ensure date is in sync:
    • ESXi set date and time:
      • #esxcli system time get 
      • esxcli system time set d|–day -H|–hour -m|–min -M|–month -s|–sec -y|–year
      • #esxcli system time set -d 19 -H 12 -m 30 -M 04 -y 2018 
    • vCSA & PSC/VxRail Manager – view & set date and time:
      • # date
      • # date -s “19 APR 2018 12:30:00”
• You can check the installed VxRail code on the ESXi host:
  • esxcli software vib list | grep marvin
• IPv6 multicast is used by the VxRail Manager loudmouth service and is required for automatic node discovery. VxRail nodes can be added manually in the event of IPv6 network limitations. This discovery network is visible only to the VxRail nodes and depends on IPv6 multicasting services to be configured on the adjacent ‘top-of-rack’ switches for node discovery purposes.
• On the first host of the cluster set the VxRail Manager VM IP via IDRAC/DCUI (vxrail-idrac-esxi-shell-access-via-dcui):
  • vxrail-primary --config --vxrail-address 192.1.20.50 --vxrail-netmask 255.255.255.0 --vxrail-gateway 192.1.20.1 --no-roll-back --verbose
• Check if VxRail Manager VM is running:
  • esxcli vm process list
• Enter the console (ALT+F1) and change the VLAN ID for the “VM Network” as per PEQ. As per this example the ‘VM Network’ portgroup is tagged with VLAN ID 120:
  • esxcli network vswitch standard portgroup list
  • esxcli network vswitch standard portgroup set -p "VM Network" -v 120
• Restart loudmouth on each ESXi host:
  •  /etc/init.d/loudmouth restart
• Restart loudmouth on the VxRail Manager vm:
  • systemctl restart vmware-loudmouth
• Power cycle the VxRail Manager VM:
  • vim-cmd vmsvc/getallvms
  • vim-cmd vmsvc/power.shutdown vmid
  • vim-cmd vmsvc/power.getstate vmid
  • vim-cmd vmsvc/power.on vmid
  • vim-cmd vmsvc/power.getstate vmid

Check that VxRail Manager can discover available ESXi hosts; login to the VxRail Manager vm via SSH and run the following commands:

• /usr/lib/vmware-loudmouth/bin/loudmouthc query | grep -o "applianceID=..............." | sort
• /usr/lib/vmware-loudmouth/bin/loudmouthc query | egrep -o "EMC........-..|..............-..-.." | sed 's/ {"//g' | sort
• curl --unix-socket /var/lib/vxrail/nginx/socket/nginx.sock -H "Content-Type: application/json" http://127.0.0.1/rest/vxm/internal/do/v1/host/query -d '{"query":"{availableHosts{moid ,name ,hardware{sn psnt} ,config{network{vnic{device, ipv6}}}}}"}' |json_pp

Switch Validation

• Confirm NIC’s are up:
  • esxcli network nic list
  • esxcli network vswitch standard list

• Confirm the mac address of VxRail Manager is showing up on the switch. You can find the VxRail Manager mac address from the ESXi console & compare to mac table on switch:
  • vim-cmd vmsvc/get.guest vmid | more
  • show mac-address-table

• Ensure your switch ports are configured identically for all VxRail port connections.
• Ensure each port has the required VLANs configured and trunked to all ports. In this example ESXi mgmt 120, vMotion 122, vSAN 123 and default Private Mgmt Network 3939.
• In a dual switch configuration ensure that the VLANs are configured on the ISL/VPC peer link.
•  Sample DELL s4048 VLAN port config:

!
interface Vlan 120
ip address 192.1.20.1/24
tagged TenGigabitEthernet 1/1-1/32
no shutdown
!
interface Vlan 122
ip address 192.1.22.1/24
tagged TenGigabitEthernet 1/1-1/32
no shutdown
!
interface Vlan 123
ip address 192.1.23.1/24
tagged TenGigabitEthernet 1/1-1/32
no shutdown
!
interface Vlan 3939
no ip address
tagged TenGigabitEthernet 1/1-1/32
no shutdown
!

Note On Cabling: For VxRail nodes with RJ45 NICs use Cat 6 or higher Ethernet cables. In the case of nodes with SFP+, SFP28, or QSFP28 NICs for Twinax connectivity use supported Twinax Direct-Attach-Copper cables with SFP+, SFP28, or QSFP28 connectors and in the case of FC use supported fiber cables SFP+, SFP28, QSFP28 optical transceivers.

Configure VxRail (Step-By-Step)

VxRail Software used in this example: 7.0.100 build 16850804

Browse to the VxRail Manager IP address which in this example is https://192.1.20.50, VxRail Welcome screen is displayed, click on ‘Get Started’ and accept the EULA:

CURL Command to monitor install progress from VxRail Manager SSH console:

curl -kX GET --user administrator@vsphere.local https://127.0.0.1/rest/vxm/v1/system/initialize/status | python3 -m json.tool

Another Day1 monitoring option includes leveraging the built-in Swagger UI:

Post Install Validation

Here are some quick checks after a successful bring-up of VxRail. These checks are confirmed by logging into vCenter Server HTML client:

Check Logs

Monitor VxRail dayone.log , short.term.log & firstboot.log:

• dayone.log – detailed information in relation to initial first run configuration. (/var/log/microservice_log/dayone.log)
• short.term.log – micro services related information. (/var/log/microservice_log/short.term.log)
• firstboot.log – detailed information in relation to micro services boot up during initial configuration. (/var/log/firstboot.log)

cd /var/log/microservice_log/

# cat dayone.log
# tail -f dayone.log
# tail -f dayone.log | grep ERROR
# tail -n 100 dayone.log
# more dayone.log

Confirm DNS entries in resolv.conf & DB:

cat /etc/resolv.conf

curl -X GET --unix-socket /var/lib/vxrail/nginx/socket/nginx.sock http://localhost/rest/vxm/internal/configservice/v1/configuration/keys/system_dns -H "accept: application/json" -H "Content-Type: application/json"

NTP Validation:

timeout 2 bash -c "</dev/udp/10.104.0.254/123"; echo $?

Returns 0

In terms of viewing a “Configuration complete!” equivalent in the logs the string is :
/var/log/microservice_log # cat dayone.log | grep "notify {'level': 'workflow', 'state': 'COMPLETED', 'progress': 100}"