Dell EMC ViPR 3.6 – VMAX Export Path Adjustment

A very useful new feature in ViPR 3.6 is the ability to increase/decrease the number of fibre channel paths on an existing VPLEX or VMAX export (host/cluster). The following example showcases the ability to increase the path count for a VMAX backed ViPR Export Group.

Note: If you want to adjust the path for an export where there are pre-existing masking
views for a host or cluster in the export group, first ingest any volumes exported to
the host. Example ViPR Ingest Procedure.

As can be seen from the following screen captures the export group consists of 2x ESXi hosts with each host having 2x initiators and each host presently configured for 1x FC path per initiator, as part of this example we will double the total fc paths per host from 2 to 4.

Export Group named ‘EHC’ details:

ViPRExpandPaths0

Continue reading

DellEMC ViPR 3.6 – Install&Config for VMAX AFA

This post details the wizard driven installation steps to install ViPR Controller and configure for a VMAX AFA system.

Recommendation when deploying on a vSphere cluster is to deploy the ViPR Controller on a minimal of a 3 node ESXi DRS cluster, and to set an anti-affinity rule among the
ViPR Controller nodes to, “Separate Virtual Machines,” on available ESXi nodes.

Begin by downloading the ViPR Controller packages from support.emc.com

ViPR 3.6 Install & Config1

This ova will deploy three VMs in a 2+1 redundant fashion allowing for the failure of a single controller without affecting availability. There is also a 3+2 ova available. Continue reading

EMC VMAX – Fully Pre-allocate TDEV

By Fully Pre-allocating a TDEV all the tracks associated with the device are reserved, this may be useful for mission critical apps or avoiding any write miss penalties.

Example SYMCLI:
Single TDEV example:
symconfigure -sid xxx -cmd “start allocate on tdev 0c66 end_cyl=last_cyl allocate_type=persistent;” commitCliPreAll1

Range of TDEVs:
symconfigure -sid xxx -cmd “start allocate on tdev 0c6e:1116 end_cyl=last_cyl allocate_type=persistent;” commitCliPreAll3

CliPreAll2

Example UNISPHERE:
From the Unisphere GUI navigate to storage>volumes right click the device you wish to modify and select ‘Start allocate’.

UniPreAll1

UniPreAll3

UniPreAll2

UniPreAll4

EMC VMAX3 – CLI Cheat Sheet

Guest post by the VMAX Guru – Paul Martin @rawstorage

VMAX3 CLI Cheat Sheet

Disclaimer, this is not a comprehensive how to, just a toe in the ocean of VMAX3, there is always more and there is always why. The information here is not a substitute for the product guides which have been consolidated into a single downloadable PDF documentation set please download and refer to the documentation set for full feature descriptions.

https://support.emc.com/docu59402_Solutions-Enabler-8.0.3-Documentation-Set.pdf
Also see the new features paper for more details on VMAX3 and features in general

https://www.emc.com/collateral/technical-documentation/h13578-vmax3-family-new-features-wp.pdf

FAST with SLO

One of the major changes with V3 is the way we provision storage. FAST has been enhanced to work on a more granular level (128KB track level) and we have abstracted a lot of the internals so that the end user need not be so concerned about the mechanics of the array they can simply provision capacity and set a performance expectation which the array will work to achieve.
In VMAX3 FAST is always on and the majority of the configuration is pre-configured, available SLO are dictated by the disks available in the array and Storage Resource Pools are defined in the bin file.
Provisioning storage on a VMAX3 is easier that on previous Symm/VMAX arrays, we are no longer required to create meta devices to support larger devices and the SLO model makes provisioning intuitive and easy. From the command line it’s pretty much a three step process:

1. Create your storage group and assign your SLO and workload (optional), if no SLO or workload is specified FAST will still manage everything but your SLO will be optimized. The storage can represent your applications devices as a whole and can be used in SRDF and Timefinder meaning if you design storage with application==storagegroup snapshot/srdf design becomes simpler later on too. VMAX3 supports 64K storage groups so there is no reason not to configure 1 per app.
symsg –sid 007 create myapp_sg –slo gold –workload oltp
2. Create and add your devices, here I am creating 5 x 2048 GB devices and adding to my storage group. Note I can just create 2048 GB devices, no meta is created. At present we can create devs up to 16TB soon to be increased further.
symconfigure -sid 007 -cmd "create dev count =5 config=tdev, emulation=fba size=2048 GB sg=myapp_sg;" preview
3. Present to the host via a masking view, no change from VMAX here.
symaccess –sid 007 create view –name myapp_mv –sg myapp_sg –pg myapp_pg –ig myapp_ig

Here I will highlight a few of the key commands to gather information about the configuration and interaction with the SRP and SLO.
NOTE:- Monitoring and Alerting of FAST SLO is built into Unisphere for VMAX. SLO compliance is reported at every level when looking at storage group components in Unisphere.

Viewing SRP Configured On The Array

Most VMAX3 arrays will only have a single SRP however it is possible to have multiple, if you are using FAST.X or ProtectPoint you may have an additional SRP in the config, the following command shows you what is available:
symcfg list –srp

Note the default SRP is set to be usable by RDFA DSE, this is normal. There is no need to configure a separate pool for DSE in VMAX3, we can reserve and cap some space from the default SRP for this purpose.

VMAX3-CLI1
VMAX3-CLI2

Viewing the Available SLO

symcfg list -slo
VMAX3-CLI3

To get a more detailed look at the SLO’s and the workloads that can be associated with storage groups you can run the following command. The output shows the approximate response time for each.

symcfg list –slo –detail –by_resptime –all

VMAX3-CLI4

SRP Capacity Consumption

In order to get an idea of how your storage is being consumed from the command line you can run the CMD:
symsg list –srp –demand –type slo
this will show you how your SRP is being consumed by each of the SLO, it will also list how much is consumed by DSE and Snapshot, remember this capacity all comes from your SRP so it’s worth keeping an eye on.
VMAX3-CLI5

Listing SLO associations by Storage Group

The previous command gives us a good idea at a high level, but if we want to see from a storage group level which storage groups are associated with each SLO we have a command for that too:
Symsg list –by_SLO –detail
this shows each storage group and whether or not it is associated with an SLO, we also get some detail about the number of devices but we don’t see much regarding the capacity.
VMAX3-CLI6

Additionally you can see consumption on an individual device level on the application storage group.
VMAX3-CLI7

You can see the full breakdown of your SRP including drive pools and which SLO you have available as well as TDAT information. The output below shows all the thin devices (TDEVS) bound to the SRP and how much space they are each consuming.
VMAX3-CLI8

Changing SLO On Existing Storage Groups

Changing Service Level Objective to Platinum and Workload to OLTP_REP for a storage group test:
symsg –sg test -sid 123 set –slo Platinum -wl OLTP_REP

Solutions Enabler 8.X also allows for moving devices between groups non-disruptively
• Moving devices between child storage groups of a parent storage group when the masking view uses the parent group.
• Moving devices between storage groups when a view is on each storage group and both the initiator group (IG) and the port group (PG) elements are common to the views (initiators and ports from the source group must be present in the target).
• Moving devices from a storage group with no masking view or service level to one in a masking view. This is useful as you can now create devices and automatically add to storage group from CLI, so a staging group may exist. Command is:
symsg –test –sid 123 –sg staging_sg move dev 345 gold_sg

VMAX3-CLI9

SnapVX – Space efficient Targetless Snapshots
I’m not going to go into the full details of SnapVX and what makes it revolutionary in the VMAX3, we have a very good technote that already covers this in detail. Needless to say, taking snapshots on VMAX3 is quicker, more efficient and easier than it has been on any previous generations. See the technote for full details.
Like most features in the VMAX to access the functionality simply put the word sym in front of the feature name. SnapVX is controlled with the symsnapvx command set. Really the only command you should need is symsnapvx –h, this will get you the full set of options. I’ll highlight a few of the main commands here.

Creating Snapshots

SnapVX is simplest when your storage has been designed with an application per storage group, you can still use device groups or files if you want but VMAX3 supports 64K storage groups, that is enough for one per application in most environments and means only managing a single entity for each application for provisioning as well as local replication and remote replication. You can snap multiple applications together using a cascaded storage group containing all of the child storage groups for each application. SnapVX snapshots are consistent by design so no need to specify any additional flags to obtain a point in time image of a live system.
To create a snapshot simply grab the storagegroup name which contains all the devices for your application and execute the establish command, the example below will create a snapshot hourly snapshot and will automatically terminate the same snapshot 24 hours after it was created:
symsnapvx –sg test–snapshotname hourlysnapshot establish –ttl –delta 1 –nop
You could run the command above in a cron job or batch file every hour and snapvx will create a new generation each time (gen 0).

Listing SnapVX Snapshots And Capacity Consumed

In order to see which storage groups are consuming the most space we can run the following cmd:
symcfg list –srp –demand –type sg
The output lists the storage groups showing their subscribed capacity (how much potential space they can consume) as well as their actual allocated capacity. A Particulary useful output here is the SnapShot Allocated (GB) Column, if you are in a bind for space you can quickly identify which storage group has consumed the most snapshot space and terminate some snapshots to return space to the SRP.

Note your storage group will only show up in this command output if it is FAST managed. Although everything in VMAX3 is under fast control it is possible to create storage groups that are not FAST managed for various use cases. A storage group is FAST managed if you explicitly specify the SRP and or assign an SLO. Shown below SourceSG1 has a large capacity of snapshot allocated storage.
VMAX3-CLI10

To find out more about your snaps you can run the following cmd:
symsnapvx –sid –sg groupname list –detail

VMAX3-CLI11
If I want to link off and access a snap I can use a storage group which I have pre-created with the same number of devices as the source/target devices can be same size or larger..
VMAX3-CLI12

For deeper dive and more on the internals please see the Technote on EMC.com
https://www.emc.com/collateral/technical-documentation/h13697-emc-vmax3-local-replication.pdf

Useful Commands For Everyday Use!:

This information is at your finger tips with symcli -v

SYMCLI BASE Commands:

symapierr- Used to translate SYMAPI error code numbers into SYMAPI error messages.
symaudit – List records from a Symmetrix audit log file.
symbcv – Perform BCV support operations on Symmetrix BCV devices.
symcfg – Discover or display Symmetrix configuration information. Refresh the host’s Symmetrix database file or remove Symmetrix info from the file. Can also be used to view or release a ‘hanging’ Symmetrix exclusive lock.
symchg – Monitor changes to Symmetrix devices or to logical objects stored on Symmetrix devices.
symcli – Provides the version number and a brief description of the commands included in the Symmetrix Command Line
symdev – Perform operations on a device given the device’s Symmetrix name. Can also be used to view Symmetrix device locks.
symdg- Perform operations on a device group (dg).
symdisk – Display information about the disks within a Symmetrix.
symdrv – List DRV devices on a Symmetrix.
symevent – Monitor or inspect the history of events within a Symmetrix.
symhost – Display host configuration information and performance statistics.
syminq – Issues a SCSI Inquiry command on one or all devices. Interface.
symipsec – Administers IPSec encryption on Gigabit Ethernet connections.
symlabel – Perform label support operations on a Symmetrix device.
symlmf – Registers SYMAPI license keys.
sympd- Perform operations on a device given the device’s physical name.
symsg- Perform operations on a storage device group (sg).
symstat – Display statistics information about a Symmetrix, a Director, a device group, or a device.
symreturn- Used for supplying return codes in pre-action and post-action script files.

SYMCLI CONTROL Commands:

symaccess- Administer Symmetrix Access Logix. (Mapping and Masking of devices)
symacl – Administer Symmetrix access control information.
symauth – Administer Symmetrix user authorization information.
symcg- Perform operations on an composite group (cg).
symchksum- Administer checksum checks when an Oracle database writes data files on Symmetrix devices.
symclone – Perform Clone control operations on a device group or on a device within the device group.
symconfigure – Perform modifications on the Symmetrix configuration.
symconnect – Setup or Modify Symmetrix Connection Security functionality.
symfast – Administer Symmetrix FAST (Fully Automated Storage Tiering) policies, associations, and the FAST Controller.
symmask – Setup or Modify Symmetrix Device Masking functionality.(Older Symmetrix Pre 5977)
symmaskdb- Backup, Restore, Initialize or Show the contents of the device masking database. (Older Symmetrix Pre 5977)
symmigrate – Migrates the physical disk space associated with a Symmetrix device to a different data protection scheme, or to disks with different performance characteristics. (VMAX 10K/20K/40K)
symmir – Perform BCV control operations on a device group or on a device within the device group.
symoptmz – Perform Symmetrix Optimizer control operations.
symqos – Perform Quality of Service operations on Symmetrix Devices.
symrcopy – Perform Symmetrix Rcopy control operations on devices in a device file.
symrdf – Perform RDF control operations on a device group or on a device within the device group.
symrecover – Perform automated SRDF session recovery operations.
symreplicate – Perform automated, consistent replication of data given a pre-configured RDF/Timefinder setup.
symsan – List ports and LUNs visible on the SAN
symsnap – Perform Symmetrix Snap control operations on a device group or on devices in a device file.
symsnapvx- Perform Symmetrix Snapvx control operations.
symstar – Perform SRDF STAR management operations.
symtier – Create and manage storage tiers within a Symmetrix.
symtw- Manage time windows for the Optimizer, FAST and FAST VP controller within a Symmetrix. (VMAX 10K/20K/40K)

SYMCLI SRM(Mapping) Commands symhostfs- Display information about a host File, Directory, or host File System.
symioctl – Send IO control commands to a specified application.
symlv- Display information about a volume in Logical Volume Group (vg).
sympart – Display partition information about a host device.
symrdb – Display information about a third-party Relational Database.
symrslv – Display detailed Logical to Physical mapping information about a logical object stored on Symmetrix devices.
symvg- Display information about a Logical Volume Group (vg).

EMC VMAX – SMI-S Configuration & Discovery

The following are some configuration notes for configuring the ‘VMAX Management HOST’ for communication via SMI-S for purposes such as ‘ViPR’ or with ‘VCE Vision’. Before proceeding ensure you have presented and configured the ‘VMAX Management HOST’ with gatekeeper volumes from the VMAX to allow for in-band communication over Fibre Channel. EMC SMI-S provider is included as part of ‘Solutions Enabler with SMIS’ install package which can be downloaded from ‘support.emc.com’.

Begin by installing SMI-S Provider, ensuring you select the ‘Array provider’ (Windows does not require Host provider) and chose the option for SMISPROVIDER_COMPONENT:
VisionVMAX1

From the windows services.msc console check that both the ‘ECOM’ and ‘storsrvd’ services are set to automatic and in a running state:
VisionVMAX2
Check that EMC storsrvd daemon is installed and running from a Windows cmd prompt using stordaemon.exe:
stordaemon install storsrvd -autostart
stordaemon start storsrvd
stordaemon.exe list

VisionVMAX3
Or using the SC (service control) command you can query/start/config the ECOM and storsrvd services:
sc query ECOM.exe
sc query storsrvd
sc start ECOM.exe
sc start storsrvd
sc config ECOM.exe start=auto
sc config storsrvd start=auto

VisionVMAX4

Run netstat -a and check the host is listening on ports 5988 5989:
VisionVMAX5

UPDATE ENVIRONMENT VARIABLES:
Add the SYMCLI installation directory path (DRIVE:\Program Files\EMC\ECIM\ECOM\BIN) to the list of system paths:
VisionVMAX2a
Or use the windows CLI to add the SYMCLI and ECOM directories to the PATH environment variable:

setx /M PATH "%PATH%;C:\Program Files\EMC\SYMCLI\bin;C:\Program Files\EMC\ECIM\ECOM\bin"

Perform a Symcfg Discover followed by Symcfg List to ensure communication is present between the VMAX and the VMAX management server.

If experiencing issues such as the ECOM service failing to start it is worth rebooting the mgmt server at this stage.

ECOM SERVER: Adding A NEW SMI-S Provider User 
Provided all the validations are successful then proceed to login to the ECOM server and create the user you would like to use for connectivity:
Open https://localhost:5989/ecomconfig
Login with default credentials of: admin / #1Password
VisionVMAX6a

Select the option to add a new user and create the Vision user with administrator role and scope local:
Visionvmax7ab
VisionVMAX8ab

Windows Firewall
If the Windows firewall is enabled then rules will need to be created to allow ECOM ports TCP 5988&5989 and SLP port UDP 427. For example using the windows command line netsh to create rules for SLP and ECOM:

netsh advfirewall firewall add rule name="SLP" dir=in protocol=UDP localport=427 action=allow

netsh advfirewall firewall add rule name=”ECOM” dir=in protocol=TCP localport=5988-5989 action=allow

netsh advfirewall firewall show rule name=SLP
netsh advfirewall firewall show rule name=ECOM

VisionVMAX11

Confirm VMAX Connectivity VIA SMI-S (TestSMIProvider)
Open a Windows cmd prompt session as admin user, if the environment variable has not been set then you will need to ‘cd’ to c:\Program Files\EMC\ECIM\ECOM\BIN. Type TestSMIProvider.exe at the prompt, from here chose all defaults except for the Vision user and password created through the ECOM console:

VisionVMAX9

At the prompt type ‘dv’ (display version info) command to confirm connectivity between the VMAX and SMI-S:
VisionVMAX10

For any troubleshooting please refer to: ‘C:\Program Files\EMC\ECIM\ECOM\log’

Note: When configuring Vision please ensure to use the ‘VMAX Management HOST’ IP address for all VMAX entries in the Vblock.xml configuration file.

EMC VMAX³ – Front-End FC WWPN’s & Zoning Considerations

Please begin by downloading the attached PDF here.

As you can see this is a fully compiled listing of all 256 possible FC WWPN’s available on a VMAX³ system. A 100K system can cater for a maximum of 64 FC Front-end connections, a 200K has a possible 128 ports, while the flagship 400K can have up to 256 FC Front-end connections.

VMAX3-WWN0

VMAX3-WWN-Decode1-a

Note: for demonstration purposes I am using X’s and a ? to explain the unique identifiers of a VMAX³ system. Please refer to the .pdf listing to help understand the concept.

X:XX:XX:X = System-wide Unique ID – as you will see from the provided WWPN listing this value is the unique identifier per VMAX³ system (a follow-on post focusing on decoding VMAX³ WWN’s shall explain this further). On a per VMAX³ system the X:XX:XX:X value will remain the same for all FC WWPN’s associated with that VMAX³ system.

There is a notable change from the previous VMAX usage of WWPN’s; there is now a unique identifier labeled as ? this uniquely identifies a pair of engines:

? = Unique ID For Engines 1&2 | 3&4 | 4&6 | 7&8

On previous VMAX generations all the X’s and ? were consistent across all FC Port WWPN’s with only the last 2 hex values of a WWPN acting as the unique port identifier, with the VMAX³ the unique port identifier is now the last three hex values. With VMAX³ the key point to note is the ? value remains the same throughout directors 1-4 then increments by one hex value for the next four directors, for example if C:04 is the unique ID for Director1 Port4 then for Director5 Port4 the C changes to D and remains at this value for directors 5-8 etc; so given this information and referring to the list provided:

Director1 Port4 has a value of 50:00:09:75:58:01:EC:04
Director5 Port4 has a value of 50:00:09:75:58:01:ED:04
Director9 Port4 has a value of 50:00:09:75:58:01:EE:04
Director13 Port4 has a value of 50:00:09:75:58:01:EF:04

There are two choices of FC Font-end I/O Modules to choose from:
• 8 Gbps four-port FC-module (Glacier) Non-Bifurcated, operational at speeds of 2/4/8Gbps, Populated left to right (Slots 2,3,8,9).
• 16 Gbps four-port FC-module(Rainfall) Bifurcated(meaning that the 8 lanes of PCIe are split into 2 connections of 4 lanes each), operational at speeds of 4/8/16Gbps, Populated right to left (Slots 9,8,3,2).
VMAX3 uses PCIe 3.0 thus allowing for maximum available port speeds.

Dual Fabric:One approach for cabling is to connect Even director ports to Fabric-A & Odd director ports connect to Fabric-B. Engine-1 Example:
VMAX3-WWN3
When using this approach in a single engine system the I/O ports from each director evenly span both SAN fabrics.

HOST or Cluster FA Port Usage: in order to ensure a balanced approach is maintained, connect a Host or cluster to 2xDirectors in a Single Engine system or 4xDirectors in a VMAX with greater than 1xEngine.
Single Engine example:Zoning a Host evenly across 2 directors and across both fabrics using ports 1D:4, 1D:31, 2D:28 & 2D:7:

VMAX3-WWN4

Two Engine example: Zoning a Host or Cluster evenly across 4 directors and across both fabrics using ports 1D:4, 2D:31, 3D:28 & 4D:7, this will spread load for performance and ensure fabric redundancy :

VMAX3-WWN5

These examples are a guideline for evenly balancing port utilization across all available director ports. See below for additional reading.

VMAX³ ACLX GK: The first physical FA port on the array will have the show ACLX flag set; thus any host attached to that port will be shown the ACLX device as LUN 000.

Hopefully these considerations and lists may assist you with planning (or automating) your zoning scripts for VMAX³ systems.

SYMCLI List all FA WWN’s: symcfg -sid xxx list -fa all -port -detail

Useful References:
VMAX3 Family New Features – A Detailed Review of Open Systems White Paper
http://www.emc.com/collateral/technical-documentation/h13578-vmax3-family-new-features-wp.pdf

VMAX3 Reliability, Availability, and Serviceability Tech Notes
http://www.emc.com/collateral/technical-documentation/h13807-emc-vmax3-reliability-availability-and-serviceability-tech-note.pdf