2/12/2016

HP xl170r, NVMe WS2012r2 on M.2 SSD Sata

The xl170r Proliant Server Blades that fit into a (2U) sized hosting chassis have an NVMe or PCIe RAID-1 "chip drive" its not listed in the BIOS on first glance. Installing Windows 2012r2 to it can be difficult. Here is how to do that.


First its not a typically sized mSATA drive (its 1/2 sized), second its hosted not at the end of a Controller, but directly on the PCI Express (PCIe) bus. That's great for performance but normally means you need a special device driver to be loaded and enumerated in order to access it.

HP has two types of SCSI/SATA host controllers, hardware backed (based) and software backed (based). To HP (all) SCSI/SATA host controllers are [potentially] RAID host controllers by default.

The hardware versions are what most people are familar with called "SmartArray" and generally come with Operating System specific device drivers and add-on drivers to work with installation software, like for the WinPE installer from Microsoft.. or as a kernel module for Linux.

For some time HP (now HPE) has been moving towards a converged UEFI hardware platform that integrates downloading and providing device drivers to the hosted operating system on their server blades [at the time of install] or as needed by manifesting additional mass storage devices on the PC extended buses (mscdex, usb, pci, pcie, iscsi or pxe) as needed.

The NVMe support in the new HP servers definitely takes a strange turn for most people familar with laptops and other servers.

Basically like the old CCISS approach of creating a vendor specific "bus" to support their "vision" of the hardware.

HP has used their [Software (backed)] B170i RAID controller to {host} the NVMe but normally all RAID controllers under HP require the user to configure them at UEFI> BIOS> Host Controller BIOS> setup [before] they are visible or enumerated in the boot loader order. When it is properly setup as a RAID-1 software RAID device "backed by NVMe hardware" it will appear as an additional boot option, but won't even be listed as an option until the LUN is created in UEFI BIOS (note: Not Legacy BIOS!)

Unlike other vendors implementation of the NVMe specification, they are treated as a SCSI device by the HP hardware (Not PCIe devices~!). This both complicates and simplifies things.

It complicates things because the NVMe descriptions for "native" PCIe bus support do not apply to the NVMe on HP servers.

It simplifies them because the operating system can treat them as another SATA/SCSI device.. and the subsequent operating system only needs the device driver for the generic Software based RAID specific to HP products.

While this still requires you to [Inject] [F6] the special HP provided device driver when in the WinPE menus, it does eventually discover the new B170i controller managed "M.2 SSD devices" as if they were merely another SCSI LUN hanging off the Software RAID controller made available by the HP device drivers.

You can use the "normal" HP plethora of extended media options to get the HP boot time device driver for WinPE at the time of install.. namely (iLO) mscdex, usb, pci, pcie, sd media.. or their physical options as provided by your servers physical form factor.. aka.. external "ports" on the surface or backside of the machine.. or a USB/KVM dongle.. custom wim images with the device driver pre-injected into the WDS or SCCM services are also possible.. to each their own.

Windows WinPE will then copy the runtime device driver into the eventual native store after install and setup the device as a bootable device automatically.

The "behavior" is you begin to Install Windows 2012r2 with WinPE bootable media and an install source, it proceeds normally [Up to] where it lists the "available storage devices" and asks where you would like to install Windows 2012r2.

If you have already created a RAID device in the hardware based SmartArray controller, that will be listed. If you have not created a RAID device in the software based B170i controller, then the M.2 SSD RAID device will not be listed -- and you will have no hint that there is a "missing" storage device of 64 GB.

Even if you use the F6 procedure to inject the B170i device driver into the WinPE install, unless the B170i has "already" had a RAID device setup before you began the Windows 2012r2 install procedure.. it will still not be listed as an available 64 GB RAID device made from the M.2 SSD hardware.

The only way to (properly) expose the M.2 SSD hardware is "first" to boot into the UEFI BIOS intelligent provisioning provider menus. They start a Linux derived GUI and present a representation of the available RAID controllers and the hardware hanging off of them.

In this "diagram" will be the (two) 64 GB M.2 SSD devices, which can be selected and combined into a RAID-1 device which will automatically create a SCSI LUN which the subsequent Windows installer "can see" once the F6 procedure is used to "inject" the B170i device driver into the runtime image.

With the LUN pre-setup, and the F6 procedure followed, WinPE will take a few "minutes" to rescan the available RAID storage controllers, and it will "see" the B170i controller, and that it has a RAID-1 LUN and present that as "available" storage media and a valid target for completing the install of Windows 2012r2 server. (Don't expect "Spectacular" drag racing style install speeds.. rather most of the pedantic Windows install process is about thoroughly auditing hardware busses and enabling hardware then testing it.. and contructing a registry from scratch.. all of this is memory intensive.. not disk io bound).

NOTE: Until you manually go into IPP storage provisioning from the BIOS UEFI screen and manually select and [Create] a RAID-1 array the M.2 SSD devices will not be listed or available to boot [from]. In fact they are never made [directly] available to the operating system as a PCIe device, they are treated as (one) B170i RAID device.

If you wanted to use the PCIe SSD as a cache store in a Mac Fusion"y" kind of "hybrid drive" manner to boost the access speed of a partnered hard drive.. that is not what the default is intended. I believe a special License key must be purchased to enable the HP io store accelerator.. or something named like that to provide an enhanced RAID cache to increase disk I/O.. different product for a different purpose.. and I believe its only available with the hardware (based) SmartArray products.

Rather this solution is meant to host the bootable Operating System (only) and in a mirrored RAID fashion, such that one half the M.2 Sata RAID can "fail" and notify you before the situation becomes "critical". A maintenance window can be scheduled, replacement parts can be shipped ect..

As delivered M.2 SSD devices can be many sizes, but at 64 GB the base install for a recent server I helped setup, this was more than enough after the Windows 2012r2 installation rounded out at about 20 GB leaving plenty of room for subsequent updates and feature installs.

Another good reason for this beyond speed, redundancy, and reliability, is these drives are relatively small and unobtrusive close to the motherboard with no moving parts.. so their fault tolerance should be superior to that of the data drives.

The Moonshot/Apollo platforms are a medium sized converged platform which hosts a maximum of (4) servers in a "mini-chassis" with a quorum of drive slots allocated per server on the front of the chassis. LFF only allocates 3 slots per blade server, and if two slots host another RAID-1 configuration and the third is used for hot spare failover.. its best to use these only for bulk storage and data provisioning.

Since the Operating System has effectively now been taken "inside" the blade server it modularizes.. and opens up the possibility of swapping blade servers [whole cloth] when troubleshooting.. and leaving the data drives where they are.. to be picked up after replacing the server and os all at once.

Someday such an arrangement may even lead to "hotswap" of the blade server and os, but I am inclined to think even a partial scenario like that is possible today with Clustered CSV and Clustered Nodes on the current failover system built into windows.. with VMs and Docker Containers it becomes even more likely. Swapping rapidly turns into a "Time Share" problem where jobs are merely "paused" rather than torn down and rebuilt after each reboot.