Zeo Mobile, hailing frequencies open

Real progress. I compiled a custom win32 app which mimics the 'Zeo Android' service and received a connection from the device over bluetooth direct to the windows 7 pc.

That binary string of unprintable characters in the output is what the actual Zeo headband sent to the windows pc before the simple program declared victory, attempted to print the output to the screen and ended the program.

The bluetooth "comm channel" is now open.. its just a matter of interpreting what the headband is sending, and in theory vice versa.

The same "emulation" can be made to work both-ways.

A pc program can now pretend to be a Zeo Mobile and send status reports to the Zeo App on Android to help determine what constitutes a diagnostic packet.

HMSG = prefix of a "messaging" protocol, something like 'headband msg' ?

The program was written towards windows xp and only using  microsoft bluetooth api's so it should work on just about anything windows.

rats.. I thought it would be harder

Zeo Mobile, intermezzo

So why am I hacking the Zeo, or [hacking / learning how it works] ?

I think there are two ways of acquiring new skills and learning things.

The first is practice and failing, and demonstration. Which sort of supports the idea that you learn something best when you try to explain it to other people.

The second is curiosity, or scratching an itch.

Where you might learn something specific with the first method, the second can lead you in wild and woolly ways around bends and twists and turns you never expected.

I've a very broad skill set but its hard to get motivated to practice in depth, or push further than the superficial without a project or slight goal in mind. Ultimately our portfolio of work is the sum of our projects and what we can present from them.. see what I did there? 1st leads to 2nd and back to 1st...

I knew the Zeo had an Android app, and an iOS app (would that was still available.. gee) so I would be getting deep into Android app databases, possibly programming and scripting. And I had a slight notion the headband would be using WiFi or Bluetooth. WiFi is mostly easy these days but power hungry.. Bluetooth sips power so it was more likely. Maybe even BLE or Bluetooth 4.0 but due to the age of the device.. it really wasn't likely.

Bluetooth is a fantastic technology made really hard by very poor user docs. The politics of the Special Interest Groups (SIGs) read like legal documents and are often the basis of lawsuits.. lawyers in suits.

Worse the technical documents are often demonstrations of the prowsess of the researchers producing them and very hard to understand. And some people really don't realize they make egregious mistakes in the documents.. which are easy to over look because they are so opaque and dense.. sometimes arbitrary.. which the Bluetooth docs very much are arbitrary in some ways.

The best way then to learn is to "poke around" and see an implementation.

At first I thought to simply understand and compile the OpenZeo > ZeoCSV demo app on Github, but quickly found out it was incomplete, and very old.. and Google had recently ripped the wholey stuffing out of the development community by pushing everyone away from Eclipse with ADT and towards Android Studio.. and removing documents that had rested on the developer website for years.. people are really crying out there that Google tore a hole in the fabric of the web and space time.. it makes compiling or understanding old apps very hard... Archive.org is crippled with respect to this and isn't really of much use.

To make matters far worse.. Google tossed a document called Migration.html out on the web and didn't make much of it.. except if you try to use the old ADT with Eclipse beyond SDK version 22 (Android 5.0 was version 23) things break all over the place.

Marching on, I did acheive a ZeoCSV version that worked using another persons Github as the base for an app that did compile and sideload. Then found an older ZeoCSV by Nicholas Barry did it even better and he had uploaded that to the Google Playstore.. great I thought.. until I realized people would have problems distriguishing them from one another and the Amazon Fire tablet while running Android doesn't have access to the Playstore unless you first root it and then sideload the Google Playstore app.. gee.

So now there is a growing confusion over the ZeoCSV apps, sideload or playstore version, and who made theirs, and when, and which version supports which features. Clif-notes are Nicholas Barry's Google Playstore version with the Purple background is the best one for now, it shares [all] of the sleep records with your choice of method through secondary apps on your device to your destination, be that your email box, google drive, one drive, amazon drive, own cloud, dropbox, boxnet, or the local file system of the device itself. The key is [Look for the Purple background] once you install the app. If its purple, you have the correct app.

Then there is the ZeoViewer application for Windows desktop. This is also an alienrat production that is freely downloadable. It installs and leaves a shortcut on your desktop. I tested it on Windows 7, but did not test it on Windows 8.x or Windows 10.

Nicholas released an older ZeoCSV (only side load available now) that exported files that ZeoViewer cannot read, he fixed that this year, very recently and the new ZeoCSV app [Purple background edition] which exports 'complete' sleep record csv files which ZeoViewer [can open].. actually it [Imports] them into a local database on the Windows 7 pc, once imported they stay there and you can refer to them again and again after opening and closing the ZeoViewer program. This makes the ZeoViewer a central [hub] for your sleep records. The ZeoViewer can also re-Export the records, or a slice of records into a CSV file which you can open in any spreadsheet program like Excel if you like later.

While all of that was going on.. I've been going down the path of learning [better] and fixing my Windows 7 desktop [Bluetooth] stack.

Much of my problems were understanding the diferences between XP and Windows 7 bluetooth stack implementations. The [User Interface] Microsoft created for it in XP was [ripped apart] in Windows 7 as the company transitioned from a [GUI] user interface into a [WUI] user interface.

GUI was a XeroxParc invention more or less, that Apple adopted and Microsoft adopted in Windows 95, and ever since its been very popular the world over.

Iconography taught by "showing" or brokered on "intuition" as the common human language to enable people to learn to use programs.. that eroded with the [data management hysteria] surrounding Word with the ribbon user interface. A few people became panicked over deprecating older iconography and were very [verbally] oriented and felt they had to catalog every function on a ribbon bar.. that lead to [english words as a search navigation] paradigm.. and not everyone is fluent in "words" that a program or industry uses for its jargon..  (let alone national cultures).

So effectively Windows 7 [-hid-] much of the [GUI] user interface.. this got really [warped] with the Windows 8 user interface.. when they kept word navigation.. but hid the [search] tools.. and did not communicate even their own confusion.

Basically what happened was in a graphical Windows 8 ... [GUI] was dead. [WUI] was the backup interface.. for the new [TUI] touch interface.. which no one understood.. the iconography was terrible.. (massive understatement) and gyrating and animated (it was multi-dynamic icon) so the whole thing shifted like a blurry mess. Turning things off, disabling animated tiles started a word fight between Microsoft and Users that ended with the apocalyptic and childish press release [ "We're not stubborn" ]

- the key with WinXP is [use your intuition 'Luke', icon navigation]
- the key with Win7 is [use your search tool 'Leia', word navigation]
- the key with Win8x is [use your touch page 'Data', page navigation]

Back to Bluetooth - the XP interface was simply a Control Panel applet and a [system tray Applet] for accessing [the Radio], [the Device Installer] and [the Settings menu]. It could not have been simpler. The Windows 7 interface [-hid-] the Control Panel applet and dynamically made the system tray Applet.. "appear" and "disappear" at seemingly random intervals.. never mind the plugin play device installer did the same [Hide and Seek] games when actually searching for device drivers. Gosh.

I discovered there are actually registry entries for restoring the XP behavior in Windows 7 and things started to make sense again. This information came as a benefit of windows server admins attempting to copy or [port] the Bluetooth services and tools from desktop to server.. because many preferred to run server on the desktop for higher network performance and connection space.

Once I could [-use-] the user interface for Bluetooth again.. gosh darn it microsoft.. gosh darn it.. it was immediately apparent the Zeo was using SDP - service discovery protocol, to advertise it has two serial port services for SPP and iAP device type drivers for anyone to use. It was also apparent any Android device running the MyZeo app was advertising via SDP - a Zeo Android service.

.. to be continued

Zeo Mobile, interview with a vampire

Switching over to look at the [ zeo android ] service on the Mobile app using linux, sdptool retrieves:

I tried connecting to the [ zeo android ] service with rfcomm from linux.

Each time the Zeo Android app on the mobile device indicated  a state change in the Headband and then a loss of connection as something closed the connection. Presumably the Zeo Android app.

This leads me to think that the [ zeo android ] service is actually a server waiting to hear from the Headband with concise reports and state changes.

Reversing this state of affairs, it should be "possible" to setup a similar service on a PC to listen for reports from the Zeo Headband, and thus receive whatever it has to communicate.

[One Step Beyond ]

Tried connecting a 'Serial Over Bluetooth Link' driver to the Zeo Android service on the mobile app, which produced a COM6 port on the PC. Then used PuTTY to open the port, and this is what appeared:

Further the Zeo App indicated a Headband was now connected:

The Zeo Android app [ Diagnostics page ] indicated it was the Windows PC connected, but apparently part of the connection proctocol from Zeo sends its status, which the Zeo Android app decoded as not very useful.

Closing the PuTTY connection to the Zeo Android app Serial port on COM6 immediately changed the Headband status:

So it appears the [ Zeo Android ] service is the 'Headband' service, and it appears we can 'fake' a Headband module connecting over Bluetooth by using a Windows, Linux, or OSX connection.

A little more investigation.

Bluetooth development on many platforms is at various stages of ease.

Linux has tools for starting a listener on a particular RFCOMM channel and connecting that to a device in its /dev inventory. The BlueZ stack which was in then out, then in again, in the Linux Kernel has commandline tools for changing the SDP service record for a service.. but lack an important feature. UUID16 for a service can be changed with the Linux tools, but not a UUID128.. which is what the Zeo App on Android is using.

The reason is because the Bluetooth SIG organization recommends using UUID16 for commonly defined bluetooth services that everyone provides with their default bluetooth development "stacks" but also recommends creating a unique UUID128 "waay out there" so as not to accidentally collide with someone elses service. So if your developing on your own and inventing the latest Bluetooth service application.. they would appreciate it if you do it in "another solar system" please.

The Linux command line tool sdptool will let you define UUID16 even UUID32 or a string, but not UUID128. The reasoning went something like "well that's kind of fringe, and don't bother me.. I'm busy writing code.. " -- really it wasn't as nice as that.. but .. yeah.

Zeo Mobile, bluetooth let mein

Using bluetooth on Windows 7 you can discover and connect to the Zeo Mobile headband without Android.

It was originally designed to partner with a bedside alarm clock. It then evolved to use iOS and then Android.

When it is paired with a desktop it exposes two bluetooth profiles, one for iOS and one for other devices.

The iAP bluetooth profile is Apples proprietary version of the SPP profile for the exclusive use of its manufacturing partners.

The SPP bluetooth profile is a specification for emulating a Serial Port connection over bluetooth using the RFCOMM protocol.

To expose these to applications on a desktop local device drivers must be assigned to each connection.

These are connections initated by the desktop to the device over bluetooth.

While the connections have been established and drivers brought online. I have not had success in opening communications with the device. I am deficient in some crucial bit of information.

Within the same time frame of setting up the outbound connections to the Zeo, an unsolicited [inbound] connection attached to the desktop and spawned a new comm port.

The new comm port however also did not surface information. It could be a timing issue however since I had nothing watching the any of the comm ports.

When I did get around to attempting to connect to them with PuTTY I got error messages like this one:

[Huh - This: Note that RFCOMM/SPP only allows one connection from a remote device to each service. So do not use BluetoothClient and a virtual Serial port to connect to the same service at the same time; the second one will fail to connect. - orthogonal observation, I have several devices paired with same zeo.. any one of which might be holding the port open.. and denying access. - also noticed that in hcitool, (iAP) is on RFCOMM channel 1 (SPP) is on RFCOMM channel 2 ]

It also appears installing the Zeo app causes android devices to publish a bluetooth service called [ Zeo Android ]

There is little known about this [ Zeo Android ] service, except that it would make a good place for a serial port debugger if app developers wanted to create such a thing (pure speculation). It could also be a PIM or OBEX service.. I have no idea. But it is an interesting mystery.

OSX on Mac looks the same:

Windows - automatically provisoning a Virtual COM port when Adding a Bluetooth Device

MSDN [ Installing a Bluetooth Device ]- seems to be saying the bthci service will drive Pnp to install a device driver based on the contents of an inf file. The inf file used will be determined by a matching identifier pulled from the SDP discovery packet, the simplest being based on the discovered 

BTHENUM\{ServiceGUID}

Each service port has a unique guid, therefore a unique inf file should be created for each and installed on the operating system. Many Bluetooth devices seem to skirt this issue by advertising "commonly recognized" ServiceGUIDs like 

GenericSerial, BTHENUM\{00001101-0000-1000-8000-00805f9b34fb}

Since Zeo advertises unique ServiceGUIDs its ports will need special inf files. They should look like the default bthspp.inf file contents with a change to the ServiceGUID identifying the SPP port on the Zeo.

Zeo SDP also does not advertise much more about the device than its name.

Windows also does not have an iAP device driver - the closest in the default windows driver base is for Apple: Apple Mighty Mouse

The purpose of this would be to drive the plug-n-play system such that anytime a Zeo is detected its spp serial port would automatically be assigned a virtual com port and the proper device drivers to support that virtual com port would be loaded. The assumption is of course that while the Zeo mac address will change "slightly" (the last few digits), the SPP ServiceGUID would be a static "assigned" sequence by the manufacturer.

It is also possible that it is not a Serial type of communications, the only thing that makes this look likely is the SDP advertisement. It could also be some other type of protocol. But its unlikely.

I rather think its a protocol that does use RFCOMM but intermitently. That is while on the PC side a virtual serial port must persist, or a program that uses RFCOMM needs to listen, the actual connection and flow of information would be on the zeo terms.. it would waste too much battery power on the zeo to keep the connection up all of the time. So it probably makes a connection, delivers its information, and disconnects.

In the other direction, the pairing operation probably assumes the zeo is in pairing mode, exchanges the passkey '0000' and then closes its connection. The zeo app on the mobile does not appear to proactively reach out and communicate to acquire headband activity, but it does seem to update status.. perhaps the headband reports "status" when a state change occurs. Zeo might be 'broadcasting' its state changes.

A key change that seems 'triggerable' other than 'pairing' is attaching the charger. The app status changes to 'charging' and this might be a good place to be monitoring when the zeo changes state.

On a different laptop with a WIDCOMM chipset.. and that bluetooth stack, I noticed whenever plugging or unplugging the icon for the zeo would change from red to green briefly. While WIDCOMM is old an venerable the tools to work with it are few. But appeared to indicate a state change had occurred.. such as at least the device had established a connection, then disconnected.

Zeo Mobile, hack from the future

Feb 13, 2016 one of the original forks of the ZeoCSV demo app has come back to life.

Quantitative Self - Zeo Sleep Manager Data Export

It has now been updated and made available in the Google Play store - ZeoCSV

I do not intend (at this time) to publish a Zeo app to the Google Play store, this app by Nicholas would be the best for now. It works on my Android 6.0 device without disabling Draw over apps.

Its key advantage is the CSV format has been updated to work with the "ZeoViewer"

Here is an example:

An interesting side effect of this development is once the data is Imported into the ZeoViewer database, it can then be exported into one of three formats CSV, Plain text or Zeo DAT file.

The export and import from Android into the Desktop database transfers all records, and persists them in the desktop database.

ps. interesting tidbit, the MyZeo apk on Android isn't encrypted or obfuscated, it seems if you wanted to by-pass the application and talk directly to the headband that is something that could be done.

Zeo Mobile, first data share apk

This android app can be side loaded on Android 4.2 through 6.0

ZeoCSV.apk

disable any apps granted the [Draw on other apps] privilege while installing.

go > [Settings] > [Apps] > [Gear] > [Draw over other apps]

1. Look for any apps set to [-- yes --], stop or otherwise disable them.
2. The Zeo Sleep Manager app should be installed before the ZeoCSV app.
3. The Zeo Sleep Manager app should have at least one sleep record in its database.

 Tested share options include, email, file system, gdrive and others.

References:

Zeo CSV 2012-08-06 - [Alvaro F Boirac] author of the ZeoCSV java.main routine in this apk
Open Zeo 2011-11-28 - [ Zeo Inc.] original source of the original ZeoCSV demo apk

Zeo Mobile, The hackening

The first nights sleep was rather easy to collect. While the data produces nice charts. The next step was to see how to download the data from the android phone. Here is how to do that.

 

The Android application connects to the headband over Bluetooth. The raw data is post processed via an FFT into a "profile" of 30 second and 5 minute bins which are used to produce the charts. This data is recorded in a dataset per sleep session called a sleep record in a SQLite3 database file on the Android device in the protected path: 

[/data/data/com.myzeo.android/databases/zeo.db]

Prior to Android 6.0 "Marshmallow" it was rather easy to install and then grant limited "root" access to Apps permission to access an Apps data directory using something like the SuperSU application. However as of Android 6.0 that path is much more difficult to enable.. and carries significant security risks if achieved.

Rather the preferred method is to construct an App which "requests" access to a co-installed Apps data store and explicitly "grant" it permission to do so.

The OpenZeo sample projects demonstrated how to do this:

Essentially it is intended to use the Android feature set to access the Apps sqlite database and returns the results of a query intending to save it to the local file system and or share it through other APIs with a user.

Unfortunately the sample program has a few problems and crashes.. however the compiled APK does run on Android 6.0 up to the point at which it attempts to share data.

This is likely a symptom of the age of the App since it was originally compiled to work on probably the Android 4.0 KitKat platform.. a lot has changed since then.

During the Android 4.0 "era" the ADT Eclipse plugin was loaded into an install of Eclipse and the Android SDK provided libraries a build environment and AVD (android virtual device) to complete a development environment.

During the Android 6.0 "era" this has been replaced with Android Studio.

In both cases, the best option is to install "git.exe" such that the original OpenZeo GitHub sample project can be cloned directly and brought in as a Project and compiled. In some cases "Warnings" must be disabled in order to allow the project to compile. It can then either be run on the AVD virtual device, or side loaded into an actual Android Device.. if the device is Android 6.0 it will request access to the MyZeo Apps datastore upon installing which must be granted. It should be noted that (1) it assumes the MyZeo App will be installed and (2) that the MyZeo db has at least one sleep record to access.

Another "successful" method is to use a different Android device which has Android 4.4 Kitkat installed and for which "#SuperSU" (in the GooglePlay store) can be used to allow a file browser app like "ES File Explorer" (in the GooglePlay store) to travel to the:

[/data/data/com.myzeo.android/databases/zeo.db] 

directory and copy it to a  directory where it is more easily accessed or can be uploaded or synced, ES File Exporer has a 'built-in' Remote Manager > it is an FTP server -> Settings > Network >V - Remote Manager (works well with WinSCP if you 'remember to select FTP'). You can then browse the Android device from a desktop or any wireless device on your network to retreive the file... scary huh?

Then using a Windows program like [sqlitebrowser-3.8.0-win64v2.exe] the data can be browsed and exported.. or other sqlite tools can be used to work with the data.

A final method would also be to install the Android ADB and use the Android Backup command to take a backup of the apps and their databases and store them offline. Then use a program like BackupExtractor (on GitHub) to retrieve the App database file from the backup.

I believe all things being equal its probably best to modify the ZeoCSV example app to export records to a file on a regular basis and email, replicate or copy those to a central repository as individual sleep records.

The OpenZeo project also documents the format of the SleepRecord such that all of the fields available from the sqlite database are defined and easily interpreted.

Sleep Record Field Definitions

 

The sad thing so far, is that, this is an [intermediary] method of accessing the Zeo data, post processed. The actual EEG data was turned into this format before it was stored in the sqlite3 database. While that is probably the most useful for most people. The pre-processing means the device is still a highly sought out commodity and a single point of failure. We can only hope that the company that bought the intellectual property returns to the market at a similar price point, or that other IoT devices are brought to bare on the problem.

The Zeo bedside model had been "opened" such that raw EEG signal data could also be obtained.

Since the headband unit both transmits in "buckets" during the night and "deluge" mode in the morning to round out a sleep record. It is probably where the FTT analysis is conducted. IoT devices are becoming more powerful all of the time and its often only a sketchup away from providing FFT services.

Observing the bluetooth signals would be an interesting study.

In the [other] direction.. where to put all of this data.

A couple of Cloud services come to mind, for free and cost, Ubidots, AWS and they offer enhanced algorithmic processing of the data that appears in a dashboard.. very similar to the previous Zeo dashboards.

Dashboards as a Service is more or less a commodity business these days.

It might be worth mentioning there are many formats for the data file:

1. EEG raw

2. XML post processed

3. sqlite2 post processed

4. sqlite3 post processed

5. pseudo-CSV, XML export format from various Apps (definitely post processed)

6. whole Spreadsheets of imported source data post processed

7. emails of results

The decoder/viewers from various sources appear to only work with "their variant" and its often not well documented which format they use. On the whole the XML formats appear easier to read and are often what you see in "print" on the web, this is because binary blobs are just not webshop "photogenic".. whatever the format the data is in it is usually not that difficult to figure out.. just put your Sherlock hat on and start puzzling it out.

Another thing to consider is the product has gone through two to three design "re-fits" or releases, so there is a lot of information available on the web that only applies to the earlier models. 

Essentially, the Bedside model went through a period when the data was safe guarded and encrypted, which for the time was probably a good idea. But then they seemed to re-think this and opened it up, retro-fitting the older gear with firmware upgrades to store the data un-encrypted.. hence the three firmware upgrades you see floating around. 

The Mobile version was a redesign with the same goals and similar interfaces, but as far as I can tell it "never" encrypted your sleep data, hence no need for firmware upgrade to get the un-encrypted data for study.

While the iPhone, iPad apps are gone, as far as I can tell they also used the sqlite database, although it may have been version 2 (I don't have access to the iOS apps so I can't confirm).

The Android apps just used the Google APIs which used a local sqlite file per apk stored in the apps private directory on the phones file system, which has become harder and harder to access.. and now its better to use the APIs put in place to support cross-app accessing their datastore.. rather than pilfering the db files outright.

If you attempt to compile the Android sources on the Github projects with Android Studio or Eclipse ADT beware the platform SDK the source expects. android-8 was "2.2 Froyo" if you don't have the SDK the source is expecting as available in the compiliation environment it will fail with many mysterious class undefined errors and warnings. I succeeded in recompiling the old code and running it on Android 4.4 and Android 6.0 but it wasn't totally obvious at first. Most of the code was checked in around 2013... we're talking "Stargate Ancients" here .. in cell phone terms.. that was a "looong" time ago.

I have a full day today, but this evening I'll see if I can throw together a little YouTube video of how to compile the Github apps in Eclipse ADT and Android Studio.. if someone else doesn't preceed me. Jimmy Neutron I'm not.. but I found it a little difficult to navigate Googles new Alphabet soup.. it looks like they sent out a few Gravity Waves of their own when they ripped apart ADT last year.

References:

Grc Zeo Page

Zeo – In Depth Product Review - Pictures of the Bedside model and Sensor Pads

Zeo Sleep Manager Review  - Pictures of the Mobile model, Box and Cradle

Zeo, The Adventure Begins

Zeo is a sleep monitoring device which collected Electroencephalographs (EEG) data from a a headband worn while sleeping. It wirelessly downloaded the data when the band was removed and set to charge the next morning. It is no longer being made, however a few final production devices were made available on eBay this month.

This is an exploration regarding one of those final devices.

First it comes in two models a set with an Alarm clock, or a set (as show) called a Mobility kit.

I only have the Mobility kit, which is designed to be used with an App for an iPhone or Android device.

The iPhone App has since been removed from the App Store.

The Android App is still available on the Google Play store.

The data collection device is a small arduino like device in the central plastic "brownie" package, which has three button snaps on the bottom to attach it to the headband in the silver package.

The instrument package has a yellow "Spinner" or "Rising Sun" logo on the front, which makes one appear to be Cosplaying the central character from Karate Kid.

Cliff Notes: "Sweep the Leg Johnny - No More Kings"

The instrument package also has a micro-usb port under a soft rubber pull-out cover on one edge, and a rectangular colored LED and rectangualr button along the opposing edge.

The other two components are obviously the USB to micro-USB cord and a slim 110 volt wall charger with female USB port. The first thing I did was plug the cord into the wall charger, then the wall and then the instrument pack. It immediately began flashing "green".

Bluetooth Pairing was a bit of a "challenge".

The Zeo App for Android did not automatically "pair" with the Zeo. I chose Zeo "Mobile" -

Then "Pair Your Headband"

And it searched for the headband, but consistently "failed" to find it

I finally went into the Bluetooth Settings for my Nexus 5X Android "Marshmallow" operating system and found it as an "Available device" labeled "Zeo" so I selected it there and it proceeded to pair.

Returning to the application it detected that the headband had been paired

HP xl170r, NVMe WS2012r2 on M.2 SSD Sata

The xl170r Proliant Server Blades that fit into a (2U) sized hosting chassis have an NVMe or PCIe RAID-1 "chip drive" its not listed in the BIOS on first glance. Installing Windows 2012r2 to it can be difficult. Here is how to do that.

First its not a typically sized mSATA drive (its 1/2 sized), second its hosted not at the end of a Controller, but directly on the PCI Express (PCIe) bus. That's great for performance but normally means you need a special device driver to be loaded and enumerated in order to access it.

HP has two types of SCSI/SATA host controllers, hardware backed (based) and software backed (based). To HP (all) SCSI/SATA host controllers are [potentially] RAID host controllers by default.

The hardware versions are what most people are familar with called "SmartArray" and generally come with Operating System specific device drivers and add-on drivers to work with installation software, like for the WinPE installer from Microsoft.. or as a kernel module for Linux.

For some time HP (now HPE) has been moving towards a converged UEFI hardware platform that integrates downloading and providing device drivers to the hosted operating system on their server blades [at the time of install] or as needed by manifesting additional mass storage devices on the PC extended buses (mscdex, usb, pci, pcie, iscsi or pxe) as needed.

The NVMe support in the new HP servers definitely takes a strange turn for most people familar with laptops and other servers.

Basically like the old CCISS approach of creating a vendor specific "bus" to support their "vision" of the hardware.

HP has used their [Software (backed)] B170i RAID controller to {host} the NVMe but normally all RAID controllers under HP require the user to configure them at UEFI> BIOS> Host Controller BIOS> setup [before] they are visible or enumerated in the boot loader order. When it is properly setup as a RAID-1 software RAID device "backed by NVMe hardware" it will appear as an additional boot option, but won't even be listed as an option until the LUN is created in UEFI BIOS (note: Not Legacy BIOS!)

Unlike other vendors implementation of the NVMe specification, they are treated as a SCSI device by the HP hardware (Not PCIe devices~!). This both complicates and simplifies things.

It complicates things because the NVMe descriptions for "native" PCIe bus support do not apply to the NVMe on HP servers.

It simplifies them because the operating system can treat them as another SATA/SCSI device.. and the subsequent operating system only needs the device driver for the generic Software based RAID specific to HP products.

While this still requires you to [Inject] [F6] the special HP provided device driver when in the WinPE menus, it does eventually discover the new B170i controller managed "M.2 SSD devices" as if they were merely another SCSI LUN hanging off the Software RAID controller made available by the HP device drivers.

You can use the "normal" HP plethora of extended media options to get the HP boot time device driver for WinPE at the time of install.. namely (iLO) mscdex, usb, pci, pcie, sd media.. or their physical options as provided by your servers physical form factor.. aka.. external "ports" on the surface or backside of the machine.. or a USB/KVM dongle.. custom wim images with the device driver pre-injected into the WDS or SCCM services are also possible.. to each their own.

Windows WinPE will then copy the runtime device driver into the eventual native store after install and setup the device as a bootable device automatically.

The "behavior" is you begin to Install Windows 2012r2 with WinPE bootable media and an install source, it proceeds normally [Up to] where it lists the "available storage devices" and asks where you would like to install Windows 2012r2.

If you have already created a RAID device in the hardware based SmartArray controller, that will be listed. If you have not created a RAID device in the software based B170i controller, then the M.2 SSD RAID device will not be listed -- and you will have no hint that there is a "missing" storage device of 64 GB.

Even if you use the F6 procedure to inject the B170i device driver into the WinPE install, unless the B170i has "already" had a RAID device setup before you began the Windows 2012r2 install procedure.. it will still not be listed as an available 64 GB RAID device made from the M.2 SSD hardware.

The only way to (properly) expose the M.2 SSD hardware is "first" to boot into the UEFI BIOS intelligent provisioning provider menus. They start a Linux derived GUI and present a representation of the available RAID controllers and the hardware hanging off of them.

In this "diagram" will be the (two) 64 GB M.2 SSD devices, which can be selected and combined into a RAID-1 device which will automatically create a SCSI LUN which the subsequent Windows installer "can see" once the F6 procedure is used to "inject" the B170i device driver into the runtime image.

With the LUN pre-setup, and the F6 procedure followed, WinPE will take a few "minutes" to rescan the available RAID storage controllers, and it will "see" the B170i controller, and that it has a RAID-1 LUN and present that as "available" storage media and a valid target for completing the install of Windows 2012r2 server. (Don't expect "Spectacular" drag racing style install speeds.. rather most of the pedantic Windows install process is about thoroughly auditing hardware busses and enabling hardware then testing it.. and contructing a registry from scratch.. all of this is memory intensive.. not disk io bound).

NOTE: Until you manually go into IPP storage provisioning from the BIOS UEFI screen and manually select and [Create] a RAID-1 array the M.2 SSD devices will not be listed or available to boot [from]. In fact they are never made [directly] available to the operating system as a PCIe device, they are treated as (one) B170i RAID device.

If you wanted to use the PCIe SSD as a cache store in a Mac Fusion"y" kind of "hybrid drive" manner to boost the access speed of a partnered hard drive.. that is not what the default is intended. I believe a special License key must be purchased to enable the HP io store accelerator.. or something named like that to provide an enhanced RAID cache to increase disk I/O.. different product for a different purpose.. and I believe its only available with the hardware (based) SmartArray products.

Rather this solution is meant to host the bootable Operating System (only) and in a mirrored RAID fashion, such that one half the M.2 Sata RAID can "fail" and notify you before the situation becomes "critical". A maintenance window can be scheduled, replacement parts can be shipped ect..

As delivered M.2 SSD devices can be many sizes, but at 64 GB the base install for a recent server I helped setup, this was more than enough after the Windows 2012r2 installation rounded out at about 20 GB leaving plenty of room for subsequent updates and feature installs.

Another good reason for this beyond speed, redundancy, and reliability, is these drives are relatively small and unobtrusive close to the motherboard with no moving parts.. so their fault tolerance should be superior to that of the data drives.

The Moonshot/Apollo platforms are a medium sized converged platform which hosts a maximum of (4) servers in a "mini-chassis" with a quorum of drive slots allocated per server on the front of the chassis. LFF only allocates 3 slots per blade server, and if two slots host another RAID-1 configuration and the third is used for hot spare failover.. its best to use these only for bulk storage and data provisioning.

Since the Operating System has effectively now been taken "inside" the blade server it modularizes.. and opens up the possibility of swapping blade servers [whole cloth] when troubleshooting.. and leaving the data drives where they are.. to be picked up after replacing the server and os all at once.

Someday such an arrangement may even lead to "hotswap" of the blade server and os, but I am inclined to think even a partial scenario like that is possible today with Clustered CSV and Clustered Nodes on the current failover system built into windows.. with VMs and Docker Containers it becomes even more likely. Swapping rapidly turns into a "Time Share" problem where jobs are merely "paused" rather than torn down and rebuilt after each reboot.