linux

AMD RAID on Ryzen

bsilvereagle

• 6 min read

Warning : This guide is not for the faint of heart. Backup data before proceeding. Here be dragons. This process involves a) compiling a kernel b) compiling a kernel driver c) adding that driver to the kernel d) creating a new LiveUSB installer containing the custom kernel+driver. Here be dragons.

Warning #2: As of November 19, 2017 hard drives using the rcraid.ko driver do not respond to smartctl, hddtemp, or hdparm commands. I have contacted AMD support regarding the lack of S.M.A.R.T. support in Linux and am waiting for a response.

Warning #3: It seems that HDDs using rcraid.ko do not hibernate and continuously spin.

Setup assumptions
  1. AMD Ryzen processor & X370 motherboard (preferably Ryzen 1700 and Gigabyte X370 Gaming 5
  2. The BIOS is configured for SATA AHCI
  3. A working Linux (preferably Fedora 27) is installed
  4. An empty 8+GB flash drive is available
Download the drivers

AMD provides drivers for AMD fakeRAID on their website. Binary drivers for RHEL7-3.10.0-514 and Ubuntu 16.04-4.4.0-31 are provided. If you are not using those exact distributions and kernel versions (uname -r), the compiled drivers do not work. Fortunately, AMD also provides source code and binary blobs to compile the drivers on other distros. If compiled successfully, a rcraid.ko kernel driver will be built that can be swapped with the ahci driver to interface with AMD fakeRAID drives.

Remove ahci from the kernel

My distro of choice for this endeavor was the newly released Fedora 27. Unfortunately, Fedora compiles the ahci driver into the kernel instead of leaving it as module. If ahci is compiled into the kernel, it cannot be removed and replaced with the rcraid driver. To check if ahci is a kernel module, run modprobe -r ahci; if ahci is removed successfully, you may not need to recompile the kernel.

Fedora provides a decent but not thorough guide to compiling a custom kernel on the Fedora Wiki.

  1. Install dependencies:

# dnf install fedpkg fedora-packager rpmdevtools ncurses-devel gcc

  1. Clone the Fedora kernel:

$ fedpkg clone -a kernel

  1. Build kernel dependencies

# dnf builddep kernel.spec

  1. Create a new branch for the RAID drivers and base it on Fedora 27

$ git checkout -b raid origin/f27

  1. Edit the kernel.spec to designate this kernel as being RAID-enabled.

$ vim kernel.spec

%define buildid .raid

  1. Configure the kernel to build ahci as a module instead of a built-in.

$ vim kernel-local

CONFIG_SATA_AHCI=m

  1. Configure fedpkg to only produce a release kernel

$ make nodebug`

  1. Build the kernel and go to lunch

$ fedpkg local

  1. Install the fresh kernel on your existing installation

dnf install --nogpgcheck ./x86_64/kernel-$version.rpm ./x86-64/kernel-core-$version.rpm ./x86-64/kernel-modules-$version.rpm ./x86-64/kernel-modules-extra-$version.rpm

  1. Reboot to verify the kernel functions properly
  2. Make sure ahci is a module in the new kernel

$ modprobe -r ahci

Build AMD RAID Proprietary Driver

Unfortunately, the AMD RAID license has a very nasty license:

/****************************************************************************
*
* Copyright © 2006-2008 Ciprico Inc. All rights reserved.
* Copyright © 2008-2014 Dot Hill Systems Corp. All rights reserved.
* Copyright © 2015-2016 Seagate Technology LLC. All rights reserved.
*
* Use of this software is subject to the terms and conditions of the written
* software license agreement between you and DHS (the "License"),
* including, without limitation, the following (as further elaborated in the
* License):  (i) THIS SOFTWARE IS PROVIDED "AS IS", AND DHS DISCLAIMS
* ANY AND ALL WARRANTIES OF ANY KIND, WHETHER EXPRESS, IMPLIED, STATUTORY,
* BY CONDUCT, OR OTHERWISE; (ii) this software may be used only in connection
* with the integrated circuit product and storage software with which it was
* designed to be used; (iii) this source code is the confidential information
* of DHS and may not be disclosed to any third party; and (iv) you may not
* make any modification or take any action that would cause this software,
* or any other Dot Hill software, to fall under any GPL license or any other
* open source license.
*
****************************************************************************/

Notice (iv) you may not make any modification or take any action that would cause this software, or any other Dot Hill software, to fall under any GPL license or any other open source license

This tidbit is really unfortunate since the code that is provided requires some modifications to compile. Unfortunately, those changes can not be redistributed. With some hints, the necessary changes should be obvious.

  1. Download and extract the drivers
  2. Attempt to compile and fail

$ cd driver_sdk/src

$ make

  1. The sigfillset function can be found in linux/signal.h. This may need to be included in the appropriate file.
  2. It appears that the AMD driver is not-GPL compliant. You can read more about MODULE_LICENSE if needed; the macro call is in rc_init.c
  3. The module will need compiled against the kernel source used above.

$ make KDIR='/path/to/kernel/kernel-$version/linux-$version

  1. Verify that rcraid.ko was compiled successfully and that it loads

# modprobe rcraid.ko

At this point, there is quite a conundrum. When the SATA mode is changed from AHCI to RAID in the BIOS, the physical size of the disk reported by the HDD changes. This will cause existing Linux installations to break. If your lvm game is strong and you're comfortable in a dracut console, feel free to prove me wrong:

Warning #4: You will probably lose all your data and have to start from scratch! If you do not want to lose all your data, skip to the next section.

  1. Copy rcraid.ko to /path/to/kernel/kernel-$version/linux-$version/drivers/scsi/
  2. Run depmod
  3. Slip the module into the initrd with dracut

# dracut --add-driver rcraid --force

  1. Reboot

  2. Use lvm tools to attempt to recover the messed up partitions. The following commands may be your friend

     lvm pvdisplay
 lvm vgdisplay
 lvm lvdisplay
 lvm vgscan
 lvm vgchange -ay
 lvm pvresize --config "global {locking_type=0}"

  3. If you manage to recover from this let me know

Build a LiveCD Installer

The disks need to be in RAID mode before the installation begins. There are two ways to accomplish this: with inst.dd and by spinning a new installation disk with the kernel+driver installed. Building a driver disk and loading it with inst.dd may be the better solution to this problem; however, it was unsuccessful in my attempts. Fortunately, Fedora provides an easy way to build custom LiveCDs with kickstart. Unfortunately, including a custom kernel driver in the initramfs is not immediately apparent. This can be fixed by adding the AMD kernel module code to the Linux source tree.

The AMD RAID license prevents the patch to the kernel from being distributed. However, you can build it yourself.

  1. Change directories to the drivers/scsi folder

    $ cd kernel/kernel-$version/linux-$version/drivers/scsi

  2. Create a git branch for the AMD RAID changes. This is necessary to generate the patch

    $ git checkout -b raid

  3. Create a folder for the AMD code and move it over

    $ mkdir rcraid

    $ cp driver_sdk/src/* rcraid

  4. Create a Kconfig file for the RAID driver

    $ vim rcraid/Kconfig

     config AMD_RCRAID
 	tristate "AMD fakeRAID"
 	depends on X86_64
 	help
 		"Proprietary AMD driver for AMD X370 fakeRAID"

  5. Modify drivers/scsi Kconfig to recognize rcraid

    $ vim drivers/scsi/Kconfig

     source "rcraid/Kconfig"

  6. Add the rcraid Makefile to drivers/scsi Makefile

$ vim drivers/scsi/Makefile

    obj-$(CONFIG_AMD_RCRAID)   += rcraid/

  1. Add all of the files to git and commit

    $ git add drivers/scsi/rcraid/* drivers/scsi/Kconfig drivers/scsi/Makefile

    $ git commit -m "Patch commit for AMD fakeRAID driver"

  2. Create the patch and copy it to kernel directory

    $ git format-patch master

    $ ls | grep patch

    $ cp $patch.patch /path/to/kernel/

  3. Add the patch to kernel.spec. Find END OF PATCH DEFINITIONS. Just above, there will be say Patch498: another-patch.patch. Add a line beneath it with the patch created above

    $ vim kernel.spec

    Patch499: 0000-amd-raid.patch

  4. Add the AMD_RCRAID configuration option to kernel-local

CONFIG_AMD_RCRAID=m

  1. Recompile the kernel

$ fedpkg local

  1. Create a dnf repo in the output directory

$ cd ./x86_64

$ createrepo .

  1. Get the dependencies to build the Fedora LiveCD

# dnf install livecd-tools spin-kickstarts

  1. Copy the example kickstart files to a convenient location

$ cp /usr/share/spin-kickstarts/* .

  1. Modify fedora-live-base.ks to use the compiled kernel in ./x86_64/

    kernel-$version.fc27.x86_64
kernel-core-$version.fc27.x86_64
kernel-modules-$version.fc27.x86_64
kernel-modules-extra-$version.fc27.x86_64

  2. Create a local.ks file and add our kernel repository information to it

    repo --name=newkernel --baseurl=file:///path/to/kernel/x86_64
%packages
kernel-$version.fc27.x86_64
kernel-core-$version.fc27.x86_64
kernel-modules-$version.fc27.x86_64
kernel-modules-extra-$version.fc27.x86_64
%end

  3. Modify fedora-live-workstation.ks to include the repository kickstart file

    %include local.ks

  4. Build LiveCD

    # livecd-creator --config=fedora-live-workstation.ks --fslabel=Fedora-RAID --cache=/var/cache/live

  5. Burn Fedora-RAID.iso to a flash drive located at /dev/sdc

    # dd if=Fedora-RAID of=/dev/sdc

  6. Reboot the machine with the USB flash drive plugged in, enter the BIOS, and change the drive type to RAID.

  7. At the GRUB boot menu, hit e to edit the kernel boot paramters and blacklist the ahci module

    linux .... modprobe.blacklist=ahci

  8. Hit Ctrl+x to continue booting to a Fedora installer that will recognize the RAID disks!

Did this information save you hours of needlessly recompiling, rebooting, and almost returning your Ryzen for an i7? Are your disks working in RAID? Say thanks and help keep this site ad & analytics free by using my Amazon Affilliate URL. Or, consider donating some BTC (1DNwgPQMfoWZqnH78yt6cu4WukJa3h8P1f) or ETH (0xf3c4a78c24D34E111f272Ac2AC72b1f01ba52DF3).