This is a simple guide, part of a series I'll call 'How-To Guide Without Ads'. In it, I'm going to document how I set up
bcache on a Raspberry Pi, so I could use an SSD as a cache in front of a RAID array.
bcache is sometimes used on Linux devices to allow a more efficient SSD cache to run in front of a single or multiple slower hard drives—typically in a storage array.
In my case, I have three SATA hard drives:
/dev/sdc. And I have one NVMe SSD:
I created a RAID5 array with
mdadm for the three hard drives, and had the raid device
I then installed
$ sudo apt-get install bcache-tools
make-bcache to create the backing and cache devices:
$ sudo make-bcache -B /dev/md0 UUID: eb360a2d-4c62-451d-8549-a68621c633e5 Set UUID: c8b5c63c-0a44-49f3-bb65-cd4df9b751a0 version: 1 block_size: 1 data_offset: 16 $ sudo make-bcache -C /dev/nvme0n1 UUID: 15bf54e9-be21-4478-b676-a08dad937963 Set UUID: dea419ba-d795-4566-b01f-bb57fa96eb21 version: 0 nbuckets: 15261770 block_size: 1 bucket_size: 1024 nr_in_set: 1 nr_this_dev: 0 first_bucket: 1
Then I tried to look in
/sys/block/md0/bcache/ so I could attach the cache to the backing device, but I realized
bcache isn't loaded into the default Raspberry Pi OS kernel... so I'll have to compile that in.
bcache on Raspberry Pi OS
I cross-compiled the Raspberry Pi Linux kernel, and when I did it, during the
menuconfig portion, I selected the following option:
> Device Drivers > Multiple devices driver support (RAID and LVM) > Block device as cache (BCACHE)
I recompiled the kernel and copied my updated kernel to the Pi, then rebooted.
At this point, I could see the
bcache0 device was working:
[email protected]:~ $ lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 3.6T 0 disk └─md0 9:0 0 7.3T 0 raid5 └─bcache0 254:0 0 7.3T 0 disk /mnt sdb 8:16 0 3.6T 0 disk └─md0 9:0 0 7.3T 0 raid5 └─bcache0 254:0 0 7.3T 0 disk /mnt sdc 8:32 0 3.6T 0 disk └─md0 9:0 0 7.3T 0 raid5 └─bcache0 254:0 0 7.3T 0 disk /mnt mmcblk0 179:0 0 14.8G 0 disk ├─mmcblk0p1 179:1 0 256M 0 part /boot └─mmcblk0p2 179:2 0 14.6G 0 part / nvme0n1 259:0 0 7.3T 0 disk
But if I checked on the status of the cache, it said there was no cache:
[email protected]:~ $ cat /sys/block/bcache0/bcache/state no cache
Attaching the SSD cache to the backing device
Finally, it's time to attach the SSD cache to the backing device:
$ sudo su # switch to the root user # cd /sys/block/md0/bcache/ # echo dea419ba-d795-4566-b01f-bb57fa96eb21 > attach # cat state clean
The UUID in the
echo command above comes from the 'Set UUID' output from the
make-bcache -C command earlier.
Creating a filesystem and mounting
To actually use the device, I formatted it and mounted it to
$ sudo mkfs.ext4 -E lazy_itable_init=0,lazy_journal_init=0 /dev/bcache0 $ sudo mount /dev/bcache0 /mnt
To avoid the initialization when making the filesystem, you can omit the
-E option entirely. But for RAID arrays I typically let it go full blast on first initialization, because I don't like relying on
ext4lazyinit on a RAID array—it can take days at its reduced rate, and affect RAID performance that whole time!
You can check the stats from
$ tail /sys/block/bcache0/bcache/stats_total/* ==> /sys/block/bcache0/bcache/stats_total/bypassed <== 563.1M ==> /sys/block/bcache0/bcache/stats_total/cache_bypass_hits <== 0 ==> /sys/block/bcache0/bcache/stats_total/cache_bypass_misses <== 0 ==> /sys/block/bcache0/bcache/stats_total/cache_hit_ratio <== 13 ==> /sys/block/bcache0/bcache/stats_total/cache_hits <== 132 ==> /sys/block/bcache0/bcache/stats_total/cache_miss_collisions <== 0 ==> /sys/block/bcache0/bcache/stats_total/cache_misses <== 849 ==> /sys/block/bcache0/bcache/stats_total/cache_readaheads <== 0
Switching the caching mode
There are multiple caching modes, including
none. The most performant (but most dangerous, especially if you're using a single SSD and not a set of SSDs in RAID 1 for safety) is
writeback, which caches reads, and writes data to the SSD first (considering a write 'complete' once written to the SSD), then asynchronously copies that data to the backing device.
Check the current caching mode with:
$ sudo cat /sys/block/bcache0/bcache/cache_mode [writethrough] writeback writearound none
To change it, for example, to
$ sudo su - -c 'echo writeback > /sys/block/bcache0/bcache/cache_mode'
Dropping the cache
If you want to pop the SSD off of the backing device, and use it again for other purposes, you have to first de-register it (otherwise you'll get errors like
probing initialization failed: Device or resource busy):
$ sudo su # cd /sys/block/md0/bcache # echo 1 > detach # Prints a 'cached_dev_detach_finish' message in `dmesg` log # cd /sys/fs/bcache/dea419ba-d795-4566-b01f-bb57fa96eb21 # echo 1 > stop # Prints a 'cache_set_free ' message in `dmesg` log
Then if you want to use the device for something else, wipe it with
# wipefs -a /dev/nvme0n1
See the kernel documentation for bcache for more detail and usage examples.