servers

Cores are the new megahertz, at least for enterprise servers. We've gone quickly from 32, to 64, to 80, to 128, and now to 192-cores on a single CPU socket!

Amazon built Graviton 4, Google built Axiom, but if you want your own massive Arm server, Ampere's the only game in town. And fastest Arm CPU in the world is inside the box pictured above.

It has 192 custom Arm cores running at 3.2 Gigahertz, and in some benchmarks, it stays in the ring with AMD's fastest EPYC chip, the 9965 "Turin Dense", which also has 192 cores.

High-core-count servers are the cutting edge in datacenters, and they're so insane, most software doesn't even know how to handle it. btop has to go full screen on the CPU graph just to fit all the cores:

This is the BliKVM PCIe, a full computer on a PCI Express card. This is an IP KVM (Internet Protocol Keyboard-Video-Mouse) that can be put inside another computer or server.

Most server motherboards already have remote 'lights-out' management functionality built in. Most frequently this is referred to as IPMI (Intelligent Platform Management Interface, but Dell calls it iDRAC, and HPE calls it ILO.

That title is awfully specific.

But I was building a new FreeBSD server with a used SuperMicro motherboard with IPMI. The default password was changed from ADMIN (or maybe it's a new enough board that it's a random password), and when I was booted into FreeBSD, I wanted to reset the IPMI settings so I could be sure I was starting fresh.

ipmitool that came with my FreeBSD install doesn't seem to be able to reset IPMI to factory defaults, so I tried running ipmicfg from Supermicro's website (which is annoying to download—you have to fill out a form and a Captcha for the privilege).

I haven't had time to write up the details yet, but I wanted to share a project that's been many months in the making: The Petabyte Pi Project on YouTube.

I'm still doing follow-up testing based on feedback from Broadcom storage engineers, and will put out a much more in-depth blog post later, but the gist is:

Can a single Raspberry Pi cosplay as an 'enterprise' storage server, directly addressing 1 PB of storage?

Now... caveats abound here. What does 'enterprise' mean? And what does 'directly addressing' mean? Those things are all answered in the video linked above.

But to give a tl;dr: The Pi does not perform swimmingly. But... I did get a single array of 60 hard drives—20TB Exos HDDs to be exact—working in a 45Drives Storinator XL60 chassis, controlled only through a single Raspberry Pi Compute Module 4. Of course I had to rip out the Xeon guts and replace them with said Pi:

Now that I have a half-height rack and a 3D Printer, I figured I should finally move all my Raspberry Pis from sitting in odd places in my office to the rack. And what better way than to print my own 1U Raspberry Pi Rack mount unit?

The rack unit you see above was assembled from 6 'frames', 6 hot-swappable Pi carrier trays, 2 rack mount ears, and a couple lengths of threaded rod for rigidity.

It was printed from these plans from russross on Thingiverse; Russ Ross also made an assembly video, and shows how you can build a 2U 12-Pi enclosure using the same basic design, with interchangeable Pi trays!

Video

There is more detail and a full walkthrough of my home rack in this video:

For the past few years, as the number of servers I manage has increased from a few to many, and the services I operate have required more flexibility in terms of adding and removing similarly-configured servers for different purposes, I've been testing different deployment and configuration management tools.

Many developers who are also sysadmins have progressed much the same way as I have, beginning by building everything by hand without documenting the process, then documenting the build with text files, and ultimately scripting builds with bash scripts. However, none of these techniques allow fast provisioning, continuous configuration management, or the flexibility required to make constantly-evolving applications adapt to the requirements of the day.

In recent years, 'DevOps' (better integration of development and operations) has become a hot buzzword and mantra of companies espousing agile development methodologies.

A Very Brief (and woefully inadequate) Philosophy of DevOps

Source: DevOps.com

Servers, like instances of applications, should be managed via version-controlled configuration, and should be disposable (a common war cry: Trash Your Servers and Burn Your Code. If a server blows up, or if another few application servers are needed, they should be able to be provisioned or decommissioned in minutes, not hours (much less days or weeks!), and should be able to be provisioned and decommissioned automatically, without human intervention.