Earlier today I did a livestream on my YouTube channel to attempt using an Nvidia GeForce GTX 1080 on a Raspberry Pi Compute Module 4.
It's only been a few hours, but I've already gotten good suggestions for better debugging than I was able to do on the stream. And someone pointed out it might be the case, due to 32-bit memory limitations on the BCM2711's PCIe bus, that no GPU with more than 4 GB of onboard RAM could work. Though it's hard to confirm there'd be no software workaround—even 1 and 2 GB graphics cards (AMD and Nvidia) are crashing the kernel in similar ways.
The full livestream is available on replay and is embedded below: