I actually did go with the route of using a desktop as my server. I custom built it with the expectation of it being a server I would remote into for games. I have proxmox on it with a windows VM I remote into with parsec instead of sunshine. It allows me to keep a light laptop while still getting the beefy power of a desktop.
A custom PC running proxmox:
MOBO: Asus ROG Strix Z790-E gaming wifi
RAM: 4x 32GB Ripjaws S5
CPU: i9-13900k
GPU: Gigabyte RTX 4090
GPU2: EVGA GTX 1070
HDD: 4x 8TB WD red plus in raid 10
SSD: 2x 2TB Samsung 980 pro in raid 1
PSU: Super Flower Leadex Titanium 1600 W
Case: Fractal Design R5
I run everything on it: homassistant, Plex/*arr, pihole/unbound, my windows gaming VM, etc.
I recommend QEMU/KVM with GPU passthrough (will require two GPUs but an iGPU will suffice for the host Linux install if you don’t need a powerful GPU for it). You don’t get complete bare metal performance but it’s close.
You can do USB passthrough for your gaming peripherals with the windows-only drivers installed on the VM.
I actually do this now for video games but with a windows VM running on a headless proxmox host that I remote into with Parsec. I had to scrap and remake the VM a few times in the beginning while figuring everything out but the VM has been going strong without breaking for around 1.5 years now.