I've been working on something similar. The plan is to have a working "bridge" in my basement so that I can include my children in my various hobbies. There will be 3 main stations and a few other minor ones. The two stations for my kids are essentially fidget boards with sound effects unless I enable them with a switch on my console. That will allow their switches to perform various functions in game such as discovery scanner or targeting.
For each board, I'm feeding everything into 4 Arduino Unos. These connect via USB to my main computer through USB switches(This allows me to 1. Turn on an off the additional panels, or 2. Redirect the input from each panel to a laptop of my choosing so that other functions can be mapped to various ED Tools from 3rd parties.)
The signals eventually make their way to the target system (either my own or a laptop) and are fed into vjoy with all of my other peripherals.
As the full setup has more buttons, switches, toggles, dials, doodads, and gizmos and ED has room for in it's various configurations, the plan is to have a lot of each panel control (through Arduino #4) various aspects of the "bridge". These functions include turning on and off fans, adjusting lighting, locking/unlocking the cabin door, opening/closing various shutters.
I am certain you can achieve the same with your RasPi, my recommendation is to implement vjoy on your system and feed everything in there.