It's tangent time.
I may be off track here but isn't is more usual to filter out noise at the power supply rather than in the signal path though?
Power supply for mixed analogue and digital designs is tricky, and strictly speaking those two domains each get their own supply. For example, it's a common mistake to even have the same ground even though pretty much every DAC or ADC will have separate pins for digital and analogue supply and ground. When you just go "oh they're both 3.3V and ground" and solder them together, the digital parts can and will inject nasty interference into the nice and clean analogue circuitry.
I suspect that's the case with my cheap USB audio interface (a Behringer UCA-222 which I highly recommend except for this issue): it plays back without any audible noise floor so I can crank up both the mixer it goes in and the power amp and get nothing; it also records very clean. However, enabling the recording ADC causes very notable high-frequency noise on the line out.
Ideally, audio power would be derived from a nice big and inefficient linear supply with a toroid transformer and well-filtered linear regulators providing something like ±15V for opamp rails and whatever else from separate windings. In consumer electronics except dedicated amplifiers or receivers that's of course not an option for several reasons, so you get little cheap and relatively noisy switchmode supplies with varying quality everywhere.
With onboard audio you also have the problem that was already mentioned that the inside of a computer case is just about the worst place to put that stuff because it will be sitting next to the CPU, graphics parts, USB, and whatever else comes along with wildly unpredictable emissions trying to muddle up the place. That gets even worse when you look at microphone inputs which take an extremely weak signal and amplify the hell out of it, with extra bonus for using case front connectors that are attached to a long antenna lead that will pick up so much more, and people wonder why (apart from cheap headsets) they sound like crap over VoIP. With front connectors I've seen it get so bad that,
even without a microphone connected, the leads would pick up the headphone signal at a level that made it echo back; that was fun to diagnose.
All in all, you need to avoid and potentially filter noise and interference everywhere.
The TL;DR is: get audio outside the case digitally in whatever way, be it the digital interface of the onboard solution or even an affordable (not "cheap") USB interface, and you will usually get better quality. Power amplifiers for headphones or speakers are another if similar story. Even more tangentially, if you want digital out, the coaxial "electric" interface is better than optical.
