Relative to consumer CPUs, they have ECC ram support, and higher models support having more than one in a system. Previous to Skylake there was some overlap and people got Xeons as cheaper alternatives to consumer CPUs, although since Skylake I understand Intel have forced the difference between them and Xeons will now only work in server chipset systems.
Depending on what you are doing, there are some potential bargains out there. I got a used E5-2683 v3 which is 14 core, 2.0 GHz base, and so far I got it to turbo all cores to 2.3 GHz. All at a cost lower than a i5-6600k. Now, the clock is obviously way down compared to a high end consumer CPU, but so many cores! For software that can handle it, it does provide a lot of performance for not a lot of money. That scored 1585 in Cinebench R15, whereas my i7-6700k at 4.2 GHz was around 900 ball park. The even older E5-2670 (2.6 GHz base, 8 core) is even cheaper, but this isn't so helpful as there are not many motherboards still around for it at any sane price.
