Yes I went from 2x 970's Sli like yourself to a Titan X, call me crazy I know
But i had several reasons......
1st
Although i didn't really seem to notice at first was the latency issue, in-fact I first started noticing it when viewing pages on the net. Particularly on forums when scrolling down a page there was always a short spell of inactivity before the page started to scroll
Then I started to notice more in ED, little hitches here and there and not the usual stutters that we are all familiar with.
I have had sli setups in the past but that was way back when they launched 8800 GTX when i built a rig ready to play the first Crysis but that was a long time ago
2nd
The heat, with having sli my sound card needed to go in the middle of both cards, which was fine for the lower card and ran considerably lower around 60c but the top card it really restricted the air flow from the fans and I saw temps of up to 85-90c which then started to down clock the cards
3rd
TBH I probably got a little carried away with Rift obsession, trying to cure issues that were out of my or any hardware's control and in all fairness i should of waited for the 980Ti but thought what the hell i have the money so why not
So will you notice any performance differences between 2 x 970's and a Titan X ? honestly probably not but you may find your head movement smoother and game play smoother
You would never benefit from the ridiculous 12GB of ram, the most Iv'e seen my Vram usage go to was 3.7GB in Dying light running at 1440p ED uses around 2 - 2.6GB of Vram with the rift
The other negative apart from the cost of a Titan x is the reference cooler, you really need to create a fan profile with msi afterburner to keep temps below 79c so downclocking doesn't start.
My reference model does actually clock higher than what nvidia stock setting say out of the box, as I said above if you keep the cooling below 79c my clock speed will sit at 1202Mhz all day but if you cross over into the 82 - 85c it will down clock to 1189Mhz or even back to 1001Mhz
If you looking to go to one card from sli get the 980Ti with board partners own cooler designs means you can quite easily achieve the same performance as a Titan X and beyond
If you have any more questions I'm more than happy to give you a honest answer in comparison