If you have 8GiB or more VRAM, adding more probably won't help. In my opinion and experience, 3-4 is too little for a good experience, six is fine for middling settings, and eight or more makes capacity, in and of itself, generally a non-issue.
The current game still has some CPU utilization issues, but on the GPU side is pretty well balanced and no longer has a major architectural bias toward Ampere (prior to update 8 my 3080 had a major edge in concourses because of the excessive shading going on in concourses and GA102's extreme shader count, but this is no longer the case). So, just buying performance is going to get you performance, if you can overcome that CPU limitation (or throw so much eye candy at things that you bury it in a GPU bottleneck).