Yes, they'll collide with the skin unless you have collision turned off in the game menu. I mentioned that on the Steam workshop page.
FWIW everything else in my workshop is designed to work with collision on, but so many people are making collision off blueprints that I figured I'd give it a try. It does open up new possibilities by having it off though, especially with regard to using water in builds.