Dumb question, how do you get into the caves of Umbra veil?

It's not dumb at all! This is my first festival that I figured it out!

So, there is a named in front of a cave entrance. If you defeat them, you can move forward and interact with the cave (unlike the other named's buildings). Click "Enter Cave" and you're all set!

(If you need help finding it, the thread { Tori's Maps } shows you where everything is. Tread lightly though, as there are spoilers)

If you mean the door at the end of the path, you can't go in. Maybe some day, Krin will do something with it, but for now it is just a decoration. The real caves are north of there as mentioned in the above post.