I haven't been able to get to a Gathering either. And probably never will since they always seem to occur while my wife is working. She can't take take off from work in the first few weeks of the school year. It's sad.
But I get where you're coming from. There is always a bit of anxiety when it comes to traveling. Especiallywhen you're going somewhere that is completely different than your normal view of the world. If you've ever been to a chill festival, think of it like that on a grander scale. Not a festival where everyone is on crazy drugs like pills, molly, coke, everything else. But like a folk festival where everyone is kind to eachother, and always ready to lend a hand. You don't have to stay long. And you can leave early if you feel uncomfortable. It's not like you bought a ticket and don't want to lose your money's worth.
Plus, Gatherings have been going on for ages. I'm sure they've had their fair share of people freaking out. And the type of folks that show up are ready to help calm you down, or know the right person for you to talk to. No one would look down on you for not liking it and wanting to pack up and go home.