Let’s talk about open versus closed playtests. As a game designer, my preference is for a closed playtest. There are a few reasons for this. For a start, I’m much more comfortable in an environment that’s more like a writing group. They’re a group of people that I know, and whom I trust to give me useful feedback. I know their biases and their preferences, so I can tell whether something is objectively wrong or subjectively just not to their tastes. Because they know me, and they all know how to deliver critique, they can be honest. They’re neither trying to flatter nor to wound. The people that I select will tell me what I need to hear, and know how to tell me in a way that I will hear it. None of them will get hurt, though, if I don’t follow their suggestions and go with my own instincts.
There can be advantages to doing an open playtest. You will get perspectives that you hadn’t considered. More eyes, in theory, catch more mistakes and contribute more possibilities. For me, having tried it a couple of times, there are more disadvantages. It can be a lot of material to go through, and that’s time consuming. As a one-person operation, I don’t necessarily have that time. While I can do some polling, I don’t know the backgrounds of each and every playtester. How much weight can I give to them? Do I even take that into consideration, or just test out ideas that “feel” like valid improvements or concerns. There’s also the issue of egos, and the expectations individuals may have that contrast the expectations that I set.
There is only one reason why I’d consider an open playtest, and that’s marketing. You can put a free sample into the hands of your audience. They can read it, play with it, and talk about it. Hopefully, that turns them into loyal fans. It can create a connection with your audience, and make them feel involved. Those are all good things, but they still fall under the heading of marketing. I still prefer to keep my creative process a closed one, so that I can make what I want to make, the way I want to make it. As an indie publisher, that’s part of the reason I do this.