Playable prototypes are essentially small experiments to clarify the path forward. We’re usually testing whether a new feature or iteration meets its stated goal.

The data that will inform our conclusion is either directly measured results (parry attempts, successful parries, etc.) or player feedback (whether internal or external). Giving good feedback takes some skill and some effort.

Ask Clarifying Questions

To provide the most relevant and valuable feedback, always make sure you understand the intended goal. Unrelated feedback can be valuable too, but not always and not at the expense of evaluating critical experiments.

It’s great to ask clarifying questions when you’re unsure. Sometimes designers or producers simply want open feedback or want to avoid asking leading questions. If so, make sure that’s the case.

Keep an Open and Attentive Mind

It’s generally fine to say, “I don’t like this yet” based on first impressions. It’s unfair to say, “This direction isn’t worth pursuing” unless you’ve thought through all the possibilities.

Word choice matters. Even if you’re skeptical of an idea or its viability within a certain time frame, many people may have worked on it and they get caught in the crossfire. In creative work, it’s important for everyone to feel comfortable proposing ideas.

Naturally, some folks will have concerns when evaluating bold experiments or complex systems. With every bit of negative feedback, we’ll need to determine whether it can be resolved or whether it exposes a fatal flaw in the concept.

People are generally risk-averse, so opinions can shift slowly. Exploring new ideas with hazy implications always feels somewhat uncomfortable. But the alternative is only staying in familiar territory and never trying anything new in the open.

Provide Reasoning

Playtesting is part of our job and we get paid to be thoughtful and specific with our feedback. Without reliable data, we can’t evaluate an experiment and can’t move forward with any degree of confidence.

It’s okay to share blanket positive or negative reactions, but it’s far more useful to provide reasoning. Take a moment to trace your impression down to the delight or pain point at its source. Otherwise those working on the feature have to dig deeper to uncover or infer the reasons.

Unexplained feedback isn’t really actionable. If we act on it, we’re effectively deferring the decision until the underlying reasons emerge. Whenever that happens, expect the same choices to come up again for re-evaluation or revision.

Not engaging with new additions that are explicitly presented for feedback focus is also obviously unhelpful. But sometimes it’s fair to say that you need more time to consider, or that there’s been too much new/recent change to absorb.

Caveats for Leaders

If you’re in a leadership position, always specify whether each of your notes is a directive, a suggestion, or simply food for thought. Skipping this step always creates uncertainty and chaos. There’s no way around it.

Be careful with blunt or unfiltered feedback, especially when aimed at feature owners in a group setting. Not only can it undermine their confidence, but also their team’s trust in their ability to drive future iteration. Leaders don’t need to be harsh to get their point across or their priorities carried out.

Consider keeping unrelated, low-priority, or future polish feedback to yourself. Jot those notes down in a private file and revisit them when appropriate. Assuming you’ve hired talented people, they’re fully aware of everything that feels unpolished within their craft.

If you’re concerned enough about the quality bar staying too low, book some formal time for fine-tuning through your production process. Once those days are established, you can safely bring up some areas of focus. You may even want to schedule these hardening cycles on a regular basis. In the absence of clearly budgeted time, leadership complaints about minor bugs and polish tasks can be extremely disruptive to mainline priorities.

How do you know when you’re ready to take a prototype into production with a bigger team? Or when you’ve prepared enough to start playtesting a live service game with external players?

These are tough questions to answer, partly because there isn’t much experience to go around. Dev teams typically need to stay small until an early prototype has been validated. A natural consequence is that most folks in our industry have never been involved in the early stages of a prototype that goes on to be expanded and refined by a large team.

I don’t claim to be an expert, but i’ve been fortunate enough to work on a few such projects. What i’ve noticed is that the team’s ability to make sound decisions and iterate quickly are more important than what’s actually on the screen – even several months or years into development (depending on scope).

Genre Evolution

When trying to redefine an existing genre or innovate toward a new genre, expect to spend at least a year answering questions that you would’ve thought were too basic to ask. The truth is we don’t have a very good understanding of what makes most mature game genres shine. No one’s ever sure that their unreleased game will hold up for ten thousand hours.

Most major titles essentially evolved from absurdly large sample sizes (thousands of prototypes?) with absurdly long play times (billions of hours?) poured into evaluating and refining and doubling down on working patterns. That’s not a design process. That’s straight up natural evolution.

Setting Attainable Goals

Regardless of how relevant genres reached their current state, becoming sufficiently familiar with their core tenets and bounds is half the “genre-defining” challenge. I think we’re trying to do two things here:

1. Craft a fun game that’s complex enough to take players a long time to figure out
2. Become the world’s most capable team at improving that game in real time before (and rapidly after) shipping.

If we release a game that’s awesome at first glance and throughout the first 100 hours, we’ll gain a lot of player trust and a lot of leeway to continue growing it. That depth is the first step toward making players recognize our expertise as developers, to establish a constructive relationship.

We’ll need to keep building upon that foundation under real time pressure from players (and from deeply engaged internal devs as our team scales up). Trust is much harder to regain than to maintain, so it’s important to avoid major mistakes that could undermine their trust.

Parsing Player Feedback

All player feedback is valuable to some extent, but not all suggestions are. Sometimes we’ll have to pump the brakes a bit when heated discussions start jumping straight to solution space, before articulating the underlying pain points clearly enough.

Ultimately, dedicated players just want to know that the biggest fires are being prioritized. And they want to be able to follow the reasoning for any implemented solution.

To meet these needs, we probably have to become the world’s leading experts in our genre. I think that’s the right bar to set. If the established experts can’t solve a problem quickly, players will have to be somewhat understanding and patient, instead of demanding immediate (and short-sighted) answers.

Steady Communication Is Key

Preserving that patience requires open communication. We’ll earn space to operate by demonstrating our expertise.

For any given problem, players will propose their own theoretical fixes, and they’ll want those suggestions considered. Hearing us explain how several potential solutions wouldn’t work because of unseen limitations or fatal implications, and why the direction we ultimately chose is sound will go a long way toward building confidence in the team.

Sometimes players will find perfect answers too. In fact, it’ll probably happen more and more frequently over time, until players are generating more viable ideas than our team can internally.

But the hard part is always evaluation. Our job is to separate the good suggestions from the flawed ones, quickly and reliably. So there’s no getting around that expertise requirement.

The Struggle Is Real

Of course there will always be the practical struggle of staying sane in R&D as we’re constantly swinging back and forth between “Oh no, we don’t know what we’re doing, let’s learn stuff” and “Oh no, we haven’t done anything in forever, let’s build stuff!”

Just remember that both phases are valuable, and genuine creativity requires cycling between them to some degree. Whatever game you think you want to make at the outset, trying to build it will almost certainly make you reconsider your theory.