Yay, Extra Credits finally talked about QA! I’d been looking forward to this for years. Now that I’ve seen it I’m… it’s there. It’s fine.

Boy, this took me a while to get to… *checks* Over a month!? Damn… Well, let’s get to it!

Eighty Percent On

This is about 80% there. They point out a lot of truths, even using the same words that I use at times.

QA is under-appreciated, under-respected, maligned, and mis-understood, even within the industry. This is true. Testing is (obviously) not “just playing games all day.” Skills in testing are akin to a detective, tracking down steps to reproduce to allow the developers to understand what’s actually going wrong.

Ad-hoc testing is not the entirety of testing. Important, though not mentioned directly, is the running of test plans. These give the testing structure to verify that each feature, each item, each character, etc. work as intended. This is a part of how testers are tasked to elements of a game.

Isolating steps to reproduce a bug is a very important part of the job. Especially if it’s complicated, one will want to make sure the engineer has NO room to claim that they can’t reproduce the effect. It’s a tremendous thing to take a 5% reproduction rate issue, and find the missing pieces to make it 100%.

There are places that do not take care when hiring testers. Yes, those are bad places to work, but that’s not limited to QA. ANYWHERE that’s careless when hiring is a place to avoid.

“Testing might be the most varied discipline in the games industry.” Not might be, it is, because we touch everything. I’ll come back to this thought.

Automation will continue to grow in game testing. This will make testing the “golden path,” or the intended methods and areas of play, much faster, more complete, and more reliable. It’s the difference between having testers run every mission in Skyrim vs walking over every square inch of the map in Skyrim. It is true that this will allow for smaller, more specialized and focused teams. Hopefully this will also mean greater respect for QA.

Twenty Percent Off

Ad-hoc testing is still very important. While automation is growing in importance, and that won’t stop, it follows a script and cannot intuit, adjust, or get creative. Many issues will continue to be found by testers being actively adversarial to the very system of the game. This will always be necessary because the players will also be adversarial in the same manner. Ad-hoc testing is the only way to simulate actual user behavior.

For those who follow me on twitter, you may have spotted me taking exception to the line “…it is impossible for a tester to have a master level understanding of the game’s programming, scripting, and art…” You may be surprised, because QA is actually in the best position to see how everything fits together. In terms of exact code logic, game QA rarely has access to the code itself, so that doesn’t apply. However, QA does use the code more than the engineers making it. We see the art move more than the artists making it. We hear the music and sound effects more than the audio designers. QA knows the exact status of the project, and can see all of the pieces fitting together.

Hot take:"While it is impossible for a tester to have a master level understanding of the game's programming, scripting, and art"
You would be surprised. Devs may know the code, and the intent, but QA knows better about how it's actually acting, providing cross-referenced insight

— Kynetyk (@KynetykKnows) January 18, 2018

While the steps to reproduce are important, the summary is even more important. The one line that is the header of the report, and often the only thing an engineer will read. It is SO critical to get to the point as quickly as possible there.

QA does get outsourced a lot, and this does cause communication issues, but those issues don’t require QA to be in a separate office. This can happen within the same office, as a result of QA being treated as a burden or as second class citizens. Engineering and QA are opposing disciplines. It doesn’t have to be adversarial, but engineers will often want to dismiss issues and QA has to force recognition of the issues.

Testers do get attached to bugs they write. This is true, but I must note that this is usually a sign of immaturity, or a lack of professionalism. Testers who are too attached tend to think that they are the designers, the ones who make the final call of what goes into the game, and this isn’t true.

Developers testing their own work is a TERRIBLE thing. As I mentioned above, these are fundamentally different disciplines. Writers need editors who are different people. This practice is how Atlassian products became the dumpster fire that they are. Maybe Valve has had success, but their structureless structure is so strange I don’t think they should be an example of anything.

I mean, look at this article. I wrote it and didn’t have anyone else editing it. I’m sure there are many ways this could be improved, but I’m too close to my own work to have an appropriately critical eye.

The Pedantic

It’s QA, not Q+A. I wouldn’t be a good tester if I didn’t spot that one.

That’s attention to detail, right there.

Kynetyk is a veteran of the games industry.  Behind the Line is to help improve understanding of what goes on in the game development process and the business behind it.  From “What’s taking this game so long to release”, to “why are there bugs”, to “Why is this free to play” or anything else, if there is a topic that you would like to see covered, please write in to kynetyk@enthusiacs.com or follow on twitter @kynetykknows