Quality Assurance

Hello everyone!

Since we announced our revamp of the art style earlier this year we’ve taken you on a whirlwind tour of the solar system and its inhabitants. We’re going to take a break from talking about art to discuss a critical but, often misunderstood part of game development: quality assurance testing.

Quality Assurance

For those who are unfamiliar, Quality Assurance (QA) is the field of testing a product to ensure that it meets expectations for quality and usability. Our development team at Mobius is working directly with a quality assurance studio by the name of HUWIZ, the ‘Quality Assurance Wizards’, who are based in Saint-Jérôme, Canada.

It’s important to the team at Mobius to show a glimpse of what our Quality Assurance team contributes to the development of Outer Wilds. As developers, we want to shed some light on this underappreciated cornerstone of video game development. The team at HUWIZ has been working long and hard at helping make Outer Wilds the best it can be and we want to acknowledge the importance of their work on the game.

Broadly speaking, the goal of QA testing is to actively try to ‘break’ the game in every way possible. This means pushing the limits of the game and its systems to see if any crashes or strange bugs occur. Smaller bugs can ruin the sense of immersion when playing a game, while crashes can outright prevent people from playing and ruin their experience. The goal of good QA testing is not to be noticed: hopefully players will never encounter such bugs because the QA testers have found them in time for the team to fix them before players ever have a chance to encounter them.

In order to effectively communicate and organize the hundreds of unique bugs found through QA testing, Mobius and HUWIZ share a database where bugs are recorded, catalogued and prioritized. Each entry in the database has very specific information that is aimed at communicating how the bug can be reproduced; how frequently it occurs; and how much this interrupts the tester’s progress or enjoyment of the game. These notes are usually accompanied by screenshots and video capture to ensure that our team can quickly figure out what’s causing the issue and start working on a fix. This database is an incredibly useful tool to ensure we prioritize the correct bugs, as well as helping to track the overall stability Outer Wilds.

An example of the general layout of a bug entry and all of the components that are full of detailed information.

While tracking bugs is a large part of what our QA team spends their time on, it’s isn’t everything. The HUWIZ team sends Mobius and Annapurna a daily report with what their latest findings are, including performance benchmarks and advice pertaining to various certification requirements. With all of the hard work HUWIZ pours into helping us make the best game we can, we would like to share a write up from one of our testers.

The following is a letter from the testers at HUWIZ to discuss the joys and challenges of testing Outer Wilds and what sets it apart from other projects.

On testing Outer Wilds:

QA can be a repetitive process, and the many things we love about Outer Wilds have truly kept us motivated, focused and entertained.

First of all, the music in Outer Wilds is a major highlight for the team. Music and sound is integral to any game or movie, and in Outer Wilds, the use of music is taken a step further (no spoilers!), which is something our team is delighted by. Over the months the team has come together with excitement whenever a new musical element was implemented or discovered - and, after all these months, we can confidently say that we are still not bored of it - in fact, we still LOVE it. Not many games, musicians, albums or movies can have that effect on people, so hats off!

It may seem like that was a lot of emphasis on music, but when doing QA and wearing headphones all day, the simple fact that the music is so good and engaging adds a lot to the testing experience.

The game itself takes the user through all of the emotions at some point or another, and this also keeps the testing process interesting for us. For example, Outer Wilds can come across as cute, scary, profound, and actually quite funny (we have all laughed out loud at some point), and even sad. These emotional variations keep the testing process very interesting. In fact, the first impression of every tester who works on the game is exactly that - interested; this game is extremely mysterious and unique.

The one-of-a-kind factor that we've discovered in Outer Wilds really adds a lot of pleasure to the testing process, however, this factor also adds to some of the biggest challenges we have come across. All of the testers who embark on Outer Wilds very quickly acknowledge the complexity and ambition involved in this project (and that it does not really compare to any other game). The simple fact that Outer Wilds takes place in an open world solar system with a variety of planets, each with their own physics, adds a lot to the challenges one might encounter in QA. Not only does each planet and environment have a unique set of characteristics and physics, there are also environment systems based on alternate dimensions, black holes and quantum physics. These combined environment systems in an open-world make for an infinitely complex project to comb through as QA analysts.

Another great challenge stemming from the environments and physics come from some time-based locations that can only be accessed during certain moments in the cycle of the solar system. This added to the challenge of testing Outer Wilds especially when the project first arrived in house. The team would suddenly come together in disbelief after discovering a "new" location for the first time after thinking we had made a thorough sweep of the solar system. For this reason, among others, testing Outer Wilds makes the QA team question ourselves regularly.

This game takes brains! And in the video game QA world, we all become experts after a few months; it's our job. Outer Wilds continues to make us question our intelligence when we think we know the game so well and still find ourselves connecting dots months later. It's honestly mind-boggling and sort of frustrating! Outer Wilds is ridden with intricacy and lore, so much so that some of it can be easily missed. The difficulty in this game comes in being able to notice the details, and that's what makes it so satisfying to test and to play.

The overall experience, opinion and impression from the team: Charmed.

That’s the End of This Update

Join us in another two weeks for another development update. Have a great weekend everyone!

Cheers,

Mobius

Facebook

Twitter