Since we announced our revamp of the art style earlier this year we’ve taken you on a whirlwind tour of the solar system and its inhabitants. We’re going to take a break from talking about art to discuss a critical but, often misunderstood part of game development: quality assurance testing.
For those who are unfamiliar, Quality Assurance (QA) is the field of testing a product to ensure that it meets expectations for quality and usability. Our development team at Mobius is working directly with a quality assurance studio by the name of HUWIZ, the ‘Quality Assurance Wizards’, who are based in Saint-Jérôme, Canada.
It’s important to the team at Mobius to show a glimpse of what our Quality Assurance team contributes to the development of Outer Wilds. As developers, we want to shed some light on this underappreciated cornerstone of video game development. The team at HUWIZ has been working long and hard at helping make Outer Wilds the best it can be and we want to acknowledge the importance of their work on the game.
Broadly speaking, the goal of QA testing is to actively try to ‘break’ the game in every way possible. This means pushing the limits of the game and its systems to see if any crashes or strange bugs occur. Smaller bugs can ruin the sense of immersion when playing a game, while crashes can outright prevent people from playing and ruin their experience. The goal of good QA testing is not to be noticed: hopefully players will never encounter such bugs because the QA testers have found them in time for the team to fix them before players ever have a chance to encounter them.
In order to effectively communicate and organize the hundreds of unique bugs found through QA testing, Mobius and HUWIZ share a database where bugs are recorded, catalogued and prioritized. Each entry in the database has very specific information that is aimed at communicating how the bug can be reproduced; how frequently it occurs; and how much this interrupts the tester’s progress or enjoyment of the game. These notes are usually accompanied by screenshots and video capture to ensure that our team can quickly figure out what’s causing the issue and start working on a fix. This database is an incredibly useful tool to ensure we prioritize the correct bugs, as well as helping to track the overall stability Outer Wilds.
An example of the general layout of a bug entry and all of the components that are full of detailed information.
While tracking bugs is a large part of what our QA team spends their time on, it’s isn’t everything. The HUWIZ team sends Mobius and Annapurna a daily report with what their latest findings are, including performance benchmarks and advice pertaining to various certification requirements. With all of the hard work HUWIZ pours into helping us make the best game we can, we would like to share a write up from one of our testers.
The following is a letter from the testers at HUWIZ to discuss the joys and challenges of testing Outer Wilds and what sets it apart from other projects.
On testing Outer Wilds:
QA can be a repetitive process, and the many things we love about Outer Wilds have truly kept us motivated, focused and entertained.
First of all, the music in Outer Wilds is a major highlight for the team. Music and sound is integral to any game or movie, and in Outer Wilds, the use of music is taken a step further (no spoilers!), which is something our team is delighted by. Over the months the team has come together with excitement whenever a new musical element was implemented or discovered - and, after all these months, we can confidently say that we are still not bored of it - in fact, we still LOVE it. Not many games, musicians, albums or movies can have that effect on people, so hats off!
It may seem like that was a lot of emphasis on music, but when doing QA and wearing headphones all day, the simple fact that the music is so good and engaging adds a lot to the testing experience.
The game itself takes the user through all of the emotions at some point or another, and this also keeps the testing process interesting for us. For example, Outer Wilds can come across as cute, scary, profound, and actually quite funny (we have all laughed out loud at some point), and even sad. These emotional variations keep the testing process very interesting. In fact, the first impression of every tester who works on the game is exactly that - interested; this game is extremely mysterious and unique.
The one-of-a-kind factor that we've discovered in Outer Wilds really adds a lot of pleasure to the testing process, however, this factor also adds to some of the biggest challenges we have come across. All of the testers who embark on Outer Wilds very quickly acknowledge the complexity and ambition involved in this project (and that it does not really compare to any other game). The simple fact that Outer Wilds takes place in an open world solar system with a variety of planets, each with their own physics, adds a lot to the challenges one might encounter in QA. Not only does each planet and environment have a unique set of characteristics and physics, there are also environment systems based on alternate dimensions, black holes and quantum physics. These combined environment systems in an open-world make for an infinitely complex project to comb through as QA analysts.
Another great challenge stemming from the environments and physics come from some time-based locations that can only be accessed during certain moments in the cycle of the solar system. This added to the challenge of testing Outer Wilds especially when the project first arrived in house. The team would suddenly come together in disbelief after discovering a "new" location for the first time after thinking we had made a thorough sweep of the solar system. For this reason, among others, testing Outer Wilds makes the QA team question ourselves regularly.
This game takes brains! And in the video game QA world, we all become experts after a few months; it's our job. Outer Wilds continues to make us question our intelligence when we think we know the game so well and still find ourselves connecting dots months later. It's honestly mind-boggling and sort of frustrating! Outer Wilds is ridden with intricacy and lore, so much so that some of it can be easily missed. The difficulty in this game comes in being able to notice the details, and that's what makes it so satisfying to test and to play.
The overall experience, opinion and impression from the team: Charmed.
That’s the End of This Update
Join us in another two weeks for another development update. Have a great weekend everyone!
Now that we have zipped through the solar system, we’re going to take a look at the playable characters alien race, the Hearthians.
The alien race known as the Hearthians are an amphibious race with four eyes, little sense of self preservation, and great curiosity for the outside world. Their home planet is Timber Hearth, which was discussed in a previous update. On their redwoods-inspired home planet, the Hearthians work together through to reach a greater understanding of the solar system they live in.
Porphy, a Hearthian who brews sap wine.
Hearthian culture was inspired by the unusual mix of Apollo-era NASA with backpacking and camping at a national park. Sit out on the porch at sunset and roast marshmallows while you watch your neighbor test out their new model rocket thrusters. Because Timber Hearth has geysers powerful enough to send things into orbit, early Hearthians learned through trial and error how to launch things into space, and it wasn’t long before they started to build ships to explore their tiny solar system.
Marl is more than up to the task of cutting all the lumber required for any new projects.
The Hearthian space program, Outer Wilds Ventures, was founded by Hornfels, Gossan, Slate, and Feldspar. During its early years, Outer Wilds Ventures collected objects of interest (often Nomai in origin) and brought them back home, further enriching the culture and technology of the Hearthians. Many of these technologies have been used to enhance Hearthian exploration technology, granting systems such as artificial gravity or the ability to remotely retrieve a scout with warp technology.
The observatory’s lower floor functions as a museum of curiosities brought back from other locations in the solar system.
As the latest Hearthian traveler, you are the first to be equipped with a tool that will allow you to translate Nomai text - the first of its kind. You and your friend Hal developed this technology to gain a greater understanding of the ancient mysteries that lie dormant in the solar system.
The upper level of the observatory is where Hornfels studies the stars.
Before you can take off on launch day, however, you need to grab the launch codes from Hornfels up at the observatory, so make sure to say your goodbyes to the other villagers as you make your way up. You can play with Mica’s model rocket, hear about the other travelers from Gneiss, or play hide and seek with Tephra and Galena. Make sure to stop by your old flight coach, Gossan, for a refresher on your zero-g training, and check out the nearby scout launcher to get a feel for the planet outside of your home crater. Once you’re ready, grab the launch codes and go explore the Outer Wilds!
Updates on our games, our process, and the joys of being Mobius Digital.