Video Game Testing Automation: Myth or Real Power?
Imagine one of the games you’ve ever played. It can be WoW, Counter-Strike, The Witcher, The Sims, etc. How long can it take to verify the correctness of all the elements manually? How many people should be on the team to check each update and to make sure it doesn’t conflict with previous ones?
That’s right – eternity.
That is why the question of videogame testing automating is so relevant for its developers. The game production industry in this sense is a very promising field, but yet not very experienced in using automation tools, despite the fact that for some of their colleagues from other spheres of development it is mainstream now.
In this article, we will try to dwell a little more on test automation for game developers, and talk about the prospects for its implementation. But at some points, we will have to dispel empty expectations.
Some Basics
Test automation and testing itself are different concepts that are not mutually exclusive, but rather supportive. Manual testing is responsible for developing test cases that identify as many bugs as possible and then performing these cases manually.
Automation allows executing some of these tests by using special tools and scripts. This means that it is relevant for repetitive tasks, allows you to reduce the cost of man-hours at this stage, speeds up the testing process, and therefore the product launch.
Game developers, especially those who use CI/CD methodology in their work, can reap significant benefits from automation. Most of such products are designed for a long lifespan. Therefore, it is often an endless process of testing and releasing patches, extensions, updates, and versions.
More importantly, games always have buyers (gamers) who invest in a product and expect smooth operation, adaptation, and fast bug fixing.
Game Testing Automation: What Can and What Cannot?
What results can you expect from the automation of any process? First of all, of course, its acceleration and cost minimization. When it comes to games, developers expect post-deployment releases to be delivered much more frequently due to the “more tests in less time” formula.
But, this is partly true and not entirely true.
The reason is that automation is often idealized, and given advantages that it does not have. Not every test needs or is suitable for automation. And having paid all the money in the world for the implementation of this service, without receiving the expected results or setting up the process incorrectly, the developer can become frustrated and refuse to automate. And this is also not correct. You just need to distinguish clearly between what automation can and what it cannot, where it is more effective, and where the good old manual is better.
| QA Game Automation | |
| More reliability because of human factor elimination | All manual test can (must) be automated |
| Possibility of test scripts creation and reusage | No need for ramp-up time |
| You can do more tests in less time | Fast financial benefits |
| Resource optimization: you can distribute team and device pool more efficient | If bug reports can be automated, we don’t need stuff |
| Stress and load testing with less number of QA engineers | There are numerous ready-to-use tools that can be purchased of the shelf and don’t need adaptation |
Things to Consider If You Want to Implement Automation in Your Game Testing Process
Production costs
Any implementation is costly. To get the ultimate benefit, you will need to hire new specialists, spend time on writing scripts, adapt tools, educate and reorganize the team. And that is a significant investment. Do not forget that testing is also a product with a code, which may contain its own features and bugs, which will complicate the implementation and increase its cost.
Infrastructure
The only possible way to make tests automated is to have a solid infrastructure. So you should include this point in the production costs. Only the bulletproof system can be integrated with game development and adjust test code to the changes in a gaming product.
Reusability
The ability to create scripts and reuse them is one of the greatest benefits of automation. However, the gaming sector has its own nuance, which makes the possibility of reusability more difficult. Unlike other products, each game is built on its own new code. Therefore, to get the most out of automation, developers and testers will have to “hide” new game code details from the automated tests.
Manual testing
Automation does not relieve you of a team of in-house or independent testers. In the end, some tests simply cannot be automated (Ad hoc for example). Test automation will simply allow you to devote much more time to manual testing of the playability, balance, and other issues that ultimately will lead to an increase in the quality and speed of product release.
The scale of changes
There is a proven fact: the more aspects of testing you want to automate, the more you need to be ready for fundamental changes in operational processes. There are no magic tools, so automaton processes will require long-term investments without fast return, searching for expensive and experienced engineers, and further optimization of the QA team.
What testing processes can be successfully automated and in what kind of games?
Some games can benefit from part-automation, another can successfully implement it full-scale.
- Multiplayer Games: you can use automation for testing recurring actions, server integration, and server load.
- First-Person Shooters: it can be used for imitating a player shooting activity, and player recreating all paths through scenarios to detect clippings, sticky spots, or invisible walls. Also, automation can help in retesting the GUI with every new game build.
- Puzzles: automation is good for testing non-specified steps of gamers (what will happen if the user will make such an unexpected move)
- Mobile games: you can use scripts for testing game screen displays, press actions, and menu actions.
So it is obvious that the game developing industry can have certain advantages from testing automation and that is not a myth. But dev companies shouldn’t consider automation as a magic wand with advantages only. If they want to succeed in it, they should take into account such points as high investment rates, operational changes, production costs and so on. In another way, it can turn out in a huge resource loss.
