This week I’ll write about a topic which may seems a little bit austere, but truth is that it can easily become quite fun: testing
How to test a game?
When we talk about test, it may refer to several things. In the case of WinThatWar!, we distinguish three test levels:
- unit test
- integration test
- test in real conditions
Test in real conditions
Let’s start with the last one, it’s the simplest: the best way to test a game is obviously to play it. It allows you (and me also) to find out unexpected issues, not only bugs, but problem of use as well (ergonomy, feeling, gameplay possibilities…)
When you’re coding a game, you can’t stop telling yourself that you really need to improve it and you forget to play it “seriously”. That’s why, at Insane Unity, we schedule regular test sessions of WTW, that we intensified when the release is about to come out.
The problem is, after a while, we know the game really well, we play the same way again and again by force of habit, and because of that we’re becoming unable to hit the broad side of a barn. That’s why we are constantly looking for new users. Newbies allow us to easily find out ergonomy issues and to make a game which will really be easy to play. Talented RTS gamers allow us to estimate how hard to master is the game, and they bring out balancing issues and the efficiency level of the interface.
Once again, I take this opportunity to thank you all, all the alpha-testers. If you weren’t there to help us improving the game, WinThatWar! would be of a much lower quality than it is today. That’s your game too!
Unit test responds to a very simple problem: not to go round and round in circles everytime you’re coding something, undoing what was already working.
Before comitting any change in the code, the developer have to execute tests that will check if every module’s basic fonctionnalities work well. For example, some path-finding tests check that two groups of units can cross without striking each other, some others exist to check that our maths library’s convertion functions produce the expected result, etc.
Generally speaking, a unit test is written BEFORE coding the function it will satisfy. It’s called TDD. This is one of the fundamental practices of agile development. Unit tests are very useful to “re-factorize” the code, in other words to deeply overhaul it without changing its functionality, in order to get it ready for the add of a new feature, or to make it simpler and less subject to bugs.
If a dev ever commit a change that “broke” one of these tests (called regression), he received an e-mail from our buddy Tim (that’s the name of our integration server) that gently insults him, and keeps doing so as long as the problem isn’t solved.
Finally, the integration test takes place between the two other kind of tests I just wrote about. While unit test limits itself to test a module, integration test consists, as the name suggests, on making all the modules working together.
At Insane Unity, that means make 4 AI playing in 2vs.2 every night. Here again, Tim is in charge. Forward, we’ll check that we meet the expected balancing (for example, that AI lvl 5 always beat AI lvl3 in less that 10min.) But, for now, we’re using these tests for two things:
- check there is no crash during the game
- make sure our simulation is desterminist
RTS engine determinism
Determinism, WTH is that? Once again IT engineer gibberish…actullay a program is called determinist if it provides the same result every time it’s launch with the same starting conditions. It may sounds a bit “dumb”, but that’s not that simple. When you launch a program twice in a row on a modern computer, that’s never exactly the same starting state. So, if you ask your computer to do several things at the same time, it can choses, on its own, to launch the tasks in a different order every single time.
So what? Is that really a serious problem? Yes! When two players launch a RTS game, the world state isn’t sent by the network: it would take too long. Instead, we just send every player’s commands. Starting with exactly the same initial situation, and if we apply the exact same order, at the very same time, every players actions, we manage to keep exactly the same state, the same “reality” on every device.
But here it is, if one of the computer choses to execute the operations in a different order, the reality of every player will start to split, and that where the troubles start.
The first versions of WinThatWar! were massively using parallelism to take part of the molticore modern processors. We quickly realised theses kinds of optimizations didn’t get on well with the network mode of the game. If you want to explore further, you can read this excellent article about Age Of Empire.
So, from now on, everytime we use multithreading, we have to make sure the operations order have no impact on the final result. And how to be really sure about that? Of course, we asking Tim for help! I told you that Tim plays WinThatWar! every night, at a rate of ten per night. That’s not all: it plays every game twice in a row, starting from the same seed and checking that every step of the world simulation is exactly in the same situation during both of the two games.
From now on, if you test a LAN pre-version of WinThatWar! with some of your mates, and if you suffer a crash, be curious and go get the log.txt file to watch it’s written on it. Now, you should know what that “Simulation out of sync!” message means. 😉