My name is Aleksandr Pindyk, and I am the Lead Engineer of the AI team at Wargaming studio in Kyiv. For more than five years, our team has been developing AI (artificial intelligence) for one of the most popular online games - World of Tanks. I want to talk more about the profession of an AI developer in game development, the specifics of his work, and how to become one.
First, let's define what game AI is. If we move away from the dry definitions from Wikipedia, the game AI is a system that imitates the behavior of a character or player. Depending on the genre and purpose of the game, different technologies and approaches are used in the development of AI. Let's look at the main types of games and the AI goals.
Games where the goal of AI is to defeat the human player
Let’s take a look at games we all know. A good example here would be chess and Go. This type of game can use artificial intelligence based on neural networks. Since the primary goal of AI is to defeat the player at all costs, it does not need to imitate human behavior. The actions of such an AI often look incomprehensible and sometimes even frighteningly unnatural. Even now, a human cannot defeat AI (AlphaZero) in the same chess and Go.
In addition to these two classic games, experiments with a similar approach are ongoing in card games like poker and in competitive games like StarCraft and DOTA 2. Many scientists and programmers from Google and OpenAI are working on building this. These however would most likely be a cloud solution since a lot of resources are needed to train artificial intelligence.
This approach is not justified in game development because of its complexity and the conflicting goals for which AI is used.
In in-game development, the goal is usually not to beat the player at any cost. And do not forget about server capacities. Another disadvantage of this approach is the need to train the AI again after introducing new mechanics.
PVP games
These are games, the main cycle of which is the confrontation of players or teams with each other (player versus player). One might wonder what the role artificial intelligence is in this scenario? But there are several areas where AI is used in such games:
-
Bots for beginners
They are used both in the initial training and as opponents in the game's early stages. If there is no matchmaker according to the player's skills or when it is impossible to balance the teams, bots allow you to increase the level of difficulty for the players smoothly. And eliminate frequent losses against players with much better skills. Bot training modes in World of Tanks and Hearthstone are good examples.
-
Launching a project or opening new game regions
PUBG Mobile used such a trick at launch. When it was impossible to assemble teams of 100 people, bots threw themselves into battle. Battlefield 2042's 128-player mode uses bots with up to 64 opponents. When the server covers a large territory with different time zones, bots can smooth out the online drawdown for remote regions. For example, when one server covers Europe and Asia.
-
Training mode
It is often needed in games that have been released for a long time and in which there are a lot of pro players, which are challenging to play against without good preparation. In addition, such games do not always separate beginners and pros in a matchmaker, and playing with bots become almost the only way to gain a skill. A good example here would be Escape from Tarkov. While playing with bots, you will not be able to complete missions, but at the same time, you will not lose your equipment, learn the level, and practice equipment combinations and shooting skills.
From a development point of view, such games need to be able to fine-tune the AI difficulty and debug the decision-making logic of the bots. AI itself works on the game server. The development team implements the behavior of bots as much as possible similar to the behavior of a person who knows how to play well. At least bots try to be like such players 😁
PVE games
In the main cycle for these games, the player's interaction is with the environment and AI (player versus environment). These can be both purely single-player offline games (Witcher 3) and online PVE (Genshin Impact), as well as special events and modes in PVP games (Mirny-13 for Halloween in World of Tanks). In such games, AI does not have the goal of defeating the player at any cost. Even vice versa. One excellent game designer said: "The main goal of AI is to give up beautifully." So that a human player can get positive emotions from victory, in this case, AI is one of the elements of setting the complexity of the game.
Balancing between the number of enemies, their "thickness", and the ability to play, you can create the right level of complexity and interest for a person. Also, depending on the game, the importance of the visually logical behavior of the AI bot will increase or decrease. In games like Genshin Impact, bots are just cannon fodder, and the player doesn't always pay attention to how logical they behave.
Another option when using bots is for simple allies and player abilities. For example, mobs in DOTA 2 are pretty primitive and have only one goal - to go forward and attack the enemy. When the game is single-player and offline, AI most often works locally for the player in the client. In the case of games like World of Tanks, AI and PVE modes work on the server. It is necessary because the game is online, and the server controls all actions. It limits the CPU and RAM resources that the AI can use.
One of the critical tasks in developing such artificial intelligence is the ability to create engaging behavioral scenarios for bots. For example, players must get to the position, defend it for some time, and after receiving reinforcements, go on the attack and defeat all enemies. Game AI script can consist of several parts. In the first part, bots are weak, allowing players to get to the desired position quickly. Then, the bots that attack the players become stronger and start using tactics that prevent the players from going on the offensive ahead of time. After moment X, the bots make some mistakes, which allows them to visually, without reducing the complexity, give the players an advantage. Having realized this, they will go on the offensive and be able to win.
A tool that helps in game development
In addition to gameplay, artificial intelligence aids in game development in several areas:
-
Level design and testing - For example, bots can walk or drive across the entire level and thereby reveal impassable areas, jams, and other nuances in the design of the level geometry.
-
Level balance testing - If you fight bots against each other, then by collecting statistics, you can understand how good the new level balance changes are. An essential condition for such a test is the skill of bots. To get the most relevant statistics, bots should play at the level of the average player. The absence of the need for a large team of level testers dramatically speeds up the balance polishing process.
-
Testing new items and skills - The developer can use Bots as test opponents when creating new equipment, items, etc., in the game. For example, you can add bots to the map when introducing a new rifle and check how well it performs its task.
-
Functional and load testing - AI helps to test different game components and even test itself 😀. For example, you can create a scenario in which a bot from each type of weapon will shoot at all types of surfaces and collect statistics on which surface, how it breaks through, and whether the damage is done to the enemy behind the surface. You can load test the client. Send bots to walk around the map and see if there are FPS drops or even the whole game. And for the server, you can perform load testing and thereby check how the server will behave during an “unexpected” peak load.
How AI works in games
We have considered how game AI helps game development, and now we will consider how it works and what approaches are used in its development.
AI works in a cycle: Sense >> Think >> Act to achieve its goals. Whenever new information about the environment appears, it is impossible to make a decision, or the execution of the action is completed/interrupted, AI enters a new round of the Sense >> Think >> Act cycle.
Let's look at each of the major steps in the AI action cycle.
Collection of information (Sense)
First step. AI (agent or bot) collects information about the environment to make any decision. For example, it can be information about opponents and allies. Their positions, condition, equipment, etc.
Decision making (Think)
The next step is the most complex and resource-intensive part of AI. Since all the information collected by the AI agent is processed and analyzed at this stage. Depending on the elective decision-making principle, the complexity of calculations and the accuracy of the result will increase. Consider the main approaches to decision-making:
Decision Trees - Jokingly, you can call it "if" driven development. The easiest way to build AI is as it creates many "if-then" checks. It can also be called a method that instantly reacts to external events. From such checks, a tree is created through the nodes that the AI agent passes until it finds a node in which there will be an action. Usually, the information gathering and analysis steps are combined into one block and are usually an "if" test condition. In this approach, if the check result is "False," then both the transition to the next check and the execution of the action are possible.
Finite state machine - A case when the AI agent is in one of the possible states and has information on which of the states it can go. For example, the bot was in a patrol state. He can go into the state of enemy attack or retreat from this state. The choice of which state to go to may depend on how many hit points the bot has and the ratio of allies/enemies within a radius of X meters. The number of states and transition conditions is not limited. In each new state update cycle, usually every frame (sometimes, to optimize resource consumption, the transition conditions are recalculated less often), the transition conditions for each state are checked. And if the conditions are met, the transition is activated. The advantage of this approach is that we have a clear state transition logic.
Utility System - A system is based on assessing the usefulness of a particular action or transition. The logic is that the AI agent has a set of actions. He chooses which one to perform depending on the current situation on the battlefield, on the inherent priority coefficients for performing an action or transition from one state to another. For example, a bot met the enemy while patrolling the area. It will be more important for this bot not to attack the enemy but to hide, raise the alarm, and call for help from other bots. In this case, his coefficient for avoiding a direct collision will be higher than the desire to attack the enemy. And if the bot does not have a critical advantage over the enemy, it will not choose the behavior branch associated with the attack.
Behavior Trees - It is a convenient add-on for state machines. It allows you to take out state management for the state machines themselves. Thus, we can create a tree of conditions by which we will switch state machines for the AI agent.
-
Goal-Oriented Behavior - It is a concept that allows bots to play, focusing on the most appropriate role and goal for the AI agent and, depending on this, choosing which Behavior Tree to work on. That Behavior Tree determines the conditions for the transition of the bot to different states and the execution of actions.
In this case, the bot uses fuzzy logic based on calculating the coefficient of importance and usefulness of a particular role. For example, a role for a bot with a weapon or a shield. The coefficient for becoming a "tank" will be higher than for the role of scout. Next, after choosing a role, there is a calculation of which goal is now the most useful. If there is a magician who is preparing a spell next to us, then the best goal will be to protect him at all costs. Next, based on the current goal, a Behavior Tree set is selected, which allows the bot to implement the behavior to fulfill the goal.
This approach allows you to create bots that do not obey commands but choose one or another action based on their capabilities, current combat situation, and developers' priorities. We use this approach in World of Tanks when developing AI.
Action (Act) - Final step. It is the implementation of the task. Usually, this includes finding a path and moving along the path, implementing shooting, taking cover, etc.
So, what does an AI developer do? The main task is implementing the Sense >> Think >> Act cycle. The AI developer works closely with other team members since the implementation and optimization of AI behavior algorithms is a joint work of the programmer, game designer, and QA. Let's go through the AI cycle again but from the developer's point of view.
Collection of information (Sense) - n this part of the cycle, the AI programmer is responsible for collecting and storing information about the situation on the battlefield. Together with the game designer, the developer determines what information is collected, how often it is updated, and how it is distributed between AI agents of the same team in the case of a team game.
In team games, some of this data is common to all AI agents, and bots can exchange this information, allowing for greater realism for AI bots. Often the collection of information does not consume many resources, except in cases of frequent ray casts, which the developer should optimize.
Also, some of the information can be static. For example, in World of Tanks, we have created several layers of data on the map. One layer is responsible for shooting from point to point throughout the map. It allows you to understand how much a position is protected or which areas of the map can be controlled from this place.
The other layer tells you how well the area is hidden from detection, which allows you to understand if those bushes are good for a recon tank or not. The other layer is the NavMesh, which allows the bots to move around the map. It is only part of the data created in advance, and in addition to that, we also can do pre-calculations. For example, we can store not only a map of shelters but also against which directions they protect.
For large maps, we can pre-calculate long paths from one point of interest on the map to another. At the same time, you can calculate how good the path is in terms of length, terrain complexity, etc. During the battle, you can already superimpose dynamic information about potential enemies along the route. Thus, you can save a lot on CPU and RAM resources when choosing a path. It is quite an essential and exciting part of AI development. In addition to the apparent benefit from optimization and adding new features to bots, the developer gets an excellent field for development, since such layers need to be invented, and algorithms need to be improved and adapted for their project.
Decision making (Think) - The developer usually does not implement the logic of the bot behavior on his own. He does this together with the game designer.
In addition, the developer implements a system that is responsible for the execution of the Behavior Tree (we can tentatively call the AI engine). Together with the game designer, they design an API for working with the Behavior Tree, and then the developer implements the logic for this API. For example, methods that allow you to calculate the probability of hitting an enemy, the attractiveness of a shelter, or assess the threat in a sector. It is the most resource-intensive part of the AI work.
One of the options for solving the problem of performance and resources is the development of efficient algorithms and the efficient use of statically calculated data. Based on them, AI decides the optimality of specific actions. At the same time, the decision-making system must be sufficiently flexible and stable because it is not always possible to guarantee that the bot has all the available subsystems. Or how well they are adapting to the role played by AI. And sometimes, a well-built AI foundation helps her survive partial damage.
We had a funny case in World of Tanks when we decided to try to use the logic of a conventional tank for artillery, adjusted for the ability to shoot like artillery. We played a regular fight in which our QA decided to prove that he plays better than bots (in fact, he does, but not everything is so simple).
He could bypass the bots, arrive at the enemy base, and decide to destroy all the enemy artillery, three bots. He ignored that he was discovered by enemy vehicles not far from the base. When he entered the line of sight of the artillery, he immediately went to observe the battle in free camera mode. It turned out that the bots, having seen the approach of a dangerous enemy, decided to shoot at him, having calculated in advance the place where he would most likely appear. And as soon as our hero emerged in the line of sight, the bots fired at him. And everything would be fine, but all three artillery did this at the same time. And we discovered an unexpectedly well-coordinated game of bots on artillery, even though it is not implemented yet!
Action (Act) - In this part, the developer implements mechanisms that allow the AI agent to execute instructions after making a decision. Implementing modules responsible for navigation (for example, adapting A * to the game's features), shooting, etc. For example, you can take the A * algorithm and use it in your product. But most often, you will have additional conditions, such as cliffs. If not optimized, then the path for a heavy unit can be built too close to the cliff, leading to its fall. In this case, it is necessary to introduce additional coefficients for approaching a cliff or other inconvenient type of terrain and implement a smooth smoothing of the route. Otherwise, the bots will walk the "ladder." This area is very close to the gameplay, as it uses its mechanics to complete the tasks.
Tools The AI developer, in addition to the implementation of the cycle (Sense >> Think >> Act), creates tools for himself, the game designer, and QA. Sometimes, to implement a particular mechanic, it is easier to create a tool with which you can configure and use the functionality and only then proceed directly to implementing the task itself. A game designer needs to be able to analyze the logic and sequence of AI decisions. For these purposes, we at World of Tanks have created tools that allow you to record all the actions of bots in battle and then create replays that visualize the steps of bots, the sequence of decisions made, and the data based on which these decisions were made.
Developers are also creating systems that allow AI to help develop the game. You can create a toolkit for level designers that will enable you to quickly arrange bots by level and set scenario goals for them.
For example, go through the entire map and check if everything is fine with the passing ability. For QA, it is often necessary to create tools that allow you to check individual subsystems of the game using bots.
In World of Tanks, we use AI to customize how the AI plays on the map. To do this, we have created a statistics collection tool. Next, we change the map, such as how the strategic zones for bots look and where the transfer points between these zones are. Next, 1000-3000 battles are launched on the server. After that, we look at the battle pattern using statistics and heat maps. The game designer adjusts the map settings and repeats them in a circle based on this data. At the same time, using enough server capacity, you can get such statistics for the night of bots and analyze and make changes in the morning.
How it works in practice
Let's use a real example from World of Tanks to look at the complete cycle of creating new functionality using AI. We will also be able to observe how a programmer, game designer, and QA interact with each other.
Let's start with this task: we need bots that will ram the player and explode, causing him, and a small radius around him, damage. Let's call them "Hedgehog." These "Hedgehogs" will go faster than the player under additional conditions. They will not shoot, will be similar in shape to a ball, can be blown up with one shot, must ram only players, and ride in a group of X units. It would seem, what could be easier?
Having received this data from the game designer, the developer remembered the math and how to calculate the lead on the player's movement and together with the game designer created a Behavior Tree logic set that directed the bot at the enemy and made him ram. A particular set of methods was implemented responsibly for undermining and area damage on the programmer's part. To check, they created a special script where they placed one ruff bot and one regular bot.
The first successfully rammed the second, and all was well. We decided to give it to the in-game test within the team and collect statistics. And it showed that the bots on the real map do not reach the players. It turned out that the bots received damage because of the high speed of bouncing on the bumps and when falling. After several such falls, they exploded and caused damage to neighboring ruffs. In turn, it created a chain reaction. I corrected the number of live bots and added a logic scattering drive. I also tweaked the test scenario based on a real map with hills.
After running the second testing stage we realized that the bots were very "polite." The thing is that they rammed the players only along the movement path, i.e., head-on. But if they approached from behind, they began to go around the player on a big arc. Having discussed the options with the game designer, the programmer changed the formulas for choosing the ramming trajectory. After conducting a new series of tests, QA gave feedback that the bots were too fast and accurate, and the players had no chance. Therefore, I went back to the formula and corrected it so that it sometimes gave an error and modified the behavior of the approach to a new circle after the bot missed the player.
The bot's trajectory to turn and ram back increased, allowing players to reload and shoot back at them. It created exciting gameplay moments when bots sometimes flew off the map or sometimes missed and managed to hit them in the last seconds. Sometimes they successfully trapped the player and attacked from different directions. QA appreciated the gameplay experience of encountering ruffs, and this functionality went into the player release.
You can see from this example that throughout the development of the functionality, the programmer, game designer, and QA constantly communicated, which made it more efficient to find any problem areas of the feature and fix them before they got to the players. Also, the programmer and the game designer worked together to develop the algorithm for ramming the player, while the QA gave feedback on how much fun it was to play against such bots. It is a real example from PVE-mode Mirny-13 by our team and the creators of the Silent Hill series.
What it takes to become an AI developer
First, love the games! Seriously, if you don't love games, why even do it?
Second - it's essential to play your product a lot because only by putting yourself in the player's shoes will you understand how he plays and how bots should play.
Otherwise, you'll be "coding" the tasks of a game designer, and you won't be able to become an AI developer that way. Last is the desire to understand why something works this way or that way. AI for complex games doesn't use clear logic, and the result of the Sense >> Think >> Act cycle won't always be 100% predictable and understandable. It's not rare that the developer, together with the game designer and QA must analyze the data on the decisions made by bots to understand whether the AI acted correctly or not. And there's no need for Hard Skills and Soft Skills.
Hard Skills (this is a set of technical knowledge). And the first on the list is knowledge of scripting programming languages, usually written gameplay game logic. For example, in World of Tanks, it is Python, and for Unity, it will be C#. The second essential item will be C++. Without it, it won't be easy because most optimizations are made on it.
Now the next step.
A good understanding of data structures and algorithms. AI development involves a lot of calculations and choosing the best strategy for data storage and processing can save a lot of time and CPU and memory resources. A knowledge of 2nd-year university-level math will also be necessary. Of course, it will be good to know the basic approaches in AI development (the AI cycle and approaches in decision making).
Soft Skills (skills that allow you to work in a team and engage in constructive dialogue). There are quite a few valuable Soft Skills, but in AI development, I would like to single out one:
Teamwork!
Game designers, programmers, and QA work very closely together, and it is often impossible to complete a task or solve a problem without combining their experience and knowledge. That's why skills like listening to other people, discussing rather than arguing, and asking questions are fundamental. Sometimes it's better to ask a game designer or QA about a particular nuance in the game or mechanics instead of inventing and doing it yourself and then painfully redoing it for a long time. Good work of AI in the game is always the merit of the whole team, not just one specialist!
As you can see, creating AI for games is difficult, but it's so much fun and enjoyable! Among other things, this is a good area for developing both Hard Skills and Soft Skills.
Let's make different and exciting games, and AI will help us.