Interesting and creepy AI experiment with Q3

No, they cannot learn.

Designing a neural network -pretty much the only widely-studied "learning" computational mechanism available today- is pretty hard: it must have precisely set inputs, output and design goals, and requires a lot of time/steps to train, and most of the time their behavior is marginally better than a traditional, procedurally-designed AI (and it usually starts off as unusably worse).

Just think of an apparently simpler scenario -designing a Doom bot that can find the exit in any level, including finding switches, keys, teleporter shortcuts etc. (this is actually discussed in several bot threads here in DW, just do a search).

It would alreaby be hard to design such a bot procedurally -aka setting out clear-cut rules for its behavior. Having it learn "blindly" by randoming would take a lot of time and an enormous number of "takes" -retries etc.

Quite overkill to include such a mechanism in Q3 bots. They don't even need e.g. a mechanism to detect campers or your preferred weapons, as they can already find you whenever you are simply by asking the engine for your x,y,z coordinates at any time.

The question is if one wants bots that learn to play as well as possible by human rules -including limited field of vision, interface limitations and access to information- or bastards that can dodge even railgun shots by dodging at the very last moment -that can be easily coded-in.

All I remember about the q3 bots is that they were pretty stupid in general. They have a tendency to, in ffa, clump up at the bottom parts of maps and avoid other areas completely, typically areas higher up but not always. Some maps demonstrate this pretty blatantly. You can also see this exact behavior in other q3 engine games that use the bot code.

I know I played a game once in q3dm13 and there is spiral staircase and a hallway just off of the main open area with the quad and I sat in that hallway and got up and left for 15-20 minutes and the bots never touched me. Reason being is that when they spawn they always beeline to the hole in the main area and drop down to try and grab the armor there and the teleporter at the bottom moves them to the opposite side of the map and then they run back to the hole...

Bots usually work no different than the computer opponent in Artillery-type games: a human has to go by successive tries, while the computer could get you all of the time from the very first shot -very evident in e.g. Worms, where the player can't match the computer's aiming with a plain bazooka/grenade even when using homing missiles).

The only thing truly complicating FPS games for bots is movement -too many factors to compensate for simply by having uncanny aiming- and the different characteristics of weapons. The railgun and hitscan weapons in general are the one most suited to the bots' abilities, while weapons that require closing in/surprising/leading ahead are less suitable, because they require adequate combination with movement: an approach/cover strategy, knowledge of spawn spots, etc.

The railgun OTOH just requires one tic of mutual visibility -that's enough for the bot to bag you.

However in an open arena with no cover and with the only determinant factor being marksmanship, bots can destroy any but the most hardcore human opponent from inhuman distances -again, think of Xaero: can anyone truly defeat him by consistently out-sniping him from accross the map? That map is bordering dangerously on the "open arena" situation, but fortunately you can move and take cover -about the only thing that makes it beatable.

Maes said:

No, they cannot learn.

Designing a neural network -pretty much the only widely-studied "learning" computational mechanism available today- is pretty hard: it must have precisely set inputs, output and design goals, and requires a lot of time/steps to train, and most of the time their behavior is marginally better than a traditional, procedurally-designed AI (and it usually starts off as unusably worse).

Just think of an apparently simpler scenario -designing a Doom bot that can find the exit in any level, including finding switches, keys, teleporter shortcuts etc. (this is actually discussed in several bot threads here in DW, just do a search).

It would alreaby be hard to design such a bot procedurally -aka setting out clear-cut rules for its behavior. Having it learn "blindly" by randoming would take a lot of time and an enormous number of "takes" -retries etc.

Quite overkill to include such a mechanism in Q3 bots. They don't even need e.g. a mechanism to detect campers or your preferred weapons, as they can already find you whenever you are simply by asking the engine for your x,y,z coordinates at any time.

The question is if one wants bots that learn to play as well as possible by human rules -including limited field of vision, interface limitations and access to information- or bastards that can dodge even railgun shots by dodging at the very last moment -that can be easily coded-in.

I <3 AI. Maybe this isn't the place to post such a thing, but I'd say to make a Doom bot mimic player activities, clear enemies that are currently a threat, then look for any items not collected that can be collected, then look for switches that haven't been flipped. If there is nothing left to do in the area, move to the next unexplored area. Flipping switches / doors would result in new areas or shortcuts being unlocked.

The x,y,z coordinates of bots knowing where you are is a cheat as you've mentioned before. maybe that's good for very hard difficulty. But I'd prefer a sight cone and a last known area radius coupled with how long it would take the player to leave the radius. But in games with mics, 2 teammates could communicate together and coordinate an attack if not flank. instead of 2 from the front, they can have 1 from front and 1 from behind at the risk of the frontal assault dying. But then 2 coordinating AIs could wait for the behind AI to be in position before the attack begins. Attack from behind, then when the player turns, have the other attack.

About the uncanny aiming, that can all be programmed in and I've seen it happen with bots in games. Like easy = 50% accuracy if that. Hard = 90% accuracy. Very hard = 99% accuracy. Some aimbots I've seen as cheats / hacks have 90% accuracy so people can't say oh you're clearly cheating or they have smooth aimbotting that instead of snapping directly to an object, it will rotate the cursor and shoot. These are hacks similar to TF2's sentries. The sentry knows something is there, and it turns to shoot 100% accurately. You still have a chance with 'smooth aimbotting.'

Another issue for bots is taking cover. Would the bot rather stand out in the open or get to a chest high wall?

All of this means processing power though. So do you want the most advanced humanistic bots ever at the risk of slowing down the gameplay while your PC calculates what their next move is?

geo said:

The x,y,z coordinates of bots knowing where you are is a cheat as you've mentioned before. maybe that's good for very hard difficulty.

Actually, it would be an extra difficulty upon the programmers to make them ignore it and rely only on "visual" and "auditory" cues to locate the player. That would just make the bots appear even more stupid, unless followed by drastic improvements in other departments -most notably human-like "cunning".

geo said:

About the uncanny aiming, that can all be programmed in and I've seen it happen with bots in games. Like easy = 50% accuracy if that. Hard = 90% accuracy. Very hard = 99% accuracy.

Just like knowing where you are, it takes conscious effort by the programmer to make bots miss sometimes. It's easiest to leave them hard (heh).

As a rule of thumb, think of any purely mechanical aspect of playing, like going from place A to place B in a straight line, or aiming precisely at object with coordinates x,y,z: these are tasks very easily coded to be done by bots (and actually it's easier and more natural to make them "perfect"), but only approximated by humans. Bots need to be deliberately "toned down" by the programmers to appear less robotic/mechanical in their approach or become less annoyingly deadly. The more complex the game mechanics, the harder it's for the programmers to make an AI that can exploit all aspects of the game as well as a player -the shortcut is cheating.

But, come think of it, usually a very skilled player appears like cheating to a scrub, so the "shortcut" is not all that bad ;-)

geo said:

Another issue for bots is taking cover. Would the bot rather stand out in the open or get to a chest high wall?

That depends on how well covering works in a particular engine. In games where there's full freeaim, cover is pointless: just your head sticking out of cover is enough to get insta-fragged (usually by a headshot, which is also worth extra damage). In vanilla Doom, some types of cover may be very effective against hitscan weapons or rockets (allowing you to see or aim through them, but not being hit easily with standard autoaim on). On servers with full freeaim however, this may not hold true.

In any case, true AI occupies the same "niche" as jetpacks, all-solar powered vehicles, 100% perfect speech recognition, 100% perfect robotic vision, infinitely parallelizable computers etc. : either "cool but impractical", "stumbling upon some natural or logial law that limits progress beyond a certain point", or "requiring other major, yet unknown intermediate breakthroughs" in other to be feasible. Nobody could envision e.g. a practical space rocket would work when the steam engine itself was still a finnicky affair.

FYI, the Quake 3 bot is largely documented. An example link of Mr Elusive's thesis is here: http://www.cs.rochester.edu/~brown/242/docs/QuakeIII.pdf

It does learn something; it uses genetic algorithms to optimize weapon preferences. I didn't follow that detail closely though. But it doesn't use neural networks.

Maes said:

If they wanted to, they could have coded-in an "evasive maneuver" routine even in vanilla Doom: each monster checks for the movement of projectiles within a certain distance of itself (let's say...2 or 3 map blocks away) and determine if any of them are moving too close to it. If so, temporarily modify its movement pattern in order to avoid them (I think ZDaemon bots kind of do that already, by quickly strafing whenever you shoot at them point blank).

Dodging projectiles? I'm really interested to have that perfected. AutoWolf gets its ass handed to it by Hitler Ghosts because it doesn't have a very good projectile evasion logic. Sometimes it avoids going near jets of fire and succeeds; other times it starts trembling and gets hit (and dies). Other times, because of bad position and careless strafing, it causes the Hitler Ghost to spread-fire in the entire room, trapping the bot and dooming it.

printz said:

It does learn something; it uses genetic algorithms to optimize weapon preferences. I didn't follow that detail closely though. But it doesn't use neural networks.

That can be as simple as keeping a stat telling it with which weapons it stays alive longer or made the most kills -and thus preferring to use those, if available. That's already a crude first approximation of how a genetic algorithm works: trying to favor "successful" options among a multiple choice.

I doubt this is long-term knowledge though -it may be useless in a different map or against different opponents.

Auditory cues are good. Hearing gunfire around the corner, but also smell.... Not all games would need this, but if you have animal AI for a single player, wolves and attack dogs would be able to smell your trail where you've been and follow the trail to you, but really would anyone care about that? Wow the dog finds a line of a trail where you were and follows it only to just get shot by you? When the game can just spawn a dog behind you.

As for 'missing' yes it is easier to make it hit you 100% of the time, but that's where accuracy cones come in. You can have weapon accuracy cones and bot accuracy cones. Does the gun fire absolutely straight or does it fire 0.01% at an angle? Shouldn't be that tough. And even then the player can walk into the inaccurate bullet. Kind of like in Doom when you're shadowed, if I strafe dodge projectiles pre preemptively, I often get hit by what I'd normally dodge. I need to learn to wait to see where the projectile is going and not where the enemy is about to projectile at (while shadowed).

I've heard the aim accuracy cone for the game RIPD is hilariously wide. Shooting right at you, yet the bullets go no where near you.

Even games like Morrowind adapt for strafing. If you're strafing left, the AI will shoot where you will be and not where you are, so you'll need to stop strafing or get hit.

Another issue with hitscan and AI in general is that humans don't have that quick of reflexes. Even pressing a button from a visual cue takes 2 tenths of a second. And well sometimes that 2 tenths of a second makes a difference.

And finally with cover I'd hope all cover would block, but yes dead eye aim can just head shot you, but lesser aim would end up hitting the cover. I'd rather hide behind cover and reload than stand out in the open :-) Uncharted was really good at using cover and coming out of cover in wild angles. Sometimes diving out, our jumping out with a wild crouched down position, peaking out from cover and sniping from over top or the side. It was a nice diversity that felt unpredictable. But that's not really bots so much as enemy AI.

If I were to train AI bots, I'd take cues from how dominant human players behave in MP games. Some ZDaemon maps (e.g. brit10) lead top-players to develop very specific play patterns, which include grabbing specific weapons, spamming specific spots etc. down to questionable tactics like camping -which however still grants you easy kills with least "cost" for you -aka high K/D ratio.

I don't know if such a thing such as a "globally skilled" or "versatile skilled" player truly exists, or if they all rely on shortcuts or map specific/noob-trap tricks to some degree. I'd bet for the second, and teach the AI to try and mimic this behavior, rather than trying to be "globally good".

As I said, a very good player can appear as cheating to a scrub ;-)

And that's good for the bots to know dominant player tactics, but also how to counter act them. So that way top players can be challenged without just 100% accuracy. Like if specific weapons bait in dominant players, instead of just racing for the dominant weapons, use them as bait. If spaming specific spots is an issue, the bots can come from a different area, but that boils down to map design. TF2 map design is good that it prevents camping. As much as someone can sit and kill everyone from 1 direction, there is always an alternate route to take out the campers.

And even if you study dominant players, those dominant players could change their tactics over the months.

I also think as a sense of 'learning' the game can keep track of how many shots a player misses vs hits and adapt. Like if the player is hitting 100% of the time. Then the AI should be too. If the player only hits 1/10 shots then the AI can adapt to that too. If you get better, so do they. Unless its a difficulty setting issue like some hardcore gamer just wants to play on easy for the LOLz or a newb player wants to jump into the deep end and get obliterated.

geo said:

And even if you study dominant players, those dominant players could change their tactics over the months.

Once the optimal tactic for a specific map is discovered, especially in competitive FFA over ZDaemon, it hardly ever changes.

Counter-camping/spamming/griefing tactics also don't help much, because even if you devote your game to stopping one specific "ace" player in his tracks, he will rack up more kills than you do anyway, and still win. This would only work if ALL the other players formed an anti-camper alliance -even if temporary- and gang-up on a single player, exclusively, until he got the message. It's hard to get this level of cooperation even in team DM or CTF, let alone in a FFA.

geo said:

I also think as a sense of 'learning' the game can keep track of how many shots a player misses vs hits and adapt. Like if the player is hitting 100% of the time. Then the AI should be too. If the player only hits 1/10 shots then the AI can adapt to that too. If you get better, so do they. Unless its a difficulty setting issue like some hardcore gamer just wants to play on easy for the LOLz or a newb player wants to jump into the deep end and get obliterated.

This is called Rubberband AI :-)

And now that I know this I will now utterly destroy nightmare Xaero with knowledge that he changes tactics that ultimately fail.

I guess I've seen rubberband AI before in other games. Sometimes to an infuriating level, but never noticed it with FPS bots. Maybe its nothing I really notice though in a FPS. Like how people told me how great Deus Ex's AI is and reacts to sound and stuff. Well I'd just make a sound... then they'd all line up single file and come at me for me to shoot them down instead of trying to flank and surround me. The AI for Half Life Blue Shift had some great AI that would surround and flank you, but that could just be the map setup would encourage something like that.

From the ace player option I guess then the trick is really to get the ace position first. Its sad in a way there is no rock / paper / scissor system to take out an ace.

For an example of how ridiculous games with an unrestricted "adaptive" or "intelligent" component can end up (usually simple rubberbanding/genetic algos), try Warning Forever and Inside a Star-Filled Sky.

In the former, the bosses eventually get so large that you die simply by them occupying the whole screen, and in the latter, the better you get the better the enemies get too (let alone that they have access to some powerups and "behavior modificators" that you don't, like e.g. superior speed than yours, maxed-out powerups to a level you cannot achieve, extreme bullet-dodging skills etc.)

Then again, the concept would be interesting to apply to Doom monsters -make a "generic" type of monster which randomized abilities (speed, attack, health points, number of projectiles etc.) and let the best of them "breed" and see how they evolve level after level of killing them. Again, this would need some restriction on "resources", otherwise all monsters would evolve towards flying high-HP archviles or cyberdemons with infinite rockets ;-)

That sounds like a good project. You can change the color of enemies based on how much HP they have. Change the projectile color too with its speed. Maybe instead of letting things evolve into flying arch viles, keep the enemies as they are, but just upgrade them. Suddenly zombie soldiers machine gun pistol at you and take 2 rockets to kill them. But really having them do more damage seems effective enough to up the challenge.

Then to top that, why just quick rockets, why not 3 at once in a wide spread. Or imps that don't just throw projectiles, but do splash damage or not just splash damage, but a splash that stays. Walk int he splash and take damage until the splash dissipates. Like tear gas in a way. Or even blinding / fogging your screen. But now I'm just throwing out ideas that probably wouldn't be fun.

The real challenge in such projects is thinking beforehand of all things that could/should be parametrized, including e.g. duration or radius of splash damage/explosions, number of projectiles etc.

Also, there should be some limited resource allocated to each base monster class -so that an imp cannot match the power of a cyberdemon no matter how much it evolves.

A good way to start would be to have "base types" being like the ordinary monsters, and allowing some random variation in their base stats. The "breeding" procedure would e.g. show if slightly faster or slightly tougher imps would be more fit to survive, but once again, the sum of all of their abilities should not exceed what their "gene pool" allows. E.g. it might be possible that a random imp is both tougher, faster and more damaging than all the others, in this case it's OK, and it will very slowly rise what can be considered as "average" stats for a population, but there shouldn't be any jumps in stats (e.g. an Imp becoming as strong as a Baron in a single generation).

There are other real-world biology considerations e.g. higher strength and HP should correlate to higher body mass, which should normally translate to lower speed and reaction time -unless strength goes to their feet, in which case it becomes higher moving speed etc.

Allowing x-monster breeding is another possibility, in which parental traits would carry over to the offspring following Mendelian rules (this means e.g. Imp x Pinkie would have a chance to be both fast-walking and throwing projectiles and getting a decent melee attack, but also of getting short-changed e.g. "inheriting" imp's speed and health, pinky's melee delay, and imp's damage, and no projectile).

Hmm...maybe that's an interesting side project...Genetic Doom ;-)

With the parental Doom Imp + Pinkie that would need to be represented with a skin color change like a brown demon or a pink imp. Or an imp with a pinkie head.

At some point years ago I had made something to mix and match image parts amongst enemies like paper dolls. It was never actually programmed into Doom like where barons would hover because half of them would be caco demons.

With Imps going up to Cyber demon levels, I guess you could always do a (enemy baseHP * 1.1% * level of play = enemy HP ) That way a level 10 cyber would always be stronger than a level 10 imp. I guess the catch there would be level 20 imp might be stronger than a level 10 cyber. The level 20 imp would appear red so you know its got more HP. In fact you could have eye color change based on the level and that would be more subtle, because it would look strange having blue imps and red imps.

Otherwise like you said bigger mass could mean more HP. We'd end up with some giant imps.

RPGs with enemy leveling usually keep the enemy level near yours. You can randomize it where it'll be a level under you or a level over you, or if its a boss it will always be 5 levels over your everything else.

I've also played A LOT of Oblige. Their levels of difficulty increasing are more enemies, bigger enemies. Based on what you want when you generate the maps instead of on the fly.

Unfortunately, a sprite-based 2D monster engine would severely limit the possibility of mixing monster visuals, let along scaling or distorting them (well, scaling would actually be possible with a minor hack, so that e.g. stronger imps appear slighty larger than the rest, weaker ones appear smaller etc.). Imagine being swarmed by angry, super-fast micro-imps *lol* In any case, scaling and palette swaps would be pretty much the only way to visually differentiate monsters, otherwise a system of pop-up stats could be used (after all, even RTS games with unit levelling up seldom have different appearances for high-level units, if they are 2D-based).

A vector-based engine would be certainly more flexible, but that'd mean using a fully 3D engine with (special) models.