How Do monsters choose between melee or their range attacks?

During their walking animations, monsters repeatedly call a function called Chase. Every time the function is called, it checks if the distance between the monster and its target is lesser than melee range, and if so, the monster will always immediately enter its melee attack animation. Otherwise, the function checks if there is a free line of sight to its target, if its "reaction time" is zero, if RNG gives a good value, and a few more special case checks, and if so, the monster will enter its ranged attack animation.

The choice between both attacks is purely distance based. Some ports can set this value per monster, others can not.

The decision whether to shoot a missile or not is a bit more complicated with some randomness involved, but this decision is only being made if a melee attack has already been ruled out.

Some monsters in Heretic have attack functions that call melee first on the condition of distance, if no, then shoot projectile. This is like the HKs, which will melee or throw a projectile based solely on where you are when their attack actually occurs (not when the animation starts). Other monsters, like the rev, behave differently. I'm getting fuzzy at this point but I think a good parallel is the mummy leader from Heretic. Heretic is nice, since we have its source code in a way that isn't totally questionable.

Yeah, in my previous post I was explaining how monsters decide when to begin their melee/ranged attack animations. After the animation begins (whether melee or ranged), it usually plays some "attack preparation" frames, and then comes the frame with the attack function itself. Its behavior depends on the specific attack function: Some always cast a melee attack (Demon/Spectre, Revenant's punch), some always cast a ranged attack (Arachnotron, Mancubus, Cyberdemon, Revenant's missile, hitscanners), and some check the distance to their target at the moment when the attack function is called, and cast a melee attack if the target is close enough that the melee attack will hit it, and cast a ranged attack otherwise (Imp, Cacodemon, Hell Knight/Baron). Actually, the latter monsters in particular don't even have separate animations for melee and ranged attacks, but a single animation for both, and attack functions coded to choose melee or missile only when the immediate attack moment comes.

It mostly depends on the distance between the player and monster. There's usually a small random factor added that can override a decision eg distance=large;attack=range...final=melee.

I can't post any code to support this, as I'm still not that familiar with the code.

Ya, its has thrown me off before as it'll look like they're about to throw a fireball, so I sr in then get punched in the face XD.