k8vavoom: no good thing ever dies!

51 minutes ago, ketmar said:

and some good news: by commenting one line of code, i managed to reduce RAM consumption for decorate-heavy mods by a factor between 5 or 7. yep, you got it right: before, complex doom+lca were barely fitting into 3 GB of RAM, and now they need only ~500 MB. considering that windows builds are 32 bit, this matters alot. also, this should make some huge difference for Switch port too.

...What was that line of code supposed to do? It sounds like something was seriously wrong if it caused 6x RAM usage.

4 minutes ago, KVELLER said:

What was that line of code supposed to do? It sounds like something was seriously wrong if it caused 6x RAM usage.

no, not wrong, just premature optimisation that bites back.

internally, when creating new class method, compiler pre-allocates a space for 1024 VM instructions. which is (was) fairly good move -- that allowed to avoid memory fragmentation from resizing instruction array, and most functions in VaVoom code are quite big to justfy that number.

but then, decorate comes. basically, if you have to call decorate action, you have to create an anonymous class method. like, `A_SomeAction(Height+42)`, for example, require calculating `Height+42` in runtime, pushing it onto stack, and calling VM method. the easiest way is to create anonymous "wrapper" method for that.

so far so good, but now imagine that we have some decorate-heavy mod, where almost 2/3 of states are calling actions. oops. now we have ALOT of small anonymous methods, each only several VM instructions long. yet we're still allocating a space for 1024 instructions for each one. it easily sums up to gigabytes of unused RAM.

we have MUCH better memory allocators these days, so there is no need to "help" them with such small pre-allocating. so i just removed that, and BOOM! running vanilla wads now require less than 128 MB of RAM, and decorate-heavy wads still fits under 1 GB easily.

10 hours ago, ketmar said:

tnx, @kb1. so you suggesting to implement lockstep-emulation mode for scripts, and if more than one script is going to execute, then execute them in the order they were executed last time (with a smart trick that stores execution time in delay counter). yeah, i thought about that (and even half-implemented that solution), but tbh, i don't like it. not that there is something wrong, but it feels so hacky and dirty, that i just don't want that into the engine.

 

besides, that really helps only a little. all other things are still not "lock-stepped", and while timing issues becomes more... "predictable", in reality it glitches even more. like, HUD messages are not lock-stepped too, for example. so scripts will still miss the timing (many, if not most scripts are creating "fadeout" HUD messages, and just replacing 'em in the very last game tick). so next step will be locking of HUD. while it is not a hard task too, i foresee that next i will find another small thing i'd better lock. and then next one. and... and this is all for want of a nail.

 

thank you for suggesting the idea, though. it is always nice to know that solution you invented is something other people thought about too -- it means that you're not crazy. or at least that other people are crazy like you. ;-) yet for this case i think i'll stay with slightly flickering HUD UI (after all, creating user UI with ACS HUD messages is a gross hack by itself! ;-), and with simplier engine code. meh. after all, The Right Way to create UIs in k8VaVoom is VaVoom C. i know that in the future the world will forget about all other sourceports anyway, i just have to wait a little. ;-)

 

 

p.s.: just to clarify: most problems are with scripts that does `delay(1)` 60 times, for example. this is where error accumulates. "locking" script delays to game ticks solves that, but due to fairly random delta times, HUD updates still missing the target.

Yeah, there's not much you can do with poorly written scripts that don't properly handshake with one another...it sure seems like specialized methods to handle such script synchronizations might alleviate such issues.

However, there should be some documented way for a script programmer to know in what order scripts will execute. In other words, any timing roundoff errors should drift the same amount for each script, so the script programmer is guaranteed that, in such situations, script execution order is maintained.

Stated differently: Roundoff errors in timing should not change execution order. The pseudo-code I provided was an attempt to globalize the roundoff errors into a single variable that all scripts depend on. With the proper care, this idea could spread to your actor free step model,, without damaging the concept. Everything in your engine could feed off of this global timer. You'd have to maintain actor position at the last integer frame position and offset off of that. Like, if your last Frame is 9.75 and your current frame number is 10.1, your delta = (10.1 - 9.75), or 0.35. You'd also maintain an integer delta: deltaInt = (10 - 9) = 1. You'd also maintain 2 sets of actor position variables, like, for X, you'd have X, and XInt. Instead of updating actor->x += (actor->momx * 0.35), you do the following:

actor->XInt += (actor->momx * deltaInt);  deltaInt = 1

actor->X = actor->XInt - (actor->momx * delta);  delta = 0.35

In other words, for each frame, your actor is moved as if your running 35 fps, and then adjusted by the fractional portion of the current frame number. There's no roundoff error, because the actors are first moved the vanilla way (as if frame number is an integer), and then they are adjusted. Any roundoff error is corrected each frame. The timer source for scripts, actors, and everything else is that same global floating point timer. Free step mode still works, but it becomes "synchronized step mode". There's no script or actor position drift, and no accumulated roundoff anywhere.

Again, I haven't seen your code yet, but, when time permits, I am very interested to check it out. Regardless, what I am suggesting is not too difficult to do, but you'd probably have to apply it in a lot of places. It doesn't have to be a dirty hack either. If done cleanly, with a lot of research, it can be very tight and clean, and it can provide perfect sync between scripts and actors. Script programmers can always write crappy script code, but the best you can do is provide a stable, synchronized platform for their scripts. You know your engine much better than me. Doing this right does require some thought and care, but if done right, I know it can work. Just an idea. With what I've seen you do with this engine, I'm sure you could implement it elegantly. Good luck!

currently, script ordering is right (in the sense of compatibility) and predictable -- scripts are run exactly in the order they did so in the original games, and there is no execution order change due to timing. it is just their timing and HUD timing are slightly off. as i don't have strict frame times anymore, scripts that expecting it (strict HUD timing, that is) are missing a point a little. that's why HUD messages in SkullDash EE are working as expected with free stepping, for example (it rely on strict script execution order). it is when i tried to "lockstep" scripts, the ordering broke (due to some scritps missing frames -- tnx to accumulation errors). that could be solved by something you proposed in your previous post, but... too much work and complex code for too little gain.

"free step mode" is ok for game logic part -- as all game logic steps with same delta time (this delta can be different between steps, but it is constant for one given step), so no desyncing is possible there (this is why most ACS-based mods are worked after i implemented missing opcodes). but HUD messages are completely separated from game logic, and lives by their own time -- that is why HUD scripts are off sometimes. but as i said, this is purely cosmetic issue -- some interface blinking, nothing more. 'cause scripts can only ask engine to draw some HUD element, but cannot query its status, so no game logic decisions can be based on that. HUDs are so separate that pausing the game for some time wasn't affecting HUD message timeouts at all. ;-)

most of the time i can just add "0.01" (or something) to HUD timeouts to remove flickering, but this is hack too, of course. ;-)

also, i want to give proper credits: it is Janis who did most of the hard work on VaVoom engine, including converting it to "free step mode". he left a solid thing that is pleasure to work with. i don't think that we can call it "my engine" -- not yet. ;-) that's why i mostly refer to it as "VaVoom", "k8VaVoom" (when i want to point that we're mostly talking about features i broke), or "the engine", but never "my engine". ;-)

That's very noble of you to credit Janis - I imagine he would be thrilled by your respect for his design, and honored by your appreciation. Programmers rarely get honored for their brilliant code structuring. Most people use their compiled exes without ever knowing how awesome the program internals are set up.

You can get to know a person by checking out their code - you're in their head space, more so than most normal human conversation. Please realize that your ability to recognize a pleasing structure earns you some respect in my book, that's for sure!

It may not be "your engine", but neither was the original linux 1.10 doom source code release. You've taken the source, modified it, added new cool stuff, fixed bugs, compiled it, released it, and people have played it. It's yours now!

EDIT: Anyway, it seemed like you were reaching out for a possible solution. If your happy with it, I'm happy with it!

did you ever seen a programmer that is happy with his code? ;-)

not that i am happy, but "it is good enough for now". after all, this is minor issue, and we have much more important things to fix -- like ROR rendering, water portals, and such. they're really affecting gameplay, and there is only 24h in a day+night. ;-)

before starting k8VaVoom, i was waiting for almost 10 years for somebody to pick up VaVoom and continue development, but nobody came. so i had to do it myself.

hey, people, somebody knows how to fork our universe? i have such sights to show you great ideas for it!

intermediate build. as it was said on Mighty No. 9 dev stream, "in the end of the day, it is better than nothing". so, "better then nothing" build is here for you.

* Strife voices pack is optional now
* state label resolver should be case-insensitive
* damage factors in decorate were completely ignored
* MAJOR lowering of RAM usage! before, complex doom+lca were barely fitting into 3 GB of RAM, and now they require less than 500 MB of RAM. yay.
* internal change: moved weapon slot definition to PlayerPawn class, so different player classes can have different weapon slots
* more decorate fixes (i should just make this item permanent)
* Strife dialogs should be visible on all resolutions (it looks ugly, but it is better than nothing)
 

@ReaperAA what exactly is wrong with HUD? besides looking very ugly on various resolutions, of course? for me, both status bar and NPC conversation UI are visible and working (at least for first two NPCs). if something is absolutely broken for you, please, tell me what and how, so i can at least make a workaround. ;-)

2 hours ago, ketmar said:

what exactly is wrong with HUD? besides looking very ugly on various resolutions, of course? for me, both status bar and NPC conversation UI are visible and working (at least for first two NPCs). if something is absolutely broken for you, please, tell me what and how, so i can at least make a workaround. ;-)

Well the status bar was not appearing for me in older build. But with today's build it appears fine so all is good now.

Also how do I load Hexen's mission pack pwad (Hexdd.wad)

38 minutes ago, ReaperAA said:

Well the status bar was not appearing for me in older build. But with today's build it appears fine so all is good now.

please, feel free to report such bugs. i usualy don't test the engine on all resolutions, and HUD code is somewhat broken (i started to rewrite it to "look good in different resolutions", but it is far from finished), and i may just never notice that something is wrong/broken/missing. for now i know that Strife status bar sometimes has a black "hole" between it and actual game screen, this will be fixed. but otherwise it should work. if it doesn't, please, ring the bell. ;-)

37 minutes ago, ReaperAA said:

Also how do I load Hexen's mission pack pwad

i guess that simply starting the game as "-hexen -file hexdd.wad" should work. i am not a Heretic/Hexen fan, so i usually not testing 'em properly. that is, if something is broken, feel free to report it too -- chances are i simply never noticed a breakage. ;-)

14 minutes ago, ketmar said:

"-hexen -file hexdd.wad" should work.

My bad, I was using "-hexdd" and "-game hexdd". It works now.

Also I haven't yet noticed any "black hole" in strife. Im running the game at 1600x900 res.

On 11/28/2018 at 3:53 AM, ketmar said:

that's why i mostly refer to it as "VaVoom", "k8VaVoom" (when i want to point that we're mostly talking about features i broke), or "the engine", but never "my engine". ;-)

Maybe that's a bit too obvious, but why don't you just call it "KaVoom"?

i like the original name. "VaVoom" is... nice. and "k8" is just a shortcut for my name: ketmar -> keith -> k8. when i started the project, it wasn't a full-featured VaVoom "rebirth", i just wanted to make several small fixes, so VaVoom would compile cleanly and work on my current system. and i needed a name for source directory, so without further thinking i just named it "k8vavoom", which simply means "my(k8) modifications to vavoom engine". i usually using this naming scheme when i have no plans to turn project into something bigger. i also created repo on repo.or.cz, 'cause it is my way of doing backups: simply upload my git repo to internet, and let site admin care about proper backups. ;-)

and when i found that the project has grown to full-featured fork, it was too late to invent a new name: several people already seen (and used) it, some people know it under this name, and i was too lazy to find a better one. so it stuck. ;-)

1 hour ago, ReaperAA said:

My bad, I was using "-hexdd" and "-game hexdd". It works now.

great! ;-)

1 hour ago, ReaperAA said:

Also I haven't yet noticed any "black hole" in strife. Im running the game at 1600x900 res.

it will be there if you'll change resolution without restarting the game. also, it is there in some resolutions, but not in other. and i see now why you had UI glitches: "900" as vertical resolution is pathologically low for my current code (something i have to fix): the Real Thing now starts from 1024, anything lower is very badly tested. i don't mean that you did anything wrong, of course -- it is about my lazyiness in testing the engine in resoultions different from one i prefer.

7 hours ago, ketmar said:

"900" as vertical resolution is pathologically low for my current code (something i have to fix): the Real Thing now starts from 1024, anything lower is very badly tested.

Then I'm amazed it worked as good as it did in my 1366x768 display the one time I tried this :P

@ketmar How's this for a compromise?: How about "co-author"? You know, I've found that tightly-woven code is nice to work with, but it requires a programmer to understand the whole system, more so than with average code.

Don't cut yourself short: You must be a great programmer to be able to do the enhancements you're doing on such a tightly-written masterpiece. I see you as nothing short of "co-author", carrying the torch. We all write bugs, but if we persevere, we get 'em fixed, and produce a better product.

14 hours ago, KVELLER said:

Then I'm amazed it worked as good as it did in my 1366x768 display the one time I tried this :P

me too. ;-) the root of all UI problems lies in hardcoded 640x480 virtual screen. original VaVoom forced that vritual resolution for all UI. of course, if you will turn off UI filtering, then anything except intergral ratios will look awful. and there is not much reasons to such hardcode. so i changed the code to use something that is not too far from 640x480, but with integral scaling factors. the bad thing is that 640 and 480 were simply hardcoded across the whole source code as magic numbers. this includes various precalculated numbers too, like `210` instead of `640/3`. so it is hard to automatically find and replace all of this, and in many places code is still assuming 640 and 480.

1280x1024 gives 640x512 -- a very good approximation. coincidentally, this is the resolution i am using to play. so it is fairly good tested. too bad that 1280 is 640*2: most of the old hard-coded values are working flawlessly for me. ;-)

@kb1 over years, while using various FOSS code, i developed a nice skill: "writing patches without being conscious". it helps alot. i also managed to cause alot of radius damage with my first patches: i still wonder how the engine survived some of those. ;-)

as for terms... i think "developer" is good. that is what i am, after all. tbh, i don't care much about mine authorship: i attached my name to the project so people will know who to blame for broken features.

thank you for your words. but, people, don't be distracted: i still prefer posts like: "ketmar, you idiot, you broke XYZ!" ;-)

1 hour ago, ketmar said:

 

@kb1 over years, while using various FOSS code, i developed a nice skill: "writing patches without being conscious". it helps alot. i also managed to cause alot of radius damage with my first patches: i still wonder how the engine survived some of those. ;-)

 

as for terms... i think "developer" is good. that is what i am, after all. tbh, i don't care much about mine authorship: i attached my name to the project so people will know who to blame for broken features.

 

thank you for your words. but, people, don't be distracted: i still prefer posts like: "ketmar, you idiot, you broke XYZ!" ;-)

Heh - my boss still gets pissed off at me for being able to release time-saving enhancements I don't understand, to programs I completely don't understand, for people that think I know what I'm doing. "unconscious patches"... I totally understand that term!

Ketmar, you idiot, you released something that kicks ass! (I guess I need some practice)  :)

"Co-developer"? Roll with it, man. Wear it!

Now, for something serious. About this hard-coded UI: In my home port, I think I did something similar: I scale everything with 640x400 as a base, meaning "full screen". So, to put a dot in the middle of the screen, you'd use (320,200), regardless of the actual display resolution.

Does your UI get smaller as the resolution goes up? Or is your code, like mine, just using 640x480 as a way to normalize your coordinate system?

in the original VaVoom, all UI code was using 640x480, with scaling to real screen size. now the engine tries to use something close to 640x480, but with integral scale factors. for example, for 1280x1024 it will use 640x512. there is a problem, though: for vertical resoultions less than 960, there is no integral scaling factor to get anything close to 480. so you may get 800x300, for example, which distors fonts alot.

(p.s.: i cannot go to less than 600 for horizontal resolution, 'cause VaVoom UI -- menus and such -- are expecting to have at least 640. to change that expectation, i have to write a new UI code, and this is, again, not a highest priority now. yet for vertical resolution i can just use scrollable menus.)

that is, original VaVoom had the same visual UI size regardless of resolution, but that leads to ugly scaled fonts. now UI size deviates slightly, but fonts are nicely scaled.

still, there are several problems with that. ACS, for example, can set arbitrary virtual resolution for HUDs. and many ACS scripts are really doing that, and using images to render various UI elements. now, we have a problem: either follow what ACS want, and have non-integral scales, and ugly fonts, or try to rescale from "ACS resolution" to "engine resolution", and have integral scales, yet broken UI. like, ACS script can set 800x600 virtual resolution, and draw "mission objective" window with nice frames and such. here we realistically want to draw frames in 800x600 virtual resolution, but text in 640x480 (to get integral scale factors for fonts). at least that is what k8VaVoom tries to do. of course, this never works as expected: text is usually goes out of window dimensions, as font size is not the one script author expects.

sure, i can just turn on texture filtering, and scale UI to real screen size. but i prefer to play with texture filtering turned off, and filtered UI looks blurry. GZDoom does exactly that: is filters ACS HUD. i am still not sure what solution to use, though: i don't like filtered UI, but it seems that there is no other way to have non-broken ACS HUD output.

so for now i am using "dynamic virtual resolution" for engine widgets (menus, notification messages, etc.), and combined approach for ACS HUD. the later looks ugly, but meh... i just put that into "known bugs" section.

so, short answer:

37 minutes ago, kb1 said:

Does your UI get smaller as the resolution goes up? Or is your code, like mine, just using 640x480 as a way to normalize your coordinate system?

both. ;-)

I got you. Yes, that's a dilemma. Unfortunately, there's no 100% proper solution. I had to do the integral scaling/centering for the ammo counters and weapon indicators in the original HUD, and that was difficult itself. But to allow the ACS arbitrary virtual res/scaling, with custom images and embedded fonts, the only way you'll match another port's output is to scale exactly the way the other port does it, and to use the same font scaled the same way, blurry or not. You might be able to do scaling with anti-aliased edges, but that's a lot of work, and still not perfectly crisp (though close). I feel your pain.

Currently, I use the 640x400 base simply for positioning, combined with integral scaling w/centering for text and HUD elements. This works well for my built-in HUDs,, and for menus. But I haven't implemented ACS yet, and I don't yet allow any custom HUD stuff. So, maybe that's a future disaster waiting to happen...

yeah. this thing is a major PITA. ;-)

"No good thing ever dies!"

I'm sad that this can't be said about Stephen Hillenburg, creator of SpongeBob. At least he's alive in spirit.

it is always sad when good people dies. due to my location i never watched his creations, but of course i know about him and his work. it is a good thing, so it will never die, i believe.

3 hours ago, Vanessa said:

"No good thing ever dies!"

I'm sad that this can't be said about Stephen Hillenburg, creator of SpongeBob. At least he's alive in spirit.

That is impressively off-topic!

yep. but i love offtopics, why not? free PR.

and to get back to topic: please, do not use "uncapped FPS" in k8VaVoom. as the engine is not lockstep-based, too small time delta will break all physics. there is a reason i didn't put this into option menus. cap at 90 FPS is something you'd better not break.

p.s.: it may seem to work ok in uncapped mode. and most of the time everything will be ok. until something will go terribly wrong due to too small numbers turned into inf/nan.

3 hours ago, ketmar said:

and to get back to topic: please, do not use "uncapped FPS" in k8VaVoom. as the engine is not lockstep-based, too small time delta will break all physics. there is a reason i didn't put this into option menus. cap at 90 FPS is something you'd better not break.

 

p.s.: it may seem to work ok in uncapped mode. and most of the time everything will be ok. until something will go terribly wrong due to too small numbers turned into inf/nan.

hm maybe this is the reason why I was feeling speed issues in the earlier version, ill download the latest build and see how that feels both with and without uncapped fps

Um, where is the fps cap option? Is it only in the cfg, because I can't find it in the options menu

edit: OK so I found the cl_cap_framerate option in the cfg and it was set to 1, I set it to 0 and now it feels better, now its more like the original speed of the game and not twice as fast like it used to be :D

hm. this is definitely wrong, 'cause it should be the other way around. ;-) i can't even imagine what did go wrong in your case. maybe some CPU throttling interferes with engine's timing, and by uncapping frame rate (that is what you did) OS noticed that k8VaVoom is CPU-hungry, and stopped throttling it (and feeding it with invalind timing values to "compensate" throttling). but this is a blind shot.

Yeah I think something funky is going on for sure, when I tried to record an example video with obs 1 felt similar to 0 because my laptop sucks so the FPS was lower when recording. So you cant really see that much of a difference in the video but I can still upload it for you to see if you want

oh yeah also I guess CPU throttling might be a possible culprit considering how low spec it is and the fact I usually don't run it on the highest performance setting to save battery but i'm not sure how I would go about checking that to see if it is the problem