This is really interesting, thanks!
I wonder if it would be possible to script the DOSbox debugger in some way? Theoretically it would then be possible to dump a file containing eg. the positions of all objects in a level at every tic. This could then be compared with positions in Chocolate Doom. It would be useful both as a regression testing and debugging tool.
I already have a script for doing regression testing against Vanilla over the Compet-N archive set, but this uses a "black-box" style method that is fairly coarsely grained. Having something finer grained that could identify the precise moment a desync occurs could be quite useful.
Sometime back, I gave Quasar a hacked strife exe and corresponding VB app. The hacked Strife.exe was modified to dump the state of the RNG, and the address of the function that called P_Random(). As it ran, it filled up the unused video buffer with this info. The screenshot routine was modified to dump this buffer to file, and reinitialize. Basically, you had to run in DEVMODE, and press the screenshot key every second or so, while the demo was running. Heh!
The VB program took the dumped files, put them together, and did a lookup against the caller addresses, converting them to A_Look(), A_Chase(), etc. By writing similar code in Chocolate, you could quite easily find most desyncs, usually down to the faulty action routine.
It's a watered-down, in-exe equivalent to the sync debugging code that I used to help Maes get his Java port, Mocha Doom, to play original Doom/Doom II/Final Doom demos without desync! I don't think he's released the sync-corrected versions yet - that was a few months ago, and I haven't heard from him for a while. We got all of Doom, Doom II, and TNT working, with one desync in Plutonia. We were nearly finished when I decided to re-write my sync code to have tons of features, and crap. Since then, I haven't heard much from Maes. Real-life and all.
But, yes, imagine it: a Doom port, completely ported to Java, playing synchronized demos! Maes didn't believe it was possible at first. Maybe, he'll chime in to this thread.
Anyway, Quasar got his code working, and synchronized at about the same time that I sent him my tools, so I don't know if he got any use out of them or not. I must admit that I was a bit put off that he did not respond to me afterwards - I put a lot of time researching Strife demo sync, building him that set of tools, writing documentation on how to use them, etc. When he released his first version, I looked for myself in the credits, but I guess, if he didn't use my code, there was no obligation. Not sure why he chose that path, but I'm glad he got his code working.
That being said, if you need it, I can dig it up from my HDD and send you a copy. It's Win32-based, though. However, I could also run it on my machine, and send you dumps of the final output, for you to compare to a dump from an instrumented version of Chocolate. It's easy to do the instrumenting in a source port. Let me know.
Let me just say: This is awesome! Great job!