Jump to content
Search In
  • More options...
Find results that contain...
Find results in...
Quasar

Research into DMX yields new info on DOOM sfx format

Recommended Posts

Seems we've skimmed over an important detail of the DOOM sound effect format all these years.

The first 16 and last 16 bytes of the sample are lead-in and lead-out and aren't meant to be played. They're used as padding so that the sound consists of an even number of "chunks" for DMX to blast to the sound card.

fraggle has found that the minimum length of sample that will play is dependent on the sample rate. For example 11025 Hz samples will not play if they are 48 in length or less. This seems to be because after the 32 lead-in and lead-out bytes are subtracted, only 16 bytes, or one output chunk, is left, and somewhere it requires that there be at least one byte more than this (I have not found where, exactly, but it has to be there, perhaps implicit in the algorithm for data output).

This is why the DSSMFIRE sample in Strife sounds like a pop in Chocolate Strife but is inaudible in the original game - DMX decides the sound has stopped before it ever tries to output any data.

Share this post


Link to post

The question is though, how many user authored samples were created with this in mind? From a mod compatibility point of view it seems to me that simply loading the sample as-is wouldn't be enough. However if we know the chunk size expected by DMX it should be possible to determine whether the sample has been padded or not.

Share this post


Link to post
DaniJ said:

The question is though, how many user authored samples were created with this in mind? From a mod compatibility point of view it seems to me that simply loading the sample as-is wouldn't be enough. However if we know the chunk size expected by DMX it should be possible to determine whether the sample has been padded or not.

Probably none, however all the samples created with it in mind have a common characteristic - the first 16 bytes are the same as the 17th byte, and the last 16 bytes are the same as the 17th byte from the end - which means they've just been padded out with that information. While it's a bit brute force, it means you can easily detect whether or not that is data, or just garbage.

Share this post


Link to post

Oh great, more stupid heuristics and guessing games the engines have to play :-(

Share this post


Link to post

I love how this sort of thing keeps being discovered, even 17 years later. Hell of a game, this. ;)

Share this post


Link to post
Quasar said:

Probably none, however all the samples created with it in mind have a common characteristic - the first 16 bytes are the same as the 17th byte, and the last 16 bytes are the same as the 17th byte from the end - which means they've just been padded out with that information. While it's a bit brute force, it means you can easily detect whether or not that is data, or just garbage.

Sounds as if the tools used to write the files just happened to write them that way.

On a broader view, here's my idea of a sweet project:
To "re-master" the Doom source code, with all of the DMX and other DOS-based "goodies" (heh), and use the original tools to compile it exactly like it was originally compiled. Do you think it would be possible to recreate doom.exe, and have it match byte-for-byte? Doing so would make it trivial to discover oddities like the DMX file padding mentioned above, but, basically, I think it would be fricking cool on it's own :)

Share this post


Link to post
kb1 said:

Sounds as if the tools used to write the files just happened to write them that way.

On a broader view, here's my idea of a sweet project:
To "re-master" the Doom source code, with all of the DMX and other DOS-based "goodies" (heh), and use the original tools to compile it exactly like it was originally compiled. Do you think it would be possible to recreate doom.exe, and have it match byte-for-byte? Doing so would make it trivial to discover oddities like the DMX file padding mentioned above, but, basically, I think it would be fricking cool on it's own :)

Almost certainly not because the version of Watcom that id used just simply isn't available, and the code as-is is not known to compile with Open Watcom.

You could probably reverse engineer a linkable DMX library out of any of the executables, but that's about it. And that alone would be a gigantic undertaking since you'd need to de-relocate all the code and manually construct .lib data structures for it all. And that's not to mention that it's almost impenetrable code. Parts of it are written in inline assembler, other parts are total spaghetti of function pointers calling function pointers, and almost all of it is a miasma of low-level device-specific code.

In the end, what would the purpose be? All you'd do is end up with a file you already had - vanilla DOOM.EXE :P

Share this post


Link to post
Quasar said:

In the end, what would the purpose be? All you'd do is end up with a file you already had - vanilla DOOM.EXE :P

There you go spoiling it by asking "Why?". Everyone asks "why":
Why are you on the computer all the time?
Why are you spending all you time working on a free game?
Why not go outside. It's a beautiful day...

Grrr... hehe. :) (Actually, I guess you didn't ask "Why" at all.)

Share this post


Link to post

It's great that you've been able to figure it out, although I wonder if we need to do some further investigation to be completely sure we've properly figured this all out.

On a side note, one thing that I would like to do would be to reverse engineer the DMX library in the same way that you've done with Strife, and get a proper understanding of how everything works so that Chocolate Doom can do all the sound stuff accurately. Apart from the digital sound effects part, this would also allow the final bugs in the OPL playback code to be fixed. I started looking at the OPL code in DMX a while back and made some good progress; unfortunately the SSD on my laptop died and I lost the file containing all my annotations :-(

Share this post


Link to post
fraggle said:

On a side note, one thing that I would like to do would be to reverse engineer the DMX library in the same way that you've done with Strife, and get a proper understanding of how everything works so that Chocolate Doom can do all the sound stuff accurately. Apart from the digital sound effects part, this would also allow the final bugs in the OPL playback code to be fixed. I started looking at the OPL code in DMX a while back and made some good progress; unfortunately the SSD on my laptop died and I lost the file containing all my annotations :-(

Well I'm very certain about the data+24 math it does on the pointer, and the fact it stores length-32 in the channel structures as well. When I get home I'll post the relevant assembly bits.

*IF* you had IDA Pro, I've already made significant inroads into DMX in the Strife 1.2 disassembly, so it would make an excellent starting point. Unfortunately the .idb is a worthless file without the program that reads it.

Share this post


Link to post
fraggle said:

IDA Pro is what I was using before, so that would be interesting.

http://eternity.mancubus.net/text/DMXcode.c

Relevant Hex-Rays output (and some assembly) for the DMX code, starting from I_StartSound and going only as far as the data originating in that function is placed into the DMX channel structs.

Share this post


Link to post

Would this explain why, "back in the day", sometimes a sound converted to the Doom format using DMAUD would end up having an audible "click" at one end (I forget if it was the start or the finish) despite the original WAV not having one - or would that be something else entirely (eg DMAUD problem)?

Share this post


Link to post

Probably a DMAUD bug. As I understand it this new oddity might remove a tiny bit of the sound but it wouldn't add it.

Share this post


Link to post

OK, thanks. I thought that it probably wouldn't be related to this but it's nice to know.

Share this post


Link to post
fraggle said:

Probably a DMAUD bug. As I understand it this new oddity might remove a tiny bit of the sound but it wouldn't add it.

Actually, that sounds to me like a very likely cause for a click sound. Ideally, all samples should start at zero-crossing (0 volts/0 potential) and will swing somewhere between max positive and max negative, crossing 0 along the way exactly 2*sample_rate/frequency. So, at a sample rate of 11025hz, a 344hz sound will be at full positive or negative potential 16 bytes into the file.

In other words, those 16 bytes that are not being played contained some frequency of amplitude swings that, instead are being chopped off. The result is that, when the sound is played you go from 0 potential, immediately to whatever potential is at byte 17 - possibly a full-amplitude sample, which is rendered as a square-wave "pop".

Of course the fix is to pad the sample with 16/sample_rate seconds (16 samples) of 0 volts (silence).

Share this post


Link to post

From what I remember, my "fix" when I got sounds like this would be to add a bit of silence to the end of the WAV file and try again. Usually that worked.

Share this post


Link to post
Enjay said:

From what I remember, my "fix" when I got sounds like this would be to add a bit of silence to the end of the WAV file and try again. Usually that worked.

Ah, the click is at the end. Same thing applies to the end. It seems more likely that a custom sample may not end at zero crossing. Padding in that case will eliminate DMAUD as the culprit, but, also, the sample may need to be altered at the end to enusre it ends at zero-crossing. Easiest way is to chop the end at the exact right spot, then, maybe add the 16 samples of padding. That should do it.

Share this post


Link to post

theres the function listing which includes the dmx library functions in the heretic source. with one interesting function i found how dmx plays midi type 0: Midi2Mus()

Share this post


Link to post
shadow1013 said:

theres the function listing which includes the dmx library functions in the heretic source. with one interesting function i found how dmx plays midi type 0: Midi2Mus()

Note that the version of DMX in Heretic is much earlier than the one used in either Strife or DOOM 1.9. The .map file in the Heretic source distrib cannot be matched up with those binaries except for a spare few functions that are still in the same order.

Share this post


Link to post
Quasar said:

Almost certainly not because the version of Watcom that id used just simply isn't available

Where did it go?

Quasar said:

and the code as-is is not known to compile with Open Watcom.

Do you know why that is?

Quasar said:

In the end, what would the purpose be? All you'd do is end up with a file you already had - vanilla DOOM.EXE :P

Well, you would also be able to create a source port based on the exact code of Doom, rather than the source code annoyingly missing the original audio code.

Share this post


Link to post
shadow1013 said:

theres the function listing which includes the dmx library functions in the heretic source. with one interesting function i found how dmx plays midi type 0: Midi2Mus()

Interesting. I never noticed this tic.map file before. It's a shame that there aren't more symbols in there or it might have been useful for disassembling the DMX code (to get more accurate OPL playback).

It's unsurprising to see Midi2Mus in there. In my investigations I've found that DMX converts MIDI to MUS internally, and Paul Radek who wrote the library was also the author of the Midi2mus command line tool. He almost certainly just integrated the code into the library.

Reaper978 said:

Where did it go?

It was never released.

Well, you would also be able to create a source port based on the exact code of Doom, rather than the source code annoyingly missing the original audio code.

Technically I guess it would be possible to extract the compiled DMX code as a binary blob library and make a port based on the LinuxDoom source that linked against it under DOS. However, such a port would be of very dubious legality. You still wouldn't have the "exact code of Doom" because the LinuxDoom code is missing even the files that interface to DMX.

Share this post


Link to post
fraggle said:

Paul Radek

Has anyone tried contacting this guy to try to get the DMX source released? Is it still used commercially? I tried googling around for information and very few useful results of "DMX sound library" pop up.

Share this post


Link to post
fraggle said:

Technically I guess it would be possible to extract the compiled DMX code as a binary blob library and make a port based on the LinuxDoom source that linked against it under DOS. However, such a port would be of very dubious legality. You still wouldn't have the "exact code of Doom" because the LinuxDoom code is missing even the files that interface to DMX.

That's where the Heretic & Hexen source, as well as the Strife reverse engineering data, would come in handy :P Strife's i_ code is 100% identical to vanilla DOOM's.

Share this post


Link to post
Vulture said:

Has anyone tried contacting this guy to try to get the DMX source released?

Yes.

Share this post


Link to post

has anyone written an open source version of midi2mus? i find it suprising with so many open source mus utilities including mud2midi, qmus2midi mmus2midi and such that i havent been able to find source for a midi2mus

Share this post


Link to post
shadow1013 said:

has anyone written an open source version of midi2mus? i find it suprising with so many open source mus utilities including mud2midi, qmus2midi mmus2midi and such that i havent been able to find source for a midi2mus



Maybe that's because nobody really needs it anymore these days. All source ports in existence can handle MIDI and even Doom.exe 1.9 can so why even bother converting. The only reason would be Doom95 but nobody cares about that these days.

Share this post


Link to post
shadow1013 said:

has anyone written an open source version of midi2mus? i find it suprising with so many open source mus utilities including mud2midi, qmus2midi mmus2midi and such that i havent been able to find source for a midi2mus


vanilla doom can handle midi.

Share this post


Link to post
shadow1013 said:

has anyone written an open source version of midi2mus? i find it suprising with so many open source mus utilities including mud2midi, qmus2midi mmus2midi and such that i havent been able to find source for a midi2mus


If you are a programmer, do it yourself. The source to Ben Ryves's mus2mid (as it appears in prboom-plus) is pretty easy to follow, and provided you have a basic understanding of the midi file format coming in, you should come out of reading mus2mid.c with a basic understanding of the mus file format.

Not sure what good it would do you though; what else natively reads mus files other than DMX? (I don't know, since google "mus file" brings up about nine million unrelated formats that happen to share the mus extension).

Share this post


Link to post

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×