Weird impy thing
Register | User Profile | Member List | F.A.Q | Privacy Policy | New Blog | Search Forums | Forums Home
Doomworld Forums : Powered by vBulletin version 2.2.5 Doomworld Forums > Classic Doom > Source Ports > Research into DMX yields new info on DOOM sfx format
Pages (2): [1] 2 »  
Author
All times are GMT. The time now is 02:31. Post New Thread    Post A Reply
Quasar
Moderator


Posts: 4615
Registered: 08-00
Seems we've skimmed over an important detail of the DOOM sound effect format all these years.

The first 16 and last 16 bytes of the sample are lead-in and lead-out and aren't meant to be played. They're used as padding so that the sound consists of an even number of "chunks" for DMX to blast to the sound card.

fraggle has found that the minimum length of sample that will play is dependent on the sample rate. For example 11025 Hz samples will not play if they are 48 in length or less. This seems to be because after the 32 lead-in and lead-out bytes are subtracted, only 16 bytes, or one output chunk, is left, and somewhere it requires that there be at least one byte more than this (I have not found where, exactly, but it has to be there, perhaps implicit in the algorithm for data output).

This is why the DSSMFIRE sample in Strife sounds like a pop in Chocolate Strife but is inaudible in the original game - DMX decides the sound has stopped before it ever tries to output any data.
Old Post 02-22-11 19:19 #
Quasar is offline Profile || Blog || PM || Email || Homepage || Search || Add Buddy IP || Edit/Delete || Quote
DaniJ
Senior Member


Posts: 1740
Registered: 08-03
The question is though, how many user authored samples were created with this in mind? From a mod compatibility point of view it seems to me that simply loading the sample as-is wouldn't be enough. However if we know the chunk size expected by DMX it should be possible to determine whether the sample has been padded or not.
Old Post 02-22-11 20:02 #
DaniJ is offline Profile || Blog || PM || Search || Add Buddy IP || Edit/Delete || Quote
Quasar
Moderator


Posts: 4615
Registered: 08-00
DaniJ said:
The question is though, how many user authored samples were created with this in mind? From a mod compatibility point of view it seems to me that simply loading the sample as-is wouldn't be enough. However if we know the chunk size expected by DMX it should be possible to determine whether the sample has been padded or not.

Probably none, however all the samples created with it in mind have a common characteristic - the first 16 bytes are the same as the 17th byte, and the last 16 bytes are the same as the 17th byte from the end - which means they've just been padded out with that information. While it's a bit brute force, it means you can easily detect whether or not that is data, or just garbage.
Old Post 02-22-11 21:58 #
Quasar is offline Profile || Blog || PM || Email || Homepage || Search || Add Buddy IP || Edit/Delete || Quote
RjY
anARCHy


Posts: 802
Registered: 05-02
Oh great, more stupid heuristics and guessing games the engines have to play :-(
Old Post 02-22-11 23:07 #
RjY is offline Profile || Blog || PM || Email || Homepage || Search || Add Buddy IP || Edit/Delete || Quote
Xaser
Senior Member


Posts: 1630
Registered: 07-03
I love how this sort of thing keeps being discovered, even 17 years later. Hell of a game, this. ;)
Old Post 02-22-11 23:59 #
Xaser is offline Profile || Blog || PM || Homepage || Search || Add Buddy IP || Edit/Delete || Quote
kb1
Junior Member


Posts: 167
Registered: 11-06
Quasar said:

Probably none, however all the samples created with it in mind have a common characteristic - the first 16 bytes are the same as the 17th byte, and the last 16 bytes are the same as the 17th byte from the end - which means they've just been padded out with that information. While it's a bit brute force, it means you can easily detect whether or not that is data, or just garbage.
Sounds as if the tools used to write the files just happened to write them that way.

On a broader view, here's my idea of a sweet project:
To "re-master" the Doom source code, with all of the DMX and other DOS-based "goodies" (heh), and use the original tools to compile it exactly like it was originally compiled. Do you think it would be possible to recreate doom.exe, and have it match byte-for-byte? Doing so would make it trivial to discover oddities like the DMX file padding mentioned above, but, basically, I think it would be fricking cool on it's own :)
Old Post 02-23-11 00:00 #
kb1 is offline Profile || Blog || PM || Search || Add Buddy IP || Edit/Delete || Quote
Quasar
Moderator


Posts: 4615
Registered: 08-00
kb1 said:
Sounds as if the tools used to write the files just happened to write them that way.

On a broader view, here's my idea of a sweet project:
To "re-master" the Doom source code, with all of the DMX and other DOS-based "goodies" (heh), and use the original tools to compile it exactly like it was originally compiled. Do you think it would be possible to recreate doom.exe, and have it match byte-for-byte? Doing so would make it trivial to discover oddities like the DMX file padding mentioned above, but, basically, I think it would be fricking cool on it's own :)

Almost certainly not because the version of Watcom that id used just simply isn't available, and the code as-is is not known to compile with Open Watcom.

You could probably reverse engineer a linkable DMX library out of any of the executables, but that's about it. And that alone would be a gigantic undertaking since you'd need to de-relocate all the code and manually construct .lib data structures for it all. And that's not to mention that it's almost impenetrable code. Parts of it are written in inline assembler, other parts are total spaghetti of function pointers calling function pointers, and almost all of it is a miasma of low-level device-specific code.

In the end, what would the purpose be? All you'd do is end up with a file you already had - vanilla DOOM.EXE :P
Old Post 02-23-11 00:32 #
Quasar is offline Profile || Blog || PM || Email || Homepage || Search || Add Buddy IP || Edit/Delete || Quote
kb1
Junior Member


Posts: 167
Registered: 11-06
Quasar said:

In the end, what would the purpose be? All you'd do is end up with a file you already had - vanilla DOOM.EXE :P
There you go spoiling it by asking "Why?". Everyone asks "why":
Why are you on the computer all the time?
Why are you spending all you time working on a free game?
Why not go outside. It's a beautiful day...

Grrr... hehe. :) (Actually, I guess you didn't ask "Why" at all.)
Old Post 02-23-11 01:20 #
kb1 is offline Profile || Blog || PM || Search || Add Buddy IP || Edit/Delete || Quote
fraggle
Super Moderator


Posts: 6000
Registered: 07-00
It's great that you've been able to figure it out, although I wonder if we need to do some further investigation to be completely sure we've properly figured this all out.

On a side note, one thing that I would like to do would be to reverse engineer the DMX library in the same way that you've done with Strife, and get a proper understanding of how everything works so that Chocolate Doom can do all the sound stuff accurately. Apart from the digital sound effects part, this would also allow the final bugs in the OPL playback code to be fixed. I started looking at the OPL code in DMX a while back and made some good progress; unfortunately the SSD on my laptop died and I lost the file containing all my annotations :-(
Old Post 02-23-11 10:17 #
fraggle is offline Profile || Blog || PM || Email || Homepage || Search || Add Buddy IP || Edit/Delete || Quote
Quasar
Moderator


Posts: 4615
Registered: 08-00
fraggle said:
On a side note, one thing that I would like to do would be to reverse engineer the DMX library in the same way that you've done with Strife, and get a proper understanding of how everything works so that Chocolate Doom can do all the sound stuff accurately. Apart from the digital sound effects part, this would also allow the final bugs in the OPL playback code to be fixed. I started looking at the OPL code in DMX a while back and made some good progress; unfortunately the SSD on my laptop died and I lost the file containing all my annotations :-(

Well I'm very certain about the data+24 math it does on the pointer, and the fact it stores length-32 in the channel structures as well. When I get home I'll post the relevant assembly bits.

*IF* you had IDA Pro, I've already made significant inroads into DMX in the Strife 1.2 disassembly, so it would make an excellent starting point. Unfortunately the .idb is a worthless file without the program that reads it.
Old Post 02-23-11 14:16 #
Quasar is offline Profile || Blog || PM || Email || Homepage || Search || Add Buddy IP || Edit/Delete || Quote
fraggle
Super Moderator


Posts: 6000
Registered: 07-00
IDA Pro is what I was using before, so that would be interesting.
Old Post 02-23-11 14:55 #
fraggle is offline Profile || Blog || PM || Email || Homepage || Search || Add Buddy IP || Edit/Delete || Quote
Quasar
Moderator


Posts: 4615
Registered: 08-00
fraggle said:
IDA Pro is what I was using before, so that would be interesting.

http://eternity.mancubus.net/text/DMXcode.c

Relevant Hex-Rays output (and some assembly) for the DMX code, starting from I_StartSound and going only as far as the data originating in that function is placed into the DMX channel structs.
Old Post 02-24-11 00:06 #
Quasar is offline Profile || Blog || PM || Email || Homepage || Search || Add Buddy IP || Edit/Delete || Quote
Enjay
ASK ME ABOUT FOOTBALL / STEAM / DEAD CELEBRITIES / THE BLAIR WITCH PROJECT


Posts: 5596
Registered: 12-00
Would this explain why, "back in the day", sometimes a sound converted to the Doom format using DMAUD would end up having an audible "click" at one end (I forget if it was the start or the finish) despite the original WAV not having one - or would that be something else entirely (eg DMAUD problem)?
Old Post 02-24-11 19:27 #
Enjay is offline Profile || Blog || PM || Email || Homepage || Search || Add Buddy IP || Edit/Delete || Quote
fraggle
Super Moderator


Posts: 6000
Registered: 07-00
Probably a DMAUD bug. As I understand it this new oddity might remove a tiny bit of the sound but it wouldn't add it.
Old Post 02-24-11 21:01 #
fraggle is offline Profile || Blog || PM || Email || Homepage || Search || Add Buddy IP || Edit/Delete || Quote
Enjay
ASK ME ABOUT FOOTBALL / STEAM / DEAD CELEBRITIES / THE BLAIR WITCH PROJECT


Posts: 5596
Registered: 12-00
OK, thanks. I thought that it probably wouldn't be related to this but it's nice to know.
Old Post 02-24-11 21:52 #
Enjay is offline Profile || Blog || PM || Email || Homepage || Search || Add Buddy IP || Edit/Delete || Quote
kb1
Junior Member


Posts: 167
Registered: 11-06
fraggle said:
Probably a DMAUD bug. As I understand it this new oddity might remove a tiny bit of the sound but it wouldn't add it.
Actually, that sounds to me like a very likely cause for a click sound. Ideally, all samples should start at zero-crossing (0 volts/0 potential) and will swing somewhere between max positive and max negative, crossing 0 along the way exactly 2*sample_rate/frequency. So, at a sample rate of 11025hz, a 344hz sound will be at full positive or negative potential 16 bytes into the file.

In other words, those 16 bytes that are not being played contained some frequency of amplitude swings that, instead are being chopped off. The result is that, when the sound is played you go from 0 potential, immediately to whatever potential is at byte 17 - possibly a full-amplitude sample, which is rendered as a square-wave "pop".

Of course the fix is to pad the sample with 16/sample_rate seconds (16 samples) of 0 volts (silence).
Old Post 02-24-11 22:15 #
kb1 is offline Profile || Blog || PM || Search || Add Buddy IP || Edit/Delete || Quote
Enjay
ASK ME ABOUT FOOTBALL / STEAM / DEAD CELEBRITIES / THE BLAIR WITCH PROJECT


Posts: 5596
Registered: 12-00
From what I remember, my "fix" when I got sounds like this would be to add a bit of silence to the end of the WAV file and try again. Usually that worked.
Old Post 02-24-11 22:26 #
Enjay is offline Profile || Blog || PM || Email || Homepage || Search || Add Buddy IP || Edit/Delete || Quote
kb1
Junior Member


Posts: 167
Registered: 11-06
Enjay said:
From what I remember, my "fix" when I got sounds like this would be to add a bit of silence to the end of the WAV file and try again. Usually that worked.
Ah, the click is at the end. Same thing applies to the end. It seems more likely that a custom sample may not end at zero crossing. Padding in that case will eliminate DMAUD as the culprit, but, also, the sample may need to be altered at the end to enusre it ends at zero-crossing. Easiest way is to chop the end at the exact right spot, then, maybe add the 16 samples of padding. That should do it.
Old Post 02-24-11 22:51 #
kb1 is offline Profile || Blog || PM || Search || Add Buddy IP || Edit/Delete || Quote
shadow1013
Junior Member


Posts: 228
Registered: 08-10
theres the function listing which includes the dmx library functions in the heretic source. with one interesting function i found how dmx plays midi type 0: Midi2Mus()
Old Post 05-11-11 02:11 #
shadow1013 is offline Profile || Blog || PM || Email || Search || Add Buddy IP || Edit/Delete || Quote
Quasar
Moderator


Posts: 4615
Registered: 08-00
shadow1013 said:
theres the function listing which includes the dmx library functions in the heretic source. with one interesting function i found how dmx plays midi type 0: Midi2Mus()

Note that the version of DMX in Heretic is much earlier than the one used in either Strife or DOOM 1.9. The .map file in the Heretic source distrib cannot be matched up with those binaries except for a spare few functions that are still in the same order.
Old Post 05-11-11 03:41 #
Quasar is offline Profile || Blog || PM || Email || Homepage || Search || Add Buddy IP || Edit/Delete || Quote
Reaper978
We find his lack of rocket launcher faith disturbing


Posts: 221
Registered: 09-08
Quasar said:
Almost certainly not because the version of Watcom that id used just simply isn't available

Where did it go?

Quasar said:
and the code as-is is not known to compile with Open Watcom.
Do you know why that is?

Quasar said:
In the end, what would the purpose be? All you'd do is end up with a file you already had - vanilla DOOM.EXE :P
Well, you would also be able to create a source port based on the exact code of Doom, rather than the source code annoyingly missing the original audio code.

Last edited by Reaper978 on 05-11-11 at 07:10
Old Post 05-11-11 07:02 #
Reaper978 is offline Profile || Blog || PM || Email || Search || Add Buddy IP || Edit/Delete || Quote
fraggle
Super Moderator


Posts: 6000
Registered: 07-00
shadow1013 said:
theres the function listing which includes the dmx library functions in the heretic source. with one interesting function i found how dmx plays midi type 0: Midi2Mus()
Interesting. I never noticed this tic.map file before. It's a shame that there aren't more symbols in there or it might have been useful for disassembling the DMX code (to get more accurate OPL playback).

It's unsurprising to see Midi2Mus in there. In my investigations I've found that DMX converts MIDI to MUS internally, and Paul Radek who wrote the library was also the author of the Midi2mus command line tool. He almost certainly just integrated the code into the library.

Reaper978 said:
Where did it go?

It was never released.

Well, you would also be able to create a source port based on the exact code of Doom, rather than the source code annoyingly missing the original audio code.
Technically I guess it would be possible to extract the compiled DMX code as a binary blob library and make a port based on the LinuxDoom source that linked against it under DOS. However, such a port would be of very dubious legality. You still wouldn't have the "exact code of Doom" because the LinuxDoom code is missing even the files that interface to DMX.
Old Post 05-11-11 09:56 #
fraggle is offline Profile || Blog || PM || Email || Homepage || Search || Add Buddy IP || Edit/Delete || Quote
Vulture
Junior Member


Posts: 151
Registered: 01-07
fraggle said:
Paul Radek

Has anyone tried contacting this guy to try to get the DMX source released? Is it still used commercially? I tried googling around for information and very few useful results of "DMX sound library" pop up.
Old Post 05-11-11 14:44 #
Vulture is offline Profile || Blog || PM || Search || Add Buddy IP || Edit/Delete || Quote
Quasar
Moderator


Posts: 4615
Registered: 08-00
fraggle said:
Technically I guess it would be possible to extract the compiled DMX code as a binary blob library and make a port based on the LinuxDoom source that linked against it under DOS. However, such a port would be of very dubious legality. You still wouldn't have the "exact code of Doom" because the LinuxDoom code is missing even the files that interface to DMX.

That's where the Heretic & Hexen source, as well as the Strife reverse engineering data, would come in handy :P Strife's i_ code is 100% identical to vanilla DOOM's.
Old Post 05-11-11 14:59 #
Quasar is offline Profile || Blog || PM || Email || Homepage || Search || Add Buddy IP || Edit/Delete || Quote
fraggle
Super Moderator


Posts: 6000
Registered: 07-00
Vulture said:

Has anyone tried contacting this guy to try to get the DMX source released?
Yes.
Old Post 05-11-11 16:09 #
fraggle is offline Profile || Blog || PM || Email || Homepage || Search || Add Buddy IP || Edit/Delete || Quote
shadow1013
Junior Member


Posts: 228
Registered: 08-10
has anyone written an open source version of midi2mus? i find it suprising with so many open source mus utilities including mud2midi, qmus2midi mmus2midi and such that i havent been able to find source for a midi2mus
Old Post 05-11-11 22:43 #
shadow1013 is offline Profile || Blog || PM || Email || Search || Add Buddy IP || Edit/Delete || Quote
shadow1013
Junior Member


Posts: 228
Registered: 08-10
does this mean the 16 bytes after the header or from the beginning of the lump should be ignored?
Old Post 05-11-11 22:45 #
shadow1013 is offline Profile || Blog || PM || Email || Search || Add Buddy IP || Edit/Delete || Quote
Graf Zahl
Why don't I have a custom title by now?!


Posts: 7130
Registered: 01-03
shadow1013 said:
has anyone written an open source version of midi2mus? i find it suprising with so many open source mus utilities including mud2midi, qmus2midi mmus2midi and such that i havent been able to find source for a midi2mus



Maybe that's because nobody really needs it anymore these days. All source ports in existence can handle MIDI and even Doom.exe 1.9 can so why even bother converting. The only reason would be Doom95 but nobody cares about that these days.
Old Post 05-11-11 22:58 #
Graf Zahl is offline Profile || Blog || PM || Email || Search || Add Buddy IP || Edit/Delete || Quote
Csonicgo
This post is probably useless


Posts: 3960
Registered: 03-04
shadow1013 said:
has anyone written an open source version of midi2mus? i find it suprising with so many open source mus utilities including mud2midi, qmus2midi mmus2midi and such that i havent been able to find source for a midi2mus


vanilla doom can handle midi.
Old Post 05-17-11 16:49 #
Csonicgo is offline Profile || Blog || PM || Email || Homepage || Search || Add Buddy IP || Edit/Delete || Quote
natt
Junior Member


Posts: 248
Registered: 05-11
shadow1013 said:
has anyone written an open source version of midi2mus? i find it suprising with so many open source mus utilities including mud2midi, qmus2midi mmus2midi and such that i havent been able to find source for a midi2mus


If you are a programmer, do it yourself. The source to Ben Ryves's mus2mid (as it appears in prboom-plus) is pretty easy to follow, and provided you have a basic understanding of the midi file format coming in, you should come out of reading mus2mid.c with a basic understanding of the mus file format.

Not sure what good it would do you though; what else natively reads mus files other than DMX? (I don't know, since google "mus file" brings up about nine million unrelated formats that happen to share the mus extension).
Old Post 05-17-11 17:12 #
natt is offline Profile || Blog || PM || Search || Add Buddy IP || Edit/Delete || Quote
All times are GMT. The time now is 02:31. Post New Thread    Post A Reply
Pages (2): [1] 2 »  
Doomworld Forums : Powered by vBulletin version 2.2.5 Doomworld Forums > Classic Doom > Source Ports > Research into DMX yields new info on DOOM sfx format

Show Printable Version | Email this Page | Subscribe to this Thread

 

Forum Rules:
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts 		HTML code is OFF
vB code is ON
Smilies are OFF
[IMG] code is ON
 

< Contact Us - Doomworld >

Powered by: vBulletin Version 2.2.5
Copyright ©2000, 2001, Jelsoft Enterprises Limited.

Forums Directory