Out of range MIDI volume bug

[nukeykt]

It is known, that TNT has a MUS lumps with out of range MIDI volumes(map02,map04,etc). Most source ports fixing it during MUS to MIDI convertation, since it is not fixed if the music is in MIDI format.

So i decided to test source ports. I've converted D_STALKS from tnt.wad to MIDI and tweaked volumes from 135 to 144.

And the result:
Vanilla Doom: MIDI plays back correctly
Chocolate Doom 2.2.1: Music broken in native mode, and in OPL with max volume.
Chocolate Doom git: Music broken in native mode, in OPL mode plays back correctly.
ZDoom git: Music broken in OPL and FMOD.

Test wad:
https://drive.google.com/file/d/0B4n8cpbZ9qicSE40Wl9KcENKdTg/view?usp=sharing

[Gez]

Well vanilla converts MIDI to MUS so it makes sense that there's no difference.

MIDI specs say that the value should be betwwen 0 and 127, so your test MIDI file is arguably a broken MIDI file, as the MUS-to-MIDI converter ought to have clamped the value.

[RestlessRodent]

I would say the out of range volume value is undefined behavior which is incorrect no matter the situation. Interpretation of the value depends if it is masked, clamped with a signed value, or clamped with an unsigned value.

[Maes]

Gez said:

Well vanilla converts MIDI to MUS so it makes sense that there's no difference

Is that really what's going on internally, or do they simply delegate MIDI playback as-is to the DMX sound library?

FWIW, the Doom 3: BFG edition's source code does indeed contain the MIDI2MUS utiility's functionality (which, let's not forget, is a third-party tool).

[Gez]

Maes said:

Is that really what's going on internally, or do they simply delegate MIDI playback as-is to the DMX sound library?

As far as I'm concerned, this is a distinction without a difference. The DMX sound library is statically linked into the vanilla Doom executable, if you look at vanilla Doom as a black box there is no relevance to whether the MIDI data is converted by Doom-code or by DMX-code. All that matters is that it's converted to MUS before being played back.

Maes said:

FWIW, the Doom 3: BFG edition's source code does indeed contain the MIDI2MUS utiility's functionality (which, let's not forget, is a third-party tool).

No, it doesn't. What it does contain is MUS2MIDI, which it uses to convert MUS format into MIDI format, so as to get the MIDI data played back by TiMidity.

On the other hand, the DMX library source code, which was discovered recently, does actually contains MIDI2MUS.

[Maes]

Gez said:

On the other hand, the DMX library source code, which was discovered recently, does actually contains MIDI2MUS.

Wait, so did the public MIDI2MUS utility come before DMX, or was it simply adopted by DMX at some point/later version, replacing their own implementation?

EDIT: I noticed that the "source code" also contains a statically linkable .lib file...does this mean that re-making Doom (almost) the way id did it, is now actually possible? With Watcom compiler and all? Or did I miss on obvious old news?

EDIT 2: NM, now I realized that MIDI2MUS was developed by one of id's guys all along. I always believed it was some third party hacker's work. Then that'd make it the first actual utility and partial Doom internal documentation that was "officially" released by someone with ties to id (until then, and for a long time afterwards, everything was done by 3rd party hobbyists).

[Gez]

MIDI2MUS has been in the archive since December 1994, and was possibly around earlier. Support for MIDI files in Doom (meaning, DMX with internal addition of MIDI2MUS) has been there since July 1994 according to the Doom Wiki. The timestamp for midi2mus.c is from April 1994, though I don't know how reliable it can be considered.

The midi2mus.c header boilerplate says the following:

/*===========================================================================
* MIDI2MUS.C - MIDI to MUS format procesor
*----------------------------------------------------------------------------
*                CONFIDENTIAL & PROPRIETARY INFORMATION 			     
*----------------------------------------------------------------------------
*             Copyright (C) 1993 Digital Expressions, Inc. 		     
*                        All Rights Reserved.		   		     
*----------------------------------------------------------------------------
* "This software is furnished under a license and may be used only 	     
* in accordance with the terms of such license and with the inclusion 	     
* of the above copyright notice. This software or any other copies 	     
* thereof may not be provided or otherwise made available to any other 	     
* person. No title to and ownership of the software is hereby transfered."   
*----------------------------------------------------------------------------
*                     Written by Paul J. Radek
*----------------------------------------------------------------------------
* $Log:   F:/DMX/MIDI2MUS/VCS/MIDI2MUS.C_V  $
*  
*     Rev 1.1   29 Oct 1993 00:02:36   pjr
*  Fixed quantinization 
*  Changed default from 280 ticks to 140 ticks per sec.
*  
*  
*     Rev 1.0   01 Sep 1993 16:44:10   pjr
*  Initial revision.
*---------------------------------------------------------------------------*/

September and October 1993, old enough to potentially have been present in vanilla Doom since the shareware 0.99 release. They probably didn't think about using it internally until modding took off.

I've also seen it claimed that before MIDI2MUS, converting the files involved using the IMUSE utility written by Romero (no relationship with LucasArts' own iMUSE). I'm not sure this information is accurate, however; IMUSE was a Commander Keen/Wolfenstein 3D era tool.

[fraggle]

Maes said:

Wait, so did the public MIDI2MUS utility come before DMX, or was it simply adopted by DMX at some point/later version, replacing their own implementation?

Internally DMX is all MUS (I realised this ages ago while reverse engineering). MIDI2MUS was written by Paul Radek initially as a conversion utility and it was presumably incorporated into DMX itself.

[kb1]

GhostlyDeath said:

I would say the out of range volume value is undefined behavior which is incorrect no matter the situation. Interpretation of the value depends if it is masked, clamped with a signed value, or clamped with an unsigned value.

If Vanilla plays it correctly, so should ports, philosophically, invalid spec or not, right?

[Graf Zahl]

Why?

Are there any broken files out there, except those known bad conversions from the bogus Evilution tracks?

[RestlessRodent]

kb1 said:

If Vanilla plays it correctly, so should ports, philosophically, invalid spec or not, right?

For compatibility with Vanilla, it should probably be disassembled to see what it actually does with out of range values and follow any code paths for them. Maybe there is some undocumented DMX thing which treats the upper bits special or similar, it could also be a keyboard or composer specific thing, or it may just be a bug.

[VGA]

?

For me in Chocolate Doom 2.2.1 the song plays. (In native mode.)