Modified prboom+ for producing videos

I guess libmad is a nice library, but it seems a bit light on official documentation. Annoying at times.

I've written the pluginish music loading system and I have the OPL emu and libmad plugging into it. Tomorrow; vorbis and dumb. Anyone have any good test files for dumb? (MOD, IT, S3M, XM).


I've noticed that the OPL synth plays the 'synth drum' track on D_E1M8 much much different than just about every soundfont I've tried. If I remember correctly, it IS being accurate to actual OPL hardware, but it's been a long time since I had my sound blaster 16.


Not-so-random question: What's up with the AllocateMIDIChannel() stuff in mus2mid.c? As far as I know, the general midi spec treats all 15 non-percussion channels the same. Why not just send 9->15, 15->9, and everything else to the same number?

natt said:

I've written the pluginish music loading system and I have the OPL emu and libmad plugging into it. Tomorrow; vorbis and dumb. Anyone have any good test files for dumb? (MOD, IT, S3M, XM).

http://prboom-plus.sf.net/tv.7z
http://prboom-plus.sf.net/valley_of_the_blindbirds.s3m.7z

natt said:

Not-so-random question: What's up with the AllocateMIDIChannel() stuff in mus2mid.c? As far as I know, the general midi spec treats all 15 non-percussion channels the same. Why not just send 9->15, 15->9, and everything else to the same number?

Probably it is not needed currently, because it was a workaround for some bad sounding with timidity ewapath (doom2 map05 iirc). That function is not present in prboom-plus 2.5.0.6 and before. Also, for the latest few versions PrBoom+ distributes with patched sdl_mixer.dll and I think that workaround is not needed anymore for windows users with patched dll (did not check it).

natt said:

Nuts 3 video

You know, it's possible to beat that level without cheating.

Creaphis said:

You know, it's possible to beat that level without cheating.

-_-

@natt: Relative to a fraps .avi file, how large are the RGB or YUV formats?

Qaatar said:

-_-

@natt: Relative to a fraps .avi file, how large are the RGB or YUV formats?

Fraps uses a special lossy codec, and I don't have any good estimates for its filesizes. I can give you the raw RGB filesize though; but you really shouldn't worry about that. The point of this project is that you can pipe to x264 or other advanced encoder directly and nothing ever ends up on your harddrive.

For example, in the nuts video I have linked at the beginning (and keep in mind, this is in full 1080P at 35fps), the total size of the RGB raw stream was over 60GB, but the size of the final output file was 800MB and the total amount of disk space used during the process was about 1.6GB (twice final size due to the second muxing step). And you can always lower the encode quality or decrease the video resolution if you're still not satisfied.

====

I am using the following encoding commands at the moment:

"oggenc2 -r -R %s -q 5 - -o output.ogg"
"x264 -o output.mp4 --crf 22 --muxer mp4 --demuxer raw --input-csp rgb --input-depth 8 --input-res %wx%h --fps 35 -"
"mkvmerge -o %f output.mp4 output.ogg"

I know I said I wanted to stick with something simple, but these work fine with none of the issues in the other setup, and because ffmpeg is so bloated, the combination of oggenc2.exe + x264.exe + mkvmerge.exe + all supporting dlls is only slightly larger than the static link ffmpeg.exe.

The following modules are in place and have passed basic testing:

mid and mus files:

portmidi player, dependency libportmidi (output to system midi output; so could be drivespace synths, sound card hardware, or even midi out port on a soundcard)
fluidsynth renderer, dependency libfluidsynth (uses a lot of cpu!)
OPL emulator, no extra dependencies (can technically play any midi file, but chews a lot of the non-MUS ones up)

other formats:

mad player, dependency libmad, plays MP3 files
dumb player, dependency libdumb, plays various tracker formats
vorbis player, dependency libvorbisfile, plays vorbis embedded in ogg



I learned the whole autoconf thing as best as I could (and I definitely hate it); the configure script detects all the new libraries, and the program should function fine with one or more disabled; those formats just won't get played.

Everything but the portmidi player works with recording capture; the nature of that device means it will never support that. All of the players have file-detection heuristics, which is necessary because you don't get extensions in wad files. It's not perfect though; in particular, as far as the mus2mid code is concerned almost anything is a legal mus file if you disable the check for "MUS\x1a" at the beginning.

I still have to add resampling for the mad and vorbis players, fix the pc speaker soft synth (it currently has an unnecessary SDL mixer requirement), and do some debugging and cleanup. There's also the question of UI+settings; there are a number of options available now (preference in which MIDI player to use, output device selection for portmidi, soundfont selection for fluidsynth) that aren't exposed to the user yet.

Did you drop SDL_mixer for playing music at all or it will be available as option? It's still powerful all-in-one library. On the other hand it requires ton of additional dlls: SDL_mixer.dll + libogg-0.dll + libvorbis-0.dll + libvorbisfile-3.dll + mikmod.dll + smpeg.dll

entryway said:

Did you drop SDL_mixer for playing music at all or it will be available as option? It's still powerful all-in-one library. On the other hand it requires ton of additional dlls: SDL_mixer.dll + libogg-0.dll + libvorbis-0.dll + libvorbisfile-3.dll + mikmod.dll + smpeg.dll

Well, it's your call, but I'm against adding sdl_mixer back in. None of its sfx mixing functionality is ever used (I don't think it ever was?), and none of its music playing functionality can be used with video capture. So if it's in, you end up needing two copies of the music implementation anyway if you're going to capture video.

I changed around the sound threadsafe stuff a bit, and replaced uses of SDL_LockAudio with two separate mutexes, one for sfx and one for music. SFX locks (which occur rather often: whenever new sounds are played etc) no longer have to wait for a running music synthesis callback (which can take quite a while in the case of fluidsynth). It seems like it removed a few hiccups on my machine. Of course, none of this matters for movie capture; the sound and the video are both guaranteed to capture completely and exactly with no hiccups, skips, or desyncs, no matter how cpu intensive things are.

The SDL pc speaker emulation (it's basically a rudimentary softsynth) is working again, and it can be captured to file just like regular SFX. I'd test the win32 native pc speaker emulator (which actually beeps your pc speaker), but the code is disabled with the comment "broken" already.

Just wanted to say I think this is seriously awesome.

eternal slumber said:

Just wanted to say I think this is seriously awesome.

Thanks. Both the video capture code and the new audio code are in the current SVN releases of prboom+. Feedback is appreciated.

Finally decided to give this a try. Pretty cool stuff, the usage.txt was very clear in what you had to do to get this setup for somebody who doesn't know jack about video encoding such as myself.

Mike.Reiner said:

Finally decided to give this a try.

same here
I had no problems in having the output mkv in my folder (even if w/e name I put in strings, it always ends as "output.mkv" doing something wrong here I guess) and its without audio -like written in the usage txt.
I then tried to use the adviced programs to encode, but even if I dled the right ones for my system, it seems they dont want to run or to be opened. (I'm clueless about x264 and mkvtool in & outs anyway)

So I threw the output.mkv and the output.ogg in sony vegas, which I used till now to render the videos (recorded with fraps so far.)
all fine, beside the fact the mkv is missing the BGM music still. Im using portmidi emulation, tried changing it, and with fluidsynth it actually works in either music+ audio. but is that normal? because I hate how the music sound when not-midi emulated.

Remember this patch is obsoleted and you should use the official prboom-plus instead.

Also for what it's worth, I was never able to get the set-by-default x264/mkvtool utilities to work; my own prboom-plus.cfg contains this instead, using ffmpeg:

cap_soundcommand          "ffmpeg -f s16le -ar %s -ac 2 -i - -acodec flac temp.flac"
cap_videocommand          "ffmpeg -f rawvideo -pix_fmt rgb24 -r 35 -s %wx%h -i - -vcodec ffvhuff temp.mkv"
cap_muxcommand            "ffmpeg -i temp.flac -i temp.mkv -vcodec copy -acodec copy %f"
cap_tempfile1             "temp.flac"
cap_tempfile2             "temp.mkv"
cap_remove_tempfiles          1

blob1024 said:

all fine, beside the fact the mkv is missing the BGM music still. Im using portmidi emulation, tried changing it, and with fluidsynth it actually works in either music+ audio. but is that normal? because I hate how the music sound when not-midi emulated.

That is normal. There's no way to correctly capture (in sync and at the appropriate speed) system midi output, in general.

There are loads of different soundfonts available that can make fluidsynth sound like many different things. What midi device are you playing to?

i referred to the standard "portmidi" setting in prBoom actually.
now Im giving a go to "fluidsynth and /sdlopl2 (which Ive absolutely no idea what those are)

but i prolly didnt catch your question :p

blob1024 said:

i referred to the standard "portmidi" setting in prBoom actually.
now Im giving a go to "fluidsynth and /sdlopl2 (which Ive absolutely no idea what those are)

but i prolly didnt catch your question :p

When you use "portmidi", you're outputting to system midi, which can go to any of a number of devices that sound way different than each other, and I have no way of knowing which without more information. Without that, it's hard to say how you could make fluidsynth sound more like portmidi.

CT4MGM.SF2
It is absolutely similar to Windows MIDI

okay, gosh. I feel like I do need a guide for dummies on this now. cus cant follow any of those latest posts lol ;(

It doesn't translate 1-to-1 to PrBoom+, but this article on the ZDoom wiki might help you understand what's the difference between FluidSynth, OPL emulation, system MIDI (through PortMidi), and the rest.

Hey blob, check out this post: http://www.doomworld.com/vb/post/1032417

If you want the music with fluidsynth to sound like SDL which uses Windows' General Midi, I've converted it to a soundfont and uploaded it here
Put gm.sf2 in PrBoom+ directory and change the snd_soundfont variable in the config file to "gm.sf2"