DOOM-Crusher : Crush your PK3/WAD/PNG/JPG files!

• DOOM-Crusher shrinks your PK3, WAD, PNG & JPG files recursively -- if your PK3 contains a WAD that contains a PNG, DOOM-Crusher will optimize it
• DOOM-Crusher remembers which files it's crushed in the past, so that you can keep your WADs/PK3s small without repeating tedious crushing every update/release
• DOOM-Crusher is a Windows Batch file and associated executables. It's easy to use and easy to automate

Instructions: https://github.com/Kroc/DOOM-Crusher
Download: https://github.com/Kroc/DOOM-Crusher/releases

Enjoy!

Wow nice tool, optimization of image files may go a long way.

Nifty little tool. The Readme does a decent job of explaining stuff, as well.

Still, 3-7 hours? On what CPU, exactly? Maybe if you had an atomic toaster, but anything in the last decade or so shouldn't take that long, right?

Since you're dealing mostly with image-based stuff, there's probably the possibility of implementing neural networks/GPU based support, or at least some multithreading (or better usage of threading if you've got it - admittedly haven't run it, so can't tell) and bring that number way down. Otherwise only a handful of people will consider it worth running (overnight), especially as if it uses all CPU power, that will make the PC itself all but unusable.

This is literally an MS-DOS batch file and a whole bunch of already existing executables -- doing it this way was the least amount of hard work required to get something to work without having to spend months to develop a real program for it. Ergo, it runs very slowly because it does not process multiple files at a time. There is a benefit however that it never takes up much CPU and you can even play GZDoom and the like whilst it's whirring away in the background.

The goal is absolute minimum PNG size (lossless) so it can take an extreme amount of time for a PK3/WAD with thousands of PNG files; however since DOOM-Crusher is unique in that it remembers what files have been crushed before (unlike SLADE), it will not repeat work. If you were regularly crushing your project each week whilst you work then only the PNGs that have changed would have to be processed. In the long-term, this is much better than having to process *all* PNGs with SLADE every time you intend to release.

51 minutes ago, Kroc said:

This is literally an MS-DOS batch file and a whole bunch of already existing executables -- doing it this way was the least amount of hard work required to get something to work without having to spend months to develop a real program for it. Ergo, it runs very slowly because it does not process multiple files at a time. There is a benefit however that it never takes up much CPU and you can even play GZDoom and the like whilst it's whirring away in the background.

 

The goal is absolute minimum PNG size (lossless) so it can take an extreme amount of time for a PK3/WAD with thousands of PNG files; however since DOOM-Crusher is unique in that it remembers what files have been crushed before (unlike SLADE), it will not repeat work. If you were regularly crushing your project each week whilst you work then only the PNGs that have changed would have to be processed. In the long-term, this is much better than having to process *all* PNGs with SLADE every time you intend to release.

That much is true. And I suppose if it doesn't use much horsepower it's somewhat more acceptable, though remember that for some folks electricity is pretty expensive, and not everyone likes running their PCs for many hours a day, so an eventualy 2.0 release where you've got some accelerated functionality is still a good idea down the line.

How do you keep track of stuff that's been crushed? With stuff like PNGs it's no problem (just put a custom marker/chunk that says "This got crushed" basically), but JPGs and WAD files are a different deal entirely.

I checksum the files (SHA1) and add them to a text file of already-processed files; easy enough to do with batch-files.

30 minutes ago, Kroc said:

I checksum the files (SHA1) and add them to a text file of already-processed files; easy enough to do with batch-files.

That's pretty smart. Also accounts for if the file changed since then obviously the hash won't match.

Couldn't you just run multiple instances on the the programs (like pngcrush) in parallel to speed things up. Although I guess handling the list of work items could be a bit ugly in batch files.

Within batch code, it would be extremely difficult to do, if at all possible :/ I made DOOM-Crusher for my own purposes (shrinking the size of PortaDOOM releases) so I was not willing to spend more time on it than necessary, but if it becomes popular enough I would consider rewriting it in Golang or Rust.