Jump to content
Search In
  • More options...
Find results that contain...
Find results in...
hidfan

Doom Neural Upscale 2X [v 1.0]

Recommended Posts

Hmm, this project seems like it has a lot of potential as a base for doing a full 2x upscale mod. Some of the textures indeed look great post-conversion (e.g. BIGDOOR2 and MARBFAC3), but the rest (including all sprites posted so far) look like smudgy oil paintings. There really isn't any alternative to doing extensive manual touch-ups to the result if the aim is to keep the style Doomy*.

 

Having said all that, this is still an improvement over every upscale project attempt I've seen thus far. :P

 

 

[*well, an "oil painting Doom" mod would actually be cool to see, thinking on it. ;]

Share this post


Link to post
7 hours ago, Xaser said:

Hmm, this project seems like it has a lot of potential as a base for doing a full 2x upscale mod. Some of the textures indeed look great post-conversion (e.g. BIGDOOR2 and MARBFAC3), but the rest (including all sprites posted so far) look like smudgy oil paintings. There really isn't any alternative to doing extensive manual touch-ups to the result if the aim is to keep the style Doomy*.

which of the sprites posted are actually using the nueral upscale? is it just the first zombieman shot with the shotgunner in it for comparison?

Share this post


Link to post

Mg I didnt know that neural network can UPSCALE image! Can you share the .exe?

Btw transparency could be done by omitting all pixels that have color close to cyan by some metric...

Share this post


Link to post
4 hours ago, hidfan said:

XgCB69t.jpg

could you provide individual images or a sprite package plz?

Share this post


Link to post

you can test the first export of items and decorations upscaled here
single images are available here
haven't tested everything yet. mega sphere has an horizontal offset in one frame I think.
 

Edited by hidfan

Share this post


Link to post
14 hours ago, hidfan said:

you can test the first export of items and decorations upscaled here
single images are available here
haven't tested everything yet. mega sphere has an horisontal ofsset in one frame i think.
 

Megasphrere looks good. The invisibility sphere however has a vertical offset issue in one frame and the glowing eye (CEYExxx) and the burning barrel have sone horizontal offset glitches. Other than that pretty awesome. I'm mot sich a big fan of the wall/flat upscales as they seem to look somehow painted to me but the pistol guy and the item sprites are definately shaping up! Plz keep em' comming. Great work!

Share this post


Link to post

Sorry for my biggest offtop in this theme, but...
hidfan, how do You get access to the Nvidia's Solution (I mean Game&Textures in beta status)? I sent out a form with all the data, but nothing came.
Apparently, this is the solution that I really need to increase resolution textures in my project for fork this wonderful game.

Share this post


Link to post

I just asked for an access to the beta.

However It took a while to receive the "you're in" mail :)

 

Share this post


Link to post

This is seriously the best upscaling of traditional pixel art, I have ever seen. I really, really love how the higher res versions retain their "pixel art style" and do not devolve into a blurry mess with hard to distinguish edges (waifu2x examples case in point). Even comparing them to the originals one would assume they are hand drawn. I can't even imagine how those would look on modern maps with GZDoom and higher resolutions.

 

Having that said, I would love to see this project come to fruition and actually be finished one day. I'd love to clean up some of the resulting textures myself, which is not really hard, just a bit time consuming. Which is the crux of the whole thing, I guess. So, if you need manpower count me in, this is just too awesome to be cancelled.

Share this post


Link to post
On 2/7/2018 at 10:16 PM, Reinchard said:

I found that we can have some results even with online converter like waifu2x but only in graphics based on simple shapes and lines like Cacodeamon for example.

Sorry for bumping this but I thought that you guys might be interested. I figured out a way to get rather decent results with waifu2x if the image is smoothed a bit before processing. Here's a quick take on the Doomguy sprites:

x0Yyjso.png

This is already converted to the original palette, the edges might be jaggy a bit, I used a quick colour selection tool to remove the background. The method is described here but basically you need to resize the source image with xBRZ first, then shrink it back to the original size before processing with waifu2x (BTW I use an offline version).

Edited by MrFlibble : Fixed the link to my post where the process is described in more detail. I had mistakenly linked to this very thread instead.

Share this post


Link to post

Don't be sorry for bumping this! Those sprites look actually quite great. I would love to see this upscale come to fruition, since I think this is one of the few ways which improves the Doom textures while still retaining their original charme. I don't like other "high res" texture packs, since they most often than not look out of place and the "high quality resize" mode of GzDoom looks too random.

 

If textures need manual cleanup or something, I would be happy to work on this as well. Just upscale everything in an automated process, send me the files and I'll work my ass of to eventually get this finished. I just love how this looks.

Share this post


Link to post

So I tried to process some zombieman frames to compare with NVIDIA's results posted above:

DbpnIzb.png

 

I realised that I get better results if the image is scaled to 2x straight away instead of 4x and then scaling back. I used the pink background and then cleaned it up manually using GIMP's select by colour tool after converting the image to the original Doom palette. Here's the Doomguy processed in this manner:

3q9Hr5z.png

Share this post


Link to post

Step 1: Gather all Doom resources with the sprite fixed and offest fixed versions. Plus the missing rotations.

 

Step 2: Bulk convert them with Neural 2x and fix the transparency

 

Step 3: Release in a wad so people can use it and pixel artists from the community can work on them at their leisure

Share this post


Link to post

Exactly!

 

So... who‘s doing it? I am more familiar with pixel adjusting and general image cleanup, etc. I will try myself on that Doomguy soon, when I have some time, to show you what I mean. 

 

I really, really want to get this going. ;/

Share this post


Link to post

I figured out how to get proper sprite clipping while taking full advantage of waifu2x scaling. Here's an example:

xPIbRMv.png

 

This is basically similar to the earlier method I proposed with Duke Nukem 3D sprites, and involves resizing both images, except now both are processed by waifu2x. I'll describe it step-by-step.

 

I used the following software: GIMP, mtPaint, KolourPaint (this is optional), waifu2x and xBRZ Scaler Test.

 

1. The method involves processing two images, one with a pink background and another with a grey background (I found that a background made from Duke Nukem 3D's bathroom tiles texture works very well). Prepare a sprite sheet you want to scale, paste one copy over a pink background, another over the grey background of your choice.

 

2. Now the image needs to be softened. If waifu2x scales an image with sharp contrasts between neighbouring pixels (which happens a lot in 8 bit graphics) the results will be bad. Both copies of the sprite sheet will need to be processed in the same way. Use the xBRZ Scaler Test (link above) to scale each image to 4x its original size, save the result and open it in GIMP. Apply pixelate blur at 4 pixel radius, then change image size to its original dimensions with the nearest neighbour interpolation. Save the result.

 

3. Use waifu2x to scale up each of the softened images 2x the original size.

 

4. Convert each image to the Doom palette. I use mtPaint for this, open the image, load palette and select a file containing the Doom palette (e.g. a DOSBox screenshot). Select convert to indexed, choose current palette with no dithering. Save the result.

 

5. Now open both sprite sheets that are converted to the Doom palette. For one with the pink background, use the select by colour tool (Shift + O) to select the background and all pink/purple pixels at the outline of the sprites. Hold Shift to add the areas of desired colour to the selection. There are only four shades in the palette. Copy the selection and paste it over the other image with the grey background, deselect it, then select again and delete to free the sprites from the background.

 

The purpose of this is that while the smoothing method allows waifu2x to properly process sprite edges, it also creates undesired colours at the edges of scaled up sprites because of the background colour's sharp contrast. A neutral grey background is much better in this respect but it is also much harder to remove while keeping the sprites' details intact. So the pink background image is processed to create a mask for removing the grey background in the other image.

 

For reference, here are the two images I used to get the one above (these are raw, prior to smoothing):

pe4rFYk.png WhtHhFz.png

Share this post


Link to post

So, I have tried myself on the Zombieguy. Left picture is the original sprite obviously, in the middle we have @MrFlibble 's upscale and on the very right my "improvements" to it. I mainly tried to add "detail" where it looks too "soft" or "mushy". Mainly the armor and clothes of the guy. Additionally I removed some pixels of the edges as well as tried to add more definition to things. Unfortunately I am not really sure how the gun is supposed to look, so this would still be an area of improvement. Also, I noticed - for the first time in my life *embarrased* - that the Zombieguy is based on the Doomguy sprite. But yeah, makes sense, stupid me. xD But that could speed things up. It took me about an hour for all those changes on this single sprite. T_T

 

Mr. Flibble: Could you also try and upscale an actual texture? I can imagine those would be easier to do at first. The sprites seem a lot of work. Especially once the 16 rotations mod comes out...

 

blaaah.png.6c5bded11e8f41927350fadbfad22dc2.png

Edited by elend : fixed images

Share this post


Link to post
On 10/5/2018 at 4:11 AM, elend said:

I mainly tried to add "detail" where it looks too "soft" or "mushy". Mainly the armor and clothes of the guy. Additionally I removed some pixels of the edges as well as tried to add more definition to things. Unfortunately I am not really sure how the gun is supposed to look, so this would still be an area of improvement.

I think it makes sense to use all the animation frames as references to get a better idea of what each monster looks like. There are some good angles at which more of the rifle is seen — it appears to be somewhat like the Marine Pulse Rifle from Aliens but with a bulkier stock, judging by the unused firing and pain frames.

 

Similarly you removed the grey part near the Zombieman's neck but it's actually part of the vest's collar, which becomes clear when you examine the frames showing his back and sides.


Anyway, I did the entire sprite sheets for Doomguy, Zombieman & the Imp using the sheets from Spriters Resource. I decided not to convert them to the Doom palette yet, I guess they would be more useful in hi-colour for the purposes of editing and maybe also use in modern ports as they are (?). Also I can't vouch that mtPaint does the best job possible at converting the images to indexed palettes.

 

NOTE: It appears that the forum software automatically downscaled the images a bit (these are post attachments), making them unusable.

Spoiler

 

dguybx_trans.png

posbx_trans.png

troopbx_trans.png

 

 

Edited by MrFlibble : Hid the sprite sheets under the spoiler tag

Share this post


Link to post

Oh, yeah, the collar... you are right. Well, of course looking at the other rotations helps a lot. Which is quite nice. Now that I look at those sprite sheets... god damn, that's a lot of work. xD I mean, what do other people think. Do they actually need some cleanup or are they good to go as they are?

 

Other than that, I think the textures and flats should be mostly okay. Would love to see some of those results, too. :p

Share this post


Link to post

Here's a sample of how waifu2x does textures:

Spoiler

DZpItpG.png

This is the unaltered output. I also added some HSV noise in GIMP (at default settings) and converted the result to the Doom palette (BTW I realised that conversion to indexed mode in mtPaint seems to get more accurate if you change colour space from the default sRGB to RGB):

Spoiler

YNzjjOn.png

Doesn't seem much better, but it really depends on how this looks in-game.


I also tried applying noise to the sprites:

Spoiler

PoFtoQ0.png

Again, if there is a difference it'll probably show in-game.

Share this post


Link to post

Hmm, I am a bit dissapointed by the textures. They almost look too "oily" or "smeary" and would need extensive rework to mitigate this a little bit. :( Like adding proper detail (the skull in the poison warning sign) and reworking other details. Adding some noise to the sprites helps indeed a bit, I must say. When I manually reworked the Zombieguy I basically did the same, although to an lesser extend.

 

But by all means, I love your work and that you haven't given up on this yet. It's the only thread on Doomworld I follow, because I am really, really interested in this. :p

Share this post


Link to post

Isn't the Neural 2x method better than waifu2x?

 

 

 

...

 

 

I mean, it sounds so much cooler.

Share this post


Link to post

Yes, they do. In my opinion they retain the "hand pixelled" style better, than the "oily" waifu upscales. That's exactly what got me excited about this in the first place. Although, I have to admit, that the sprites themselves look quite okay with waifu.

 

Here the original Doom door versus the neural upscale, as well as the waifu example for comparison.


 

Spoiler


Original_Neural.jpg.59eb38461c5d25aa71f9416e55d7fbdf.jpgWaifu.jpg.457140077ed6002f6bb3995a39e7e31a.jpg

 

 

Share this post


Link to post

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×