3xBR-noblend Sprite Upscales [Update 09/21/13]

I already posted an older version of this a while back in "Death Egg's HD Sprite Project" thread (links: doomworld / zdoom). but I did it only for doom and it was splitted in too many different files which was inconvenient to use. this time it's for all iwads and I didn't convert it back to the original palette. so in gzdoom it benefits from better color transitions, in the software renderer it maps it to the original palette anyway. since it was probably overlooked in the original threads I hereby create an official release thread for it.

Update 09/21/2013:
- updated all 3xBR packs with an improved version of the scaler (level3 -> level4)
- since 2xBRh doesn't look too good in the actual game I removed the packs

Update 09/18/2013:
- there were some minor issues with the doom item graphics, missing transperancy and other things.
you can download the item graphics as a separate pack here.

Update 09/16/2013:
- replaced item graphics of doom1&2 with handmade hires ones provided by NightFright.
- added 3xBR packs for all iwads. it only uses colors from the original palette and doesn't blend them thereby keeps the image sharper.

downloads:

(Ultimate) DOOM
DOOM 2
Heretic
Hexen
Strife

(Ultimate) DOOM, DOOM 2 and Heretic requires Revenant100's Sprite Fixing Project to be loaded first:

Doom 2 Minor Sprite Fixing Project - v1.3 Release (updated 6/26/13)
Heretic Minor Sprite Fixing and Widescreen-Friendly Project - v1.0 Release

For everyone that don't know what xBR is: xBR is a pixel shader created by Hyllian over at the bsnes / retroarch forums. It made a huge impression with it's edge interpolation which easily beats hqNx, scaleNx and others. Hyllian helped me and created a standalone executable of this shader, which I used to batch convert all the sprites. yeah that's pretty much it.


here are some examples (3x Nearest Neighbor, 3xBR-noblend):







Feel free to report weird looking sprites but I also appreciate help in fixing them.

I'll never get that whole load order thing. I just ctrl-click everything I want then drag them all onto GZDoom.

Anyway, the first three look great, the robot looks a bit blurry, but I guess its the sprite itself to blame there. I'd rather have seen the acolyte upscaled, as its probably the most recognised enemy from Strife.

I would recomment using ZDL. it's a great tool for managing your source ports and the load order becomes clear.

Nice! I never noticed that his death features him trying to defend himself with his shield, only to have it shatter. I wonder how many other things like that I've missed in these games

The obvious blurring around the edges is why these filtered HQ sprite projects will never compare with manually touched up work. There's just too many little artifacts to consider it as quality work.

yeah in the original thread by death egg there were already a few users who prefered the non-hybrid version of xBR, which gives the monster sprites more the look of clay models. there is also the advantage that it only uses colors from the original image. so edges will be sharp and it's compatible with the original palette.
I'll see if I can release a 3xBR package if there is interest.

Dear god, that's TOO blurry.

sure you don't need glasses?
or is my irony detector defect? lol

Not bad. However I personally still see the same issues as evident in other comparable filters such as hqNx and which are naturally a result of their origins as full screen/image processes designed for emulators.

When used on discrete images with alpha, the fundamental issue is that pixel alpha is considered to be just another color frequency and dealt with in the same manner. I believe that better results can be achieved by using an entirely different process on the alpha data and certainly the forms described by alpha should not influence the interpretation of color forms. This issue is most evident on the Acolyte comparison.

Also, I'd suggest giving the sprite a 1-pixel wide border (or larger depending on the filter kernel size)to prevent the "square edge" issues which are evident in the Acolyte comparison. (Doomsday does the same when hq2x is enabled.)

@DaniJ
sorry for the confusion with this preview, but this was just a little example I made on the fly without considering those things.
in the actual batch process I made sure to extent the image (so that the sprite is not touching the borders) and to replace the alpha color with something that's not interfering with the scaling algorithm.
here is a comparision between the sprites how they are in the download packs (original / 2xBRh / 3xBR-noblend):



notice that I bilinear upscaled the 2xBRh one to 3x, so that they are comparable. the actual file doesn't have blended pixels at the outline of the sprite:
http://i.imgur.com/IzX4106.png

Update 09/16/2013:
- replaced item graphics of doom1&2 with handmade hires ones provided by NightFright.
- added 3xBR packs for all iwads. it only uses colors from the original palette and doesn't blend them thereby keeps the image sharper.

To be honest I have to question the application of this filter in an "offline" context. Such filters are designed to achieve "good enough" results as quickly as possible given they expect to be used in an "online" context, such as an emulator where they must quickly improve a whole frame of potentially random pixel data.

Take this fundamental speed requirement (i.e., time to compute) out of the equation and you then have no reason not to use an algorithm that can achieve much higher quality. Particularly when the results are to be distributed as a resource pack.

I totally get that it may provide some (limited) improvement over comparable scaling filters but this only makes sense when applied in the same context.

I see your point and totally agree with you, but in for a dime in for a dollar? Since I'm interested in the emulation scene I'm fairly informed about recent developments and new shaders. Yeah, there are designed to work in realtime. On the other hand, I don't know of any mentionable scaling algorithms for pixel art outside of this scene. Let alone of scalers which wouldn't work in realtime with a modern cpu on such small images. Usually the ones that are really high demanding regarding perfomance are the ones who're trying to emulate the physical process of cathode ray tubes. And scaling algorithms used in video processing aren't suitable for this kind of images.

So I'm asking you, which available scaling algorithms would you suggest who beat xBR. I'm pretty curious since to my knowledge going by edge interpolation and sharpness xBR is at the top currently (it's advance is very apparent and not limited I might say). Since it's not implemented in doom source ports (unlike hqNx and scaleNx) these packs I created are one solution to use this scaler. So to be honest I fail to see the point why it wouldn't make sense this way if it's the actual result what matters. Of course this becomes totally obsolete as soon as xBR is implemented as a resize option.
Here is a nice introduction and overview about image scaling with a big part dedicated to pixel art:

https://en.wikipedia.org/wiki/Pixel_art_scaling_algorithms#Pixel_art_scaling_algorithms

Well, it sounds like you want a solution to suit your pack rather than a general solution to upscaling pixel art per say. Given that yours is an offline problem you are free to use many different algorithms and on whichever images you deem obtain the best results.

Consequently I can't really answer the question "which is best on pixel art" because there isn't a de-facto solution for all input images. Given that you are free to mix and match there is also no point in me singling out any particular algorithm.

Once you leave behind the confined realm of realtime image processing, the far larger domain of general image processing offers all manner of fundamentally different approaches to upscaling. As I'm sure you are aware this wider field is huge and intensely researched.

https://code.google.com/p/2dimagefilter/
http://research.microsoft.com/en-us/um/people/kopf/pixelart/
http://www.wisdom.weizmann.ac.il/~vision/SingleImageSR.html
http://vectormagic.com/home/comparisons
http://www.sersc.org/journals/IJSIP/vol4_no2/5.pdf
https://biblio.ugent.be/input/download?func=downloadFile&recordOId=1061996&fileOId=1062186

I'm not up to speed on which applications offer which scaling algorithms so I don't think I can help you in that respect.

My point was simply that I think better quality can be achieved given your application.

---

It doesn't "make sense" from the point of view that your application is not constrained by the same requirements for which your chosen solution was designed. I'm not saying that this is pointless because clearly, it achieves an end.

---

Compared in context to other algorithms like hq2X and Super2xSaI the quality improvement is indeed significant.

However, your context is offline therefore I must compare the results with other offline approaches to image upscaling. When compared on this field the improvement of this approach is indeed "limited", in my opinion.

Yeah, I'm familiar with some of these links and papers. Funny you posted the one from microsoft research, cause that's is bascially what inspired xBR to my knowledge. I've never seen a concrete implementation of this paper though. Another thing is, many of these are based on vectorization, which has it's own problems (how to rasterize, curvy corners etc.). And super resolution again is something more suitable for video processing where you can also use the information of neighbored frames and motion vectors (btw there are some really miraculous achievements in the avisynth scene).

Just to be clear, I feel like if I'm defending my packs, which is really not my attention. My motivation for this was that I missed this scaler as an option in zdoom and the available implementation of xBR (and a little help of the author) made it pretty easy to realize it as an addon, which I wanted to share. I'm very open for other algorithms, but like you mentioned I don't see one specific scaler which is "better" in general. And to be honest I'm not gonna invest a huge amount of time combining different scalers to every single sprite, comparing them and picking the best one out when it's probably better invested by talented pixel artists in manual work. I appreciate your contribution though, but you're thinking in a direction which is more than this thread is supposed to be.

However, your context is offline therefore I must compare the results with other offline approaches to image upscaling. When compared on this field the improvement of this approach is indeed "limited", in my opinion.

gimme samples. I think you have vectorization in mind, which again has it's own problems and I wouldn't wanna use them for doom sprites, at least the implemenations I know of.

Indeed the super resolution link got in there by mistake. I simply plucked a random set of links from the relevant folder in my bookmarks.

Clearly you're fully aware of the bigger picture and chose to do this anyway. So I'll shut up :)

No problem, it was a nice conversation.
Have a nice evening/morning/noon ;)

updated the doom packs.

well I just don't come to an end. hyllian updated his xBR shader to level3, which detects lines in more angles. here is a little graphic which explains it:



and to show you that it's really a big improvment which justifies just another update:

3xBR: Level2 - Level3


ps:I removed the 2xBRh packs, since the algorithm generates some weird colored pixels here and there and it doesn't look too good ingame.

Just looking at the new images (posted 22/9?) I think the 2xBR versions looked nicer - but that may have been because of blending?

What does 3xBR w/blend produce? Using the image below, the 2xBR w/blend seems to hit a sweet spot of looking like something that could still be said to be hand made (the shield being the main point of comparison). The 3xBR version moves into the area of looking a bit over processed and not as hand made.



I see what you are saying about the edges and it is indeed noticable on the robot - so maybe the 3xBR needs blending too?

I wonder what a full screen shot looks like in 2xBR w/blend? if thats possible. Or perhaps upscales of textures in 2xBR w/blend?

Good stuff Sp00kyFox :o)

well the noblend version is an easy way to deal with the transparency in the pictures. if the algorithm blends colors, there are pixels at the sprite border which are half transparent which is a problem.
also 2xBRh looks good on paper and especially on big sprites, on smaller ones you see some issues with weird pixels which is just a pain in the ass to fix it all. after some testing I think that 3xBR with bilinear filtering through the source port looks best, give it a try ;)

here is a bad example of 2xBRh:


notice the weird green pixels in the imps face (probably have to zoom in). and this actually happens a lot. ingame it's quite noticeable due to the large display of small sprites.