Bauul Posted December 21, 2019 (edited) Last month the Sonic editing community produced a tool designed to interpolate the animation of sprites. The original thread is here. Thanks to @TheMisterCat for sharing the link! It's best suited for sprites that have distinct moving parts, but out of interest I loaded in a couple of Doom sprites to see what the results were. Each of these took about an hour to rig up. The Arachnotron, which has quite distinct moving parts, animates quite well. There are some small artifacts where the moving parts uncover new areas of the sprite, but it's an interesting look at an higher-frame rate animation. Original for comparison: Unfortunately it doesn't work well for more organically animated monsters. I had a go at the demon and it caused far more artifacts. It could be viable with manual tweaking, but ultimately other approaches would be likely more efficient. Original for comparison: All in all it's probably not the ideal approach for producing hi-frame-rate versions of the animations, but it's an interest experiment nonetheless. Edit: Made a bit of progress preventing the detaching body parts. Still a few graphical glitches, but it's a step in the right direction. Edited December 24, 2019 by Bauul 42 Share this post Link to post
Linguica Posted December 21, 2019 6 minutes ago, Bauul said: All in all it's probably not the ideal approach for producing hi-frame-rate versions of the animations, but it's an interest experiment nonetheless. You say that, but with some further development, it might be possible to take intermediate output from this and then use a neural net to generate convincing pose frames. 7 Share this post Link to post
RonnieJamesDiner Posted December 21, 2019 There's something genuinely spooky about that Pinky. As though it's some kind of re-animated pile of body parts and gore, loosely assembled into a new demonic form and held "together" by the powers of Hell. This is a pretty neat find though, that Arachnotron looks great! 5 Share this post Link to post
boris Posted December 21, 2019 Saw info about this program some time ago on a Discord server. Looks pretty cool. Did you try rigging one of the front animations? I suspect that the results will not be as good as side views. 0 Share this post Link to post
Bauul Posted December 22, 2019 38 minutes ago, boris said: Saw info about this program some time ago on a Discord server. Looks pretty cool. Did you try rigging one of the front animations? I suspect that the results will not be as good as side views. I had a quick go with the Imp. It worked out better than I expected actually, but I can't work out how to prevent the limps from detaching like that. Good for a puppet, not so good for an Imp. 6 Share this post Link to post
elend Posted December 22, 2019 Haha that pinky is hilarious indeed. While certainly not perfect it could serve as a base for artists to draw over, saving maybe a bit of time. Give it a couple of more years and results like that will look better, I guess. 0 Share this post Link to post
OniriA Posted December 22, 2019 (edited) This could be a great addition to something like Smooth Doom if these were to be fully worked out. Even if they look incomplete and detached in the examples, I can already picture them as a whole in my mind, and they seem to be really coming to life in this way. Almost hauntingly so. 0 Share this post Link to post
david_a Posted December 22, 2019 Does this tool always linearly interpolate frames? I don’t know the name of this concept, but I know a big thing in hand-drawn animation is that quick motions aren’t animated evenly. Imagine a character doing a sword swipe or something; you’re not going to show a bunch of intermediate frames there because the motion is too fast. One example in Doom that comes to mind is the Baron swiping at you. 0 Share this post Link to post
HorrorMovieRei Posted December 22, 2019 I reckon that filling in the holes left by the algorithm shouldn't be that hard, given that there's enough on both sides usually. I wonder how well it would work for speeding up the creation of walk animations. As in, if you only did one frame(the front frame), then just mirrored it and interpolated the two to create a rough sketch of the intermediary frames. I see a lot of potential in this tool, I'm frankly just waiting for an opportunity to use it 0 Share this post Link to post
Stopsignal Posted December 22, 2019 (edited) This is ridiculously good. As an artist, I can tell you these are really good bases to just continue drawing over. Shouldn't be hard having most of the animation work done! Probs not gonna use it out of pride, but still fantastic. Props to the devs! EDIT: Also a really good tool for people that don't know how to animate to see why their low fps animations aren't exactly working: the interpolation shows everything nicely. Animators should realize just by looking, but for everyone else it's awesome! 3 Share this post Link to post
Bauul Posted December 22, 2019 4 hours ago, david_a said: Does this tool always linearly interpolate frames? I don’t know the name of this concept, but I know a big thing in hand-drawn animation is that quick motions aren’t animated evenly. Imagine a character doing a sword swipe or something; you’re not going to show a bunch of intermediate frames there because the motion is too fast. One example in Doom that comes to mind is the Baron swiping at you. Yep, it's entirely linear. Although my observation is id did a pretty good job of having consistent movement distance based on frame delay, so it might look ok. If I can work out how to fix the marionette issue, I'll give it a go. 0 Share this post Link to post
P41R47 Posted December 22, 2019 Wow they give me the same feel as Mouldy's animations! 0 Share this post Link to post
Tormentor667 Posted December 22, 2019 This is extremely impressive. I mean with a little bit of fine tuning it works 1 Share this post Link to post
Dark Pulse Posted December 23, 2019 I have a feeling that someone can take the output of this, clean it up, and turn it into something very usable for the Doom community. Here's an interesting thought: How well does it work with the weapons? Or better yet, how well does it work with Smooth Doom's weapons? And then of course we get to stuff like Doom 64... possibilities start quickly becoming interesting. 0 Share this post Link to post
elend Posted December 23, 2019 I was wondering just now, how it compares to Smooth Doom. I thought Smooth Doom looked already quite perfect. 2 Share this post Link to post
Revenant100 Posted December 23, 2019 (edited) It looks like these animations are heavily dependent on the base sprite offsets, so problems inherent with the original values are exacerbated by the interpolation. The animations above don't actually reflect the original IWAD offsets, thus some issues are fixed and new ones have been inadvertently introduced. This is most evident in the first Arachnotron animation where, as the frames appear to have been automatically centered, the monster appears to jerk backwards during its BSPIE3B7 and BSPIF3C7 frames when its front left leg extends forward. This is the exact sort of sprite offset issue I sought to address in my sprite fixes as it accentuates the intended "smoothness" of the animations that were lost through id's automated offsets, plus I filled in the erroneous transparent pixels which evidently are also similarly exacerbated by such interpolation. In other words, the better the base offsets and cleaner the sprites, the better the interpolated results. 4 Share this post Link to post
Bauul Posted December 23, 2019 3 hours ago, Revenant100 said: the frames appear to have been automatically centered Actually that was just me! The tool requires all frames to be the same dimensions, so I just resized everything and auto-centered in the process. I'm sure if you were doing this properly you could uniformly size the frames without the incorrect offsets you describe. 0 Share this post Link to post
hidfan Posted December 23, 2019 For what I've understood (I've juste watched the tutorial), it's mostly made for 2D side view interpolation (as it asks what's front /rear). I'm wondering how it can interpolate a multiangle 360° rotation instead with back stuff becoming front stuff, or the death frames (well, actually, these ones could work, set the feet as front and the head as rear). Might be very interesting for SFX like projectiles or flames. 0 Share this post Link to post
Bauul Posted December 23, 2019 (edited) I got a bit of advice from the developer of the app around the detaching body parts - turns out the solution (overlapping limb segmenting) is pretty easy to implement. I popped in the Imp and Cybie to test. For the Imp it worked almost perfectly straight out, zero edits needed. The Cyb, being much bigger, generated far more anomalies. I manually cleaned up the rogue pixels but there's still some strange banding effects where the original frames move in and out of shadow. Still, an improvement. Edited December 24, 2019 by Bauul 15 Share this post Link to post
Mordeth Posted December 24, 2019 I truly live in the fucking future. Neural upscales, interpolation... Even at their worst, these interpolated sprites seem to provide an excellent base to work on. 8 Share this post Link to post
jval Posted December 26, 2019 I'm impressed from the results, an artist's touch after the initial conversion can provide excellent results! 0 Share this post Link to post
cambreaKer Posted December 26, 2019 try interpolating weapon sprites like the fist or super shotgun 0 Share this post Link to post
Linguica Posted December 26, 2019 On 12/24/2019 at 12:49 AM, Mordeth said: I truly live in the fucking future. Neural upscales, interpolation... Even at their worst, these interpolated sprites seem to provide an excellent base to work on. The Mordeth Effect: keep working on something long enough and eventually the future will come to you! 7 Share this post Link to post
Bauul Posted December 26, 2019 (edited) 7 hours ago, cambreaKer said: try interpolating weapon sprites like the fist or super shotgun We already have smooth animations for those, so it would be slightly redundant. Also, this tech is best suited for sprites that don't change perspective. It interpolates movement, not angles. My hunch is applying this to the gun animations would look rather strange. Edited December 26, 2019 by Bauul 0 Share this post Link to post
Lore Posted December 27, 2019 Those empty pixels looks yummy to work with! 0 Share this post Link to post
Nootrac4571 Posted December 28, 2019 (edited) Spent a bit of time playing with this last night, here's what I was able to achieve in a few hours: My thoughts: I've spent quite a bit of time trying to create tween frames for Doom animations by hand, and as everyone knows it's a hugely long winded task. This tool is great, but as the creator says, it's not a magic button that will churn out beautiful animations without needing any artwork skills. What is does is to semi-automate a very specific, and very time-consuming part of the task. And it's great at that. So, I'm psyched by this, it's going to speed up my process significantly. But it's important to temper expectations: Interprite doesn't (and isn't designed to) do anything other than linearly interpolate between keyframes, and there's a lot more to animation than that. Even with the tantalising possibilities of machine learning, this sort of tool isn't going to understand animation principles like cushioning, or smooth motion arcs, or secondary action any time in the near future; those things still have to be drawn or positioned by a human animator. In the example above: The animation has more frames, and looks ok, but it has a certain robotic quality: His feet in particular are just mechanically sliding forwards and backwards, so he looks a bit like he's ice skating. If I were drawing the tween frames by hand, I'd modify the frames just before each foot hits the ground in his forward step, so there's more of a sense of his foot lifting and dropping. And I'd also probably edit his head-bob to give him a slightly greater sense of weight. Interpolation alone can't do this. What I've made here is a great start, but I think it still needs another hour or so of manual editing before it'll be an animation I'd be properly proud of. To sum up: This tool makes a famously time-consuming task a little less (but still very!) time-consuming. The projected completion date for my own SmoothDoom-style mod has just shifted from "slightly before the heat death of the universe" to "somewhere in the distant future." Which is definite progress! 17 Share this post Link to post
david_a Posted December 28, 2019 The higher frame rate makes it seem like he’s walking faster... The jerkier movement makes his walk look much more deliberate and aggressive too. Just like with upscales, I feel like there needs to be a lot of artistic interpretation by someone that knows what they’re doing to get great results. The more I look at those the more I like your handmade version the best :/ Speaking of upscales... I’m sure I’m not the only one wondering what it’ll do with the neural net upscales. I shudder to think of how much memory high res, high frame rate sprites would use though 😳 2 Share this post Link to post
Bauul Posted December 28, 2019 2 hours ago, Nootrac4571 said: Spent a bit of time playing with this last night, here's what I was able to achieve in a few hours: I completely agree with your summation. It's a good start, but the end result often doesn't quite look as natural as the low frame rate original. A couple of thoughts: To avoid the way the sprite breaks apart (has gaps between the moving parts), when selecting the area for a part make sure it overlaps with the part in front of it. A single pixel can belong to multiple parts, just make sure the ordering of the part priority is correct. Then use the "Blend" option to smooth out the seams. This wasn't covered in the video at all but I asked the developer and he suggested this, and it works like a treat (see the Imp animation above. That has zero manual tweaks afterwards). It seems you left each leg as a single part? I suspect that's contributing to the robotic feel. I would have split that up into the feet, lower leg and upper leg. Basically anywhere there's a joint it seems to work well to break apart. Overall I agree with your assessment, it's a promising start. I feel there's nuance in the positioning of the key frame animation (the "Joints" in the program) that can contribute to a better feel. 2 hours ago, david_a said: Speaking of upscales... I’m sure I’m not the only one wondering what it’ll do with the neural net upscales. Unfortunately not very good at all. I tried it with the upscale Cyb and the results were disappointing. The higher fidelity images makes it much easier to see where the edges of each moving part is. The end result looks rather like a paper puppet: like individual flat moving parts tied together with string. Conversely the low res sprites hide it much better, it's far less obvious which individual parts are being interpolated as a block. 0 Share this post Link to post
Linguica Posted December 28, 2019 Making good normal-res tweened sprites and then upscaling the final results would probably work better than the reverse. 1 Share this post Link to post
Nootrac4571 Posted December 28, 2019 (edited) 8 minutes ago, Linguica said: Making good normal-res tweened sprites and then upscaling the final results would probably work better than the reverse. Oh yeah, definitely this. The pixel cleanup required is significant, and the more pixels you have to deal with, the more time it takes. Cleaning up a 2x or 4x scaled sprite sounds like a horrible task, which I wouldn't want to do at all. 0 Share this post Link to post