New source port: d2k

[Jon]

New-ish, commits go back to 2014. Linguica spotted it.

https://github.com/camgunz/d2k

Features basically everything ever, it seems. Based on prboom.

[printz]

Camgunz is Ladna here, right? Is it managed by one person only? Good to bring attention to this here, because it looks quite ambitious and we don't want it to be forgotten.

[HavoX]

Jon said:

Features basically everything ever. Based on prboom.

Yeah, but how long will it be maintained? That is the question.

[printz]

Why didn't he choose to start all this in Eternity? It's all basically the same idea. He even has access to Eternity!

[HavoX]

That's actually a good question, since d2k has EDF.

[printz]

I guess it's because of stuff like Lua, so nevermind me.

[dew]

Once upon a time Ladna actually was working on a multiplayer fork of Eternity (the fabled EECS), but he wanted... everything ever, hence this. It flew under the radar, because it's so ambitious that people who knew about it don't expect it to see the light of day within the next few centuries.

[andrewj]

Too bad he is hell-bent on wasting his time, Odamex could use a decent coder.

[Csonicgo]

Jack of all trades, master at none? I see this going nowhere.

There's a reason there are so many source ports. While ZDoom attempts to emulate Eternity, it will never play Vaporware properly. That's because it is made for Eternity. Same with newer Vanilla maps that abuse colors, or Boom maps that abuse the translucency and scrollers.

Source ports have been trying to be the "one stop shop" for years, and continue to fail at matching 100% with previous, once-popular ports. I love that ZDoom tries, but even Graf agrees that the closest you can get is "approximate", even if you use exact code - because the code must be changed to fit into the other port.

[Graf Zahl]

Csonicgo said:

I love that ZDoom tries, but even Graf agrees that the closest you can get is "approximate", even if you use exact code - because the code must be changed to fit into the other port.

The line will always be drawn when maps start to abuse system dependent stuff, like Boom's translucency table, which by its very design cannot translate to any renderer that works even just a bit differently.

And that goes both ways. ZDoom so much depends on vanilla bugs being fixed that trying to shoehorn all those extended features into a port that wants to preserve demo compatibility is just doomed to fail because, as I said in another thread, these two things will constantly be at odds with each other.

There's a reason there are so many source ports. While ZDoom attempts to emulate Eternity, it will never play Vaporware properly.

But that's mostly due to the different syntax of the definition files. Aside from the portals the map doesn't contain anything extraordinary. Lots of exclamation marks nonwithstanding I was able to play through it with GZDoom (ZDoom's software renderer obviously did not so great.) Should I find the time I might create a patch.

Same with newer Vanilla maps that abuse colors, or Boom maps that abuse the translucency and scrollers.

What I find a bit odd here is that some of these mappers seem to have a bit of a warped perception what engines people use. All the numbers I have seen say that outside of Doomworld, vanilla-faithful engines are a very small minority (with hardware acceleration dominating) so (ab)using these features will generate a rather diminished experience, depending on how much a map depends on it, for most users that end up playing the map (but never post here.)

It'd also be appreciated, if you know of such maps, to report incompatibilites so that they may be taken care of if possible.

[scifista42]

Csonicgo said:

Same with newer Vanilla maps that abuse colors,

What's this feature you are talking about? Modified damage/pickup/radsuit PLAYPAL / invulnerability COLORMAP range? Some ultra specific / subtle behavior of light diminishing? Displaying palette indexes 247 and 255 in modified palettes?

[Da Werecat]

Sure, abuse. Everybody and their dog knows that paletted rendering isn't allowed to have its own quirks and features. It must be an inferior version of accelerated rendering; otherwise accelerated source ports won't be able to run everything you throw at them.

scifista42 said:

Displaying palette indexes 247 and 255 in modified palettes?

Eh?

[Csonicgo]

scifista42 said:

What's this feature you are talking about? Modified damage/pickup/radsuit PLAYPAL / invulnerability COLORMAP range? Some ultra specific / subtle behavior of light diminishing? Displaying palette indexes 247 and 255 in modified palettes?

All of that and more! You won't believe what some proof-of-concept vanilla wads passed around on IRC channels from time to time can cook up. Some of it's never used in a WAD, at least... not yet...

[LexiMax]

This port is not particularly new. I even committed its initial Lua implementation a few years ago, which he seems to have subsequently ripped out in favor of depending on system Lua (bad idea IMHO, but it's his port).

andrewj said:

Too bad he is hell-bent on wasting his time, Odamex could use a decent coder.

He used to actually work on Odamex, but left to work on EE:CS. And let's be real, it's up to each of us to figure out what we want to spend development time on, and such time isn't "wasted" if you get some enjoyment out of it.

[scifista42]

Da Werecat said:

Eh?

I vaguely recall from what I've read and/or experienced that a non-black index 247 or a non-tan index 255 (if a custom palette had either of those) would display differently in different ports (some ports would display them as black anyway, or they would display a non-black color where black was supposed to be, etc).

[Graf Zahl]

scifista42 said:

What's this feature you are talking about? Modified damage/pickup/radsuit PLAYPAL / invulnerability COLORMAP range? Some ultra specific / subtle behavior of light diminishing? Displaying palette indexes 247 and 255 in modified palettes?

Since you mention it, it has been bothering me since forever that ZDoom does not analyze the PLAYPAL to guesstimate the default blend colors, one day I may get around it, so if some map comes up that actually uses this, please point me to it.

Obviously changing the entire system to actually use these palettes would defeat large portions of how ZDoom works.

And what's up with colors 247 and 255? Something about the colormap? Technically there's nothing preventing them from getting displayed, they are just normal colors, after all. Of course, they will look different if the base color is always run through the colormap, while ports which do not do that (nearly everything hardware acceletated) will always show the original color.

That said, abusing such stuff has no real value beyond showing off. Maps depending on it may easily risk getting downvoted by people who see such 'glitches' and do not know where they come from and thus declare the mapper incompetent.

Csonicgo said:

All of that and more! You won't believe what some proof-of-concept vanilla wads passed around on IRC channels from time to time can cook up. Some of it's never used in a WAD, at least... not yet...

You mean such impractical stuff like Linguortals?
Proofs of concept are one thing, but most of the time the inherent limitations make those things ill suited for an actual map.

[Linguica]

Graf Zahl said:

Since you mention it, it has been bothering me since forever that ZDoom does not analyze the PLAYPAL to guesstimate the default blend colors, one day I may get around it, so if some map comes up that actually uses this, please point me to it.

The Instadoom WADs all have custom palettes for damage / pickup / radsuit that fade towards custom colors, and the colormaps are generated to not fade to "black" but instead fade to color #0, which in some of them is pretty different from black.

[Coraline]

Is this port using the Enet-lib for networking? I wonder how well that is working out.

I have been building on HawkNL for 3DGE's network code, but have been recommended to switch to Enet by a few people now. Also thought about using SDL_Net, since my new SDL2 branch will have proper network support.

[Ladna]

Hey guys. I am camgunz, yeah. It's a little more "Silicon Valley" than "Ladna", for the sake of getting jobs :)

D2K is a project based on PrBoom+ that I and JKist3 are working on. I code, and JKist3 provides invaluable insight and expert player evaluation.

Current Status:

- Implemented c/s netcode (still need a lot of player testing)
- Added Lua and moved most of the HUD and player input processing to scripting (using FreeType, Pango, and Cairo)
- Created a demo testing framework to maintain compatibility

This version of D2K is essentially a prototype, a proof-of-concept. We wanted to show that it was possible to build a c/s port using state deltas instead of network messages, and to implement the beginnings of a scripting interface using a modern language capable of implementing large parts of the Doom engine -- thereby making it far more accessible to modders. We're not there yet, but we're pretty confident we'll get there.

Future Goals:

When we finish the prototype phase, we'll focus on making D2K capable of standing shoulder to shoulder with the current c/s Doom ports:

- Increase MAXPLAYERS from 4 to 32 (64?) while retaining compatibility
- Add spectating
- Add powerful, Odamex-style maplists
- Add RCON and authorization levels
- Build a master server
- Add WAD Downloading
- Add 3D and ZDoom physics
- Add announcer
- Add slopes
- Add teams
- Add Hexen map support
- Add c/s demos (both clientside and serverside)
- Solidify our demo testing framework
- Implement game modes in scripting
- Move as much functionality to scripting as possible
- Enable web play
- Program (awesome) bots

Other than a few differences, this is essentially what Odamex provides right now.

As has been noted, we also plan to incorporate major features from other ports. We plan to use their code whenever possible. For some things that's really easy (EDF/ExtraData, MAPINFO), others not so much (portals).

In-Depth:

D2K uses state deltas instead of network messages to keep clients and servers synchronized. Essentially, the server saves the game every TIC, and sends the difference between that save and the save for the last TIC to clients. This means that even if clients do desync (which is common in any port), they resynchronize whenever they get an update. This eliminates a huge class of bugs, and is very similar to the model used by Quake 3. We're about 90% done with this (still working on sound and various smaller issues).

We also plan to convert large parts of the engine to scripting. There's not really a reason to keep the menu in C, for example. We want modders to be able to modify nearly anything in the engine without knowing C. Largely, this means that sound, rendering, and netcode, and original compatible physics will stay in C, whereas everything else will move to scripting.

How You Can Help:

You really can't yet. We're always happy for code contributions, but since every project is short on developers, we're pretty realistic about the fact that it'll mostly be me doing this. We'd love art (although we're crazy picky), and we'll need a lot of help with documentation when the scripting interface solidifies, but we're not quite there and I'd hate to throw out a bunch of work for no good reason.

===

Aaaaaaand now I can answer some questions :)

printz said:

Why didn't he choose to start all this in Eternity? It's all basically the same idea. He even has access to Eternity!

I love EE and have a lot of respect for Quasar and SoM. I just need to make unilateral decisions. I do have a side-goal of dropping the netcode back into EE when it's been pretty well banged on in D2K though. The major changes are changes to P_PlayerThink, P_RunThinkers, and TryRunTics; everything else is largely self-contained. But, it wouldn't hurt my feelings if someone else did it... *hint* *hint*.

Csonicgo said:

Jack of all trades, master at none? I see this going nowhere.

I don't blame you, and besides, I've been at something like this for almost 6 years. We're pretty... cynical when we talk about getting D2K done. We're better at talking about current goals (increasing MAXPLAYERS while retaining compat) and issues (goddamn multiplayer sound).

D2K is ambitious, but not because we want to be ultra-cool. We think that the maps, mods, and demos that the community has generated over the last 20+ years are all vital parts of Doom. Some require Vanilla compatibility, some require Boom compat, ZDoom physics, slopes, EDF, ACS, DECORATE, Hexen, and so on.

We're also well aware of the "all talk, no follow through" syndrome when it comes to Doom source ports. We're not claiming we're going to get anything done or that anything really works, which is why we haven't made a release post or anything like that. If/When we have code that works and is tested, you'll know :)

AlexMax said:

This port is not particularly new. I even committed its initial Lua implementation a few years ago, which he seems to have subsequently ripped out in favor of depending on system Lua (bad idea IMHO, but it's his port).

It was super cool of you to contribute :) Uh, we've gone back and forth between Lua 5.2 and 5.3, and it was a big pain in the ass to keep switching out all that code. Eventually we'll settle into 5.3 (when LGI gets its shit together). I'm 50/50 on including Lua. On the one hand, I really dislike the tendency of Doom ports to include all their dependencies in their source trees (I assume this is a side-effect of using MSVC). It clogs up repos, increases build time, and is pretty antithetical to modern open source development. On the other, Lua is a critical part of D2K, and it makes sense for compatibility and support to always include its source, even if we'll never modify it.

AlexMax said:

He used to actually work on Odamex, but left to work on EE:CS. And let's be real, it's up to each of us to figure out what we want to spend development time on, and such time isn't "wasted" if you get some enjoyment out of it.

Thanks again! I left Odamex because I need to make unilateral decisions. I thought things would be different with EE, but it turns out I just suck at working in teams where I'm not the boss, haha.

Chu said:

Is this port using the Enet-lib for networking? I wonder how well that is working out.

Yeah we use ENet. Honestly the amount of code that deals with networking is very small. The trickiest stuff is syncing up clientside and sector prediction. As in, clientside you hit a switch on TIC 53, and the lift starts its movement. Buuuut, the server gets your commands for TIC 51, 52 and 53 on TIC 58, and starts the lift movement on TIC 59 (because when it gets a bundle of commands, it only runs up to 2 per TIC, otherwise players skip around like crazy). So somehow the server has to let the client know that those commands were run on different TICs, and as a result, the world looks a little different than the client predicted, without jerking the world around at all times. That took months, with me probably working like 50-60 hour weeks on it. I'm not saying it's hard and I'm super smart, it's likely it's easier than I think and I'm dumber than I think :) But still, good luck.

[Quasar]

Ladna said:

Thanks again! I left Odamex because I need to make unilateral decisions. I thought things would be different with EE, but it turns out I just suck at working in teams where I'm not the boss, haha.

I think the rather violent architectural changes that were going on in EE at the time (C++ conversion for one) didn't help and were really impeding the work you were trying to do so, personally, I totally understood when you took things in a different direction.

[chungy]

Odamex is in desperate need of a boss. Just saying... :)

[Ladna]

Quasar said:

I think the rather violent architectural changes that were going on in EE at the time (C++ conversion for one) didn't help and were really impeding the work you were trying to do so, personally, I totally understood when you took things in a different direction.

Yeah, that was a big part of it. I appreciate this, thanks :)

[Danfun64]

PrBoom-Plus's Freecam mode for demo playback is going to be retained, right? Demos created in older versions of your port will be playable in newer ones, especially concerning multiplayer? Text will be recorded? In game VOIP chat implemented (and recordable)? Q3MME/WolfcamQL/Keygrip2/etc style demo editing and/or camera tricks (like waypoints)?

[andrewj]

AlexMax said:

He used to actually work on Odamex, but left to work on EE:CS. And let's be real, it's up to each of us to figure out what we want to spend development time on, and such time isn't "wasted" if you get some enjoyment out of it.

I certainly understand wanting to do your own thing and not having the constraints of working in a team -- seems GZDoom exists as a separate entity from ZDoom for the same reasons.

I still think that if nobody ends up using Ladna's source port, then the time and effort spent developing it has been wasted. Guess he is much more optimistic than I am :)

[printz]

andrewj said:

I still think that if nobody ends up using Ladna's source port, then the time and effort spent developing it has been wasted. Guess he is much more optimistic than I am :)

It's up to Ladna/camgunz or his friends to provide a great presentation for his port. It needs a fancy website and complete documentation. ZDoom has its website and community. And its documentation has transitioned very well from the old one to the wiki.

You know when I got hooked into Eternity? When it had a cool website with a promising description and graphics (for that cancelled mod). And it also had a very nicely written documentation.

[Graf Zahl]

printz said:

It's up to Ladna/camgunz or his friends to provide a great presentation for his port. It needs a fancy website and complete documentation.

Indeed. Let's have a look at the plan here:

What does this port add, provided it gets all its features. To me the list looks like 'Most of ZDoom's features plus some demo compatibility magic.'

Having worked on ZDoom for 13 years I know in what state of flux the code is, it'll surely be a constant game of catch-up - eliminating the active ZDoom community from being the potential users, I'd imagine these people will stay where they are and not switch. So my prediction would be that the potential target audience here is precisely the crowd that right now uses PrBoom+ as their main port of choice.

So it'll either replace that or end up an obscurity like so many other ports have before. And here I agree with Printz: The only thing that matters is marketing. Being relegated to some github repo and an obscure thread on the Doomworld forum is not going to cut it.

So what does marketing mean? I'm just listing what I experienced myself when 'marketing' GZDoom after its first release in 2005.

• Biggest mistake: Promising cool features and then not delivering them. This will frustrate the users and create hostility towards the port's makers, regardless whether it's justified or not. Better do not talk about them unless you know you can deliver them in time. Having a long term roadmap is ok, but if 'soon to come' means 'in a few years maybe', people will jump ship for sure.
  
• Be open to suggestions for improvement. Especially in its first phase I got a lot of feedback about things in GZDoom that were not right - and yes, these people were mostly correct about it! So 'this sounds cool but is not on my roadmap' is a bad excuse. Never use that as justification why stuff doesn't get done. This will lose customers immediately. If you have good reasons, people will understand, but make it clear what those reasons are.
  
• Engage in discussions about the port when they come up. You might get more out of it than you may think. Many things I added in ZDoom or GZDoom were initiated by such discussions, sometimes even by rather stupid ones.
  
• Once you know your port got some traction, then a dedicated website is definitely a must, but don't make outrageous claims, especially before the port got traction. I'll just point to Vavoom's tagline 'The most advanced port...', which many found quite inappropriate for an engine that was slow and buggy, and made for many negative jokes.
  
• And let's not forget the most important one: Make sure your port has a point! At best that point should be evident from the start. If it's merely 'PrBoom with some cool stuff coming later', that 'later' should not be too far off, in today's situation you got to differentiate in a way that makes people choose your port over the one they currently use, and experience shows that people are hard to part from their pet engine, unless the new one provides a significant improvement. If it does not, they will just stick with what they use and forget about the new option. When I decided to do a public release of GZDoom I had already put multiple years of work into it, so in the public eye the entire 'soon to come' phase never got seen. I know that with a public Github repo that's not as easy anymore (which is why I'd host any project that's not ready for prime time on a server where I can keep the repo private until it gets far enough for wider exposure.
  

[Ladna]

Danfun64 said:

PrBoom-Plus's Freecam mode for demo playback is going to be retained, right? Demos created in older versions of your port will be playable in newer ones, especially concerning multiplayer? Text will be recorded? In game VOIP chat implemented (and recordable)? Q3MME/WolfcamQL/Keygrip2/etc style demo editing and/or camera tricks (like waypoints)?

Yeah, the only PrBoom+ features we plan on removing are Windows-specific things, like the launcher.

C/S netdemos will be D2K-specific (unless someone adds support for them to their port). We haven't designed the format yet, but it's likely gonna be an archive of delta-compressed state dumps, waypoints, and some metadata. The format is the easy part though; we plan to support a lot of stuff while retaining demo compatibility. That's not an easy task, so being able to play back a demo recorded with D2K isn't as simple as reading it: you need the backend support also.

VOIP isn't on our list, but I think it will make it on there. A lot of FPS' include it (ioquake3, for one), so it shouldn't be too hard to yoink. I don't know what those demo editing things are, but hey, if you want some stuff, feel free to use our GitHub Issues Page.

printz said:

It's up to Ladna/camgunz or his friends to provide a great presentation for his port. It needs a fancy website and complete documentation.

You can still just call me Ladna :) We're worried about the vitality of competitive Doom, so we know we need to attract users. We think web play and bots will be a big part of that. We hope to expand beyond the current Doom playerbase, and that means nice website, ease of use, stuff to grind on (XP), that kind of thing. It'll be a big job. We don't think we'll be the next Quake Live or anything, but we do think you should be able to read an article about Doom, have someone bring up old-school Doom/Doom II, then click on a link to start watching or playing in a huge FFA.

Graf Zahl said:

What does this port add, provided it gets all its features. To me the list looks like 'Most of ZDoom's features plus some demo compatibility magic.'

Welllllll I wouldn't say "magic". But we've discussed what maintaining compat boils down to in other threads. Our current plan is basically to swap out functions (if we have to). It's not that bad, to us anyway. Plus, we want to give modders the ability to make their own physics, so that's kind of necessary besides.

Graf Zahl said:

Having worked on ZDoom for 13 years I know in what state of flux the code is, it'll surely be a constant game of catch-up

To a degree, yeah. But "ZDoom Compatibility" isn't really our goal, "plays most of the popular ZDoom mods" is. From that standpoint, it's not too bad. Really the impetus behind this is we like a lot of Zandronum mods and would like to play them on a more "Doom-y" engine.

Graf Zahl said:

I'm just listing what I experienced myself when 'marketing' GZDoom after its first release in 2005.

This is all great advice, I really appreciate it :)

[Jon]

Graf Zahl said:

What does this port add, provided it gets all its features. To me the list looks like 'Most of ZDoom's features plus some demo compatibility magic.'

Weird, I got a completely different impression. The headlines for me were C/S with save deltas and Lua; neither of which ZDoom has.

Many of the other features listed are in ZDoom, but they weren't the headlines.

[Arctangent]

Ladna said:

To a degree, yeah. But "ZDoom Compatibility" isn't really our goal, "plays most of the popular ZDoom mods" is. From that standpoint, it's not too bad. Really the impetus behind this is we like a lot of Zandronum mods and would like to play them on a more "Doom-y" engine.

If anything, I would toss this so far down the backburner and maybe aim the Lua scripting to be powerful enough for a user to write stuff that could read DECORATE, SBARINFO, MENUDEF, etc.. It seems very strange to me to have a bullet point of "includes Lua scripting!" and then going out of your way to include other, specific scripting languages that do the exact same thing.

[Graf Zahl]

Arctangent said:

If anything, I would toss this so far down the backburner and maybe aim the Lua scripting to be powerful enough for a user to write stuff that could read DECORATE, SBARINFO, MENUDEF, etc.. It seems very strange to me to have a bullet point of "includes Lua scripting!" and then going out of your way to include other, specific scripting languages that do the exact same thing.

Lua cannot replace DECORATE, it cannot replace SBARINFO and it cannot replace MENUDEF (that is unless you really want some kludgy way of defining this stuff that's not really user friendly.)

Lua is a programming language, all the others are definition languages, this is a major difference. They are for mutually incompatible purposes. DECORATE, SBARINFO and MENUDEF may be designed to ACCESS Lua-created content, but they are in no way equivalent.