Jump to content
Search In
  • More options...
Find results that contain...
Find results in...
andrewj

Announcing AJBSP

Recommended Posts

I notice a reference to blockmap compression in the output, is this related to the ZokumBSP innovations?

Share this post


Link to post

Cool! Interested to know the motivation behind making a new nodebuilder? Is there something in particular it does that others don't?

 

Either way, really cool to see new tools. 

Share this post


Link to post

There are so many BSPs that I can't choose which to use. I use ZenNode for GZDB without problems for many time.

Share this post


Link to post

Will all of the features continue to be built into Eureka or is it going to retain only a more basic version?

Share this post


Link to post
Posted (edited)

@andrewj Long time no see :)

5 hours ago, andrewj said:

some information on how to compile this under Windows (using VSCode or whatever people use on Windows nowadays).

You can use GCC on Windows to compile, I believe you need to install MinGW and MSYS http://www.mingw.org/, I'm not that experienced with coding in general but I have used it to build a few really simple programs and other people's dlls like zlib so I know it works :)

 

Oh also i've found a few bugs with eureka which I posted on the eureka 1.21 thread

Share this post


Link to post
5 hours ago, andrewj said:

a MacOS X binary -- should be quite easy since there are no third-party dependencies.

 

ajbsp.zip

Share this post


Link to post

@Linguica Thanks for the MacOS binary.  The blockmap compression is just the basic one which glBSP had for a long time, all it does is detect duplicate blocks and uses the same index for them.

 

@therektafire I clarified the original post, I'm looking for detailed information that I can place in the INSTALL document, and possibly a Makefile or project file to add to the repository.  I will look at the Eureka thread sometime.

 

@ETTiNGRiNDER I don't expect much development to the core node building stuff, but any improvements to AJBSP will flow back into Eureka (indeed a few bug fixes and code tidy-ups already have).

 

@jmickle66666666 Main motivation is that I wanted the BSP code from Eureka, which I know works pretty well (as I worked quite hard to make it as reliable as possible, because it can run everytime somebody saves their map) -- I wanted that in the form of a CLI program. Plus it might be good if AJBSP replaced the glBSP package in Debian and other Linuxes.

Share this post


Link to post

are there significant internal differences vs glbsp?

 

would you be okay with me including this in Doom Builder X, like the other nodebuilders?

Share this post


Link to post
10 hours ago, anotak said:

are there significant internal differences vs glbsp?

Let's see....

  • AJBSP modifies a wad in-place, where glbsp created a new file with the nodes added.
  • the --fast option works differently than in glbsp 2.24, and does not look at any existing nodes.
  • if the BLOCKMAP overflows, it is left as an empty lump.  glbsp would "truncate" it, producing a valid blockmap but mostly useless since large parts of the map are excluded.
  • AJBSP supports the XNOD format, which glbsp does not.
  • AJBSP only supports V2 and V5 of GL-Nodes, glbsp supports the obsolete V1 and V3 formats.
  • AJBSP command-line options are different.

The first thing (directly modifying the input files) is the main thing to watch out for.

Share this post


Link to post

Can I use AJBSP to build nodes for UDMF maps?

Share this post


Link to post

What would it take to support ZDoom compressed nodes (ZNOD)?

Share this post


Link to post

@printz No, UDMF would require major changes (and major testing), which I don't want to do.

 

@AlexMax The code is already there for ZNOD, but why would you want it? I thought XNOD format had made ZNOD obsolete.

Share this post


Link to post

 

Awesome! I love your work so I'm always excited to see a new andrewj project. On my personal wishlist is to one day write a nodebuilder myself, simply to better understand the node structures (I don't have any particular tricks or novel ideas to try out here)

 

On 6/3/2018 at 5:59 AM, andrewj said:

Main motivation is that I wanted the BSP code from Eureka, which I know works pretty well (as I worked quite hard to make it as reliable as possible, because it can run everytime somebody saves their map) -- I wantedde that in the form of a CLI program. Plus it might be good if AJBSP replaced the glBSP package in Debian and other Linuxes.

 

This is timely. I recently adopted GLBSP but had plans to replace it, at the moment the most likely candidate would have been ZDBSP, or Zokum's (but I hadn't looked closely at ZokumBSP since moving back to the Linux desktop from OS X). I'd certainly be open to packaging AJBSP as well/instead.

 

But...

 

On 6/2/2018 at 3:03 PM, andrewj said:

The command-line interface is completely new, and is not compatible with glBSP or any other node builders.

 

This is a bit of a problem. I'd like all node-builders in Debian to provide a common virtual package (doom-node-builder or something), and to share a common minimal interface. ZDBSP and GLBSP both support "<bsp> <inwad> -o <outwad>" where inwad and outwad might be the same. As a minimal common "interface", this would mean that editors that support or rely upon an external BSP tool can default to (or hard-code) that invocation structure. (in practise this only means WadC at the moment, at least within Debian, since Eureka has its internal one)

 

It would be nice if we could have a minimal common CLI interface for node builders. I *think*, from memory, zokumbsp supports the above. haven't looked at the specifics of AJBSP yet.

Share this post


Link to post
14 hours ago, Jon said:

This is a bit of a problem. I'd like all node-builders in Debian to provide a common virtual package (doom-node-builder or something), and to share a common minimal interface. ZDBSP and GLBSP both support "<bsp> <inwad> -o <outwad>" where inwad and outwad might be the same. As a minimal common "interface", this would mean that editors that support or rely upon an external BSP tool can default to (or hard-code) that invocation structure.

Ok, I can see that a common minimal interface would be useful.

 

I am willing to add an "-o" option as a compatibility option for specifying the output filename.  The default behavior (without the option) would stay as it is now: modify the input files in-place.

 

I think it is a bad idea to check that the input and output filenames are the same and for that to enable an in-place modification.  I can't think of any other Unix tool which works that way, e.g. "cp" will produce an error if you try to copy a file to itself.  What I did with glBSP was a user interface mistake.

 

So I suggest that you leave out any mechanism for in-place modification of input files from the common minimal interface.  Just have the "-o" option to specify the output filename.

Share this post


Link to post
Posted (edited)

In general, not having a separate input and output wad is a bad idea. If there's a bug, the original data is also corrupted. There are also some special effects and optimizations that are one-way. They make the output map unsuitable for editing. I strongly recommend against this type of setup being used in nodebuilders, mapping programs etc. There's no such thing as a perfect tool without bugs. Not writing to the input file lowers the chance of a catastrophic failure.

ZokumBSP does support it, but if possible I recommend users to not do that. It's not as safe as a separate output file. There's a lot of dodgy code in ZenNode/ZokumBSP. I've fixed some bugs but there are probably others, or bugs I have ontroduced.

I find the idea of a continuous BSP calculation fascinating. It sounds like it would be really slow and I am somewhat puzzled over what the advantages are. Every moved vertex, added line, added vertex etc could mean the root partition should be different. Unless the tree is rebuilt from the ground up, you'd end up with sub-par trees. I'm curious as to how you've solved this and still kept it fast enough.

As for blockmap compression, you can just lift some of the algorithms from ZokumBSP, they're fairly easy to understand and documentet on the web site and doomwiki.org. If you dont't feel like adding all that complexity, one can just add ZokumBSP to the tool chain when testing a map. -n- -r- would make it build just the blockmap with a few basic compression algorithms turned on.

It used to be standard to use RMB for proper reject building. A tool chain of building nodes with aj, blockmap with zokumbsp and reject with RMB would be a fairly clever setup. What we call node building really is 3 separate steps. I've also added a few more steps in ZokumBSP with the geometry pass and I am also planning to add an optimization step for sidedef compression etc and some other ideas I have.

Other than that, I look forward to having more tools to play with. The more the merrier, and make sure you update the wiki about your stuff so people can find it.

Share this post


Link to post
On 6/2/2018 at 7:54 PM, riderr3 said:

There are so many BSPs that I can't choose which to use. I use ZenNode for GZDB without problems for many time.

There are a few bugs in ZenNode that I fixed in ZokumBSP. You might be better off using ZokumBSP for bigger maps. It also builds slightly better nodes and usually compresses the blockmap better due to fixing an oversight in the original code.

Share this post


Link to post

@zokum Firstly, and least importantly, I disagree about in-place modification.  When you run a DOOM map editor or even a text editor, you are directly modifying the same file.  If the file is precious, a wise person makes regular backups of it, and AJBSP provides an option to backup each file before processing it.

 

Neither AJBSP nor Eureka does "continuous BSP calculation".  Eureka builds the nodes only when you save the file, and that is optional (can be disabled).

 

I may look at your blockmap compression code, but I don't plan to make AJBSP be a tool with heaps of knobs that users can tweak -- other node builders have that covered.

 

It sounds like ZokumBSP has a strong focus on vanilla DOOM mapping, whereas AJBSP is more for modern source ports.  So I think they are more complementary than competiting with each other.

 

Share this post


Link to post
Posted (edited)

Hi folks, a few more words on the idea of a common interface, the alternatives system in Debian, and precedent.

With regards supporting "-o" and in-place modification: I'm not personally wedded to either. However, the two node builders we've had packaged in Debian (bsp and later glbsp) both implemented the two (that is, -o, and overwriting the input WAD). So, if I were trying to define a common interface that factored in the existing BSP tools that Debian has included, then it would be hard to not have to include those semantics too.

 

The way the alternatives system in Debian works is, you define a name of a binary (such as "bsp") and the alternatives system will manage symlinking that (and related stuff like manpages etc) to one of a set of installed, compatible implementations. So if we used alternatives for bsp, we'd have /usr/bin/bsp symlinked to (say) /usr/bin/glbsp, or /usr/bin/zdbsp or ajbsp or zokumbsp, depending on what was installed, and what the user's preferences were. But crucially, this is only possible if they have some kind of minimal common interface.

 

GLBSP replaced BSP entirely and they never co-existed in Debian, so we never crossed this bridge before (in fact, my memory might be faulty. perhaps we never actually uploaded bsp to Debian at all! I can't find any evidence of it. I thought we had...) The only tool these days that uses a BSP tool in Debian is WadC: Eureka uses internal instead (although @andrewj, might you change your mind on that since you are working on AJBSP now?) and Yadex was dropped in 2004 so is no longer relevant. WadC hard codes an invocation that worked with the original bsp tool as follows:

 subcmd(Arrays.asList(prefs.get("bspcmd"), wadfile, "-o", wadfile));

I think I'd be happy to adjust this so that WadC did not assume that the input and output wad files can be the same, that is, write to a temp file and overwrite the input filename only if the command returns success. That would not be a great deal of work. But I would like to avoid having to replace the tool configuration with something more complex, like a substitution-supporting string.

 

In which case, we could consider the "minimal compatible interface" as follows

  • we'd use the binary name "bsp" as the alternative that is repointed
  • the debian virtual package name might as well be a more descriptive "doom-node-builder"
  •  <bsptool> infile -o outfile, i,e., with the argument coming after the infile (matches/supported by bsp, glbsp, zdbsp)
  • no guarantee that infile and outfile can be the same file

Does that sounds reasonable? At the time of writing, bsp, glbsp, zdbsp and zokumbsp all work with the above rules (zdbsp supports the -o outfile being before infile, but bsp, glbsp and zokumbsp do not). Although bsp is completely deprecated and glbsp largely so, I'd prefer to leave them in-scope if possible, especially since bsp is the kind of grandaddy of the whole family.

Edited by Jon

Share this post


Link to post
Posted (edited)

On this topic (while we are mentioning glBSP) -- as per discussion with @andrewj, the version of glBSP in the EDGE repo that we plan on releasing externally (v2.28) will be appropriately renamed edgeBSP (or eBSP for short) once we release it stand-alone. We have made changes to the command-line and the overall glBSP code (such as supporting the building of glNodes via WADS embedded inside of archives, CMake support, C++ refactor) and other small tweaks (integrating the never-released changes from Andrew's glBSP 2.27). I am in the process of updating the GUI (glBSPX, which will just be named eBSPX). Internally to EDGE's source code, eBSP will still be named glBSP, but as far as new releases we make -- they'll be called edgeBSP/eBSP for now on. It's considered a direct continuation of glBSP, so I just wanted to give you guys a heads up to avoid any confusion there. 

 

As soon as I have everything built stand-alone and do some bug-testing, we will make a new release at the SourceForge website. 

 

However, we are planning on integrating ajBSP nonetheless (since we support edgeBSP, zdBSP, and TinyBSP for ROTT/Wolf3D) and making a small side-launcher to pick your preferred nodebuilder (instead of command-line switches). EDGE however will always default to either edgeBSP for GLNodes, unless the map is in UDMF format, which our internal version of zdBSP takes over to build nodes for UDMF). TinyBSP is always used for Wolf-based conversions as well, and at one time, was used to build glNodes in EDGE 1.32/1.36. 

 

@Jon I will look into making updates to the command line switches for edgeBSP once we release it!

Share this post


Link to post
7 hours ago, andrewj said:

@zokum Firstly, and least importantly, I disagree about in-place modification.  When you run a DOOM map editor or even a text editor, you are directly modifying the same file.  If the file is precious, a wise person makes regular backups of it, and AJBSP provides an option to backup each file before processing it.

 

It sounds like ZokumBSP has a strong focus on vanilla DOOM mapping, whereas AJBSP is more for modern source ports.  So I think they are more complementary than competiting with each other.

 

That just increases amount of backups one has to do. If the editor OR bsp tool fail, you have to restore from the editor backup, as the bsp tool just trashed the latest saved progress anyway. If you use a separate output file, the bsp tool having bugs would only destroy the temporary output. I much prefer the unix way of doing things with simple modular tools that do only one thing and do it well.

I prefer a tool chain of editor ----> myproject.wad -bsp-> output.wad ----> doom2 -file output.wad I'd like to swap out any part of the chanin with a different tool if I feel it might give me a better result.

I'm not too fond of grouping ports into modern and vanilla. Chocolate Doom is a very modern port with many features, excellent compatibility, runs on many operating systems and has an excellent code base. The term vanilla is also problematic, since it seems to mean different things to different people. For me it means doom2.exe v1.9. That includes a lot of content that is much more modern and advanced than the iwads. But I basically agree, my goal is to make a tool to make better Doom maps, not source port specific stuff. In theory, the maps should run on any vanilla compatible port.

Share this post


Link to post
Posted (edited)

fwiw doom builder 2 & children always automatically make several backups. during saving, the initial write is also to a temporary file which is only copied over the original upon successful completion of all operations.

 

On 6/3/2018 at 1:48 PM, anotak said:

 

would you be okay with me including this in Doom Builder X, like the other nodebuilders?

 

 

you never answered this question

Share this post


Link to post
On 6/6/2018 at 4:57 PM, zokum said:

I find the idea of a continuous BSP calculation fascinating. It sounds like it would be really slow and I am somewhat puzzled over what the advantages are. Every moved vertex, added line, added vertex etc could mean the root partition should be different. Unless the tree is rebuilt from the ground up, you'd end up with sub-par trees. I'm curious as to how you've solved this and still kept it fast enough.

If someone could pull it off, this would be a neat option for map editors that use a software-rendered 3D mode. It wouldn't have to be optimal - you'd still do a final full rebuild, and, maybe you'd also include a "Refresh" tool that does the full rebuild during editing. But, for simple changes, you could do the "continuous calculation". It is a fascinating idea, but it sounds very complex. You'd need an "unsplit" function, and other reverse calculations, I'd think.

Share this post


Link to post
14 hours ago, Jon said:

In which case, we could consider the "minimal compatible interface" as follows

  • we'd use the binary name "bsp" as the alternative that is repointed
  • the debian virtual package name might as well be a more descriptive "doom-node-builder"
  •  <bsptool> infile -o outfile, i,e., with the argument coming after the infile (matches/supported by bsp, glbsp, zdbsp)
  • no guarantee that infile and outfile can be the same file

Does that sounds reasonable?

Sounds reasonable to me.

 

Last night I was thinking that perhaps all that was overkill, especially when there is only a single package right now that really benefits (wadc).  But you are the Debian guy, so ultimately it is your decision, and I am still happy to support an "-o" option in AJBSP.

 

@anotak You don't need my permission to bundle AJBSP with another program, but I am Ok with that.

Share this post


Link to post

@andrewj

"Firstly, and least importantly, I disagree about in-place modification.  When you run a DOOM map editor or even a text editor, you are directly modifying the same file.  If the file is precious, a wise person makes regular backups of it, and AJBSP provides an option to backup each file before processing it."

This is just plain wrong. Most editors and tools will load something from disk and then edit it in memory. They will only overwrite the stored data on disk when you use the save function. The file is never modified unless you the user explicitly tell the program to do so. Any program that silently overwrote what was stored on disk would be considered broken by the user. I often use the save-as function when doing something experimental on a file. If I had to use save as before I started modifying it, that would be very unintuitive.

When I edited maps with Deth I usually gave the map  a 4-5 letter long name and added 000 to it and increased that number every time I saved, more or less never overwriting my old file. I'd then run a bat file that built the map with the wad file as an argument, and output the data to a fixed file name which was then loaded by doom2.exe.

Share this post


Link to post
13 hours ago, kb1 said:

If someone could pull it off, this would be a neat option for map editors that use a software-rendered 3D mode. It wouldn't have to be optimal - you'd still do a final full rebuild, and, maybe you'd also include a "Refresh" tool that does the full rebuild during editing. But, for simple changes, you could do the "continuous calculation". It is a fascinating idea, but it sounds very complex. You'd need an "unsplit" function, and other reverse calculations, I'd think.

Actually, you don't really need an unsplit. You only need to store what was there before the split. It would require a bit more memory, but in the grand scheme of things, this wouldn't be that complicated. Once you have split the tree in two, you have also lowered the amount of data that needs to be stored at the next level in the bsp tree. BSP trees aren't really suited to this though, there are probably other more efficient data structures that can be used.

Share this post


Link to post
12 hours ago, zokum said:

@andrewj

"Firstly, and least importantly, I disagree about in-place modification.  When you run a DOOM map editor or even a text editor, you are directly modifying the same file.  If the file is precious, a wise person makes regular backups of it, and AJBSP provides an option to backup each file before processing it."

This is just plain wrong. Most editors and tools will load something from disk and then edit it in memory. They will only overwrite the stored data on disk when you use the save function. The file is never modified unless you the user explicitly tell the program to do so. Any program that silently overwrote what was stored on disk would be considered broken by the user. I often use the save-as function when doing something experimental on a file. If I had to use save as before I started modifying it, that would be very unintuitive.

The difference, of course, is that the editors have a GUI, and expect the user to choose when to save. A command-line doesn't edit files until you run it - invoking a command-line tool is the implicit 'Save/Overwrite' button.

 

But, yes, I must agree that a new nodebuilder should expect to be invoked just like all other current nodebuilders, with the same arguments in the same positions, for convention and courtesy, if nothing else. Custom options can be tacked on with explicit switches. The nodebuilder author should expect that users will swap out nodebuilders for various reasons - it stands to reason that changing the executable filename should be all that is required, for the most used cases anyway.

 

11 hours ago, zokum said:

Actually, you don't really need an unsplit. You only need to store what was there before the split. It would require a bit more memory, but in the grand scheme of things, this wouldn't be that complicated. Once you have split the tree in two, you have also lowered the amount of data that needs to be stored at the next level in the bsp tree. BSP trees aren't really suited to this though, there are probably other more efficient data structures that can be used.

Storing the state before a split IS the functional equivalent of doing an unsplit, is it not?

 

The semantics are low-level implementation details, and not important during a high-level discussion of the possibility, feasibility, and usefulness of implementing such a system. As far as low-level details go, I'd be more interested in determining which branches need rebuilding. Eternity's polyobject implementation comes to mind. If built efficiently, it may even become useful to ports at runtime, in some limited form, allowing some architectural changes too involved or too awkward for polyobjects. There are lots of possibilities, and the level of efficiency would determine the practicality of attempting a runtime implementation.

 

As far as BSP not being suited to this approach, I'd like to see a BSP-like setup that accounts for 3D floors natively, to allow hidden surface removal to work as it does for 1S walls, vs. being drawn like 2S walls and sprites, which is very inefficient.

 

Share this post


Link to post
12 minutes ago, kb1 said:

As far as BSP not being suited to this approach, I'd like to see a BSP-like setup that accounts for 3D floors natively, to allow hidden surface removal to work as it does for 1S walls, vs. being drawn like 2S walls and sprites, which is very inefficient. 

Quake has BSP in 3d. All of the moving stuff in that game is a type of entity like players, monsters and ammo, and not part of the static bsp.

Share this post


Link to post

Do you have any sample on how to render the output glBSP? I would love to use Eureka + AJBSP on my own GL engine to simplify mapping. Today I use Tiled and convert to 3D prefabs, but it sucks.

 

Or maybe an option to export OBJ?

Share this post


Link to post
14 hours ago, mmx said:

Do you have any sample on how to render the output glBSP?

No I don't, sorry.

 

There may be good programs around which can convert a DOOM map to an OBJ file.  Once upon a time I experimented with it, but never finished it (for example, my program never added texturing information).

Share this post


Link to post

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×