Doomworld Forums - Packing WADs.

Doomworld Forums : Powered by vBulletin version 2.2.5

Doomworld Forums > Classic Doom > Doom Editing > Packing WADs.

Author

All times are GMT. The time now is 18:00.

Lüt
YA-HA

Posts: 12245
Registered: 05-00

I've passed that silly 30,000 sidedef limit (again) on a map I'm doing. I've heard that I can "pack" the sidedefs to reduce the total by more than half, but before I get too carried away adding extra stuff to this level, I want to know if there's any more of these limits that I have to be careful of. I heard there's a blockmap size limit too, as well as possibly nodes. Does packing the sidedefs also reduce the overall size of other aspects such as blockmap, reject and nodes, or do I even have to worry about these other aspects at all?

07-02-01 03:43 #

Profile || Blog || PM || Email || Homepage || Search || Add Buddy

IP || Edit/Delete || Quote

stphrz
I use a blowtorch to make crème brûlée

Posts: 997
Registered: 05-00

Does packing the sidedefs also reduce the overall size of other aspects such as blockmap, reject and nodes...?

The block map is used for collision detection.
The reject table is made by calculating lines of sight between sectors.
The nodes are used for sorting out what and what is not visible for any view on the map and what the engine needs to draw on screen.

So the answer to your questions would be: no, no, and no.

Sucks don't it :P

You can reduce the size of the reject table by merging sectors to lower the overall number though.

Some nodes builders are more efficient than others and can create a smaller nodes lump. There are limits to this though. The difference won't be huge. 10k or so at the most.

The only way to lower the blockmap size is to chop map structures and reduce it's overall size and number of vertices. This is because the data for each vertex is unique.

07-02-01 03:53 #

Profile || Blog || PM || Email || Search || Add Buddy

IP || Edit/Delete || Quote

Lüt
YA-HA

Posts: 12245
Registered: 05-00

So the answer to your questions would be: no, no, and no.

Sucks don't it :P

Not exactly: I haven't run out of space for those aspects yet. I'm talking about the engine being able to load the level, not the map editor. I know there's a sidedef limit, but I don't know if there's max limits for anything else or not. I really really don't want to pack sidedefs and sectors because then the level size isn't "true" but if it means I can make it full-size, then I guess I have to :(

The current map alone is 4MB, now it's gonna get killed :| I was hoping to hit 5MB before I was done. Then the last 5 levels would be bigger than the entire Doom2 game.

07-02-01 10:39 #

Profile || Blog || PM || Email || Homepage || Search || Add Buddy

IP || Edit/Delete || Quote

DeePTeam
Member

Posts: 127
Registered: 05-00

... the engine being able to load the level, not the map editor.

Both the engine AND the editor are constrained by the format of a "linedef". A sidedef reference is only a 16bit value - 64k absolute or 32k signed. ALL the limits that exist are because of 16bit level format values.

A blockmap can be 128kb only for BOOM (and one's that copied that code), but not for ZDOOM (and one's that copied that code).

Packing sidedefs does make a huge difference in map size. This step is reversable. It DOES extend the size of the level you can make - typically by at least 30 percent. You -have- to reverse it to be able to edit without introducing too many problems. Just be aware that it's possible to make a packed level you can't unpack:))

Packing sectors may make some size difference. However, that step is not reverseable. Some effects are sensitive to packing sectors (generally not a problem). Pack sectors before packing sidedefs.

Packing sidedefs and packing sector is on the file menu in DeePsea. Something that is not obvious (don't know if anyone every realized it), but CENTERING a large dimensional map is also required for exactly the same reasons as stated above.

Blockmap size is first of all controlled by the OUTSIDE dimensions of your level. So if you can trim the overall dimensions, you'll find that it makes a big difference. DeePbsp "packs" the blockmap.

When did you register?

07-02-01 16:59 #

Profile || Blog || Email || Homepage || Search || Add Buddy

IP || Edit/Delete || Quote

Lüt
YA-HA

Posts: 12245
Registered: 05-00

Both the engine AND the editor are constrained by the format of a "linedef". A sidedef reference is only a 16bit value - 64k absolute or 32k signed. ALL the limits that exist are because of 16bit level format values.

So what that's saying is that no source port will ever be able to remove the limits unless they completely rewrite the WAD code in 32-bit, which would require a different file format and different editor to use this format?

A blockmap can be 128kb only for BOOM (and one's that copied that code), but not for ZDOOM (and one's that copied that code).

This means that ZDOOM doesn't have a blockmap limit, or just has a larger one?

Packing sidedefs does make a huge difference in map size. This step is reversable. It DOES extend the size of the level you can make - typically by at least 30 percent. You -have- to reverse it to be able to edit without introducing too many problems. Just be aware that it's possible to make a packed level you can't unpack:))

I know about the sidedef packing already, I tried to fix-up a packed level - didn't go over so well :P

Anyways, I'll say it makes a different: 27,767 sidedefs packed (so far). Jeez, I went from over 30,000 to about 3,000. Now I really can make this map as big as I want :)

Packing sectors may make some size difference.

There's an understatement; it packed 3284 sectors out of a little over 4,000. I haven't had time to investigate, but I'm assuming it doesn't pack sectors which are the same except for linedef tags, right?

Blockmap size is first of all controlled by the OUTSIDE dimensions of your level. So if you can trim the overall dimensions, you'll find that it makes a big difference. DeePbsp "packs" the blockmap.

Packs it automatically? I'll try than then. Usually I use BSP because there's less oddities in maps with precise details.

When did you register?

DeePsea? About a month ago.

07-02-01 22:25 #

Profile || Blog || PM || Email || Homepage || Search || Add Buddy

IP || Edit/Delete || Quote

Lüt
YA-HA

Posts: 12245
Registered: 05-00

Now I really can make this map as big as I want :)

Scratch that. Even packed, it still crashes MBF with some kind of DOS-Prompt jibberish and ZDoom gets a "Close/Details" box. Here's the compressed data, maybe I've got too much of something:

Things: 1777
Vertices: 14107
Linedefs: 17096
Sidedefs: 3136 (31343)
Sectors: 655 (4024)
Segs: 35678
SSectors: 12040
Nodes: 12039
Reject: 53629
Blockmap: 98980

I noticed the reject dropped considerably too. Well I'm off to do more testing, maybe I'll find out what's wrong eventually.

07-02-01 23:02 #

Profile || Blog || PM || Email || Homepage || Search || Add Buddy

IP || Edit/Delete || Quote

cph
Junior Member

Posts: 208
Registered: 05-00

Now I really can make this map as big as I want :)
Scratch that. Even packed, it still crashes MBF with some kind of DOS-Prompt jibberish and ZDoom gets a "Close/Details" box. Here's the compressed data, maybe I've got too much of something:

If it crashes PrBoom, send me a copy and I'll debug it for you. Debugging is my speciality :-)

07-02-01 23:22 #

Profile || Blog || Email || Homepage || Search || Add Buddy

IP || Edit/Delete || Quote

Lüt
YA-HA

Posts: 12245
Registered: 05-00

That would be great, except that it's not done... it would be at least a little while until I actually finished the map. And after I do get to see all the stuff I've added in-game, I still may change some stuff around. Unless you only have to do the debug thing once.

Would you want the packed or unpacked version?

Also, I can't play this on PRBoom because it's for Millennium, which is a derivative of MBF. I know I can play it with ZDoom because ZDoom negates things it can't find when the level loads.

07-02-01 23:44 #

Profile || Blog || PM || Email || Homepage || Search || Add Buddy

IP || Edit/Delete || Quote

Fanatic
Old Post

07-03-01 01:07

This person is on your Ignore List. To view this post click [here]

stphrz
I use a blowtorch to make crème brûlée

Posts: 997
Registered: 05-00

I noticed the reject dropped considerably too

.

Yeah the reject table for 655 sectors only has roughly 1/37th the number of bit entries than the reject table for 4024 sectors.

I think what is happening is, if the level exceeds engine limits when it's not packed, packing it doesn't help. Like the engine is saying "you don't fool me :P." Or something. =)

07-03-01 22:01 #

Profile || Blog || PM || Email || Search || Add Buddy

IP || Edit/Delete || Quote

DeePTeam
Member

Posts: 127
Registered: 05-00

..crashes MBF with some kind of DOS-Prompt jibberish and ZDoom gets a "Close/Details" box.. Blockmap: 98980

Just got back, been out getting wet.

I assumed everyone knew about the 64kb limit so I didn't bother listing it. ZDOOM (+derivatives) ports need to redefine the blockmap offsets in the code to unsigned and all better. Right now it's still signed.

When I mentioned this to Randy, he indicated he might just drop the blockmap (not really required), but he hasn't done so yet. So ZDOOM right now can't go over 64kb. MBF should be able to go over if he copied the BOOM code (and I don't see why he wouldn't)?

I've run BOOM with 120KB blockmaps a while back when someone created this huge level that was then creating a DeePbsp error message for a blockmap over 64kb. AFAIK the other node builders don't even bother checking for this problem.

So here's what you need to do:

1. In DeePsea, enable the Big BlockMap option in F5 / Nodes if you are not using the BOOM project (it's turned on by default only for BOOM).

2. Use DeePbsp to create a blockmap over 64kb.

If you do both of these and it doesn't work please send me the level. It could be something else I'm not thinking of.

07-07-01 19:23 #

Profile || Blog || Email || Homepage || Search || Add Buddy

IP || Edit/Delete || Quote

DeePTeam
Member

Posts: 127
Registered: 05-00

Forgot to mention - all the ports can double the 16bit sizes by just making the 16bit references unsigned. -1 is a special case so the limits would be 64kb -1, instead of 32kb. Node builders also need to be changed in a similar fashion.

None of this is a lot of work - mostly grunt checking - and it's all compatible with existing WADs. But seeing how no port is willing to support tall textures (480 or so) which is a LOT more useful, I don't see any progress on this front in the near (far?) future.

07-08-01 05:54 #

Profile || Blog || Email || Homepage || Search || Add Buddy

IP || Edit/Delete || Quote

Lüt
YA-HA

Posts: 12245
Registered: 05-00

64 Blockmap Limit

Actually, I did know about that, but I also knew that most ports were supposed to have removed that. Now that I look at my other maps, none of them have a blockmap of above 64K (surprisingly enough, although they're close to the same size as this one which has a blockmap twice as large).

In DeePsea, enable the Big BlockMap option in F5 / Nodes if you are not using the BOOM project (it's turned on by default only for BOOM).

OK, this must have been on already because I rebuilt nodes with the BlockMap Max 131,070 on and off and each time the BlockMap came to 81K. I had build the nodes in BSP before and that BlockMap size was 97. I tried the level with these different configurations and each crashed MBF with some sort of stack dump and ZDoom with a Close/Details box.

glbsp in.wad -o out.wad -packsides -nogl -noreject

Just tried this; no serious warnings but 300 minor warnings. Same error as all the others.

If you do both of these and it doesn't work please send me the level. It could be something else I'm not thinking of.

How big of attachments can you recieve? I also need to send the engine and resource to go with it, it's kinda big for you to be getting over a dialup if that's what you have. I'll just FTP it somewhere and give you the link if you want that instead.

07-09-01 01:24 #

Profile || Blog || PM || Email || Homepage || Search || Add Buddy

IP || Edit/Delete || Quote

DeePTeam
Member

Posts: 127
Registered: 05-00

How big of attachments can you recieve?... I'll just FTP it somewhere and give you the link if you want that instead.

Either one will do fine. Size is not a prob. I can receive just about anything. My particular cable connection rocks - so even 50mb takes no time at all.

Oh, only ports that copied BOOM exceed the 64kb limit. ZDOOM does NOT work with > 64kb for sure.

07-09-01 17:47 #

Profile || Blog || Email || Homepage || Search || Add Buddy

IP || Edit/Delete || Quote

Lüt
YA-HA

Posts: 12245
Registered: 05-00

OK, I'll have it up tomorrow then. The unpacked version.

Cause I still have some work to do on it. Unless you want to wait till I'm totally done before trying a fix.

07-09-01 18:06 #

Profile || Blog || PM || Email || Homepage || Search || Add Buddy

IP || Edit/Delete || Quote

DeePTeam
Member

Posts: 127
Registered: 05-00

The current one will be fine - just going after whatever is causing the current problem. I hope it can be worked out.

07-10-01 03:38 #

Profile || Blog || Email || Homepage || Search || Add Buddy

IP || Edit/Delete || Quote

DeePTeam
Member

Posts: 127
Registered: 05-00

OK, I'll have it up tomorrow then. The unpacked version.

If you have it ready, could you give me the link so I can check it out?

07-12-01 17:03 #

Profile || Blog || Email || Homepage || Search || Add Buddy

IP || Edit/Delete || Quote

Lüt
YA-HA

Posts: 12245
Registered: 05-00

Oooooohhhhhhhh yeah, knew I was forgetting something :P Check your mail.

07-12-01 23:13 #

Profile || Blog || PM || Email || Homepage || Search || Add Buddy

IP || Edit/Delete || Quote

DeePTeam
Member

Posts: 127
Registered: 05-00

Ok, got the level and noticed the prob - should have noticed this sooner - although I did sort of mention it earlier:) The number of Segs exceeded 32k (35678).

This can be fixed in a rather painful fashion by reducing the number of angled lines in the level - OR- get rid of a lot of the little details "blocks" -OR- just eliminate a section. The prior version is just below the limit - so that gives you a good idea of what to do.

This is somewhat harder for the ports to solve since it involves changing the node structure pointers to 4 bytes. Doubt that will happen.

Just for kicks I forced it to 32k and the level at least loads. Then it crashes anyway, since the segs become invalid.

I've added a section to DeePbsp to warn that this has occurred.

Did you make all those levels? Nice work.

07-13-01 19:57 #

Profile || Blog || Email || Homepage || Search || Add Buddy

IP || Edit/Delete || Quote

Katgut
Post Out Of Order

Posts: 674
Registered: 05-01

Ok, got the level and noticed the prob - should have noticed this sooner - although I did sort of mention it earlier:) The number of Segs exceeded 32k (35678).

This can be fixed in a rather painful fashion by reducing the number of angled lines in the level - OR- get rid of a lot of the little details "blocks" -OR- just eliminate a section. The prior version is just below the limit - so that gives you a good idea of what to do.

This is somewhat harder for the ports to solve since it involves changing the node structure pointers to 4 bytes. Doubt that will happen.

Just for kicks I forced it to 32k and the level at least loads. Then it crashes anyway, since the segs become invalid.

I've added a section to DeePbsp to warn that this has occurred.

Did you make all those levels? Nice work.

I believe this could also be fixed by rebuilding the nodes with a splitfactor as low as possible.

(I monkey with the splitfactor on large maps to try and balance the nodetree. Lut probably should monkey with it so as to have lines broken less often into segs.)

07-14-01 00:32 #

Profile || Blog || Email || Search || Add Buddy

IP || Edit/Delete || Quote

DeePTeam
Member

Posts: 127
Registered: 05-00

I believe this could also be fixed by rebuilding the nodes with a splitfactor as low as possible.

Not really.

If anything it would be a HIGHER factor - that makes the code try longer:) He's 2900 over and that's a lot to get down. I got it down by 1000 by using DeePbsp and varying some DeePbsp options, but that's still not enough.

He also needs some breathing room, so we need to get it down to at least 31000!

07-14-01 03:04 #

Profile || Blog || Email || Homepage || Search || Add Buddy

IP || Edit/Delete || Quote

Lüt
YA-HA

Posts: 12245
Registered: 05-00

Ok, got the level and noticed the prob - should have noticed this sooner - although I did sort of mention it earlier:) The number of Segs exceeded 32k (35678).

This can be fixed in a rather painful fashion by reducing the number of angled lines in the level - OR- get rid of a lot of the little details "blocks" -OR- just eliminate a section. The prior version is just below the limit - so that gives you a good idea of what to do.

Aw jeez... segs... wtf, who needs those anyways?? Thing just can't draw the damn lines...

Well that screws over a few other levels I'm halfway through with.

Did you make all those levels? Nice work.

You mean every level in that game? No, I only did 21, 22, 23 and 24. I just noticed 23 is a old old version from 1999, dunno how that one got in there.

Anyways, thanks. You believe I put all those together with WadEd?? :P

He also needs some breathing room, so we need to get it down to at least 31000!

Grrrrr.....

Oh well, looks like the truly final version will occur if ZDoom implements Millennium support into it :(

07-14-01 10:30 #

Profile || Blog || PM || Email || Homepage || Search || Add Buddy

IP || Edit/Delete || Quote

DeePTeam
Member

Posts: 127
Registered: 05-00

Aw jeez... segs... wtf, who needs those anyways?? Thing just can't draw the damn lines...

Segs define the "visible" area. The code really doesn't need it:) If you look at the alpha levels you'll see that none of this exists. On a 386 it was critical to get the speed up, so they hired some math consultant and he taught them all about BSP trees so they could cull faster.

The ports could either drop using this -OR- do the BSP on the fly. That would take a few seconds as each level loads. Could be done only for those that need it. An auto-detect that sees there are too many segs.

Oh well, looks like the truly final version will occur if ZDoom implements Millennium support into it :(

The level isn't that far off. Straighten a few lines that won't make any diff (set the grid to 256 or so and use shift+A). There is enough "extra" detail you can remove to make it fit and not change the level very much (from my POV).

The DeePbsp option with the fewest Segs is "Type 3". This option is a -lot- slower, but it does cut it down. I eliminated the "right" outdoor section and straightened a few lines to get it inside the bounds - but I don't know what is important to you - just showing one option.

07-14-01 17:23 #

Profile || Blog || Email || Homepage || Search || Add Buddy

IP || Edit/Delete || Quote

Lüt
YA-HA

Posts: 12245
Registered: 05-00

The level isn't that far off.

But also note that what I sent is about 2/3 complete :) Those big empty rooms aren't staying that way.

I do have somewhat of an idea, I am trashing those stupid crate rooms from the beginning area and doing that one over. Zaldron's actually retexturing the whole level, so that may rearrange some scenes (like that hell building - ick - that's becoming some kinda cryotank freeze chamber) and I'm redoing a few other parts too. I guess that whole brown-secret-hallway thing sucks as well; that's from the first version of the level I made in 1995, it was inspired by that one scene in E1M4 of the same nature. Probably doesn't fit in the level now. I suppose it doesn't actually need the roads or lot either. But they just looked so cool :(

Oh well. Something'll come up. I don't suppose packing all the sidedefs reduces the segs, does it?

07-15-01 02:13 #

Profile || Blog || PM || Email || Homepage || Search || Add Buddy

IP || Edit/Delete || Quote

Katgut
Post Out Of Order

Posts: 674
Registered: 05-01

The level isn't that far off.
But also note that what I sent is about 2/3 complete :) Those big empty rooms aren't staying that way.

I do have somewhat of an idea, I am trashing those stupid crate rooms from the beginning area and doing that one over. Zaldron's actually retexturing the whole level, so that may rearrange some scenes (like that hell building - ick - that's becoming some kinda cryotank freeze chamber) and I'm redoing a few other parts too. I guess that whole brown-secret-hallway thing sucks as well; that's from the first version of the level I made in 1995, it was inspired by that one scene in E1M4 of the same nature. Probably doesn't fit in the level now. I suppose it doesn't actually need the roads or lot either. But they just looked so cool :(

Oh well. Something'll come up. I don't suppose packing all the sidedefs reduces the segs, does it?

Nope.

Roads? Lot? You have a parking lot? w0000000000000000000000000000000000t :->

07-15-01 06:46 #

Profile || Blog || Email || Search || Add Buddy

IP || Edit/Delete || Quote