Shadowcaster modding

[DaniJ]

Firstly, this isn't strictly DOOM-related so it might not belong here.

Over the weekend I found myself without an internet connection so I decided to try something I had been meaning to get around to doing for a couple of years now; Hacking apart Shadowcaster to see just how far it's engine had evolved from that used for Wolfenstein3D.

Shadowcaster can in many ways be considered as a "missing link" between between the Wolfenstein3D and DOOM engines. Perhaps not on a technical level (as there is a lot of stuff in Shadowcaster that is not present in either engine) but certainly conceptually.

The Shadowcaster engine features what I believe to be the ancestor of the WAD format as there are many similarities to support this.

Now, on to the point of this thread:

Is there any interest at all in modding for this game? Or at least being able to edit/extract the graphics? I now have enough info to at the very least extract and convert the various resources and could put together a set of utils fairly quickly if there is any interest.

[kristus]

Interesting trivia about Shadowcaster BTW. I installed it a while ago, also to play it. Had played it a bit back in the day, but never got into it really... Anyway.

It works pretty much flawlessly in WinXP, the mouse was a bit fidgety. But I had sound and it was running solid.

[Gez]

I for one would be very interested. Someone already managed to extract many resources (with tools like Xentax MultiEx Commander, Multiripper, and plain old-fashioned text editors), but not the sprites.

Getting at least the rest of the resources could allow to recreate Shadowcaster as a clone. Being able to mod the original game would be even more fun! It seems it had some qualities that Raven Software brought back in Hexen, namely hubs and some scripting capacities.

[Snarboo]

I would definitely be interested in something like this. I always thought Shadowcaster was a novel concept.

[jute]

I'd love to see something like this. Shadowcaster is definitely a gem and it would be nice to see it interpreted in a modern Doom port, or to see some mods.

[DaniJ]

The chances of a modern port similar to those available for DOOM, Hexen etc are pretty slim given that the original source for Shadowcaster was never released. In order to do such a thing would either require a complete reverse engineering of the original (which is quite feasible I might add. I can confirm that some of the core technology does indeed stem from that used by Wolfenstein3D and various subsystems of Shadowcaster do resemble those found in DOOM. This means that it would be possible to use the sources for those games for reference purposes to help speed the process) or simply writing new code for everything.

I'm not offering to do either as to be frank, I have more than enough on my plate right now what with Doomsday and the other projects I'm involved with.

What I am offering is a set of tools that would allow for the original resources to be extracted (and possibly replaced) for use with the original Shadowcaster executable.

In addition, if anyone is interesting in reverse engineering Shadowcaster or just wants to know what I've learnt through my probing I'd be happy to document it.

With regards to the game itself I must confess to never really getting into it and found the Ultima:Underworld-like interface too clumsy and it really does get in the way of me enjoying it.

[Gez]

I am very interested in tools that could extract monster sprites and in any reverse engineering pointer that would help knowing the statistics and behaviors of the various monsters and weapons. (Health, damage, speed, and so on.) That would allow to make a TC, if not a port.

[Enjay]

I too would be interested in poking around in the resources just to see what's there.

And kind of related, sort of, Cyclones... are there any tools that can pull its resources apart? I know that the sounds are simple RAW files but I don't know of anything that can read the graphics (I tried XWE which recognised the some of the file extensions but refused to open the files).

[Vegeta]

I'd love to see it's resources. I'm sure that a cool Heretic/Hexen mod could be done with them, besides other things.

[DaniJ]

I hadn't even heard of Cyclones before I googled it a few minutes ago. I could try an pick up a copy of Cyclones and have a poke around. It's engine doesn't appear to be derived from any 2.5d engine I am aware of but I can certainly take a look.

Anyway, there does seem to be enough interest to warrant me doing this for Shadowcaster so I'll knock something up over the weekend.

[Enjay]

Cyclones is at HotU. I think it was originally going to use a similar engine to Shadowcaster but they eventually went with their own ground-up engine. If memory serves, somewhat amusingly, the engine was known as Steam.

[jute]

This site, mentioned earlier by Gez, has some partial CyClones and ShadowCaster resources, ripped the hard way.

[DaniJ]

Oh believe me there is nothing easy about this. Its not like I have some magic tool that does the work for me. Its a case of knuckling down with a hex editor, calculator and a fair bit of lateral thinking.

I must admit though that in Shadowcaster's case, the job is made easier by the fact that there isn't any secondary compression on the resources.

[Kaiser]

Interesting. I never even knew Raven developed it either..

[jute]

DaniJ said:

Oh believe me there is nothing easy about this. Its not like I have some magic tool that does the work for me. Its a case of knuckling down with a hex editor, calculator and a fair bit of lateral thinking.

Oh, I don't mean to imply otherwise! I meant "the hard way" as opposed to any of the hypothetical utilities you've discussed.

[Vegeta]

jute gyte said:

This site, mentioned earlier by Gez, has some partial CyClones and ShadowCaster resources, ripped the hard way.

Hey thanks, I always wanted to get the sprites of that Cyclones' monster.

[Guest DILDOMASTER666]

How's about a port of ShadowCaster to a Doom engine? I've had the idea for a while and I've actually been able to "partially" reverse engineer the levels on a small scale. Nothing really amazing at all honestly, but a start.

[Gez]

Fisk said:

How's about a port of ShadowCaster to a Doom engine? I've had the idea for a while and I've actually been able to "partially" reverse engineer the levels on a small scale. Nothing really amazing at all honestly, but a start.

You'll need to also:
* Recreate faithfully the monsters' behavior
* Recreate the inventory system as close as possible to the original
* Recreate the morphing powers
* Recreate the XP/"surge of life and power" system
* Recreate the dynamic music system

Now, pretty much all that should be possible in ZDoom with a lot of ACS and advanced DECORATE scripting. Though the shared and separated inventories could still be a problem; it'd be simpler to just do it the Hexen way and share everything constantly.

[Guest DILDOMASTER666]

If anyone's still interested, I will post some interesting findings about the LIB files. I've not fully cracked them, but I believe I know some of the formats mostly now. I've begun extracting chunks of the LIBs (map files, item attribute lists, etc) and will post those as well if you guys want.

EDIT: I've figured out part of the .map file format, by the way. Enough to change textures and structure.

[jute]

Please post these findings!

[Guest DILDOMASTER666]

As previously mentioned by DaniJ, Shadowcaster's 2 .LIB files are basically the predecessor to the WAD format. There appears to be no valid header data, instead it just begins with the raw data and at 0x819fe you can find the beginning of a directory structure. The format is as follows:

1 byte unused, always 0x00
4 bytes size of lump, little-endian
4 bytes absolute offset, little-endian
12 bytes DOS name, zero-padded (e.g., 8 characters + extension)

Here's an archive of all of the first level's data : http://www.zshare.net/download/172104320ea0a2a2/

ruinsa.map is the map structure file. It begins with a series of floor texture IDs, which are little-endian short (2-byte) values. The little-endian value "01 00" is used for all areas that are solid walls. Eventually I will post a list of valid texture indexes. That's all I know on changing maps for now. ruinsa.ojt seems to be a list of decorations with actual names for the objects instead of an ID. ruinsa.dor seems to be a list of door textures...?

Here are the offsets for each of ruinsa's datafiles inside of SHADOW.LIB :

Unknown	: Size (dec)	: Offset (dec)	: Name (12char)

00 	: 	3866 	: 0000 		: ruinsa.scr
00	:	13312	: 3866		: ruinsa.map
00	:	6546	: 17178		: ruinsa.arc
00	:	6262	: 23724		: ruinsa.crt
00	:	832	: 29986		: ruinsa.dor
00	:	3933	: 30818		: ruinsa.ojt
00	:	336	: 34751		: ruinsa.itm

[Gez]

DaniJ said:

Anyway, there does seem to be enough interest to warrant me doing this for Shadowcaster so I'll knock something up over the weekend.

So, have you succeeded in doing so? :) (Perfectly understandable if you didn't for a reason or another, I just want to know.)

Fisk said:

The format is as follows:

Very interesting. Where is the height information?

[DaniJ]

No I didn't get around to it unfortunately. I'll try and get something done soon as.

[Gez]

Alright, thanks.

Maybe you should contact RaVeN (the guy who made this thread), he's apparently working with someone from the Xentax (ripping/reverse engineering software) forums to decode sprite format; so if you need help they may be able to help you.

[Guest DILDOMASTER666]

Gez said:

Very interesting. Where is the height information?

The height data for the entire level appears to be stored in one specific place and isn't defined for each individual cell. I'm not sure where it is, but maybe when I log on later I'll have more info for you.

EDIT: There's something that's puzzling me... Has anyone observed areas that have more than one different ceiling texture on the same level? My guess is that both bytes of the floor texture data present in the .map files are both strictly to define the floor texture, and that there's a header or some such in one of the files that defines a ceiling type (either a texture ID or a sky number). On this note, I have a theory that the .map files are composed of layers of tiles, like Wolfenstein 3D. I have made no attempt as of yet to open them with TED5 (I imagine this is what Raven used to make the .map files, but I haven't been able to get it to run, and I don't have a list of texture indexes to refer to while editing). I fear though that one of the map files is some kind of BSP data...

I also have reason to believe that the maps are 64x64 tilemaps, like Wolf3D. I'll be documenting more of my findings on the map formats in this thread later today.

Also available on Raven's site is a module for UGE which reveals some things about the savegame format: http://www.gamesdomain.com/directd/pc/dos/tools/shadcmdl.html

[DaniJ]

No there is no BSP used with Shadowcaster. Maps are indeed based on the same grid of square tiles as with Wolfenstein3D.

[Guest DILDOMASTER666]

If the beginning of the .map file is 0x00, then 0x1800 is where the walls data starts. I have no idea how it works entirely yet, but you can change textures by changing the high byte (?) of the short values after that point. A short value of "00 00" seems to be the "no wall here" code.

What is very strange is that the maps seem to be 32x32...

[Gez]

Fisk said:

What is very strange is that the maps seem to be 32x32...

In mapunits, that's 2048x2048. Plenty of place.

[Guest DILDOMASTER666]

Wall texture data is stored separately for horizontal-running and vertical-running textures. The vertically-running textures data is found right after the floor heights data. Here is my guess for the way the layers are ordered in the .maps. The offsets are relative to the beginning of the file.

LAYER ORDERING:

0x00  : Floor Textures
0x800 : ???
0xC00 : Ceiling Textures?
0x800 : Ceiling heights
0x1800: Vertical wall textures
0x1400: Floor Heights??
0x2000: ???
0x2400: Horizontal wall textures
0x2800: ???

EDIT: Have been busy with my Wolfenstein 3D project from forever ago, so updates will not be quite as frequent (or annoying). But I have a theory that textures are drawn in a kind of "perpetual lower-unpegged" mode and that assigning the proper texture type (horiz or vert) to a gap between two floors will just draw the texture properly if there is a big enough gap between the two floors. I also updated the layer ordering list.

[Gez]

Vertical and horizontal wall textures? Do you mean X-axis and Y-axis on the map?