Search In
• More options...
Find results that contain...
Find results in...

# get string of bits of wad in python

## Recommended Posts

Essentially I want a python program to open a wad file to get the data of where the points/lines etc are, edit that data in some way, and resave the edited data as a new wad.

Python seems to suck at bit/byte level stuff. My idea is convert the bits of the wad into a string of 0s and 1s, then fiddle with that, or at least test if I got the correct bits according to wad file format:

Well I made this short program.

```def getbytestring(a):
b=['0','0','0','0','0','0','0','0']
if a&1:
b[7]='1'
if a&2:
b[6]='1'
if a&4:
b[5]='1'
if a&8:
b[4]='1'
if a&16:
b[3]='1'
if a&32:
b[2]='1'
if a&64:
b[1]='1'
if a&128:
b[0]='1'
return ''.join(b)

for i in range(1000):
print i, '    ',ord(r),'    ',getbytestring(ord(r))
```
it outputs:
```0      80      01010000
1      87      01010111
2      65      01000001
3      68      01000100
4      11      00001011
5      0      00000000
6      0      00000000
7      0      00000000
8      37      00100101
9      248      11111000
...etc
```
So, I don't know, how can I test that the index 8 byte of the real wad matches 00100101 which this program output? Or does it look correct/etc?

##### Link to post

I successfully used omgifol before. It works good, but the code internals were tricky for me to understand at least. I want to implement my own simpler version so I actually know what's going on, and I could have my own program 'import' a wad without relying on external packages etc.

##### Link to post
gggmork said:

My idea is convert the bits of the wad into a string of 0s and 1s, then fiddle with that, or at least test if I got the correct bits according to wad file format

oh god please don't do this. use numpy to read binary data:

```import numpy as np

# reading unsigned short from file:
my_us = np.fromfile(f,'<u2',1)[0]

# reading an array of 10 int32s from file:
my_ints = np.fromfile(f,'<i4',10)```
similarly you could use some combination of the read() and np.fromstring() functions. For writing files back out use a combination of write() and np.tofile()

I have some scripts laying around that I use for wad I/O in python (basically because the wad format is extremely easy to work with, and I wanted to build something myself instead of using omgifol, for whatever reason) that I can PM to ya if you want.

##### Link to post

Oh yeah, I just re-found genTerrainRibbiks.py in my 2013 folder. Do you have additional stuff besides that? None of the examples I searched for on the internet to manipulate file bytes even mentioned numpy. And I was getting confused messing around with the 'struct' class, trying to decide whether to interpret the data as an int or char since the latter has less bytes, ord confusion, no knowing if file.read() returns a string or a byte or an int etc. or if all are equivalent, and having iterations of a loop completely skip on some occasions for weird reasons and yada yada ya. But if you have since made additional stuff similar to genTerrainRibbiks.py, let me know. Or I can just study that, should have remembered to do that in the first place. Probably in the back of my mind I assumed I wouldn't find it again on this comp.

The first thing I wanted to do was draw sectors that represent ceilings only, draw a separate area of sectors that represent floors only. Maybe each have a mapspot or point representing the upper left corner of each, so they can be placed on top of eachother in 2 layers and combined, so both could have unique patterns and combine into a stained glassish window, which would involve finding points of line intersections and stuff, which I've been wanting to do for awhile but never got around to.

The 'wad to string of bits' would just be too slow or something? I guess its pointless to convert to a string if you can read the file as bits in the first place, but that didn't seem easy to do after searching internet suggestions.

##### Link to post
gggmork said:

The first thing I wanted to do was draw sectors that represent ceilings only, draw a separate area of sectors that represent floors only. Maybe each have a mapspot or point representing the upper left corner of each, so they can be placed on top of eachother in 2 layers and combined, so both could have unique patterns and combine into a stained glassish window, which would involve finding points of line intersections and stuff, which I've been wanting to do for awhile but never got around to.

oy, that sounds pretty cool. but also very complicated. For arbitrary geometry it's going to be a serious challenge to keep track of all the new sectors that the intersections create. You'll also have to create a buttload of new linedefs, sidedefs, and keep track of which ones reference all the sectors of the new geometry. In short, good luck :p

The 'wad to string of bits' would just be too slow or something? I guess its pointless to convert to a string if you can read the file as bits in the first place, but that didn't seem easy to do after searching internet suggestions.

speed's not really the issue, but there's just clean builtin ways to do it (though apparently they aren't documented that well if it was difficult to find information on).

Here's a quick example script I whipped up that you might find useful for seeing the numpy commands in action. it changes every chaingunner in a map into a cyberdemon, because that sounded like fun :p

```import sys
import numpy as np

return string+chr(0)*(length-len(string))

def byteSize(obj):
# python strings add 21 bytes to their actual length on my system, so I subtract it from getsizeof() to get the actual size.
# this might be different for you...
return sys.getsizeof(obj)-21

f.seek(dirP)
lmpDir = []
allData = []
for n in range(nLmp):
[lmpP,lmpS] = np.fromfile(f,'<i4',2)
lmpDir.append([name,lmpP,lmpS])

for n in lmpDir:
f.seek(n[1])

f.close()
return allData

nLmp = len(allData)
lmpSizes = [byteSize(n[1]) for n in allData]
totSize = sum(lmpSizes)
newDir = []

f = open(outFile,'wb')
np.array(nLmp,'<i4').tofile(f)
np.array(totSize+12,'<i4').tofile(f)

for i in range(len(allData)):
newDir.append([f.tell(),lmpSizes[i],allData[i][0]])
f.write(allData[i][1])

for n in newDir:
np.array(n[0:2],'<i4').tofile(f)
f.write(n[2])

f.close()

def main():

# lets print the size of every lump present in the file...
for n in wadData:
print n[0],byteSize(n[1])

#
# as an example lets change every chaingunner into a cyberdemon in MAP01...
#

thingData = []  # thingData[i] = [xpos,ypos,angle,type,flags]
for i in xrange(1,len(wadData)):
# consult doomwiki for the byte layout of each lump. THINGS lump consists of 10-byte entries containing 5 shorts
for j in xrange(0,byteSize(things),10):
[xpos, ypos, angle, ttype, flags] = np.fromstring(things[j:j+10],'<i2')
# if type == chaingunner, change to cyberdemon
if ttype == 65:
ttype = 16
thingData.append([xpos, ypos, angle, ttype, flags])

# construct a new things lump and replace the existing one
newThings = ''
for n in thingData:
newThings += np.array(n,dtype='<i2').tostring()

# write out new wad

if __name__ == '__main__':
main()```

The code is kinda sloppy, I just put this together in a few minutes. The entire point of something like omgifol would be to make this process a bit more elegant, but I have fun with this sort of thing so I don't mind dealing with the raw binary data. How you structure the code is gonna depend a lot on what you want to do, but for this example it was pretty much:

- read raw data from each lump in the wad and store it (equivalently) as a string in a list
- iterate through list, look for lumps we're interested in: THINGS
- parse binary data of things lump, edit values of interest (we can add, subtract, modify, whatever)
- turn modified values back into a binary string in preparation for writing it to output wad

doomwiki is awesome for describing how the bytes of each lump are laid out. for this example I briefly consulted:

http://doomwiki.org/wiki/Thing#THINGS_lump
http://doomwiki.org/wiki/Thing_types

##### Link to post

Thanks, this will come in very handy and should keep me busy for quite awhile (especially since I'm being distracted by mario's picross and puzzle mode of tetris plus and tetris blast (all for original gameboy) lately). I'll let you know if when I run into more hurdles.

## Create an account or sign in to comment

You need to be a member in order to leave a comment

## Create an account

Sign up for a new account in our community. It's easy!

Register a new account