Jump to content
Search In
  • More options...
Find results that contain...
Find results in...
Sign in to follow this  
gggmork

get string of bits of wad in python

Recommended Posts

Essentially I want a python program to open a wad file to get the data of where the points/lines etc are, edit that data in some way, and resave the edited data as a new wad.

Python seems to suck at bit/byte level stuff. My idea is convert the bits of the wad into a string of 0s and 1s, then fiddle with that, or at least test if I got the correct bits according to wad file format:
http://doom.wikia.com/wiki/WAD

Well I made this short program.

def getbytestring(a):
    b=['0','0','0','0','0','0','0','0']
    if a&1:
        b[7]='1'
    if a&2:
        b[6]='1'
    if a&4:
        b[5]='1'
    if a&8:
        b[4]='1'
    if a&16:
        b[3]='1'
    if a&32:
        b[2]='1'
    if a&64:
        b[1]='1'
    if a&128:
        b[0]='1'
    return ''.join(b)


f=open('bacon.wad','rb')
for i in range(1000):
    r=f.read(1)
    print i, '    ',ord(r),'    ',getbytestring(ord(r))
it outputs:
0      80      01010000
1      87      01010111
2      65      01000001
3      68      01000100
4      11      00001011
5      0      00000000
6      0      00000000
7      0      00000000
8      37      00100101
9      248      11111000
...etc
So, I don't know, how can I test that the index 8 byte of the real wad matches 00100101 which this program output? Or does it look correct/etc?

Share this post


Link to post

I successfully used omgifol before. It works good, but the code internals were tricky for me to understand at least. I want to implement my own simpler version so I actually know what's going on, and I could have my own program 'import' a wad without relying on external packages etc.

Share this post


Link to post
gggmork said:

My idea is convert the bits of the wad into a string of 0s and 1s, then fiddle with that, or at least test if I got the correct bits according to wad file format


oh god please don't do this. use numpy to read binary data:

import numpy as np

# reading unsigned short from file:
f = open('myMap.wad','rb')
my_us = np.fromfile(f,'<u2',1)[0]

# reading an array of 10 int32s from file:
my_ints = np.fromfile(f,'<i4',10)
similarly you could use some combination of the read() and np.fromstring() functions. For writing files back out use a combination of write() and np.tofile()



I have some scripts laying around that I use for wad I/O in python (basically because the wad format is extremely easy to work with, and I wanted to build something myself instead of using omgifol, for whatever reason) that I can PM to ya if you want.

Share this post


Link to post

Oh yeah, I just re-found genTerrainRibbiks.py in my 2013 folder. Do you have additional stuff besides that? None of the examples I searched for on the internet to manipulate file bytes even mentioned numpy. And I was getting confused messing around with the 'struct' class, trying to decide whether to interpret the data as an int or char since the latter has less bytes, ord confusion, no knowing if file.read() returns a string or a byte or an int etc. or if all are equivalent, and having iterations of a loop completely skip on some occasions for weird reasons and yada yada ya. But if you have since made additional stuff similar to genTerrainRibbiks.py, let me know. Or I can just study that, should have remembered to do that in the first place. Probably in the back of my mind I assumed I wouldn't find it again on this comp.

The first thing I wanted to do was draw sectors that represent ceilings only, draw a separate area of sectors that represent floors only. Maybe each have a mapspot or point representing the upper left corner of each, so they can be placed on top of eachother in 2 layers and combined, so both could have unique patterns and combine into a stained glassish window, which would involve finding points of line intersections and stuff, which I've been wanting to do for awhile but never got around to.

The 'wad to string of bits' would just be too slow or something? I guess its pointless to convert to a string if you can read the file as bits in the first place, but that didn't seem easy to do after searching internet suggestions.

Share this post


Link to post
gggmork said:

The first thing I wanted to do was draw sectors that represent ceilings only, draw a separate area of sectors that represent floors only. Maybe each have a mapspot or point representing the upper left corner of each, so they can be placed on top of eachother in 2 layers and combined, so both could have unique patterns and combine into a stained glassish window, which would involve finding points of line intersections and stuff, which I've been wanting to do for awhile but never got around to.


oy, that sounds pretty cool. but also very complicated. For arbitrary geometry it's going to be a serious challenge to keep track of all the new sectors that the intersections create. You'll also have to create a buttload of new linedefs, sidedefs, and keep track of which ones reference all the sectors of the new geometry. In short, good luck :p

The 'wad to string of bits' would just be too slow or something? I guess its pointless to convert to a string if you can read the file as bits in the first place, but that didn't seem easy to do after searching internet suggestions.


speed's not really the issue, but there's just clean builtin ways to do it (though apparently they aren't documented that well if it was difficult to find information on).





Here's a quick example script I whipped up that you might find useful for seeing the numpy commands in action. it changes every chaingunner in a map into a cyberdemon, because that sounded like fun :p

import sys
import numpy as np

IN_WAD  = '/Users/zach/Desktop/james/doom/eaxt.wad'
OUT_WAD = '/Users/zach/Desktop/gggDemoOut.wad'

def nullPad(string,length=8):
    return string+chr(0)*(length-len(string))

def byteSize(obj):
    # python strings add 21 bytes to their actual length on my system, so I subtract it from getsizeof() to get the actual size.
    # this might be different for you...
    return sys.getsizeof(obj)-21

def readInWad(inWad):
    f = open(inWad,'rb')
    [pwad,nLmp,dirP] = np.fromfile(f,'<i4',3)
    f.seek(dirP)
    lmpDir = []
    allData = []
    for n in range(nLmp):
        [lmpP,lmpS] = np.fromfile(f,'<i4',2)
        name = f.read(8)
        lmpDir.append([name,lmpP,lmpS])
    
    for n in lmpDir:
        f.seek(n[1])
        allData.append([n[0],f.read(n[2])])

    f.close()
    return allData

def writeOutWad(allData,outFile):
    nLmp = len(allData)
    lmpSizes = [byteSize(n[1]) for n in allData]
    totSize = sum(lmpSizes)
    newDir = []

    f = open(outFile,'wb')
    f.write('PWAD')
    np.array(nLmp,'<i4').tofile(f)
    np.array(totSize+12,'<i4').tofile(f)

    for i in range(len(allData)):
        newDir.append([f.tell(),lmpSizes[i],allData[i][0]])
        f.write(allData[i][1])

    for n in newDir:
        np.array(n[0:2],'<i4').tofile(f)
        f.write(n[2])

    f.close()

def main():
    # read in wad data
    wadData = readInWad(IN_WAD)

    # lets print the size of every lump present in the file...
    for n in wadData:
        print n[0],byteSize(n[1])

    #
    # as an example lets change every chaingunner into a cyberdemon in MAP01...
    #

    thingData = []  # thingData[i] = [xpos,ypos,angle,type,flags]
    for i in xrange(1,len(wadData)):
        if wadData[i][0] == nullPad('THINGS') and wadData[i-1][0] == nullPad('MAP01'):
            things = wadData[i][1]
            # consult doomwiki for the byte layout of each lump. THINGS lump consists of 10-byte entries containing 5 shorts
            for j in xrange(0,byteSize(things),10):
                [xpos, ypos, angle, ttype, flags] = np.fromstring(things[j:j+10],'<i2')
                # if type == chaingunner, change to cyberdemon
                if ttype == 65:
                    ttype = 16
                thingData.append([xpos, ypos, angle, ttype, flags])

            # construct a new things lump and replace the existing one
            newThings = ''
            for n in thingData:
                newThings += np.array(n,dtype='<i2').tostring()
            wadData[i][1] = newThings


    # write out new wad
    writeOutWad(wadData,OUT_WAD)


if __name__ == '__main__':
    main()

The code is kinda sloppy, I just put this together in a few minutes. The entire point of something like omgifol would be to make this process a bit more elegant, but I have fun with this sort of thing so I don't mind dealing with the raw binary data. How you structure the code is gonna depend a lot on what you want to do, but for this example it was pretty much:

- read raw data from each lump in the wad and store it (equivalently) as a string in a list
- iterate through list, look for lumps we're interested in: THINGS
- parse binary data of things lump, edit values of interest (we can add, subtract, modify, whatever)
- turn modified values back into a binary string in preparation for writing it to output wad
- write out.wad

doomwiki is awesome for describing how the bytes of each lump are laid out. for this example I briefly consulted:

http://doomwiki.org/wiki/Thing#THINGS_lump
http://doomwiki.org/wiki/Thing_types


eaxt.wad



gggDemoOut.wad

Share this post


Link to post

Thanks, this will come in very handy and should keep me busy for quite awhile (especially since I'm being distracted by mario's picross and puzzle mode of tetris plus and tetris blast (all for original gameboy) lately). I'll let you know if when I run into more hurdles.

Share this post


Link to post

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  
×