Jump to content
Search In
  • More options...
Find results that contain...
Find results in...
wesleyjohnson

Copy corrupts wad files.

Recommended Posts

Stupid computer. I copied shareware Doom and Freedoom to a memory stick, to transfer it from my Linux system to a Win98 system where I was testing compiles of DoomLegacy. Kept getting strange run-time errors, which I finally traced to corrupt wad files. It seems that in 4 places in shareware Doom and about 8 places in FreeDoom, the 0xFFFF that marks that sidedef2 is not present, is getting changed to 0xFF9F during the copy to (or from) the memory stick. It is very consistent in how it corrupts the copied file.

Zipping shareware Doom into a zip file, gave me a good copy.

Anyone care to guess on what could cause this ??

Share this post


Link to post

Heh I recently bought an '8 GB' thumb drive that had a 640 KB chip and only reliably saved the first 640 KB of whatever :-p The rest became random garbage.

Share this post


Link to post
Maes said:

Heh I recently bought an '8 GB' thumb drive that had a 640 KB chip and only reliably saved the first 640 KB of whatever :-p The rest became random garbage.

That was actually a common "problem" (i.e. scam) a couple years ago. Have a look at the program h2testw, some english info is available at http://sosfakeflash.wordpress.com/2008/09/02/h2testw-14-gold-standard-in-detecting-usb-counterfeit-drives/

Share this post


Link to post

I wonder if this phenomenon accounts for all those damn wads on /idgames with the 0xFFFE blockmap terminators. Single-bit errors are the easiest to occur.

Share this post


Link to post
boris said:

That was actually a common "problem" (i.e. scam) a couple years ago. Have a look at the program h2testw, some english info is available at http://sosfakeflash.wordpress.com/2008/09/02/h2testw-14-gold-standard-in-detecting-usb-counterfeit-drives/


Yeah, I went through the whole deal and that's how I discovered I had been duped :-/ I've heard of 32 GB sticks being 512MB or 2GB in reality (so still of some practical use), but 640 KB? That's mean.

Share this post


Link to post
Quasar said:

Can this util be used on a drive that contains files, or does it have to be blank?

From the readme.txt that comes with the program:

The function of H2testw is quite simple: It fills the chosen target
directory with test data and then reads it back and verifies it.

H2testw does not overwrite or erase any existing data. It doesn't
do any low-level tricks so administrator privileges are not required.
If your hardware is working properly H2testw will not harm any
existing data.

BUT: _If_ the hardware is defective then H2testw is designed to find
that defect and might as a side effect damage existing files.
Therefore: IF YOU SUSPECT A USB STICK OR OTHER STORAGE MEDIA TO BE
DEFECTIVE, EMPTY IT AND TEST IT COMPLETELY WITH H2TESTW. Only empty
media can be fully tested with H2testw. In order to be able to
reproduce the results we recommend to format the media (quick format
will do) and then test it.

Share this post


Link to post
Quasar said:

Can this util be used on a drive that contains files, or does it have to be blank?


It should be blank if you want unambiguous results. If you really want to test every byte of its -alleged- capacity, you should start with a "clean slate", since it simply creates one (or more) big-ass files that fill all available sectors/cells/blocks/whatever, and reads them back trying to find corrupted or aliased data.

You can limit the size of these files (e.g. to see if the first 100 MB or 1 GB is good and so on) or write them with other data present, but this will run into issues with existing data occupying space that can't be tested, writes being non-continuous due to fragmentation etc. If you really want an accurate map of your drive, or to know whether the first X MB are good, it should be empty.

That's how I discovered that it didn't even get past the MB mark: writing a single MB of test data on an empty drive, caused corruption exactly after offset 0x000A0000 (or 640 KB). With files present, they got corrupted from the very first byte or wrote for some random amount. Only with a totally empty drive it was possible to test this limit.

Share this post


Link to post
Maes said:

640 KB? That's mean.

According to Bill Gates that's ten times as much as you'll ever need.

Share this post


Link to post
Wagi said:

According to Bill Gates that's ten times as much as you'll ever need.


Well 64 KB was enough RAM for many 8-bit machines, so you'd think 256 KB would also be enough for 32-bit machines, but the bloatage turned out more like an exponential function.

Share this post


Link to post

@Wagi: did Bill Gates ever say something about 64 KB?

hex11 said:

Well 64 KB was enough RAM for many 8-bit machines, so you'd think 256 KB would also be enough for 32-bit machines, but the bloatage turned out more like an exponential function.


I'd put the "sweet spot" somewhere between 4 and 16 MB: 16 MB was the practical addressing limit of many 16/32 bit machines (286, 68000 etc.), and anything beyong 4 MB was considered "high end" during the late 80s/early 90s for some exotic machines like Sparcstations, Atari TTs, Amiga accelerator cards etc. You could also play practically all MS-DOS games ever conceived, run Windows 3.1, 95, 98 and OS/2 with 16 MB, something that no modern Linux distro is capable of doing (even DSL requires 24 MB for graphics).

But less than 512 KB? Even the venerable Amiga would have had it much worse with that little RAM (remember, no dedicated VRAM on most home computers, and many Amiga games were designed to work better with 1 MB or more). Then again, Doom requires a 4 MB address space to be playable in its full MS-DOS glory without cutting down on resources, so 4 MB is all the memory you'll ever need ;-)

Share this post


Link to post

Wandering...
Copied it back the other direction does not seem to add corruption.
The Linux side does not seem to have any problems. The file on the memory stick matches the original. This leaves the Win98 read of the memory stick. It read the zipped file off the same memory stick.
All of MinGW, SDL, and countless other programs have been installed to that computer using that one memory stick and driver.

Detailed comparison of the errors shows that it is only 0xFFFF markers that are corrupted, not any other pattern. There are more of them than I thought. It selectively changes them to 0xFF9F, 0xFFBF, and 0xFF3F, and no other pattern. Does not seem to be random memory stick errors.

On the 8GB/32GB memory sticks. It does not have to be deliberate. That is the same symptom as stuck memory lines on the memory chip. If they bought untested chips or rejects, they will get many of those. Some of those manf. expect the customers to sort out the bad ones.

Share this post


Link to post
Maes said:

@Wagi: did Bill Gates ever say something about 64 KB?

There were rumors that Bill Gates once said “640K of memory should be enough for anybody". Whether he actually said it is disputed.

Share this post


Link to post
Wagi said:

There were rumors that Bill Gates once said “640K of memory should be enough for anybody". Whether he actually said it is disputed.


Well, the 640 KB quote is well known. The 64KB one you mentioned is not, however, so I was curious. Maybe he was referring to this? :-p

Share this post


Link to post

After too much testing, found the following.
Wrote 5 test copies of doom1.wad to memory stick.
The memory stick is OK, Linux can read all the wad copies and they diff as identical.
Win98 machine gets different results for each read. It returned 4 different corrupted copies (via zip files).
A second Win98 machine I tested reads the wad files OK and returned identical copies.

Final: It is one Win98 machine that corrupts wad files from USB memory sticks (but reads zip files OK and writes OK). Most likely weak hardware (PCI add-on USB card), I suspect ...
False alarm, it was NOT doom wad sensing windows code ... never mind.

Share this post


Link to post

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×