V1.1.0 - Python wrapper for wad archive data dump - fully works with compressed archive
>>> See replies below for updates to what is done <<<
Following the announcement in this thread about the closure of WAD Archive, I thought I'd put together a basic framework to present proper WAD filenames as download links, rather than the GUIDs that they are stored as.
>>> Update 10/03/24 <<<<
Changes:
- file list order to lexical
- better file loading spinner - added a file list loading spinner - better logic for UI element show/hide as loading new data - removed erroneous retrieval of source data where filename database array field was zero-length (removes `undefined` from filelist) DOWNLOAD HERE: https://github.com/smeghammer/wad-archive/tree/zipped_sources Merged with main branch: https://github.com/smeghammer/wad-archive >>> 02/02/24 <<< Added notes on adding UNC paths in Ubuntu; tweak to SCSS font module file (path was wrong) to re-enable DooM font in title. See this post below. >>> 25/11/22 <<< Confirmed that configuring a UNC path to the data works as expected - archive is not loaded into memory in it's entirety so accessing a 4GB archive over the LAN works OK. Also, some minor style changes and bugfixes;
>>> Update 23/11/22 <<< Added code to allow using just the zipped archive. No need to unzip first! This is RC1 and I'd love feedback... Please make sure you switch to the 'zipped sources' branch: https://github.com/smeghammer/wad-archive/tree/zipped_sources As long as you have all the archives downloaded, you should be able to use the application to find/download maps within the archives. You will need to set up mongoDB with the metadata that has been released already by @WadArchive, and you will need to set the configuration to match your system in the /config/settings.py file. Note I have only tested this on Windows with a local path, so any feedback for UNC paths or Linux paths would be appreciated. Have fun! >>> update 20/11/22 <<< Minor style update and altered map download; added loading spinners for AJAX loading delay if needed - see posts for today. Added current map indicator and moved image paginator to top of images. Oh, and changed to 'filter' on the button.: >>> Update 18/11/22 <<< Restyled:
>>> update! now with browseable details <<< Get the code at Github: https://github.com/smeghammer/wad-archive ## features - metadata JSON files in a mongoDB back-end (Done, Mongo database) - paged JSON endpoint listing the WAD filenames (Done, Python 3/Flask) - mapping of filenames to GUIDs (Done) - Mapping of GUIDs to archived WAD files (Done) - Web front-end to service the REST endpoints (Done, for basic details display and download link) - Download logic with correct download filename (Done) [** updated 12/11/22 **] ## And... - Show readme file if it is present (Done) - Show screenshots, maps and graphics png files, if they are there (DONE!!) The whole thing will be a Python3 and Flask web application backed by the mongo database. I'll probably make it configurable (local or remote database host, port, server IP address, filesystem path to archive location etc.) [** updated 15/11/22 **] Added paginator for images I'll update this as I do more.
- better file loading spinner - added a file list loading spinner - better logic for UI element show/hide as loading new data - removed erroneous retrieval of source data where filename database array field was zero-length (removes `undefined` from filelist) DOWNLOAD HERE: https://github.com/smeghammer/wad-archive/tree/zipped_sources Merged with main branch: https://github.com/smeghammer/wad-archive >>> 02/02/24 <<< Added notes on adding UNC paths in Ubuntu; tweak to SCSS font module file (path was wrong) to re-enable DooM font in title. See this post below. >>> 25/11/22 <<< Confirmed that configuring a UNC path to the data works as expected - archive is not loaded into memory in it's entirety so accessing a 4GB archive over the LAN works OK. Also, some minor style changes and bugfixes;
>>> Update 23/11/22 <<< Added code to allow using just the zipped archive. No need to unzip first! This is RC1 and I'd love feedback... Please make sure you switch to the 'zipped sources' branch: https://github.com/smeghammer/wad-archive/tree/zipped_sources As long as you have all the archives downloaded, you should be able to use the application to find/download maps within the archives. You will need to set up mongoDB with the metadata that has been released already by @WadArchive, and you will need to set the configuration to match your system in the /config/settings.py file. Note I have only tested this on Windows with a local path, so any feedback for UNC paths or Linux paths would be appreciated. Have fun! >>> update 20/11/22 <<< Minor style update and altered map download; added loading spinners for AJAX loading delay if needed - see posts for today. Added current map indicator and moved image paginator to top of images. Oh, and changed to 'filter' on the button.: >>> Update 18/11/22 <<< Restyled:
>>> update! now with browseable details <<< Get the code at Github: https://github.com/smeghammer/wad-archive ## features - metadata JSON files in a mongoDB back-end (Done, Mongo database) - paged JSON endpoint listing the WAD filenames (Done, Python 3/Flask) - mapping of filenames to GUIDs (Done) - Mapping of GUIDs to archived WAD files (Done) - Web front-end to service the REST endpoints (Done, for basic details display and download link) - Download logic with correct download filename (Done) [** updated 12/11/22 **] ## And... - Show readme file if it is present (Done) - Show screenshots, maps and graphics png files, if they are there (DONE!!) The whole thing will be a Python3 and Flask web application backed by the mongo database. I'll probably make it configurable (local or remote database host, port, server IP address, filesystem path to archive location etc.) [** updated 15/11/22 **] Added paginator for images I'll update this as I do more.
Yeah. What I want to do ideally is only load the first image if an array is found and load incrementally as you paginate. It may be a javascript optimisation thing but more likely a python/serverside code thing. It also may be a mongo index thing but the images are comig from the filesystem anyway.
It is also likely that the original code was having to process the zipped big archives and then the gzipped individual record. That would add a significant overhead too.
-
Reputation Points