Andromeda Posted November 21, 2020 15 minutes ago, Xymph said: Oddly, https://dsdarchive.com/wads/diamonds is missing (for https://doomedsda.us/wad1598.html) - why is that? Probably because the only demo submitted for it was a FDA, from what I know DSDA3 purged those types of demos since they don't really count as speedruns. 0 Share this post Link to post
Xymph Posted November 21, 2020 25 minutes ago, Andromeda said: Probably because the only demo submitted for it was a FDA, from what I know DSDA3 purged those types of demos since they don't really count as speedruns. Ok, that's what I suspected, thanks for the confirmation. 1 Share this post Link to post
Xymph Posted November 28, 2020 On 5/18/2019 at 12:26 PM, Xymph said: Despite automation, it is still a big task to me. Bot account actions aren't autonomous; a bot script is run on each .ini file separately, under human supervision so it can be aborted in case a malformed edit comes along (e.g. due to a non-standard entry that the script didn't take into account yet). No-one is going to review thousands of bot edits afterwards, so I believe that monitoring is important. While running across multiple .ini files can be automated further, monitoring updates on 4500+ pages (the other 1000-odd map articles don't have DSDA links) is still very time-consuming. As predicted, it took many hours spanning two weeks to update over 5700 invocations of the dsda template into dsda2, but it's done now. Remaining uses are for files not imported into the current site, and old discussions. Phew. 5 Share this post Link to post
Xymph Posted November 27, 2021 (edited) On 5/8/2019 at 10:31 AM, kraflab said: I've added an endpoint here to query records: https://dsdarchive.com/api/demos/records?wad=heretic&level=E1M1&category=Pacifist Response: {"player":"Kimo Xvirus","time":"0:27","category":"Pacifist","level":"E1M1","wad":"heretic","engine":"Heretic v1.3 + vv","date":"2009-02-22T03:06:50.000Z"} (I can add more details to the response, it's just a proof of concept.) Records sometimes have a note, like "Also Reality" or the class for Hexen. Could you add this field to the response? I may not use the former case, but the Hexen class is definitely useful. (Edit: also a very relevant example is "Plays back with 1monster_bugfix.wad".) While I realize API development remains backburnered, I hope this is a small improvement that can be added soon, as development to script-wise update records tables on the wiki has progressed quite well already, and I'll start to crunch through wads/maps within a few days. Note that I'll need to run 12 requests (one per category) per map, so you'll see API traffic soar over the next few weeks. :) Edited November 29, 2021 by Xymph : Another reason why this is needed 1 Share this post Link to post
GoneAway Posted December 1, 2021 @Xymph please ping me in January if I haven't gotten to this, assuming it will still be useful. 0 Share this post Link to post
Xymph Posted December 1, 2021 (edited) 2 hours ago, kraflab said: please ping me in January if I haven't gotten to this, assuming it will still be useful. It will be useful then, as it is already useful today and the next week+ as I work through all WADs. Currently I either have to manually add notes, which is frustrating/time-consuming/easy-to-overlook, or skip map series for now, like Hexen. Is it really more complex than querying one additional database column and adding it in the API response? Edited December 1, 2021 by Xymph 1 Share this post Link to post
GoneAway Posted December 1, 2021 It isn't complicated, but I still need to go into the codebase, which I don't do often, remember what I'm doing, add it, test, deploy, etc. It's not a lot of time, I just usually spend all my coding energy on other things :^) 1 Share this post Link to post
Xymph Posted December 2, 2021 (edited) On 12/1/2021 at 9:26 PM, kraflab said: It isn't complicated, but I still need to go into the codebase, which I don't do often, remember what I'm doing, add it, test, deploy, etc. It's not a lot of time, I just usually spend all my coding energy on other things :^) I suppose kernels are better than humans at context switching. ;-) It's just that at launch, the engine column was added that same day. Eris immediately declared the API "a godsend for the Doom Wiki". Yes, it took me 2.5 years to get around to implementing this, so cannot expect another same-day turn-around on this one extra field. But it would help having to revisit a bunch of map series (i.e. double work) next month if it's delayed til then. Meanwhile I thought of a new API endpoint: return feed entries per certain parameters, e.g. from a given date until 'now' or between two dates, and optionally in groups of variable size (i.e. default 100 entries or whatever you're comfortable with, and more with a 'limit' parameter up to a reasonable max, like 500 or 1000). I can then periodically poll this for new entries since the previous poll, and from there determine which map articles need updating. So I started updating wiki tables on Nov 27, and in 1-2 months I could query the API for new entries since that date. In absence of such an endpoint, I'd have to do the same on the web pages (have already done similar parsing of review tables on OneManDoom and Doom Underground, and of course the DSDA players list to correlate speedrunner info between the various sites) so it's feasible, but will be easier via the API. Edited December 4, 2021 by Xymph 1 Share this post Link to post
GoneAway Posted December 3, 2021 @Xymph The api has been updated to have this key: "notes": ["Array of", "Notes"] 4 Share this post Link to post
Xymph Posted December 3, 2021 6 hours ago, kraflab said: The api has been updated to have this key: "notes": ["Array of", "Notes"] Awesome, existing notes now preserved without manual editing. Thank you! 1 Share this post Link to post
Andromeda Posted December 13, 2021 I've noticed that NM 100S runs that are faster than NM Speed aren't crosslisted on the API, like they are on the table view for example. Might be something to fix perhaps. 4 Share this post Link to post
Xymph Posted January 26, 2022 I came across a strange discrepancy between website and API: https://dsdarchive.com/wads/chex for E1M1 lists 4shockblast's 0:00.97 (of 2016-02-08) as the UV Speed/Pacifist record, but the API returns depr4vity's entry of 2020-12-30 instead. How come? Could this be synced up? Also, @kraflab, could you indicate whether the feed API call is feasible within a reasonably near future? Otherwise I'll have to decide whether to parse the website feed instead. On the wiki all map articles with DSDA records have been updated during the past two months (on and off) of bot script runs, now it is a matter of not trailing too far behind again. 1 Share this post Link to post
GoneAway Posted January 26, 2022 There's no logic in place for ties, so the "winner" in such a case is nondeterministic. As far as the feed stuff, I can take care of that over the weekend. 1 Share this post Link to post
Xymph Posted January 26, 2022 (edited) 5 hours ago, kraflab said: There's no logic in place for ties, so the "winner" in such a case is nondeterministic. The first person to reach a given time could be ranked first -- the date/time stamp is already available. That is how e.g. an online racing game like the TrackMania series ranks tied entries (happen to be very familiar with that). Edited January 26, 2022 by Xymph 2 Share this post Link to post
GoneAway Posted January 30, 2022 On 1/26/2022 at 5:38 PM, kraflab said: As far as the feed stuff, I can take care of that over the weekend. On second thought, I don't really want to just hack together something here, but I also don't want to really start working on a more formal API while I'm still focusing on dsda-doom. In that case it'll probably be better if you just scrape the pages for now. Some day I will suddenly be filled with inspiration and implement these things but it could be any amount of time from now. 0 Share this post Link to post
Xymph Posted January 30, 2022 1 hour ago, kraflab said: On second thought, I don't really want to just hack together something here, but I also don't want to really start working on a more formal API while I'm still focusing on dsda-doom. In that case it'll probably be better if you just scrape the pages for now. Some day I will suddenly be filled with inspiration and implement these things but it could be any amount of time from now. That is disappointing as it'd mean extra work for me, rewriting HTML parsing into API processing later - or alternatively make the feed API a moot point (for me anyway, if I don't) by the time it arrives. Isn't it time for a break from dsda-doom after 7 releases in 2.5 months? :-) 0 Share this post Link to post
Xymph Posted April 4, 2022 (edited) The current update hiatus provided a good moment to write my feed script (using HTML page parsing), and update map records on the wiki that were added/changed since late November. So now the wiki is all caught up. Edited April 4, 2022 by Xymph 5 Share this post Link to post