I was thinking how to speed up builds auto fetch with uupdump-get-fileinfo+packs.cmd script first run might be too long for new users, so it's better to fetch fileinfo only how it works now: 1. run packsgen.php first to get packs - new users are not affected because database is epmty, but useful if WU is unstable and packs weren't generated previously 2. run fetchdetect.php to get fileinfo and packs for that build - very long operation, new builds should be processed one by one so, here's the plan: 1. run packsgen.php first to get packs - new users are not affected because database is epmty 2. run fetchupd.php to get fileinfo only - this step should be much faster, you can see a lot of builds in DB after a minute or so 3. run packsgen.php again to get packs - all stuff from the 2nd step will be generated (5-7 minutes), but you can close it and generate metadata manually on the website with one click second run of uupdump-get-fileinfo+packs.cmd: 1. run packsgen.php first to get packs - if user closed the window then all packs will be regenerated at this step 2. run fetchupd.php to get fileinfo only - fast check of new builds if MSFT released something new I think it would be better for new users but the trade off is overall it's faster to get fileinfo+packs at the same time in 7-10 minutes with fetchdetect.php
Maybe just uupdump-get-fileinfo.cmd to detect builds, then the user generate metadata manually on demand for his chosen build and uupdump-generate-packs.cmd to generate metadata for all builds in database
users want all in one solution because they're lost in 3 scripts uploaded v1.5.1 with changes above for now
Actually that can be in that GUI cmd we discussed when this all started, where you can setup the server with the desired UUP packages?
you know what, separate fileinfo and packs scripts are better because I can just add an option to call packs generation when it's done "generate metadata" on the website is stable enough I can't write GUIs and personally I think it's unnecessary because I don't like adding more clicks to this, but anyway maybe @abbodi1406 can make something
I mean a setup gui, where you can set all up and with GUI i mean something like the KMS_VL_ALL AiO cmd.
people got lost in two scripts on Discord server, I better not split fileinfo+packs script did some testing with the 1.5.1 version: 1. got 49 builds in 1 minute 2. metadata was then generated in 6 minutes looks fine to me
One question: if I have the server started, does it automatically refresh new builds as well as uudump.net? If not, what do I have to do for this?
only uupdump-get-fileinfo+packs.cmd can get new builds, on Linux I'm using cron to autorun it once in a day (ofc script is ported to Linux shell script) on Windows you can use Task Scheduler, but I never tried it myself
I'm testing an iso made with SkipApps = 1 Sideloaded Photos, Paint, Clock and WSA (and their respective dependencies) with Powershell. While WSA launches and works without issues, the other 3 don't. Is there some kind of hidden dependency or registry that could prevent launching those 3 apps?
Code: [2023-09-18 18:06:44 UTC] UUP dump v3.62.2 [2023-09-18 18:06:44 UTC] UUP dump API v1.40.2 [2023-09-18 18:06:44 UTC] Parsing database info... [2023-09-18 18:06:44 UTC] Done parsing database info. [2023-09-18 18:06:44 UTC] Generating packs for 95ef7fd2-51b3-473d-aead-7689435f4c08... [2023-09-18 18:06:44 UTC] Fetching information from the server... [2023-09-18 18:06:44 UTC] Information has been successfully fetched. [2023-09-18 18:06:44 UTC] Parsing information... [2023-09-18 18:06:44 UTC] An error has occurred [2023-09-18 18:06:44 UTC] Generating packs for dfe05d0e-cb89-4c7c-827e-e09385a7f330... [2023-09-18 18:06:44 UTC] Fetching information from the server... [2023-09-18 18:06:45 UTC] Information has been successfully fetched. [2023-09-18 18:06:45 UTC] Parsing information... [2023-09-18 18:06:45 UTC] An error has occurred [2023-09-18 18:06:45 UTC] Generating packs for 45974967-f2bf-429c-9f5b-cd2e7b9c6c6a... [2023-09-18 18:06:45 UTC] Fetching information from the server... [2023-09-18 18:06:45 UTC] Information has been successfully fetched. [2023-09-18 18:06:45 UTC] Parsing information... [2023-09-18 18:06:45 UTC] An error has occurred [2023-09-18 18:06:45 UTC] Generating packs for 08e36e9c-738f-46cc-811e-6f62260616f9... [2023-09-18 18:06:45 UTC] Fetching information from the server... [2023-09-18 18:06:45 UTC] Information has been successfully fetched. [2023-09-18 18:06:45 UTC] Parsing information... [2023-09-18 18:06:45 UTC] An error has occurred [2023-09-18 18:06:45 UTC] Generating packs for 1f27828e-0fad-4d53-8e5f-ff389a7a88d2... [2023-09-18 18:06:45 UTC] Fetching information from the server... [2023-09-18 18:06:46 UTC] Information has been successfully fetched. [2023-09-18 18:06:46 UTC] Parsing information... [2023-09-18 18:06:46 UTC] An error has occurred New method chokes on old builds in the database. The errors didn't get away until I manually deleted those. How about offering to delete such obsolete builds right away?
well, I already merged packsgen with fetchdetect in v1.5.0, v1.5.1 gives the same results in a different order in your case packs weren't generated for some reason, shouldn't happen again with v1.5.1 (packsgen -> fetchupd -> packsgen) about deleting old builds - we can't check for sure, sometimes WU throws errors for no reason
These error messages did not get away on their own and occurred two times on every run, once before and once after. Probably that's because the builds are removed from MS servers. OK, I never used 1.5.0. Also noticed you moved the database. Unfortunately only after it started to build a completely new one. Old db was in .\uup-dump\website\, new is in .\uup-dump\.
The old version's script generated packs only for the entries it downloaded fileinfo for, so, there wasn't any need to run the dedicated packs script. Maybe upon download, they had no packs or there was an error at that time. Removed now, should not happen again, packsgen now runs plenty. Maybe, but a warning or one-time migration to the new location (moving packs and fileinfo directories) would have been nice.
technically there's no difference: fetchdetect - it's getting fileinfo and packs for each build one by one, should get packs if it was missing packsgen - it's getting packs for all builds in fileinfo fetchupd - it's getting fileinfo only previously it was fetchdetect only in one script and packsgen in the second one the catch: you may miss packs due to some WU error if fetchdetect didn't do its job correctly now it's one script in 3 steps: 1. packsgen - generates missing packs, takes less than a second to parce db on empty db 2. fetchupd - very fast operation to get all builds (fileinfo), so users can see the result immediately 3. packsgen - generates packs for recently added builds and it regenerates metadata on each run new way is more simple, safe and takes the same time on new DBs
I used the old "uupdump-get-fileinfo+packs.cmd", it downloaded the fileinfo for the newly detected builds and then immediately generated the packs for them. There was no general packsgen over all builds unless you used the dedicated script (which I never had to as the builds I wanted worked fine), otherwise the errors would have shown up already (some thing were from March). Now, it will even post-generate packs for the ones that failed earlier (which unfortunately happens). Just a note to the new feature, update-only packs. I tested it and the appropriate updates were fetched fine (no useless "baseless" stuff, cab over msu). However, there's one thing I miss, as I keep checksum lists. Could you generate a text file with the SHA1 of the downloaded updates? As it stands, I still have to visit the KB page to fetch the SHA1 sums from the list.
update-only download pack is just like the general download / download&convert packs none included sha1 checksum file by default however, the download script could be modified to keep the temporary aria2_script.*.txt which include the files hashes