Working Deduplication files for Microsoft Windows 10

Discussion in 'Windows 10' started by dreamss, Oct 4, 2014.

  1. Cvolton

    Cvolton MDL Novice

    Sep 16, 2016
    2
    0
    0
    Thank you for sharing this, I have succesfully extracted the dedup packages from the latest insider build and I can confirm it works very well. No issues with extracting any packages here.
     
  2. bfoos

    bfoos MDL Guide Dog

    Jun 15, 2008
    757
    703
    30
    Just so you know, if you read more recent posts in this thread we have worked out that those instructions are technically wrong as they include 2 packages and their corresponding language packages that are not needed. Needed packages listed below.

    • Microsoft-Windows-Dedup-Package-amd64-10.0.*.cab
    • Microsoft-Windows-Dedup-Package-amd64-10.0.*-en-US.cab
    • Microsoft-Windows-FileServer-ServerCore-Package-amd64-10.0.*.cab
    • Microsoft-Windows-FileServer-ServerCore-Package-amd64-10.0.*-en-US.cab

    Build numbers replaced with * for generalization.
     
  3. bfoos

    bfoos MDL Guide Dog

    Jun 15, 2008
    757
    703
    30
  4. uncman23

    uncman23 MDL Novice

    Mar 21, 2020
    1
    0
    0
    Thank you! Do you have one that works with the 2004 version?
     
  5. bfoos

    bfoos MDL Guide Dog

    Jun 15, 2008
    757
    703
    30
    Not yet. I have been checking the Flighthub nearly daily to see if they've released a 19041 Server build but for some reason they jumped the Server IP releases to vNext builds.
     
  6. kibkalo

    kibkalo MDL Expert

    Sep 8, 2009
    1,025
    362
    60
    Is there any way to install dedup binaries from other build (not 19041). Can't access my files
     
  7. bfoos

    bfoos MDL Guide Dog

    Jun 15, 2008
    757
    703
    30
    Please provide us with more information. What build are you running right now? If it's an 19041.XXX build, you are out of luck. Roll back to to 19362/3 and disable deduplication.
     
  8. ANTONIOPS

    ANTONIOPS MDL Novice

    Aug 20, 2012
    8
    1
    0
    Can the insider preview server ISO be used to extract the packages and use it in the W10 2004 build? microsoft.com/en-us/software-download/windowsinsiderpreviewserver

    I extracted the ISO an it has this:
    amd64_microsoft-windows-dedup-chunklibrary_31bf3856ad364e35_10.0.19608.1000_none_94cbb5f5d2f08fa7

    and other folders with the dll inside.
     
  9. bfoos

    bfoos MDL Guide Dog

    Jun 15, 2008
    757
    703
    30
    Nope it won't install. There are NO 19041 Server builds available so no Deduplication.
     
  10. ANTONIOPS

    ANTONIOPS MDL Novice

    Aug 20, 2012
    8
    1
    0
    So what makes the installation fail? is it related to the hash at the end of the folders in the winsxs folder?
     
  11. bfoos

    bfoos MDL Guide Dog

    Jun 15, 2008
    757
    703
    30
    #573 bfoos, Apr 24, 2020
    Last edited: Apr 24, 2020
    It's for a higher build. If you extract the packages and try to install them, DISM will throw an error and installation will fail. Data Deduplication is a Server role and not meant to be used on client editions. The only way it works is if there is a corresponding server build matching a specific client edition. In the case of 2004 or rather 19041.XXX, there is no corresponding server build and thus no packages to enable it on unsupported client editions.
     
  12. kibkalo

    kibkalo MDL Expert

    Sep 8, 2009
    1,025
    362
    60
    #574 kibkalo, Apr 25, 2020
    Last edited: Apr 25, 2020
    I remember there was some trick unpacking CABs and installing using MUMs

    People are still installing MediaCenter from Vista on 10 using that way, I just forgot how to do that
     
  13. bfoos

    bfoos MDL Guide Dog

    Jun 15, 2008
    757
    703
    30
    Feel free to figure it out.
     
  14. ANTONIOPS

    ANTONIOPS MDL Novice

    Aug 20, 2012
    8
    1
    0

    There must be a way of install the files from any "near" version, it's just software and the files inside the .cab in the newer builds are almost identical some are also the same, like Microsoft-Windows-FileServer-ServerCore-Package, I copied all the files (a couple of .dll and a xml file) from the same package from the insider preview server (10.0.19608.1000) but had to decompress them first using this: github.com/hfiref0x/SXSEXP and it succefully intalled in the 10.0.18362 build, we just have to find what makes DISM fail at installing the .cab files and patch it, I think it must be something related to the .manifest files, the certificates inside or some sort of hashing that we must recreate after modifying the files. Maybe we can use this tool for something: github.com/stolenbytes/sxshashing , it generattes the last (hash?) of every folder inside the .cab file for the correct build and then we can use github.com/sapientcoder/CabMaker or makecab for recreate the .cab files. Renaming all the build versions in all files is a good start.
     
  15. bfoos

    bfoos MDL Guide Dog

    Jun 15, 2008
    757
    703
    30
    #577 bfoos, Apr 25, 2020
    Last edited: Apr 26, 2020
    Nobody is stopping you from figuring out. I have no interest in doing so. Happily using 18363.815 and deduplicating my files here!

    You aren't the first person to come along wanting deduplication packages for pre-release builds and I am sure you won't be the last. Fact is, 2004 isn't officially released yet and no 19041 Server build has surfaced yet and likely won't until 2004 goes GA. In my opinion, you jumped the gun by installing 2004 when you had an unsupported feature installed and enabled on a prior build of a Client edition, when a little patience and prudence on your part would have your files happily available. Roll back to 1903/1909 and wait for a new Server build and save yourself the headaches and aggravation.
     
  16. ANTONIOPS

    ANTONIOPS MDL Novice

    Aug 20, 2012
    8
    1
    0
    I know I know, actually I'm testing this in a VM, so no problem, but I guess i'll wait for the official release, because, well, as you said, the real solution is just a month ahead.:)
     
  17. What happens to the deduped volume on upgrade? Should I remove the dedup from the volume before doing the upgrade or is it enough to install the dedup package again after upgrade?

    Edit: And what happens when using the deduped volume on other machines with no dedup package installed?
     
  18. ANTONIOPS

    ANTONIOPS MDL Novice

    Aug 20, 2012
    8
    1
    0
    Well, in my tests, nothing happens, the deduped files are in the same place, deduped or not, you can still see the deduped files, but you will get an error if you try to open or delete them, it's like disabling deduplication, but they will work again after you reenable it or rollback to a previous build with deduplication enabled. So the update only removes the dedup feature but doesn't touch any deduped files and because of that, the OS does not recognize those files and thinks they are corrupted untl you reenable or install deduplication again. I didn't test opening a different drive deduped in a different windows installation though, will test it later but it should work as long as the installed deduplication packages are almost the same or higher versions, for example, W10 <-> W10 or WS2019 <-> WS2019 or WS2019 <-> 10 works, WS2019/W10 -> WS2012 won't work but WS2012 -> WS2019/W10 will work because of backwards compatibility.