Working Deduplication files for Microsoft Windows 10

Discussion in 'Windows 10' started by dreamss, Oct 4, 2014.

  1. bfoos

    bfoos MDL Guide Dog

    Jun 15, 2008
    705
    617
    30
    Pretty much as stated above, nothing happens to the files. I personally tested enabling deduplication in 1809 and upgraded to 1903, installed the 18362 dedup pack and was back up and running. The same will apply going from 1903/9 to 2004. As far as trying to access a data deduplicated volume on a machine that doesn't have that feature, the host simply will not be able to access the files on that volume... well that's technically not entirely accurate. Any files smaller than 32kb would be able to be accessed as that is the minimum file size cuttoff for dedup.

    If you REALLY can't wait to hop over to 2004 and a VM or WTG installation just doesn't cut it for you. Best practice would be to disable deduplication of whatever volumes you need to access BEFORE upgrading or clean installing. You can do that by using the start-dedupjob command with the unoptomization job type.

    Code:
    Start-DedupJob -Volume "D:" -Type Unoptimization -Full -Wait
    The above would unoptomize volume "D:" the full switch probably isn't really needed and the -Wait switch will make powershell show a progress bar in the terminal.
     
  2. luzea

    luzea MDL Addicted

    Apr 25, 2019
    621
    636
    30
  3. kibkalo

    kibkalo MDL Member

    Sep 8, 2009
    128
    10
    10
    I have unfortunately installed 2004 update on top of main PC with 1909 and dedup, where have 10TB of deduplicated files.
    Am looking for any way to install dedup package on 19041 build. Understand risks, have backup on tape.
     
  4. luzea

    luzea MDL Addicted

    Apr 25, 2019
    621
    636
    30
    Not possible for now, because there is no equivalent server build. You can downgrade your version in the settings menu if you haven’t done any disk clean.
     
  5. bfoos

    bfoos MDL Guide Dog

    Jun 15, 2008
    705
    617
    30
    There is no way until a 19041 Server build is released. For some reason, MS jumped Server IP releases from 19035 to vNext builds, bypassing 19041 entirely. They have released a corresponding Server SAC build along side every Client SAC release thus far so logically, I would assume that they will at some point release 19041 server. Who knows for sure, though? MS has been known to throw logic out the window!

    Anyway, you could create a WTG installation of 1903/9 with Data Deduplication integrated and use that to unomptomize the deduplicated files if you have the space. Or revert your 2004 installation back to 1909 if you can. Otherwise, you will have to wait and see what MS does.
     
  6. bfoos

    bfoos MDL Guide Dog

    Jun 15, 2008
    705
    617
    30
    #586 bfoos, May 12, 2020
    Last edited: May 12, 2020
    Seems Server 2004 has been released. Hopefully ISO files will show up soon so we can move Data Deduplication into the present. :)


    ***EDIT***

    Got the ISO, installing it now. 19041 Dedup pack will be up later!
     
  7. bfoos

    bfoos MDL Guide Dog

    Jun 15, 2008
    705
    617
    30
    #587 bfoos, May 12, 2020
    Last edited: May 19, 2020
    OK folks! Here is the first 19041 (en-US only for now) Data Deduplication pack! Dedup files were pulled from Windows Server 2004 and tested on my personal installation of 19041.264.

    This pack will enable the Windows Server File Server Role known as Data Deduplication on Client editions of Windows 10 2004 (19041.XXX)

    Code:
    Hashes:
    BLAKE2sp: 7EDCAE32E78363050CD08031011C380A968E357A3DB93CA6411988AA75885785
    SHA-1: 3114E31AA20F5B7C498695DC94F292DD12B310C0
    SHA-256: 403ED6041D32BF6173AA1B662B7465180F50FA765626167767291F8F21B6F4CF
    SHA-384: 2F513AF7DAB405AEE5D95EBC1EF6DA31B0FBDB2368A134DCA6E9D0DE5B3BE0C6B1518139BC947C6C87CAC71B9C4F11A1
    SHA3-224: 6EF0DF859314F3415327C60135CAFAA4532D7C9B5AE8EC267ABE5C99
    SHA3-256: C1232320ED8AABD219F1515E73F668C60178A9290A4C1118876ECCAE36477BF7
    

    New pack...
    https://forums.mydigitallife.net/th...crosoft-windows-10.57189/page-31#post-1597931

    Many thanks to @LostED for supplying the needed ISO file!
     
  8. Evengard

    Evengard MDL Novice

    Nov 9, 2007
    3
    0
    0
    Will it work fine on non-en-US editions as well? I don't care about my Windows being part English and part not.
     
  9. Canislupus74

    Canislupus74 MDL Novice

    Apr 17, 2017
    1
    0
    0
    #589 Canislupus74, May 13, 2020
    Last edited: May 13, 2020
    Isn´t the files Microsoft-Windows-Dedup-ChunkLibrary-Package-amd64.xxxxxxx missing?
     
  10. bfoos

    bfoos MDL Guide Dog

    Jun 15, 2008
    705
    617
    30
    I'm not sure if it'll work but I do know that it won't hurt to try.

    The ChunkLibrary package already exists in Client editions.
     
  11. Evengard

    Evengard MDL Novice

    Nov 9, 2007
    3
    0
    0
    I wonder does it means that client editions now can at least read deduped drives out of the box?
     
  12. bfoos

    bfoos MDL Guide Dog

    Jun 15, 2008
    705
    617
    30
    No it does not mean that. You still need to install the Microsoft-Windows-Dedup-Package-amd64 and Microsoft-Windows-FileServer-ServerCore-Package-amd64 packages to read files on deduplicated volumes.
     
  13. PCP69

    PCP69 MDL Novice

    Jun 8, 2016
    4
    2
    0
    I just installed it on 2004 German and not only does it work, but messages (in PowerShell) are in German. This might eventually be related to a previous installation before I did an inplace upgrade, though.
     
  14. PCP69

    PCP69 MDL Novice

    Jun 8, 2016
    4
    2
    0

    I tried to integrate/slipstream these in an offline 1909 but I got a

    0x800f081e error (not valid for this image).

    Any hints as to what I am doing wrong?
     
  15. bfoos

    bfoos MDL Guide Dog

    Jun 15, 2008
    705
    617
    30
    Post the full Command Prompt input/output between Code tags.
     
  16. PCP69

    PCP69 MDL Novice

    Jun 8, 2016
    4
    2
    0
    That would be the part of the script that tries to integrate Dedup:

    Code:
    @echo ***Adding Dedup>>%MyLog%
        for /R "%Dedup%" %%U in (*.cab) do (
            echo *Adding %%U to image>>%MyLog%
            "%MyDism%" /Quiet /Image:%MyMnt% /Add-Package /PackagePath:"%%U">>%MyLog%
        )
    "%MyDism%" /Quiet /image:%MyMnt% /Enable-Feature /FeatureName:Dedup-Core /All>>%MyLog%
    
    Errors (in German):

    Code:
    ***Adding Dedup
    *Adding D:\_Imaging\Dedup1909\x64\01 Microsoft-Windows-Dedup-Package-amd64-10.0.18362.1.cab to image
    Fehler: 0x800f081e
    Das angegebene Paket ist nicht fr dieses Abbild gltig. // CBS_E_NOT_APPLICABLE
    
    *Adding D:\_Imaging\Dedup1909\x64\02 Microsoft-Windows-Dedup-Package-amd64-10.0.18362.1-en-US.cab to image
    Fehler: 0x800f081e
    Das angegebene Paket ist nicht fr dieses Abbild gltig.// CBS_E_NOT_APPLICABLE
    
    *Adding D:\_Imaging\Dedup1909\x64\03 Microsoft-Windows-Dedup-Package-amd64-de-DE-10.0.18362.1.cab to image
    Fehler: 0x800f081e
    Das angegebene Paket ist nicht fr dieses Abbild gltig.// CBS_E_NOT_APPLICABLE
    
    *Adding D:\_Imaging\Dedup1909\x64\04 Microsoft-Windows-FileServer-ServerCore-Package-amd64-10.0.18362.1.cab to image
    Fehler: 14081
    Die referenzierte Assembly konnte nicht gefunden werden. // The referenced assembly could not be found.
    
    *Adding D:\_Imaging\Dedup1909\x64\05 Microsoft-Windows-FileServer-ServerCore-Package-amd64-10.0.18362.1-en-US.cab to image
    Fehler: 0x800f081e
    Das angegebene Paket ist nicht fr dieses Abbild gltig. // CBS_E_NOT_APPLICABLE
    
    *Adding D:\_Imaging\Dedup1909\x64\06 Microsoft-Windows-FileServer-ServerCore-Package-amd64-de-DE-10.0.18362.1.cab to image
    Fehler: 0x800f081e
    Das angegebene Paket ist nicht fr dieses Abbild gltig.  // CBS_E_NOT_APPLICABLE
    
     
  17. luzea

    luzea MDL Addicted

    Apr 25, 2019
    621
    636
    30
    You need to remove the dedup packages and install the FileServer packages first, then the dedup packages again.
     
  18. bfoos

    bfoos MDL Guide Dog

    Jun 15, 2008
    705
    617
    30
    I'm not sure if this is what is causing your issue, but Command Prompt doesn't like spaces in paths. Try naming your files like this instead...
    Code:
    01_Microsoft-Windows-Dedup-Package-amd64-10.0.19041.1.cab
     
  19. luzea

    luzea MDL Addicted

    Apr 25, 2019
    621
    636
    30
    That’s not the problem, I experienced this myself. FileServer has to be installed first, then Dedup.
     
  20. bfoos

    bfoos MDL Guide Dog

    Jun 15, 2008
    705
    617
    30
    Yeah, I just tested this myself. FileServer-ServerCore package needs to be installed first.