Windows 10 Compression

Discussion in 'Windows 10' started by Flipp3r, Apr 21, 2015.

  1. burfadel

    burfadel MDL EXE>MSP/CAB

    Aug 19, 2009
    2,627
    3,856
    90
    LZX compression is much slower to compress. It would be good for files that you don't use much. The physical load time from disk to memory will be reduced, as you are loading less data, but you have to factor in decompression. Decompression is relatively fast though. Basically, the Xpress modes favour speed over compressibility (although Xpress16K is reasonable for compression), and LZX favours compressibility over speed. On a fast computer, it probably wouldn't make much difference with the load times, it's more the compression time.
     
  2. robmadden1

    robmadden1 MDL Member

    Oct 21, 2009
    185
    30
    10
    #22 robmadden1, Apr 22, 2015
    Last edited: Apr 22, 2015
    Thanks for letting me know. I like compressing the windows folder to save space.
     
  3. burfadel

    burfadel MDL EXE>MSP/CAB

    Aug 19, 2009
    2,627
    3,856
    90
    Supposedly the new Xpress types are not only better than NTFS file compression, they're also faster to decompress (load). Not sure if this translates into reality though :). Even if they're not faster, it's very unlikely they are slower to actually decompress (load). This is probably why they named them 'Xpress'. LZX on the other hand, is slower than NTFS file compression, both for compression and decompression (loading).

    Note that Xpress modes are probably slightly slower to compress, especially Xpress16K, than NTFS file compression, due to better compression. The decompression (load) speed is a different consideration all together, it already knows the order of the compressed data so it's very quick.
     
  4. robmadden1

    robmadden1 MDL Member

    Oct 21, 2009
    185
    30
    10
    Thanks I am just going to use Xpress16K then to compress my windows folder.
     
  5. burfadel

    burfadel MDL EXE>MSP/CAB

    Aug 19, 2009
    2,627
    3,856
    90
    Just be careful using any of the new compression formats. They're perfectly safe to use, but keep in mind they're OS based, not file system based like I mentioned earlier. If you compress files used outside of Windows (recovery images, boot folder etc), or you compress files needed during Windows load, using any of Xpress4K, Xpress8K, Xpress16K, or LZW will mean Windows can't load! NTFS file compression is still fine on these though, as the decompressor is part of the NTFS support.

    I hope that make sense! Basically, any file that already has the system based file compression in Windows can be reapplied using Xpress16K, and any other file can also be compressed to this as long as it is only used once Windows 10 is loaded.
     
  6. Superfly

    Superfly MDL Expert

    Jan 12, 2010
    1,143
    543
    60
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  7. burfadel

    burfadel MDL EXE>MSP/CAB

    Aug 19, 2009
    2,627
    3,856
    90
    Still in the process of writing it. Actually, I'm writing version 2.0, but on release it will be the initial version :). Reason for the change is a completely different and better way of processing. I'm still ironing out a few minor issues with it. I've also done it so it won't work on system critical files (Windows folder, EFI folder etc). This includes even when you've taken ownership of these files. This is for safety, as only some things in the Windows folder can be compressed using the new algorithms without 'bricking' your system. It's pretty easy to 'unbrick' the system after doing this, but it makes sense to just disallow people applying it to these system files so people won't face this issue.

    Basically NTFS file compression (using LZNT1), which is the least efficient of the algorithms and is a carryover from earlier Windows, is available from after the initial system files are loaded. The new algorithms (Xpress4k, Xpress8K, Xpress16K, and LZX) are only available after a certain time later in the boot process (of Windows 10).

    By default, many of the system files are compressed using Xpress4k upon installation anyway, so the benefits of recompressing these files only extends to the different between Xpress4K and Xpress16K. It may even be the case that the release Windows use Xpress16K on faster machines, this won't be known until Windows goes RTM.

    Of course, I could have 'half-ar$3d it' and released it already, but I'd rather release something that works well, fast, and is unlikely to cause issues.
     
  8. Guzmanus

    Guzmanus MDL Novice

    Sep 3, 2010
    12
    2
    0
    Is the option to compress system files still enabled on the last build (10061)? because i dont see it anymore.
     
  9. burfadel

    burfadel MDL EXE>MSP/CAB

    Aug 19, 2009
    2,627
    3,856
    90
    It is done automatically upon Windows installation. Unfortunately, it only utilises xpress4k. It's true xpress16k may take slightly longer to compress, but since these files are very much read orientated xpress16k would be better. I've found xpress16k can actually be faster than uncompressed and even xpress4k. It's hard to tell with normal programs due to the small amount of data loaded, but it can be seen in some games. Yes, I've compressed some games with it, it works well! Unfortunately there doesn't seem to be any option to select this. Manually changing it to xpress16k is possible but risky on system files (fine on Windows Apps though).

    Also, xpress4k seems to be the system default too. There may be a registry entry to change it to xpress16k, but I don't know what it is :). Now, when you compact a folder using the /s: command, it compresses the files in the folder, but doesn't actually set the compression attribute on the actual folder! If you name the folder without /s:, all new files added to that folder are automatically compressed, however you cannot define the compression method. As such, the new files are only compressed in xpress4k. The downside of this is when you run compression again on the folder, the file won't be compressed as it will skip already compressed files. If /f is specified to force recompression to xpress16k, all files are reprocessed, including the xpress16k files!

    So, unfortunately it seems xpress4k will be the default, and that currently there is no option in changing that. For the script, it will NOT set the compression attribute for the folder, only compress the files in the folder. There will be an option to process uncompressed files in selectable folders in the main script application (not the explorer extensions part). I'm also thinking also a predefined folder list that can be added to :). That might come later!

    In any case, it's a great feature of Windows 10, but Microsoft could improve on it slightly.
     
  10. Soulfate

    Soulfate MDL Member

    Feb 14, 2011
    185
    51
    10
    That would be interesting to write a script to smartly compress some interesting system files.
    Will a DISM capture save the files compression? It's interesting because I "sysprep" my Windows image to fit to my needs and this option is very good for me.
     
  11. Superfly

    Superfly MDL Expert

    Jan 12, 2010
    1,143
    543
    60
    Has the system compression option been removed from 10074? I don't see it in cleanmgr options...and Treesize says Win folder 0% compr. so it's definitely not already compressed.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  12. xinso

    xinso MDL Guru

    Mar 5, 2009
    12,526
    13,516
    340
    #32 xinso, May 6, 2015
    Last edited: May 6, 2015
    On my Acer Aspire Switch 10 running 10074 Enterprise x86, the disk cleaner looks like that on Windows 8.1.

    But the WinSxS folder is

    size: 3.53 GB
    disk size: 2.26 GB
     
  13. burfadel

    burfadel MDL EXE>MSP/CAB

    Aug 19, 2009
    2,627
    3,856
    90
    #33 burfadel, May 7, 2015
    Last edited: May 7, 2015
    It has been removed from cleanmgr, and moved solely to installation time. If Windows setup detects your computer is fast enough, it compresses certain folders automatically at installation. Or at least, it toggles a setting somewhere to do it at a later time. However, I have heard from a couple of people with very decent machines that it doesn't work like it should. Manually compressing is possible, but if you do it on the wrong files you can prevent Windows from starting. Also, the default compression is Xpress4K, and Xpress16K results in noticeable smaller files and actually seems faster to read (reads less data and decompression is super fast), however it takes longer to write due to the searching required during compression.

    I'm still in the process of writing that program. Actually the base program is complete, I just want to add some more stuff to it, the ability to skip certain files etc. It's great on games, but some game files do not compress. For instance, Half-Life 2 with Cinematicmod 2013 compressed very nicely with Xpress16K, and actually loaded quicker. If there are any suggestions of game files that do not compress let me know. I've already including files like mp3, m4a, jpg, etc in the skip list. Also the Windows folder is a no-go, simply because you shouldn't do the wrong files in there. Besides that, it is already compressed for me (the auto-compression works for me).

    The downside of the auto-compression is it is only Xpress4k, it would be nice if Xpress16K was an option. This is especially true if it was in the registry somewhere... You can enable compression on a folder level (needs to be done separately), such that any file added to the folder will be compressed, but only to Xpress4K. Therefore in the script I won't be doing that since the script is to skip files already compressed. Again, registry setting or something would be nice!
     
  14. ZaForD

    ZaForD MDL Expert

    Jan 26, 2008
    1,212
    200
    60
    OK, that makes sense. I did a clean install on a HP Stream 7 from a USB drive, blowing away the previous partitions (except the 5GB recovery partition). And was surprised to find I had over 20GB of free space after installation of OS and drivers.
    Last install I only got that after using DISM with the /Compact switch.

    Can't say I've noticed the performance issues, but then again none of the TP's work that well on Tablets yet.
    And the WinSxS folder is the same size as Xinso's reported above.
     
  15. Soulfate

    Soulfate MDL Member

    Feb 14, 2011
    185
    51
    10
    Maybe you could create an option to recompress all already compressed files with the Xpress16K ?
    And do you know if a "DISM capture" keeps compression ?
     
  16. burfadel

    burfadel MDL EXE>MSP/CAB

    Aug 19, 2009
    2,627
    3,856
    90
     
  17. agent268

    agent268 MDL Junior Member

    Feb 20, 2012
    92
    33
    0
    Correct. When you capture via DISM, the outputted file is technically compressed in the WIM/SWM/ESD format. It does not keep the CompressOS flag, that only occurs when applying an image via DISM and you have to specify it. As already called out, it appears that Windows Setup will be able to dynamically identify if your PC can support "CompressOS" and will apply it for you. You can also create custom installation media with the new Windows ICD that forces the CompressOS feature.
     
  18. Soulfate

    Soulfate MDL Member

    Feb 14, 2011
    185
    51
    10
    I hope they will change that. Old NTFS compression flag is kept with a capture. It is handy to install a custom Windows image with compressed files. Maybe the new automatic compression step of Windows 10 is enough smart to do the job very well, but it won't handle custom files added by the user.
     
  19. Flipp3r

    Flipp3r MDL Expert

    Feb 11, 2009
    1,962
    904
    60
    I think a lot of people these days just deploy from a bootable usb key using dism. Can you apply with compression? What about when you create & format the OS partition? Is there a compression flag there?
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  20. abbodi1406

    abbodi1406 MDL KB0000001

    Feb 19, 2011
    16,141
    84,314
    340
    Yes, /Compact
    No flag