Mass file copy speed starts fast but slows down, then fast again

Discussion in 'Windows 10' started by eemuler, Sep 23, 2019.

  1. eemuler

    eemuler MDL Senior Member

    Jul 31, 2015
    281
    82
    10
    The process using 100% disk is - drum roll - Windows Explorer - tada!
     
  2. pf100

    pf100 MDL Expert

    Oct 22, 2010
    1,832
    2,757
    60
    @eemuler Boot another OS from flash drive or cd/dvd and copy the same file(s) to/from the same drive(s) you're doing now and you'll 100% know exactly what the problem is.
     
  3. eemuler

    eemuler MDL Senior Member

    Jul 31, 2015
    281
    82
    10
    Downloading now.
     
  4. Carlos Detweiller

    Carlos Detweiller MDL Spinning Tortoise

    Dec 21, 2012
    3,740
    3,491
    120
    Could be congestion (of the OS copy buffer). Happens if the source disk delivers faster than the target disk can cope with. The target is an OS disk (even if it's a fast one like SSD), any write I/O has to be shared between OS and copy operation (disks can only write or read one sector/block at a time). In the "fast" period, the source disk copies the excess data to the OS buffer in RAM, until it's full. In the "slow" phase, the OS empties the buffer to the target disk. Wash, rinse, repeat.
     
  5. eemuler

    eemuler MDL Senior Member

    Jul 31, 2015
    281
    82
    10
    #25 eemuler, Sep 24, 2019
    Last edited: Sep 24, 2019
    (OP)
    The target is not a system disk. The OS is on the M2 SSD (which is about 5 times faster than this SATA SSD).. The source external USB HDD is about 5 times slower than the SATA SSD.
    This problem, however, is not limited to just the SATA SSD. In my first post I listed various combinations of source and target drives.

    I've downloaded UBUNTU-MATE. Preparing a live USB pen drive. What a pain that is. The first distro I downloaded didn't work with GPT. Then Rufus had a bug. Then my persistence partition was too large. Then I forgot to change the MBR setting to GPT. Fingers crossed it will work this time.
     
  6. pcnavarra

    pcnavarra MDL Senior Member

    Nov 13, 2017
    304
    118
    10
    #26 pcnavarra, Sep 24, 2019
    Last edited: Sep 26, 2019
    Just a thought, maybe the laptop ribbon sata cable was build for an 5400rpm HDD.
     
  7. MrMagic

    MrMagic MDL Guru

    Feb 13, 2012
    6,015
    4,132
    210
    Happens to me copying locally disk to disk, think I read something about buffers filling up but was never interested enough to look more into it, but if I copy from PC to NAS (via Raspberry Pi) it is a solid transfer from start to finish maxed out only by the Pi's 100mbps LAN adapter gives around 11MB/s
     
  8. Enthousiast

    Enthousiast MDL Tester

    Oct 30, 2009
    29,094
    44,664
    390
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  9. eemuler

    eemuler MDL Senior Member

    Jul 31, 2015
    281
    82
    10
    11 MB/s is the 'low speed' I'm getting. You might have the same bug and not know it.

    Sorry guys, my Linux experiment is a total bust. I get the stdin: Invalid argument error on booting from the live USB pen drive I made. Some Googling suggests that I need a USB 2.0 port - my laptop only has two types of USB ports - 3.0 and C type. So, no live Linux testing possible. Bummer; wasted half a day on this.
     
  10. toyo

    toyo MDL Senior Member

    Aug 14, 2009
    398
    265
    10
    #30 toyo, Sep 24, 2019
    Last edited: Sep 24, 2019
    I don't know if anyone mentioned it, but most SSDs have SLC caches that offer the initial boosts in speed. After you fill that, it will drop to whatever the main NAND of the SSD is, MLC or most probably TLC if you're using consumer grade stuff, like the Evo series for example from Samsung. If it's the cheaper QLC NAND in SSDs like the QVO series, the drops in write speed can be really harsh.

    PS: Your Patriot drive looks to be TLC. 10Mb/s is really low though even for TLC.
     
  11. eemuler

    eemuler MDL Senior Member

    Jul 31, 2015
    281
    82
    10
    I did mention it in the first post itself. If using the M2 SSD as a source drive, the initial speed is huge. Settles down to about 500 MB/s after a few seconds, and then this 100% disk usage kicks in after a few files, say 4 to 6 files, have been transferred.
     
  12. MrMagic

    MrMagic MDL Guru

    Feb 13, 2012
    6,015
    4,132
    210
    No, that is the max speed of the network adapter in the Pi, not a bug
     
  13. eemuler

    eemuler MDL Senior Member

    Jul 31, 2015
    281
    82
    10
    If the bug reduces the transfer speed to ~11 MB/s, and your maximum speed is ~11 MB/s, how do you know you have the bug?
     
  14. MrMagic

    MrMagic MDL Guru

    Feb 13, 2012
    6,015
    4,132
    210
    If the MiB are invisible, how do you know they are not following you?

    Because other networked devices with higher bandwidth adapters transfer at faster speeds, and because my disk to disk on the same machine never drops to 11MB/s, it just goes up and down between 100MB/s > 180MB/s give or take
     
  15. toyo

    toyo MDL Senior Member

    Aug 14, 2009
    398
    265
    10
    I have 2 suggestions which will likely not work.
    - turn off Defender/whatever AV you might have and try the same copy operation
    - TRIM the SSDs (you don't have any HDDs left, right? If yes, defrag those). You could check if TRIM if active if needed
    You could download Crystal Disk Info and see how things look for the SSDs and if all the info looks right. Also run Crystal Disk Mark, AS SSD and Anvil on the SSDs to make sure they are running benchmarks at the advertised speeds - maybe allow the test to loop a few more additional times trying to replicate the behavior.
    That 100% disk usage might be some sort of background operation? Like garbage collection kicking in? It's really weird.
     
  16. RJARRRPCGP

    RJARRRPCGP MDL Senior Member

    Feb 24, 2010
    299
    35
    10
    Did you check the Windows event log for any drive or drive controller complaints?
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  17. eemuler

    eemuler MDL Senior Member

    Jul 31, 2015
    281
    82
    10
    Such a fun way to spend an evening. I did find something weird though, related to the SSD in question.

    Device SCSI\Disk&Ven_APS-SL3N&Prod_-1T\4&35a2a51e&1&000200 was not migrated due to partial or ambiguous match.

    Last Device Instance Id: PCISTOR\Disk&Ven_RSPER&Prod_RTS5208LUN0&Rev_1.00\0000
    Class Guid: {4d36e967-e325-11ce-bfc1-08002be10318}
    Location Path:
    Migration Rank: 0xF000FC002001F130
    Present: false
    Status: 0xC0000719

    This might be irrelevant - disk wasn't formatted when it was installed. The last device instance is probably something random - it is the SD card reader which is not even on the same bus.

    The system events log is a sorry mess, but I did notice that VSS (volume shadow copy service) seems to be running, and complained while the system was being shut down. Could this be trying to take a snapshot while the file copy is in progress? Unlikely, because system restore is disabled for that drive.
     
  18. toyo

    toyo MDL Senior Member

    Aug 14, 2009
    398
    265
    10
    Time for a proper, clean Windows install maybe? You can restore your app settings from the Users folders, and reinstalling your apps shouldn't take too much time either.
     
  19. RJARRRPCGP

    RJARRRPCGP MDL Senior Member

    Feb 24, 2010
    299
    35
    10
    If it's for an empty card reader, then you don't need to worry about that.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  20. eemuler

    eemuler MDL Senior Member

    Jul 31, 2015
    281
    82
    10
    I'm gearing up to do just that - have to back up a lot of stuff, though, and this issue isn't helping me do that quickly.