Just be glad that you found a way to boot with USB that works at all. The slow USB boot speed is just something you'll have to live with on that motherboard.
I have same problem. I haven't tried yet, but I have heard that a different brand USB thumbdrive can be the solution.