VT-d enabled motherboards and CPUs for Paravirtualization

Discussion in 'Virtualization' started by anti-t, May 1, 2012.

  1. anti-t

    anti-t MDL Junior Member

    Oct 18, 2009
    73
    41
    0
    #1 anti-t, May 1, 2012
    Last edited: May 26, 2012
  2. anti-t

    anti-t MDL Junior Member

    Oct 18, 2009
    73
    41
    0
    As I understand, VT-d was a functionnality of the northbrige chipset, but newer cpus now have on die northbrige, hence the confusion. It's a shame that intel decided allow vt-d only for non k processors.

    for the record, I've just ordered an Asrock Z77 Extreme 4 with i7 3770 (non K). I'll will report back here results asap. This mobo have VT-d options and support SLIC insertion as I've read here. Fingers crossed...
     
  3. anti-t

    anti-t MDL Junior Member

    Oct 18, 2009
    73
    41
    0
  4. anti-t

    anti-t MDL Junior Member

    Oct 18, 2009
    73
    41
    0
  5. anti-t

    anti-t MDL Junior Member

    Oct 18, 2009
    73
    41
    0
  6. SimonH

    SimonH MDL Novice

    May 24, 2012
    4
    0
    0
    Have you tried passing through the second SATA adaptor (i.e. the ARMedia one, not the one on the Z77 chipset) through to a VM? It seems to be available for pass-through (looking at nachtrab.de/2012/04/28/esx5i-z77-directpath-vt-d-asrock-z77-pro3/ I think it's the 00:1b.0 device) but this post (hardforum.com/showthread.php?t=1682499) suggests it may not work (though they weren't using the same mobo) and others reckon it's a bit iffy trying to pass through mobo SATA controllers.

    Thanks for all your posts anti-t :) - I'm building a whitebox ESXi 5 server and am most likely to buy a ASRock Z77 Extreme 4 too (I have an unopened ASUS P8Z77V-LX but that doesn't support VT-d so is going back: vip.asus.com/forum/view.aspx?id=20120519215856079). Ideally I'd like to run a desktop VM using the onboard graphics and passing through a USB controller for the keyboard/mouse, as well as a NexentaStor VM by passing through the second SATA controller with a couple of disks attached to it. Getting a compatible combination however is not easy...
     
  7. SimonH

    SimonH MDL Novice

    May 24, 2012
    4
    0
    0
    Thanks for your update anti-t! It sounds as though VT-d support is a bit variable and Intel probably don't put much as much certification/testing effort into it as compared to server processors/chipsets. I think the ASRock response is much more helpful than the "no, the Z77 chipset doesn't support it" answer I had from ASUS. I suppose vendors have two approaches: a) stick 100% to the book so that there's no scope for misinterpretation and reduce product support/RMA returns, b) try to be helpful but on the understanding that they can't say for sure but that we're sensible people and can make judgements based on the information we have. Presumably most mobos are sold to either enthusiasts or professional system builders so they should expect their customers to be looking for the best mix of features/value. It's a generalisation of course but when you look online there also seems to be quite a lot of loyalty amongst customers - once they've have good experience with one brand they seem to stick with it for future builds.

    Certainly this issue has lost ASUS some credibility for me. I've ordered an ASRock Z77 Ext4 now so will update this useful discussion hopefully in a week or so once I have my system built.
     
  8. anti-t

    anti-t MDL Junior Member

    Oct 18, 2009
    73
    41
    0
    #10 anti-t, May 26, 2012
    Last edited: May 26, 2012
    (OP)
    Hi SimonH

    I fully agree with you, Asrock at least try to help and actually provide us choice while Asus don't !!! I for sure will never buy Asus boards anymore. Asrock on the contrary, gained much respect on this. They also respond to mail and give much more useful explications. Asus is wrong as you can see below : Z77 support VT-d if you have compatible processor !

    Since yesterday, I tried what you suggested, I passed the asmedia SATA chipset AND the asmedia USB 3 controller SUCCESSFULLY to an win7 vm. I've connected a usb stick, usb hdd, keyboard and mouse the the asmedia usb controller and they all work fine ! I've not connected a sata harddisk on the asmedia because I haven't any drive left for testing. As you can't see through, the controller is working.


    Here are some screenshots :

    Clipboard02.jpg

    Clipboard03.jpg

    Later I tried the intel iGPU (HD4000 in my case), it was added and detected ok within the vm but caused BSOD while installing the intel vga driver. This is not a problem of the mainboard itself, passing VGA devices is more complicated because you have to deal with the vga bios. When I'll have more time, I'll try VGA passthrough in XEN. Stay tuned!
     
  9. SimonH

    SimonH MDL Novice

    May 24, 2012
    4
    0
    0
    #11 SimonH, May 30, 2012
    Last edited: Jun 1, 2012
    On my new ESXi whitebox server (ASRock Z77 Extreme 4 with i7-3770 and 4 Vengeance 8GB DIMMs) I've successfully flashed the BIOS to P1.60 (I used Instant Flash from USB - fortunately had none of the problems of the earlier versions) and installed ESXi 5 using an image I made with PowerCLI and Image Builder to add the latest Broadcom driver (v3.123b.v50.1). So far, so good.

    Likewise VT-d appears to be working at the ESXi level anyway - here's the ASMedia SATA controller I think:
    vt-d_asmedia_sata.jpg
    I need to rustle up a spare SATA disk to attach to it to see if I can successfully see the disk from my (Nexenta) VM.
     
  10. SimonH

    SimonH MDL Novice

    May 24, 2012
    4
    0
    0
    Hmmm - looks like the driver for the ASMedia SATA controller is only available for Windows on ASRock's website. I don't fancy my chances of getting one for Open Solaris (which NexentaStor is based on).
     
  11. k336

    k336 MDL Novice

    Jun 3, 2010
    2
    0
    0
    SimonH, I have an ASRock Z77 Extreme4-M. Just thinking you may have the wrong device selected for passthru, for me the ASMedia controller was under a PCI/PCI bridge.
    View attachment 15621

    I've successfully have the disks on the ASMedia controllers working in a Win7 VM.

    I can also confirm the problem where the ASMedia controller does not work when passing through the Intel Panther Point SATA controller. Which is a shame, was planning to use the 2 port ASMedia controller for ESXi datastores, and the Intel controller passed through to run a file server.

    On a side note regarding power management, passthrough drives are able to spindown, however idle datastore drives can not. I will be testing if using RDMs will allow for the disks to be spun down.

    My aim is to create a quiet and low power whitebox, might have to get another SATA controller if RDMs wont work :(
     
  12. kv

    kv MDL Novice

    Jun 24, 2012
    2
    0
    0
    need to double check VT-d boards!

    This link from the OP to these boards **must be read carefully. **

    source : http: // www . intel . com / support / motherb... / cs-030922.htm

    The list given for VT-d boards is actually:

    Q77 DQ77KB, DQ77MK
    B75 DB75EN
    Q67 DQ67EP; DQ67OW; DQ67SW
    Q57 DQ57TM; DQ57TML
    Q45 DQ45CB; DQ45EK
    Q35 DQ35JO; DQ35MP

    This is found after the VT-x list.

    I'm looking at coupling the E3-1240 v2 with the Q77 board DQ77M. Since I have a GTX 560 Ti card already (for CUDA calcs), the Q77 chipset seems to match what I'm looking for (home, 1st level workstation, lower power for less noise and green touchy-feely stuff, no overclocking, maybe some gaming but more for extended molecular dynamics/ab initio calcs, etc.).

    Plan to have an SSD, one 6Gb SATA hdd, and like the USB 3.0 support for external backups if desired (though a second hdd would be fine, and could network that one out for others at home).
     
  13. anti-t

    anti-t MDL Junior Member

    Oct 18, 2009
    73
    41
    0
    Thanks for pointing out that intel modified their own compability list.

    the page was updated on
    This doesn't change the fact that intel Z77 chipset is proven to be compatible, though you may want an Q77 to be sure. Also AVOID ASUS boards;
    Be aware also that nvidia cards have issues with VGA pass-through, ATi seem to be more suited. For other controllers (SATA, USB) all should be fine
     
  14. kv

    kv MDL Novice

    Jun 24, 2012
    2
    0
    0
    No problem about the update! Glad to be sure that others coming after me know to check themselves for stuff like that. ;)

    I've backed off the Asus boards totally. They show no inclination to explicitly support Linux at this time anyway. I'm headed toward a Supermicro board. The video is a non-issue.
     
  15. spacebro

    spacebro MDL Novice

    Jul 2, 2012
    1
    0
    0
    Hello, thank you guys for this valuable info.
    I want to make system on z77 and have question regarding hardware health monitoring.
    I plan to create RDM disks for freenas or unraid vm. Is it possible to monitor SMART, temperatures(cpu, disks) inside vm or in vsphere client?
     
  16. alpha97

    alpha97 MDL Novice

    Feb 10, 2011
    7
    0
    0

    I'm planning to build a new Esxi server, and first think about the Q67 from intel which offer 5 internal SATA ports and 3 PCI-E slot~ It's glad to know that the Asrock Z77 Ext4 support VT-D as well, and offer 4 PCI-E with 8 SATA ports~

    Regarding the above comment, and consider the comments from others, it seems that either the ASMedia controller or the Intel SATA controller is passed through, the other controller will not work any more.

    May I know if we can pass through both of the ASMedia controller and the intel SATA controller ? such that I can pass both to a VM running freenas.

    Many Thanks!
     
  17. k336

    k336 MDL Novice

    Jun 3, 2010
    2
    0
    0

    That's something I was thinking of testing too. However I am unable to test at the moment as I'm sending the RAM back for RMA. What were you planning to use as datastores to host the VMs?

    I'll post again when I get my RAM back in a couple weeks.
     
  18. alpha97

    alpha97 MDL Novice

    Feb 10, 2011
    7
    0
    0
    Many Thanks.

    I am thinking for the below 2 ways:

    1> The VM with the dedicated HDD will be a freenas instance, and a raid 1 disk can be shared out as datastore thru iSCSI~

    2> Buy a cheap two bay nas, and shared out thru iSCSI ( less likely to happen)