It's great! When I read the comments from other, either one will work for pass through with the other one not available. This information could help many of us! I will have a look, it would be helpful with more features and being stable. Good, thanks for the idea! I first plan to use 1 usb for Esxi itself only, and let the freenas share a raid 1 storage as datastore for ESXi thru iSCSI, while I forgot the freenas required a separate disk for freenas OS installation which cannot be installed to the datastore this time. Another dedicated USB will solve this with the pass through feature. Yeah, seems I can soon start my plan, finally have all the things in 1 machine instead of 2 now. One more question, may I know the disk performance of the freenas under such virtual setup? Thanks
I don't run it full time because this mainboard is my work computer, still I tried copying a 4GB file (4*275*585*028 bytes exactly) from another computer on the lan. The file has been copied (via samba) on a slow 2.5" hdd connected to the asmedia sata passed through. It took about 122 sec ~ 33,42Mb/s. This is very good IMHO considering the weak hdd used. Also take in consideration that the network controller was bridged in ESXi (not passed through in the nas vm) so I loosed additionnal perfs here. Unfortunately, I haven't another (Gb) network controller to perform more tests. My guess is that the passed through sata controller run at native (or near native) speed at the vm level.
Hello everyone, came across this thread during some searching. I am trying to find a board that supports nested virtualization that passes Intel IO-MMU and VT-xd through to multiple layers of VMs created on ESXi. I know Xeon chips and chipsets can do this, can anyone recommend a consumer level board that supports this same functions and features? Thanks!!
May be some people here have tried this scenario already (I don't !) As far as I know, once a device or controller is passed through and a vm is using it, it become unavailable for sharing to the other vm's. VT-x is CPU virtualization, it is always available. For ex, you can run a vmware workstation (under linux, windows) inside ESXi and use a passed through controller in this vmware VM. All you need is a board that support VT-d. I would recommand Asrock here, and for more support, you may pick a Q77 like the Asrock Q77m V-Pro.
I have gotten the Intel DQ77KB motherboard with an I7-3770S CPU to work with ESXi 5. There are a couple of gotchas to be aware of, but on the whole it works fine. The biggest gotcha is a BIOS bug that's present as of version 0043. ESXi 5 installs to a GPT partition by default, and the board has a BIOS bug that apparently won't allow automatic booting to a GPT partition. You get an error saying that there is no boot partition available, or if you have any other bootable drives or devices attached it will always boot to one of those. However, if you interrupt the boot sequence by pressing F10 and then explicitly picking the drive that has ESXi on it, then it will boot from that drive just that one time. The fix for this is to reinstall ESXi 5 and use Shift-O during the install to enter additional options. Enter the option: formatwithmbr That will cause EXSi to install on a MBR type partition and after that it will boot normally. ESXi 5 only recognizes one of the NIC cards. It doesn't recognize the one that's marked for use by the Intel AMT software. That might be for the best because I've read that although you can supposedly share this NIC card between an OS and the AMT firmware, there is often a performance hit and/or lost packets if you do. I did test the AMT functionality and I was able to boot the machine, enter the BIOS, and configure ESXi from the console successfully. I also did some testing with the vt-d functionality. I installed an Intel 5300 wireless card in one of the mini express card slots on the motherboard and I also tested it out on the native USB controllers. It works just fine with the Intel card, and I was able to assign the card to a VM and use it with the native drivers which worked just fine. To answer a question earlier in the thread, if you assign a device as being eligible for assignment to a VM, then ESXi can't use it, even if no VM is currently using it. When you make changes to which devices are eligible to be assigned to a VM you have to reboot ESXi afterward to make it effective. Please note that if you assign a device that ESXi needs, for example the disk controller that has ESXi itself on it, then you can end up with a non-bootable system and will likely have to reinstall, so be careful what you choose. Once a device is eligible to be assigned to a VM, it can be moved from one VM to another without rebooting, but can only be assigned to one VM at a time, and you have to have the VM powered off to add or remove a "PCI" device. I played around with the USB controllers in assigning them to a VM, as I want to be able to put some USB hard drives for backup purposes on the box for use by the VMs. If you use the ESXi native USB support the performance is really bad, so I wanted to assign the whole controller to a VM in the hopes that it would perform at near native speeds. I got some mixed results trying this. First, although ESXi shows several USB controllers as being assignable devices, which makes sense given how many USB ports are on the board, it seems as if assigning even one of them to a VM causes all the USB ports to be assigned to the VM. Furthermore it didn't seem to matter which USB controller I picked. I was expecting to have to test and see which external USB connectors went with which USB controller, and was hoping to only assign one controller to VMs, and leave the others assigned to ESXi. However, it appears to be an all or nothing proposition. Furthermore the VM (I tried both a Windows 7 and Windows 8 VM) always sees the controller as a USB 2 controller instead of USB 3. I could tell that both from device manager and from the color of the LED on one of the portable USB drives I was testing with as it's blue when connected as USB 3 and green when connected as USB 2. The good news is that the device did transfer data at near native USB 2 speeds, which is a big improvement over the throughput you get if you use the native ESXi USB support and assign it as a USB device. The other thing to be aware of is that since all the USB ports are taken away from ESXi when you assign any one of the controllers as eligible to be assigned to a VM, you lose the ability to communicate to it via the keyboard on the main console. Also, the Intel AMT functionality was affected in that I could still see the console from AMT, but I couldn't type, so the AMT support is somehow pushing keystrokes into the USB controller simulating a keyboard I guess. To get around this issue I've ordered a PCI express USB 3 card and an adapter that will allow me to connect the card to the second mini-pci express slot on the motherboard, which has a cutout on the backplane to allow the card to stick out the back. That way hopefully I can assign the new card as a PCI device eligible to be assigned to VMs, and I can leave the onboard USB controller(s) under the control of ESXi. I also hope that will allow the VMs to see the new card as USB 3. I'll post an update once I get the card and have a chance to test it out. mudtoe
May I know if you make it on ESX 4 or ESX 5? Latest update: seems ESXi5 do not allow creating VM without datastore....
Problem solved For ESXi4, the esxi installed usb drive could be used as local datastore, and one may setup a vm to host the freenas instance to managed the dedicated disk controller, such that the localstore would be accessed thru iSCSI offered by the freenas on the same machine. For ESXi5, the esxi installed usb drive could no longer used as local datastore, and the freenas instance is no where to install if one dedicated the disk controller for the vm supposed to host freenas. One of the solution is to buy a little NAS with iSCSI, such that we can setup the "freenas vm" on the NAS with the dedicated disk controller of the ESXi machine.
Maybe this could be helpfull. I can confirm, that vt-d is working on DZ77GA-70K. I have managed to pass secondary nic and marvell sata controller to guest OS. On the other hand built in 1394 controller fails, but i don't need it
Hi anti-t, can you help to ask the AsRock CS person again which AsRock he tried to enable vt-d with 2500k (k-series CPU)? Unfortunately I read about vt-d too late, by now I can only return the ASUS motherboard and cannot return my 2550k CPU already. I assemble this new computer just for vt-d. So please help me to save from wasting my money for nothing. Really need your help! Thanks a lot!!