I think that using all nics in one vSwitch is the best way if you want to use the PCI Nics in order to obtain failover in case than one type of nics fall(onboard/PCI).
Another scenario could be using PCI nics for backup traffic, and the LOMs (virtual nics) as separate networks, i mean, maybe you can use 8 GB for LAN ( trunk), 1GB por MGMT, and 1GB for VMotion... but always you need to use VLAN Tagging.
hope this help you
Regards