Virtualization (VMware, VirtualBox, Hyper-V, ...)

radditsu

radditsu

Silver Knight of the Realm
4,676
826
ronne said:
You're vmotioning VMs between physical datacenters? Madness I say!

Anyway, you positive all the nodes you are using have 10g links on them, and the 10g links are the nics assigned for vmotion and management traffic? Our nodes for example have 10g pci nics in them, but that connects to storage only, they run over the onboard 1g for vmotion and management traffic etc, but none of that is leaving the confines of their DC so it never really matters. If you have a 10g link between these two DCs and it tests at 10g speeds outside of vmotion traffic on these nodes, occams razor says its the nodes obviously. You on the network team it sounds like?
Click to expand...


Well we aren't exactly a huge operation, we have a failover rack in another location ~ a mile away. We have got the VM support involved. They can never figure out anything.

It is on a isolated management VLAN so ONLY layer 2, I got into the cli of the VM's yesterday and poked around and everything looked fine. Going into deep dives could it be some cache setting maybe? I was going to mirror the port today and see if there was some sort of MTU issue, but I find it hard to imagine a MTU causing 90% degradation in speed. Could it be a HDD write speed? IS there a way i could do an IPerf like test between ESXI management hosts in the CLI? So many questions. The more I dig into it the more I don't THINK its a network thing. But I don't want to be the asshole.
 
  • 1Solidarity
Reactions: 1 user
Vinen

Vinen

God is dead
2,782
486
I'd validate there is no MTU Fragmentation.

Any of you out at VMworld right now? Need to find someone other then co-workers to hang out with for a night.
 
ronne

ronne

Nǐ hǎo, yǒu jīn zi ma?
7,900
7,041
I'm stuck working while others got to go, such is life.

Radditsu, it could be storage speed, depends on what you're using. If your replication environment isn't on a SAN or other high end storage and is just writing to local disks you could easily be hitting a chokepoint there. Do you see this slowdown anywhere but during vmotion? Like, if you just make a 50gb tar file on a node and copy it to other via console how quick does it move then?
 
Alkorin

Chancellor Alkorin

Part-Time Sith
<Granularity Engineer>
6,029
5,915
MTU could easily cause a 90% performance drop, sure, but you'd notice it everywhere, and you'd be able to see it easily. Just attach a Fluke or something like it (a laptop with Wireshark will do) to the interface where the vMotion traffic is, and if you're being inundated with fragmentation re-sends, that's your problem. The question is why your MTU would be set that way, and whether or not you have control over the setting on both sides of the WAN.

As ronne said, though, often this is just a result of (really) poor storage IO. What storage are you using and is there constant replication involved here (like, SAN-level replication), or is this copying the file over the WAN in its entirety every time you vMotion? How loaded is your IO on a regular basis? If you're sitting at a reasonably high level of IO during normal operation, trying to vMotion without shared storage could easily push you over the edge.
 
Frenzied Wombat

Frenzied Wombat

Potato del Grande
14,730
31,802
We have two Hyper-V 2012 R2 clusters each running about 20 VM's each on Netapp storage. I've honestly never touched anything beyond the basics when it comes to VMWare, but Hyper-V has been rock solid once we got the host drivers and hotfixes sorted. Hyper-V replica is also gold from a DR perspective as you can basically have fault tolerant replicas on dissimilar storage and not have to worry about shipping snapshots.
 
Breakdown

Breakdown

Gunnar Durden
5,803
8,015
Ive used Citrix Xenserver for a few years and love it. They lost their HP certification recently I think, so they are fucked if thats true.

Been demoing Nutanix hardware recently running OpenXen. Pretty interesting little product.
 
Alkorin

Chancellor Alkorin

Part-Time Sith
<Granularity Engineer>
6,029
5,915
Frenzied Wombat said:
Hyper-V replica is also gold from a DR perspective as you can basically have fault tolerant replicas on dissimilar storage and not have to worry about shipping snapshots.
Click to expand...

Yep. This is one thing that 2012 R2 has going for it. No shared storage? Yes please. Before this feature, I wouldn't have touched Hyper-V in any business environment.
 
radditsu

radditsu

Silver Knight of the Realm
4,676
826
No Mtu issues. Wiresharked and the only packet fragmentation was the ones i created. We did notice that if we Vmotion multiple servers at a time there is a definitive increase in network utilization, than one at a time.
 
tyen

tyen

EQ in a browser wait time: ____
<Banned>
4,638
5,164
Found out the other day that VirtualBox is an absolutely terrible option for a MacVM. Using VMWare, great for making an Apple App.
 
Frenzied Wombat

Frenzied Wombat

Potato del Grande
14,730
31,802
BTW, fuck VDI in its ass. Never seen such an overhyped tech that will garner such intense hatred from your CFO and user base. Hey, but at least you only have to patch one golden image right? This shit has no business being installed outside of schools and hospitals.
 
Big_w_powah

Big_w_powah

Trakanon Raider
1,887
750
Frenzied Wombat said:
We have two Hyper-V 2012 R2 clusters each running about 20 VM's each on Netapp storage. I've honestly never touched anything beyond the basics when it comes to VMWare, but Hyper-V has been rock solid once we got the host drivers and hotfixes sorted. Hyper-V replica is also gold from a DR perspective as you can basically have fault tolerant replicas on dissimilar storage and not have to worry about shipping snapshots.
Click to expand...

With hyper-v replica---Does the replica VM have to be running along side the primary site VM? Or is it off, getting updated, and only turned on when failover is needed?
 
Frenzied Wombat

Frenzied Wombat

Potato del Grande
14,730
31,802
Big_w_powah said:
With hyper-v replica---Does the replica VM have to be running along side the primary site VM? Or is it off, getting updated, and only turned on when failover is needed?
Click to expand...

It has to be off and can only be brought online during a failover, and gets incremental updates at defined intervals (default 15 minutes). You can also make these replicas VSS aware if you're running something like SQL, maintain staggered replicas, as well as setting up "chained" replicas where you can actually have a tertiary site also receiving a copy.

We've tested it multiple times and it works great. In fact, despite having Netapp and Commvault doing whole volume as well as VM specific snaps, we still choose to go the hyper-V replica failover route because it's just plain easy.. You can even pre-program the alternate IP configuration as part of the failover.
 
A

a_skeleton_03

<Banned>
29,948
29,762
This server is on a ProxMox host for those interested.

I will take screenshots later of how I have it running in case there are any suggestions for things to change.

This server and everything running on it is self taught in the last 6 months or so. First foray into VM as well.
 
gogusrl

gogusrl

Molten Core Raider
1,359
102
This is my little homelab
Advantech AIMB-272 mini-itx, with a i7-2760QM (4c8t), 16gb ram, 120W pico-psu, 256 ssd as datastore, Lsi9211-8i passthrough to a VM with 1 x 128 ssd as lvm-cache for 1 x 6tb for torrents and 2 x 2 tb for backups. Everything is stuck in this case PowerCube 712 | Computer Cases | Products | Spire Corp - Powered by Innovation

Running ESXI on it with Mikrotik CHR for routing 1 gbps, a Debian for Sonarr, RuTorrent, Plex, etc and a Windows 10 VM for work stuff. Can route 1gbps in one VM, download at 1gpbs in the other while encoding a Plex instance or two. All at ~75W max load and ~20W idle.

Thinking of switching to ProxMox because I haven't played with it yet and looks interesting.

rLi7fPH.png
 
Big Phoenix

Big Phoenix

Pronouns: zie/zhem/zer
<Gold Donor>
44,550
92,986
Whats a good way to learn about virtualization? Would something like this;

TS-251+ :: QNAP

Be good for getting your feet wet or would it be better to build an actual pc with an intel i7 and use hyper-v? Really want to pickup a skillset so I can move on from just being a help desk monkey.
 
A

a_skeleton_03

<Banned>
29,948
29,762
Big Phoenix said:
Whats a good way to learn about virtualization? Would something like this;

TS-251+ :: QNAP

Be good for getting your feet wet or would it be better to build an actual pc with an intel i7 and use hyper-v? Really want to pickup a skillset so I can move on from just being a help desk monkey.
Click to expand...
So I bought a Dell R720 for $2k and slapped ProxMox on it and just figured it out.

You could go cheaper like an R710 for under $500 and do the same.
 
Vinen

Vinen

God is dead
2,782
486
Big Phoenix said:
Whats a good way to learn a

bout virtualization? Would something like this;

TS-251+ :: QNAP

Be good for getting your feet wet or would it be better to build an actual pc with an intel i7 and use hyper-v? Really want to pickup a skillset so I can move on from just being a help desk monkey.
Click to expand...

Intel NUC are what most in my circle use.
For example, NOTE: Removed. This site is a piece of shit :). Turns it into a media link or something which doesn't work.

I would recommend learning vSphere/ESXi if you can. Hyper-V is out there (my product is multi-hypervisor) but I haven't seen many customers with it. Nobody ever got fired for using Microsoft or VMware. They did for using the others.
 
Last edited:
A

a_skeleton_03

<Banned>
29,948
29,762
Vinen said:
Amazon.

There I told you :D
Click to expand...
You have Adblock on then, whitelist us and all your problems are resolved!

Yeah a lot of people on r/homelab use NUc's from what I have seen.

The reason I used proxmox is because I liked the interface and was wanting to play with something a little more open. I regret it but it works just fine. The thing I regret is that any problem I might have the answer isn't quite a single search away.
 
You must log in or register to reply here.