Well zombies you're forced into STR stacking, which obviously translates into a lot of EHP. Skeletons you could build glass cannon but no real reason to. In fact I'd say there's no reason not to build tanky on summoner regardless of the skill you're using. All my recent summoners pathed all the way down to marauder shit and got like stun immunity, 250% life, a layer of block/evade via flasks, one of them had 2min end charges, Soul of Steel for more phys reduction and 1% max, the spectre amulet for big EHP(although currently I think you might be able to do better with the life% rolls on amulets) and so on. 8k raw life, some ES on top(if using phys summons you can use spirit offering for great survability), some avoidance, some regen via stone golem, leech via baron if zombie, can use MoM with an enduring mana flask for big EHP/"regen" and so on. Current trend is to use cyclone with cast on channeling to pop spirit offering more often, while providing high life on hit(imperial claws), holy relic procs(more healing and damage) and fortify easily, with immunity to stun(from cyclone).
For clear speed though anything skeleton is going to be kinda sucky. You have to resummon every pack and while they'll kill the pack instantly there's the delay for their out of ground animation, so it's not super great. Zombies should feel pretty alright though, but for random map clear you can more or less do a 4L +3 spectre in helm with solar guards or slavers and just clear white mobs with that.