So, it looks like the long dungeon queues could be the result of a bug. Haven't been able to find much info on this, but here's what I've been experiencing. Queue for dungeon (or pvp) get group, teleport to instance. Someone disconnects or drops group, so I leave party. Re-queue get a party, game doesn't tp me to dungeon, and won't let me leave party, unless I log out. Now, this has a cascading effect, because any group I join and can't get to or leave will be shorthanded, so they will leave party, re-queue and get stuck, etc etc.
I think it could be related to the cooldown on changing map instances, but that still doesn't explain why you can't leave party.