Two different approaches to this that I liked were Rift and Throne & Liberty. In Rift, traditional raid bosses were instanced, though there were open world bosses labeled as "world events" that didn't encounter lock, so anyone could participate in killing them. (The loot kinda sucked though, so...