The MOMA (Museum of Modern Art) is pretty cool too.
I saw Ratatat play a show at the Guggenheim back in '06. It was awesome. Definitely a cool museum, I'd hit that up too.
Madison Square Garden isn't much to behold from outside, might as well go in. And if you're in the mood for pizza go...