I agree, it's not too big of a stretch that they lose credibility in going that route. It's just a barrier that shouldn't be understated from both a technical and storytelling perspective.
I talked about why it's a big fucking deal from a storytelling perspective above that boils down to: the ultimate race we know, the reapers, aren't from another galaxy and don't talk about going to another galaxy so there's a bit of a paradox. If the answer to that is, "Oh suddenly traveling across galaxies now is a thing." it has to be done carefully.
From a technical perspective the mass relays are limited in the distance they can transmit mass with. The citadel, being the biggest known mass relay, is the only one known to be able to transport mass into 'dark space' outside our solar system, and in the pictures of the reapers they are indeed in dark space and not fucking up some other galaxy.
Most of the relays transmit mass on the order of thousands of light years. And the duration of the trip is related to the distance. They aren't wormholes ala Interstellar, so any trip to another galaxy would be constrained by this fact.
Just to put it into perspective, earth to andromeda is 2.5m light years. Earth to our nearest star, Proxima Centauri, is 4 light years. The distance from the earth to mars is .005 light years, or 5 one millionths of a light year. So in terms of distance, the ratio between traveling from Earth to Mars and traveling from Earth to Proxima Centauri is the same as travelling from Earth to Proxima Centauri and Earth to Andromeda.
In other words, saying, "Oh you send a rover to mars? You can use the same tech to go to Proxima Centauri." is as ridiculous as suggesting you use mass relays to go to Andromeda.
However, they can just wave their hand at this ala Interstellar and say "no mass relay, we use wormholes.". another classic way around this problem is to just have a fleet of spaceships travel to different parts of Andromeda in stasis mode carrying a bunch of incubatable human eggs. I think this would be more interesting because you'd have absolutely no contact with the Milky way and have to deal with the fact that everyone but you and your fleet is dead.