You should have an Eternal Ferrystone that you can use to warp back to Gran Soren, Cassardis, or Bitterblack Isle, or any of your Portcrystals whenever you want. Outside of those locations, yeah, Portcrystals are limited until you reach the postgame, when you can buy them in shops.