I think just based on this cliff notes version of your relationship that doing nothing would leave you with regret. So, I say make more of an effort (go visit, try and call more, whatever) but have some clear lines in your own mind that when crossed you can say to yourself "Fuck this, I tried"...