Echoing what is said here, the first part of the Halligtree is pure Fuck You. Run past it to the first grace, as there is nothing unique of value that is worth the headache. Of course if you have Heima Cannon, its a lot easier, but the bubble fuckers still can facefuck you from miles away with their autotracking horse shit. The lower you get, the tougher the enemies, but the less the environmental screwover becomes. Plus the place is a gold mine of resources (3 Somber Dragon Stones can be obtained here, among other very good unique items and talismans) you will want. The last boss is a mixed bag and worth one item (if you are high Dex or High Faith) and a Somber Dragon Stone, so skipping her is fine. The rest of the place, particularly the run between the first boss and the drainage, is loaded with goodies.
The good news is that you can put it off as long as you want, even past the end of the game if its pissing you off that much. Don't slack on the snow mountain areas, though. There is way too much stuff for ALL builds to just skip over it. And its actually pretty fun outside of the snowblind dragon boss. Fire Giant seems rough, but it is doable by nearly any build, especially if you keep up on Alexander's quest or have Poop Man as a summon.
I would follow Ranni and Fia's questlines to the end before tackling either, however.