Silver Knight of the Realm
ha. honestly like 40% of the game occurs in that town. There is a ton of stuff to do, talk to everyone.
As for getting destroyed. this is a common mistake. Guards, and npcs will worn you, but its easy to misread, or overlook. Exit west, out of the west gate, (or secret exit.), but not to the western beach. the beach is like level 8, or so iirc. When leaving town its west, north, west-beach, east iirc. then on to new area. level 3 is fine for the actual western forest.
you can have a full 4 members by the time you leave town. starting two, and 4? npcs, to pick 2 from.
The only companions who matter are Medora and Jahan the other two suck.
Madora pump leadership and survivability. Jahan pump water and air and dual wield. Staffs suck. He may also be a decent wand build but I never use him that way.
Also once you get the halls where companions go you can buy nonverbal companions...also can stick the ranger chick next to an oil, ooze, cooking pot, anvil, and whetstone and pump crafting for infinite money. Axe + log then branch+branch = staff that sells for $$$$