I always took it that Jedi were like Space Marines in 40k. In a sense.
I mean this in that while people have heard of them. The average person has no real idea what they can do or anything and the average citizen of the Imperium/Republic can easily go through their entire life never seeing one...