Interestingly, you're both right and wrong at the same time. It's called the art of medicine for a reason. Imagine this scenario. A patient comes in with pain and swelling in her right calf. An ultrasound reveals a deep venous thrombosis. Patient is admitted to the hospital and started on Lovenox (anticoagulation, to prevent further clot build up.) While bridging the patient to warfarin, she experiences the sudden onset of difficulty breathing and pain in her chest. The most likely scenario is that she threw a pulmonary embolism, but we don't know this for 100% certain. We could do a CT of her chest and expose her to radiation (which would increase the likelihood of some type of thoracic cancer by almost 0.5-1%) to prove that she has a pulmonary embolism, or we could presume she has one, because proving it would not change the course of treatment (you treat a pulmonary embolism the same way as a DVT.)
Similarly, when someone presents with unilateral facial pain in the distribution of the trigeminal nerve, you think trigeminal neuralgia and start treating for that. TN is mainly a clinical diagnosis, as there aren't any blood or imaging tests to back it up. With his symptoms, you don't think "cerebral hemorrhage" and do a CT scan of the head. You don't run every test in the book for every complaint. No medical system on planet earth could sustain itself if medicine were practiced this way. Yes, sometimes mistakes are made, but if you follow the best evidence based guidelines, then 99 times out of a 100 you're going to be ok. It's actually a big problem with doctors and mid-level providers (NPs, PAs, etc) with poor diagnostic skills that work in ERs. People will present with certain problems, and instead of being able to diagnose the problem they run every test in the book, which skyrockets the cost of healthcare and exposes people to unnecessary radiation. CT scans can cause cancer. I have one patient that has hemiplegic migraines (migraine headaches so bad that they give her stroke-like symptoms), and every time the poor girl comes into the ER she gets a CT scan of the head, because of the fear that one time she may actually be having a real stroke, but the poor girl may eventually develop brain cancer. (Note, CT of the head is the first test run on any patient with stroke to see if it's hemorrhagic or ischemic, the latter receiving aspirin and being managed medically [I don't push TPA, poor neurosurgery coverage where I am], the former of course requiring immediate neurosurgical intervention.)
Basically, you treat common problems commonly. In the case of the OP, I would have been suspicious of giant cell arteritis, which is a medical emergency, but my physical exam and the course of the disease would have led to the correct path. I probably would have gotten a biopsy of his temporal artery and checked a ESR/CRP to be safe.