This is likely so that Siri knows where you are in relation to the contact, and can appropriately dial using whatever international and/or local prefix/area codes are needed.
Sure, you know that 'Uncle Joe' lives down the road and you can call him via his phone number, but Siri doesn't, and there might be an area code or international code requirement in order to complete the call. THAT is why Siri wants your location.
If you don't care about this, you can turn off Dial Assist (Settings -> Apps -> Phone -> Dial Assist) and Siri should just dial the number as entered in contacts (although I haven't tried this to see if it no longer requires Location Services, but logic says it should).
If you're really, really worried about your phone tracking you, turn it off. Seriously, there are so many other apps and ways of tracking you, this seems like a hard hill to die on, and for the vast majority of users, dial assist is useful. In either case, your cell phone provider knows where you are when you make a call (they have to for the call routing), so there's no hiding.