In my point of view it is all about communication. Not only street photography but any thing you do in the street even when you are not speaking to people, it is all about communication. As street photographer it is all about the communication with your subject, it applies for any kind of photography, even still photography is a communication between photographer and subject. I feel that it is what most photographers fail to realize.
Back to street photography it depends on what you are telling people with your attitude and body language. If you in in the street photographing people passing by you, you are just photographing people passing by you in a public space. You are not blocking them, you are not following them, not invading their personal space or doing anything personal. You are only interested on people in society, their culture, their fashion, their body communication, the moment in history and society, and so on. In case the person feel uncomfortable about have their image in your camera, they are free to come to you and ask to not publish their photo anywhere or to delete their photo from your camera. You do as they wish and life go on. It is not a big deal.
Many photographer are afraid of people reaction. This is the reason many photographer feel uncomfortable to be seen by their subjects. But as far as you are not doing anything wrong or mean to the person, there is no reason to feel uncomfortable or uneasy about what you are doing and show it to them.
I was in Dublin photographing people legs in the streets. When I was going home I saw a woman with beautiful legs, wearing a beautiful shoes. I approached her and asked if I could photograph her legs. She obviously felt uncomfortable with the unusual request. So I explained her that I photograph legs as representation of people expression and personality, without their face, hands, etc. I also showed her the photos of many legs I took that day. She than understood that I was not a pervert and it was nothing personal about her, and she became pleased to let me photograph her legs. Honestly, the number of people who told me "thank you" to me because of my interest on their style, look, etc are huge. Many people see as complement my interest on photograph them.
Some people don't like and some people will feel threatened think it is something personal and become rude. But again, show show them that you don't care about them at all, it just happened they were there when you was there photographing people. Many people are insecure, paranoiac or won't be able to understand what you are doing. And that's ok, but not your problem. Just delete the photo you took from them if they wish and go on with your activity. Never argue and never try to cleaver. Only explain once or twice and if they don't get it, let the paranoiacs and insecure ones think what they want to think. As far you are doing nothing wron you have no reason to worry about them. All that matters is you photography project and knowing that you know what you are doing.
I think it is bad when you surprise people jumping from nowhere in front of them, blinding them you your flash in a close up distance, scaring them and blocking their way. Than you being invasive and giving reason to people get annoyed.
Or when you are with a tele-lens or behind the bush photographing people without they noticing they are being observed and photographed. It is not only creepy but also unfair because they can't ask you to not use their image of they wish and there is not dialog (body language dialog). It is more a voyeurism than street photography, in my opinion. Voyeurism is when there is no communication between photographer and subject, or observer and observed. See without be seeing as observer. Or be seeing without knowing you are being observed as subject.
The problem is that many photographers are afraid to communicate with their subjects or don't know how to communicate.
It also depends on the situation and culture. In Ireland people usually just ignored me and they could see that I was just photographing people passing by, so they usually don't take it personal. As far as you don't block people way or follow them it is not considered rude or invasive. After all they are in public space doing nothing what you wouldn't expect to see in public.
In Germany is the opposite. Blocking people way is not considered rude. In fact people do it all the time because they don't have what in other cultures we call personal space. On the other hand, because of the introvert culture they easily feel invaded by just your presence or with small talk (and they don't pay attention to body communication). So instead of ignore you they will stare at you even if you are not photographing them, to watch you in case you invade them (despite the stare to be considered a invasion when you are being seen but not when you are the one staring).
So Ireland has a more social e communicative interaction which make street photography easier and more pleasent, while in germany is more individualist and no communicative social interaction, which makes it harder and less pleasant, but not impossible for street photography.(I have a big collection of German Stare street photographs, their stare is as invasive as my photographs which I find an interesting cultural subject to approach).