My adviser called me to his office as he was considering two candidates (both Indians) for a Ph.D. position. He wanted my opinion on who he should choose. I had taken a class with both of them. So, I knew both of them quite well. One of them was from IIT and another from a local college. I told him the pros and cons of choosing each candidate, but, my final recommendation to him was "Choose the candidate not from IIT!"
What did my adviser do: "Chose the candidate from IIT!", just because he was from IIT. So, it does matter.