I think you are misinformed. I took a lot of classes in Physics department of IIT Delhi and the teachers were as good as the electrical engineering department. In fact, I did my B.Tech. project in the physics department.
Whether you should do your masters from IIT is entirely up to you.
I like to blog about various topics ranging from education in IITs to social issues in India and US. All opinions and views presented here are my own and not of my employer.