Whats with the controversy and bickering with interleague play in the MLB?

To this day I still don't understand the complaint of interleague play in the MLB. In all other professional sports league in America, (NFL, MLS, NBA, NHL) it's perfectly normal and expected that teams from one conference will play against other teams in the other conference. Simple as that. No debate there. What's the deal? Is it because of trying to fit so many home and away series against teams from their own league in the regular season?