It's kind of the same reason some folks have a beef with religion. It's all great until it grants one group special privileges or actively limits the possibilities for another. Same affect as affirmative action.
Comparing Masons to affirmative action is pretty disgusting.

