The problem with empire... you are on the good side of argument.
Look at empire' type of government. They have autocratic leaders. While the empire would be successful when they have capable leaders, the empire goes horrifically wrong if they have incapable leaders. Can people choose the capable leader? No, because leaders are not selected based on talents and capabilites.
As well, the difference in terms of culture, language and other things make the empire to function inefficiently in many cases. For instance, English Empire would make sure that Britain gets the best interest, and place tariffs and other fees to discourage her colonies and give advantages to British companies. Colonies are suppressed, ensuring that they would not work too efficiently that would improve their colonies but not the motherland. Empire limits the advancement of the colonies.
Colonies are used as bargaining chips by the empires. These native people have no say with their land. If the empire wishes it, their land could be taken away and sold to other nations/empires. If you were part of my empire, then I have every right to take away every goods you possess and sell you off as slavery, by the Imperial order. You will not have any right to debate with me in the name of the emperor.