"Suppose there should be a law made that all black men should be imprisoned, it would be unreasonable. We have as little reason to quarrel with other men for being of different opinions than as for being of different complexions."