it's standard fantasy setting ever since tolkien that humans outnumber all the other major intelligent races: elves, dwarfs, halflings (I think, I need to check), beastkin/lizard-kin, etc.: being balanced (no major weaknesses), adaptable, etc.: so survival of the fittest.
Being balanced, humans are also good at negotiating with other races, and getting more benefits from trades (like getting dwarven weapons), whereas elves' pride means they can't take advantage of these benefits. humanity's balance: means we develop in all areas: magic, metal-work like weapons, all elements, etc. whereas elves only develop magic/nature, and dwarves only only metalwork, halflings mostly develop only rogue skills (and don't work on magic at all), and beastkin/lizard-kin only develop fighting/hunting techniques.
you can think of isekai slavery setting as the point in which humanity's development has overshadowed the the other standard races (except for demon kind with its demon lord) too much that humanity will definitely win against them if war breaks out. many of the more prideful races like elves still cling to their (past) pride, but the wiser individuals already realize it's too late to beat humanity, so even if they know about the enslavement of a some members of their race, they don't actively start wars against humanity.
if you think that the other races are too stupid or something, please remember that the novels we read are written by humans for humans, and that isekai novels with slavery are often supposed to be wish-fulfillment. Also, note: japan is a pretty homogenous country: elves with their pride is like their metaphor for caucasian foreigners, other races is like a metaphor for other types of foreigners (except more beautified for the japanese readers' wish fulfillment), so japanese readers probably identify with humanity more