They compromised on a lot of issues. Slavery they simply gave in on. That's part of a larger compromise, but it isn't like, say, the anti abortionist and those who simply agree it shouldn't be used in lieu of birth control both agreeing on that point or on minimizing the number through education. At the heart not every issue leaves room for compromise but most do, in some form or fashion. Jefferson could have tied his agreement into a demand for more humane laws about treatment, separation of families (with an eye to a slave free future) but he didn't and a tragedy was compounded.
In fairness to the political tests on point, my left lean is pretty minuscule and disappears on a number of large issues, from abortion to gun control. It's more of a mistrust of conservatism to protect the weakest, least empowered (slavery being a prime example of that, holding true through the civil rights movement and beyond).