Here's a partial transcript:
Clarke: I would say to them, Set me free
Colbert: Slavery, really? I didn't realize there was slavery in Brooklyn in 1898.
Clarke: I'm pretty sure there was.
Colbert: Sounds like a horrible part of the United States that kept slavery going until 1898.
Colbert: Who would be enslaving you in 1898 in New York?
Clarke: The Dutch.
The New York Daily News notes New York legally abolished slavery in 1827. And the Dutch ceded control of New Amsterdam in 1674.