The Civil Rights Act of 1866 and Reconstruction

People commonly say that the Fourteenth Amendment was ratified, in part, to constitutionalize the Civil Rights Act of 1866. Indeed, there is an ongoing debate among scholars about how the 14th Amendment did that.

But why is the premise correct? Congress enacted the Civil Rights Act of 1866 on the authority of the 13th Amendment. And in the Civil Rights Cases, the Supreme Court passed on the question of whether the Act was a valid exercise of congressional power under the 13th Amendment.  

It is true that John Bingham did not think that the 13th Amendment authorized the Civil Rights Act of 1866. But he was alone among Republicans in Congress in believing this when the Act was passed. Why then do people keep saying that one part of the original public meaning of the Fourteenth Amendment was constitutionalizing the Civil Rights Act? Even if other members of Congress besides Bingham said this when the 14th Amendment was under discussion, I'm not sure that's enough evidence. Or at least I'd want to see how many people said that and who they were before I made up my mind.