After some more digging, it looks like the class name being assigned is wrong. I can see the styles in a style tag (the element selected is what should have those styles) - but I don't have the common error in console about the class name being wrong

Also worth noting, I just removed styledComponents: true, from the next config and the error about classname appears. So the config is definitely working