In this article, Julia Rone and I provide an overview of conflicts over embedding social values into generative AI systems. We highlight that these struggles add a social values dimensions to the politics of AI. As governments and businesses try to shape the values of AI systems, this leads to tensions both within and between societies and both on the level of AI system design and governance. Governments and businesses furthermore engage in values signaling, attempting to instrumentalize that certain social values and political ideas are necessarily embedded in AI systems.