If every gem is made to serve a specific leader, it must have been shocking for a huge chunk of Homeworld to lose their purposes. Everything on Homeworld seems to be orderly, organized, and perfect. Losing a Diamond would create imperfect rifts, and in addition, all of the forces of Pink Diamond would need to be either shattered or reassigned. But when you only have one purpose, how could you possibly be reassigned?