Peridot reported back about the gems living on Earth, and was sent to destroy them. When she failed, and eventually joined their ranks, Homeworld only sent a few ruby soldiers. But they were only after Jasper! So since Homeworld wiped out most of the Crystal Gem forces some 5,000 years ago, are they actually concerned about their existence?