Spirit Gem
Spirit Gems are items that are hidden in Phantom Hourglass that can be found by opening chests, solving puzzles, purchasing, checking the mail, or completing a certain event. There are three in total: Power, Wisdom, and Courage, and each has a different effect depending on which Servant Spirit is equipped, and of those there are 20 total gems of each version to collect over the expanse of the World of the Ocean King.
It should be noted that, for the gems to have any effect, they must first be taken to Spirit Island.
Spirit Gems look almost the same as Goddess Pearls from the The Wind Waker, and should not be confused.
Enhancement and Skill Upgrades
When Link collects 10 of a certain gem version, he sees an ability enhancement on the part of his sword or in his physical skill set, however, upon collecting all 20, Link is upgraded with the full extent of the Servant Spirits' power.
Power Gem
For the Spirit of Power, securing 10 gems ignites Link's sword and doubles its attack power. Getting all 20 quadruples the amount of such power. Hitting a Phantom or Reapling from behind with Oshus's Sword with the Spirit of Power equipped will stun it.
Wisdom Gem
For the Spirit of Wisdom, obtaining 10 gems doubles Link's defense. Also, the Shield will be upgraded, able to block more powerful attacks and able to stun weaker enemies by simply walking into them. Getting all 20 quadruples the defense ratio. If hit by a Phantom, less time will be lost from the Phantom Hourglass if the Spirit of Wisdom is equipped - instead of 30 seconds, it will be 15 seconds with 10 gems, or only 5 seconds with all 20.
Courage Gem
For the Spirit of Courage, getting 10 gems powers Link's sword with the Sword Beam technique. Getting 20 doubles the width of said beam. This has no known effect on the Phantoms or Reaplings.