1. Complete Paladin-specific quests
Christian listed these as his number one recommendation and I'm going to have to concur. At some point early in the game's design, Blizzard thought it would be cool to have players do quests that added a special flavor to their class. It was a little thing that sent Shamans talking to spirits or Warriors cavorting with whirlwinds. In addition to infusing some class-related lore, those quests made each class feel unique.
Unfortunately, Blizzard has mentioned that such class-specific quests had a poor ratio of development time to player ratio, with only a tenth of the classes getting to experience it. This means we won't see class-specific quests very often, so doing what Paladin quests are available is just making the most out of the game. Fortunately for Paladin players, we've got much more lore in the game and consequently richer quests than most other classes.
It has to be said that the Alliance and Horde quests feel completely different from each other, and it's a shame that most new Horde Paladins will never get to experience the how it is to draw power from a chained-up god. That quest has been replaced with drawing power from one of the magisters in stupor, which is functionally the same but doesn't quite have the same punch.
The path to Redemption
While there are actually class-specific quests as soon as you start out the gate, the real thing begins at Level 12, as both Alliance and Horde have short quest lines to obtain the Redemption spell. For the Alliance, Humans and Dwarves start from The Tome of Divinity (same name, different quests for either race), which starts off a chain of quests that culminates in the Paladin learning Redemption. Draenei have it much simpler, starting with a "talk to..." quest called Jol. Blood Elves also have a different chain that starts with a Summons from Knight-Lord Bloodvalor.
Is it any wonder that many Paladins enjoy Retribution? For a lot of Paladins, their first taste of a really cool weapon was through the quest chain obtained at Level 20. The Tome of Valor kicks off the quest for Alliance characters to obtain the dangerously-spelled Verigan's Fist. It's an exciting quest chain that requires traveling to another continent as well as enter one the Paladin's first instances. The Horde equivalent, which starts from The Second Trial, is equally exciting and has a more sinister tone. It awards the Blood-Tempered Ranseur, which is lower in DPS than Verigan's Fist, but has Strength and looks way more badass. Both quest lines award Sense Undead along the way.
Blizzard has made it so that players can simply learn the spells - and with the implementation of Patch 3.2 where players can learn epic mounts much sooner - so the quests will amount to pretty much nothing now, the absolute best class-specific quest lines are the ones formerly needed to learn the Charger, or epic mount, spell. It is a great shame that Paladins no longer need to do these quest lines, but I highly recommend it simply because it's extremely fun, fulfilling, and infuses your gameplay with a good bit of lore.
The WoW.com guide to getting the Alliance mount is somewhat dated but will still prove handy, as will the guide for the considerably easier Horde quest line (which has the added bonus of a cool-looking tabard). While the mounts will be available from trainers, the quest lines are about the richest Paladin-specific content in the game. Any self-respecting Paladin player should at least get a taste of the few great, specialized quests Blizzard ever designed.
Trinkets and such
Optionally, there are the afterthought quest lines that drive players to enter the extremely confusing Temple of Atal'hakkar. These quests start off from the Western Plaguelands, in Chillwind Camp and The Bulwark for Alliance and Horde players respectively. Alliance players get a choice from three shiny blues, and Horde Blood Knights get Scourgebane, which has excellent Stamina for an item of its level.
2. Make a Death Knight and complete the starting experience
I've raved about the Death Knight starting experience back during Wrath beta, and I'm happy to say that the opinion still holds. It's one of the best things Blizzard has ever designed and will be one of the most fun players will ever have. What does this have to do with Paladins? Well, the Death Knight experience plunges you right into the heart of evil - essentially the corruption of everything that a Paladin stands for. You'll be forced to do all manner of despicable things, and in the course of breaking free from the Lich King's thrall, you will understand the path of redemption.
While this perhaps reeks of some RP and lore-nerdism, the experience is invaluable to understanding just what a Paladin means in the game. Why? For one thing, the absolute greatest Paladin moment in the game (so far) can only be witnessed through playing the Death Knight starting experience until the end. As a Paladin player, I got goosebumps throughout the battle in Light's Hope Chapel... from the talk about consecrated ground, to the wielding of Ashbringer, and the - there's that word again - redemption of key characters in lore. One cannot understand how truly bright the Light is without knowing the darkness. The bottom line is that it's fun. Do it. And turn on your weather effects.