Gauntlets of the Skullsplitter - You'll likely find these on the AH, so check often. These are available at 64, but will easily last you until 70. They lack stamina, but have a good amount of strength, agility, and most importantly, hit rating.
Ragesteel Gloves - Like much of the Ragesteel set, these gloves are lacking in stamina. They make up for it by having a lot of strength and crit rating, but remember -- you can't get the set bonus unless you're a blacksmith.
Adamantite Plate Gloves - These are more balanced than Ragesteel, as they carry both strength and a good amount of stamina, but they lack crit. However, they come with 2 sockets (a red and a blue), so you have some flexibility. Considering that some of pieces in your starting set may lack stamina, you may opt to take these.
Zaxxis Gloves - These have a nice mix of all the stats you're looking for, plus some hit rating. They don't really excel in any one area though, so I'd replace these when you can.
Gauntlets of the Vanquisher - Part of a long quest chain in Netherstorm, some of which will require a group, but these gloves are totally worth it. A lot of strength, stamina, agility and hit rating -- pick these up if you can.
Gauntlets of Cruel Intention - Drops off of Thorgrin the Tender in Botanica. These gloves pack a good amount of strength, stamina, and crit rating, so grab these if you get the chance.
Girdle of Siege - You can probably pick this up from your local auction house. Once you hit 63 you can use this belt, which has bonuses to strength, stamina and crit rating.
Engraved Cattleman's Buckle - Available from a quest chain in Netherstorm, this belt is almost so good it could be blue. Bonuses to strength, stamina, crit rating and hit rating... it's got it all.
Borak's Belt of Bravery - From the "Ciper of Damnation" quest chain in Shadowmoon Valley, this belt comes with strength, agility and stamina.
Deathforge Girdle - Drops off of Talon King Ikiss in Sethekk Halls, this is one of the better belts available. Bonuses to strength, stamina and crit rating, plus 2 gem slots for flexibility.
Legguards of the Shattered Hand - Carries a good amount of strength, agility and hit rating, but lacks stamina. However, some Clefthoof Leg Armor will make up for that, and get you some additional agility as well. Considering that you can find these at the AH, I would make an attempt to pick them up as they should last you for a while.
Runed Sketh'lon Legplates - From a quest in Shadowmoon, you really can't go wrong with these, as they've got what you need -- strength, stamina, crit rating.
Midrealm Leggings - Similar to the above, except you get them in Netherstorm. Compared to the Sketh'lon Legplates, these trade in some stamina but pick up some hit rating. If you're not yet at the hit cap, consider picking these up.
Ornate Leggins of the Venerated - Blackheart the Inciter, the 2nd boss of Shadow Labyrinth, drops these. No crit rating, but a good amount of strength, as well as some intellect. The spell damage really isn't helpful, but these do come with 3 gem slots, so you can just socket in the stats you need more of.
Boots of Zealotry - These come with a decent amount of strength and crit rating, but lack stamina. If you're having trouble finding a good pair of boots, consider picking these up from the auction house.
Vortex Walking Boots - Requires Exalted reputation with Ogri'la, but these boots are worth it. Strength, stamina, crit rating and 2 sockets are your reward for a long rep grind.
Expedition Footgear - Available from a quest early in Zangarmarsh, this is a greater starter piece, but should be replaced later.
Kirin'Var Defender's Greaves - From a quest in Netherstorm, these boots have a little bit of everything -- strength, stamina, agility and hit rating.
Obsidian Clodstompers - Drops off High Botanist Freywinn, the 2nd boss in the Botanica. These boots are an excellent choice, carrying good amounts of strength and stamina. They also come with 2 gem slots, so socket it what you need.
Lightwarden's Band - Achieving Revered status with the Aldor grants you this fine ring with agility, stamina and attack power. It's good, but lacking a little in attack power.
A'dal's Command - You've got to hit Exalted with the Sha'tar to get this ring, but it comes with a good amount of strength and a bit of agility and stamina.
Ogre Slayer's Band - From a quest chain in Nagrand, this is a well balanced ring with a little of everything.
Aggressor's Mark of the Sha'tar - From a long quest chain in Shadowmoon Valley. There are several DPS ring rewards for this quest, but this one has bonuses to crit and attack power.
Overseer's Signet - Available from a quest in Netherstorm, this ring comes with strength, agility and hit rating.
Ravenclaw Band - Drops off of Talon King Ikiss in Sethekk Halls and comes with agility, stamina, hit rating and attack power.
Ring of Umbral Doom - From Thorngrin the Tender in the Botanica, this ring has a good amount of stamina, crit rating and attack power.
Felsteel Boar - The use effect doesn't help out much, but this trinket carries a nice passive bonus to attack power. This trinket is BoP, and as such is only available to Jewelcrafters.
Nightseye Panther - Ignore the stealth bonus on this one. The passive bonus to attack power isn't as great as the Felsteel Boar, but the use effect is far better, increasing your attack power by a whopping 320 for 12 seconds. As with the boar, this trinket is BoP.
Ancient Draenei War Talisman - Available fairly early in Terokkar Forest, this trinket carries a nice bonus to crit rating, plus a use effect that increases your attack power.
Core of Ar'kelos - From Netherstorm, this trinket increases attack power by 52, and 200 for 20 seconds on use. You can use this or the above, depending on whether you need more crit or more attack power.
Bladefist's Breadth - You can pick this up almost immediately after entering Hellfire Peninsula, and you'll likely use it to 70 and beyond.
Skyguard Silver Cross - Requires Exalted with the Sha'tari Skyguard, but this trinket is well worth it -- a massive amount of crit rating, and has a 50% chance to increase your attack power by 140 for 30 seconds after killing an enemy.
Libram of Zeal - Not the best choice, but will help out a bit by enhancing your SotC.
Libram of Righteous Power - From a quest chain in Shadowmoon, this Libram will increase the damage of your Crusader Strike by 18. Not a huge boost, but is a nice alternative if you have nothing better.
Libram of Avengement - Not for starters, as this drops in Heroic Blood Furnace, but needs to be mentioned since this is arguably the best Libram for a Retribution Paladin. This causes JoC, JoR, JoB or JoV to increase your crit rating by 53 for 5 seconds, so work towards getting this as soon as you can.
The Oathkeeper - Very popular with retribution paladins due to its slow speed and high top end damage. The additional strength and stamina helps out as well. Most of all, this can be found at the AH if you're having trouble finding a weapon.
Blade of Misfortune - Better for Blood Elf paladins using SoB due to the speed, but can be bought at the AH and used starting at 60. Increases strength and stamina, and has a massive amount of crit rating.
Thunder - This is BoP, so you'll have to be a high level blacksmith to make this, but it's well worth it. The slow speed and high end damage is ideal for SoC, and can be upgraded as you start running heroics. Also consider Lionheart Blade or the Lunar Crescent depending on what stats you need the most of.
Hellreaver - Available in Hellfire Ramparts, this polearm has bonuses to strength, stamina and crit rating.
Shaarde the Greater - From Tavarok in Mana Tombs, this sword grants increased strength, stamina and crit rating.
Crow Wing Reaper - Drops off of Talon King Ikiss, the last boss of Sethekk Halls. This axe comes with strength, agility and stamina as well as 3 gem slots.
Firemaul of Destruction - From Warbringer O'mrogg in Shattered Halls, this mace has a good amount of strength, a large amount of stamina, and resilience. The resilience is only really useful for PvP, but the high end damage and slower speed is good for SoC.
So there's the list -- get out there and get started! You can put together a pretty decent starter set just by browsing the auction house, but remember to upgrade when you can!