Know where to look
The most important thing you'll need in order to collect 100 mounts is a list of all the mounts that are available to you. There are many online and in-game addons and such that will help you keep track of your mounts. But I like a good, old-fashioned checklist.
To create that checklist, I go to Wowhead. If you just do a raw list of all the available mounts, you'll get over 290 available mounts. Now you need to start filtering. There are 201 mounts available to the Alliance and 202 mounts available to the Horde.
These are just raw lists. There are some mounts here that you can't obtain if you're not a particular class and other mounts that aren't available anymore. Others aren't available to players. But still -- the point of this list is to get started so that you know where you are.
Print out (or save to a spreadsheet) all the mounts on your list. Now you can get started.
The broad strokes
When you're collecting mounts, it helps to get the "broad strokes" out of the way. That means picking up all the racial mounts available to your faction, then getting all the PvP mounts. It helps to make sure you're exalted with all of the races that make up the Horde and Alliance, since that gives you carte blanche to collect everyone's mounts. To make it a little easier, look at the list of Horde-specific mounts and Alliance-specific mounts.
The PvP mounts might be the most painful to you, if you don't enjoy the battlegrounds. However, PvP really is one of the quickest ways to pick up a handful toward your goal. Each mount takes 2,000 honor to get. You get the Alliance mounts from Lieutenant Karter in Stormwind and the Horde ones from Raider Bork in Orgrimmar.
The next obvious place to start looking for mounts is the Argent Tournament. Each race has an additional mount available there, and the tournament itself offers a mount. Picking up these mounts will take a while, since you get the currency to buy them from daily quests. But give it time, and you'll get them all.
There are still a lot of one-off mounts living in instances. Here is going to be where you spend a great deal of time grinding instance after instance. Whether the mounts drop in heroic or normal mode, just chalk yourself up for a lot of time in the dungeons. If it helps soothe your frustration, make sure to sell all the cloth and greens on the auction house; it'll make the time spent feel a little more worthwhile.
There are about 15 mounts that are reward from achievements. However, these require a lot of work outside your ability to solo, so don't put too much effort into them. Reserve these mounts for when you're done everything else.
It takes time
Lastly, remember that the ultimate mount achievement is 100 mounts. That's a lot. It's going to take a lot of time. It's meant to; this is supposed to be a big deal. Take your time and work down your printed list one by one. Look at your progress along the way, and remind yourself that you're getting there. Leave us a comment when you're successful; I'd love to hear that people are getting this done.
Visit the WoW Rookie Guide for links to everything you need to get started as a new player, from how to control your character and camera angles when you're just starting out, to pulling together enough cash for mid-level expenses such as mounts, to dungeoneering and travel tips for lowbies.