There are various factors that ought to be considered when making such an incredibly, earth-shatteringly important decision. The biggest consideration is the summon time for a mount, firstly, which requires that your character stand still for 1.5 seconds. That time is time wasted, time which could be spent running towards your destination. And what's more, many characters have speed boosts, like Blink, Heroic Leap, Feline Swiftness, Body and Soul, and so on. Those all have to be factored into your decision: to mount or not to mount?
I posted this question on Twitter last night, and among the discussion, a mathematical force emerged. Jarre, druid blogger at Rank4HealingTouch, leapt in with some calculations. They obtained their numbers for a character's base run speed, and put together what they referred to as "a bit of napkin math".