Starting the fight
As a caster class, mages will generally want to keep some distance from their enemies, enabling them to cast freely and without fear of interruption. One common tactic seen in PvP is a mage starting things off with a Polymorph and casting a long cast spell such as a Pyroblast or Frostbolt. If such spells land, they'll usually take a large chunk of your health and gets the mage started on the right foot. Don't let them. Unless you're a healer or have ways to recover lost health or put up some immunity to even the odds, having a mage get off a long-cast, high-damage spell on you puts you at a disadvantage. Even as a healer, be mindful that mages have Counterspell at the ready, so letting them damage you also means they'll be waiting for that heal.
Most mages will start with Polymorph as a way to bait your trinket. How you deal with this is discretionary -- mages can easily re-sheep you (albeit with diminishing returns) as well as have a variety of snares they're likely to unload after you've blown one cc break. A rule of thumb you can follow is that if you'll be within range to do some damage after breaking sheep, or close enough to interrupt a second cast, use the trinket. Of course, it is ideal to have an ally remove the Polymorph with a dispel or hope that some splash damage in the area will break it, but sometimes the best course of action would be to simply "eat" the incoming high damage spell and save the trinket for later use.
Mages hate up close and personal combat. They hate it. It's generally a bad idea for a mage to be in the thick of things so they will do everything in their power to get away from it. Although they have instant cast spells and close range caster-centered AoE, getting close to them disrupts their flow. A mage performs best when at a distance, running, stopping to cast, then running again while firing off a few instant cast spells. They hate close range even more than hunters (even hunters will want to keep combat at minimum range against a mage) because they wear cloth and their spells can be interrupted or pushed back. A lot of your effort will be spent closing the gap... this is par for the course. Be prepared to use all your mobility tools to establish combat at a favorable distance.
Their bag of tricks
Mages have a wide arsenal to use in PvP, although there are a number of spells that will see frequent use on the battlefield. Polymorph, as mentioned, will be used and abused to no end. It is a 1.5 second cast spell, so it is reasonably interruptible. It is from the arcane school, so spell lockout will restrict their use of Counterspell or Blink, but still enable them to cast frost or fire spells. It can be recast at will but is subject to diminishing returns, so mages tend to cycle their sheep targets to optimize crowd control. It also regenerates the target's health, so you can sometimes use this to your advantage by letting the regeneration tick for a few times before breaking it, ideally putting up a defense of some sort as you do.
Blink is another spell that you should watch out for. Aside from mobility, Blink breaks the mage free from stuns and snares, and mages will use it instinctively to break free of such spells. It is on a short, 15-second cooldown, so you should time your stuns or snares to land no longer than a few seconds after its use. If you can silence, time it right before when you expect them to use Blink (hint: if you're in melee range, that will be almost every cooldown). This will confuse and frustrate them. You can keep track of their cooldowns with AddOns such as Afflicted. Better mages will trick opponents by running towards them and Blinking right before melee range and reappearing behind their enemies. If a mage is running towards you, be prepared to whip around and face backwards so you won't lose pursuit time.
All mages regardless of spec will use Frost Nova. It is one of the game's best peeling spells and mages won't hesitate to use it when in enemies have gotten into an uncomfortable range. Many mages will use this spell followed by a Blink to create some distance. Not only does it root enemies, but they are also considered Frozen, which means they are vulnerable to a Shatter combo. Shatter is a frost talent that increases critical strike chance by up to 50%, and getting crit by a mage is seriously bad news. Frost mages will often follow up a Frost Nova with Ice Lance, while others will cast longer cast, higher damage spells. Basically, Frost Nova is a bad thing. Get out of it or be prepared to get hurt. A lot.
Cone of Cold
Similarly, all mages will use Cone of Cold to slow opponents down. It is on a 10-second cooldown (8 if talented) and is an instant cast spell, which means mages will almost always have it ready and can't be prevented from using it. With the proper talents, the spell can do respectable damage, although most mages will use it only for the snare component. The good news is that the spell has three basic limitations: it is a cone effect in front of the caster, which means the mage must be facing their opponents to use it; it has a relatively short range (12 yards with Arctic Reach); and it is expensive.
Even though mages are casters, it's easy to overlook the fact that they're also one of the best classes against fellow casters because of Counterspell. Healers in particular will have to watch for the short cooldown spell and try to bait it with several jukes or fake casts or suffer from a long spell lockout. Try to bait Counterspell with a spell from a different school (sorry, paladins, you're screwed here), allowing you to cast heals freely. Good mages won't fall for it, however, and will wait usually only counter nature or holy spells. Many PvP mages will also pick up Improved Counterspell, which isn't too deep in the arcane tree. Always keep track of Counterspell's 24-second cooldown.
Although they don't have offensive dispels, mages have access to Spellsteal, which is expensive but works even better. They tend to use this against buff-heavy targets, although the buff stolen is random. Be extremely wary of using spell power increasing buffs, and protect it with other buffs if possible. Shields and other absorption spells are also attractive targets for a Spellsteal.
And then there's Arcane Explosion, one of the most popular and recognizable mage spells that sees frequent use in the Battlegrounds, particularly around flags in Arathi Basin or the Isle of Conquest. You would do well to remember that a cornered mage engaged up close can still inflict a lot of pain with Arcane Explosion. It gets increased damage from a few talents, but ultimately is dangerous because it is spammable in quick succession. It will be expensive for the mage, but they would rather lose mana -- which they can recover -- than health.
Mirror Image and Invisibility
Although they have three minute cooldowns, Mirror Image and Invisibility are powerful clutch spell that mages often in battle. In desperate situations, a mage can cast Mirror Image to throw off enemy targeting, although a /targetlasttarget macro should allow you to reacquire the mage. Some use it to escape -- even pairing it with Invisibility, although most will use it to unload massive damage in a bid to end a fight. Invisibility gives mages a chance to reposition themselves, if not flee. Note that Invisibility is not stealth, and stealth detection will not work on invisible targets.