Personally, I know I won't be throwing away my Abyss Shard come the expansion. For those non-warlocks out there, the Abyss Shard allows you to cast your summon voidwalker spell without the need for a soulshard. It doesn't sound that useful, but I still find myself occasionally out of shards after a raid, and the voidwalker summon comes in handy. It isn't the most uber trinket, but it is certainly worth keeping around. On my priest, I will be certainly be taking Mike's advice and keep my Hydrocane.
What about you guys? What will you never give up? Is it a trinket as well? Or maybe it is that hard-earned epic mount that regardless of what kind of neat rides await us across the Dark Portal, will never leave your bank?