To listen to your voicemail through your headset, just navigate to Phone > Voicemail. Once there, click the "Audio" button in the top right -- you will be presented with some options. If you have your headset paired with your iPhone, you will see a headset option along with the iPhone and Speaker Phone options.
Clicking any one of these will immediately transfer the audio to the output specified.
Thanks for the tip, Brandon!
Want more iPhone tips and tricks like this? Visit TUAW's iPhone 101 section today!