The contrary principle
At the heart of challenging characters is a very simple concept. You pick something that you usually do when creating a character, and then you do the opposite.
This isn't to say that can be the only thing that's different about a given character, but it's a fine place to start. We've all got habits that we tend to fall into in terms of character types. Maybe you tend to play men with a real dominant streak, maybe you tend toward seductresses, or maybe you've got a knack for quirky jokers. It doesn't matter. Try playing a character who is actively the opposite of your stock types.
Be wary of hitting the opposite button too hard, though. It's enough to simply change one aspect of your play; if you usually play aggressive men with a lot of physical strength, try playing one that's weak. Even if everything else is the same, the fact that he lacks muscle will force you to develop his character in other ways. When you notice one trait that's consistent across several characters, invert that rather than just creating a straight inversion of one person's traits.
It's also possible to do this with things that are just assumed naturally. It's assumed, for example, that your character can hear and speak, belongs to certain religions, and knows how to interact appropriately with the setting as a whole. You can invert that as well. What if your World of Warcraft character just happens to belong to a benevolent but heretical religion far from the mainstream? What if your Star Trek Online captain comes from a culture with radically different values compared to the Federation as a whole? Don't assume these are fixed points; you can move them around.
Critical existence challenge
Let's assume for the moment that you've tried the above and you can now play all sorts of characters. Your boundaries have expanded greatly, you have a greater range of available character types, and the world is your oyster. You get the idea. But now you want to move to the next level.
This is when you can start playing around with a character's self-enforced codes. Roll a character who always tells the truth but still manages to be untrustworthy and duplicitous, for example, or a paladin who lives by a code rather than following the rules of authority. Or a mercenary who will genuinely turn on anyone if the money is right.
Instead of forcing yourself to play against something familiar, you're forcing your character to face a pretty consistent internal conflict. Mercenaries are mercenary, but most human beings can't just turn on former allies without some emotional response. You want to be perceived as honest, but you still want to hold on to secrets and deception. The code you follow is ancient and outdated, but you respect the virtue and dedication so much you want it to be mirrored in your life.
A lot of fiction sets up rules like this as a sort of lingering Chekov's Code, like the man who refuses to kill right up until the plot demands that he kills someone. But it's far more interesting to see someone go to great lengths and sometimes lose out because there are rules to be followed. To see your paladin uphold his code to defend innocents who don't deserve it, your mercenary assault people he still considers friends, or your honest liar cornered into telling the whole truth even though she'd rather lie. And it helps push you further as a roleplayer to realize that an out exists that your character simply cannot take.
One of the ultimate goals with challenging characters is to help yourself find the voice of your characters, which is doable only when you're not thinking about it. As long as you're trying to hunt down someone's voice, you're always going to be missing it. Voice is an unconscious thing, a state wherein you know full well what a character would do in a given situation even if you know it's a stupid idea.
Challenging characters force you to think in ways contrary to your natural impulses. The closer your character is to yourself, the more likely you are to fall into familiar thought patterns. But characters that challenge you aren't familiar and don't conform to your patterns. You have to think about what they do and rearrange your thoughts so that it makes some kind of logical sense. You teach yourself to think like someone else.
Once you manage that, you've got your voice. And as you get more practice, it becomes easier to find a voice for a new character because you're more comfortable with the whole process. You just have to challenge yourself to get to that point.
Feedback is welcome in the comments below or via mail to email@example.com. It's time to look at fencing in your characters with next week's column, and the week after that I'll reveal to you why no one wants to RP with you.
Every Friday, Eliot Lefebvre fills a column up with excellent advice on investing money, writing award-winning novels, and being elected to public office. Then he removes all of that, and you're left with Storyboard, which focuses on roleplaying in MMOs. It won't help you get elected, but it will help you pretend you did. If you need a refresher, check out the Storyboard Library.