Relmstein tries to make a spectrum between professional production and a strong script, but I'm not sure those are on opposite ends-- it's definitely possible to have both. I'd say a good, strong script is important for sure no matter how your movie looks. Humor tends to be popular, but I think that's only because non-humor stuff tends to be overly dramatic RP stuff so far-- it's hard to convey real emotion, as my friend recently said when I showed her the Code Monkey video, with characters who don't change facial expressions.
And maybe this is just me, but I kind of have a pet peeve with some machinima-- I really tend to dislike overuse of the Model Viewer to create improbable situations. It has it's place, I think-- Hardware Store is a great video that uses the Model Viewer almost exclusively for all sorts of fun. But I am more impressed by the folks who are able to shoot great stuff inside the actual game engine rather than flooding the screen with their own creations, effects, and graphics. Yeah, it takes skill to come up with all your own stuff, but if I'm watching a WoW video, I'd rather it was made with WoW, not five or six other programs just using the WoW models.
But maybe that's just my own personal opinion. What do you think makes a great machinima video?