I don't want the iPhone form factor, but I will admit that the sweet spot in terms of size was the Motorola Photon 4G. I really miss that phone. I like the Note II, but man is it hard to use with one hand. :/
I have large hands and my thumb can reach across my nexus 4 :S
That being said, The Sony Xperia Ray (now an older phone) had a nice design in a form factor that was more compact. I almost bought one of those. As Frank said, the HTC One Mini and Galaxy S4 Mini are the modern day equivalent of a reasonable sized phone. Sony continues to make smaller android phones (just don't look at the Z series)
I know. I was holding one and it literally is ludicrous. It dwarfs the Note 3 but not the Galaxy Mega
I was referring to the Xperia SP, J, and M which all seem reasonable size. Anything but the Z series haha
Sadly I think most Android OEM's are going this route now, and trying to compensate by offering three sizes. Perhaps a HTC One Mini or Galaxy S4 mini would suffice.
Why is this a sad thing? I don't want a phone bigger than my face.
Edit: I've never owned an iOS device btw and do not intend to. I've gone back to an old Sony Ericsson (non Android) after getting tired of Android flagship devices.