First off, procs are random. I know, random is the enemy of fun, right? Well, it can be, sure. A random proc that doesn't proc very often is awful. But tuned properly, a random proc can be a ton of fun. The proc on the Blackblade of Shahram, for example, is ludicrously fun, because it procs fairly often and because it is beyond a doubt a visual treat when it procs.
This is the second great thing about procs - they do stuff. A weapon with a proc can summon a big critter, or make a tentacle appear, or suddenly a huge burst of light appears. It's an effect that isn't immediately apparent in statistical terms - it's not just haste or crit to be stacked. Now, at times procs aren't very interesting, it's true. Proccing more crit or more haste is just another way to stack. But when your proc weapon summons a gigantic infernal or makes a ring of stars appear around your head, while blasting all around you with their furious light? That's pretty awesome.
This being said, I recognize that proc items can be hard to deal with, hard to simulate/model, hard to know if they're better than a dependable "adds X to Y stat" item. Procs also have other disadvantages (sometimes cool ones, but still) -- they can alter your rotation by changing up how often something happens, increase your reliance on a specific ability (a weapon that does elemental damage with a chance on hit benefits a shaman more than a warrior, for instance) -- they provide challenges both on the design level and on the player level. And of course there's the PvP issue we described above. No PvPer who was serious about PvP as a primary or even total focus of their gameplay likes having to run PvE to get a BiS item.
Heck, I only PvP'd once I had a Gurthalak back in Cataclysm, and that's a huge indictment of game balance. I loved it as a sometimes PvPer/mostly PvE player. I would have loathed it were I PvP only. My proposed solution to this is one that I have no illusions about having created. Namely, put in PvP proc items with weird PvP specific procs like damage shields. Maybe a weapon proc that summons Ivus or Lokholar from AV? There's potential here.
Ultimately, what I love about proc items is, they're not static, they're not boring if done right, they're game changers that do something cool and new. Come on, we all love having something weird and awesome happening in front of us. I'd like to see more of them.