There are a few reasons why I think this. First and foremost, you cannot progress in PvP without first completing a large potion of the PvE content. You start out at level 1 and progress up to level 70. You don't level up by PvPing against one another. You level up by fighting against the environment. Put simply, without the PvE there would be no PvP.
Secondly, PvP is an addition to the game. If you remove PvP from the game entirely, the game itself would not fundamentally change. However if you remove the PvE elements, the game would be nothing like it is. Everything would just exist like the Arena Tournament server. That might be fine for some people, and this is evident in the success of the Arena Tournament server. Even I enjoy spending a couple hours a week on there, but by no means would I want to just exist on a server where the only thing to do is kill one another.