Maybe it would help to think of /Applications/ like a public library: everyone gets to use what's in there; whereas apps stored in ~/Applications/ are just for you. Some people prefer to keep their /Applications/ directory as clean as possible, and having a separate directory for your personal apps lets you do that easily.
Now, of course you don't need to store all your applications in /Applications/ or in ~/Applications/. You can keep them anywhere you want, but just like it's easier to keep all your pictures in ~/Pictures/ and all your music in ~/Music/, it's easier to keep all your applications in one (or two) places. It helps to make sure that you don't end up with two versions of the same application. Not to mention that when you need to find an app, you'll know where it is.
Finally, here's a tip that falls a little bit outside the Mac 101 purview, but that you might find useful anyway. Personally, I use /Applications only for the default applications, apps which come with installers and Dropbox. My ~/Applications/ folder is actually a link to ~/Dropbox/Applications/ where I keep all my other applications, which don't require installers. That way I have all of my apps on all of my folders, and when an application needs to be updated, I only have to update it once. I've been doing this for more than six months and have not had any problems. That said, if you decide to do that, do so at your own risk. When you are done working on one computer, quit all apps, logout or shut down the computer to minimize the chance of running the same app at the same time on different computers.
The Applications folder is a powerful element of the OS X interface -- there's a lot of benefit and possibility in keeping all of the Applications on your Mac in just one or two places.
Below: the ~/Applications/ directory, which is in the Home directory for your user account.
And here, below, we see the /Applications/ directory, nestled just inside your Macintosh HD itself, or at the "top" of the hierarchy. These are applications available to all users.