Mac users quickly come to know the /Applications/ directory, where you will find all of the default applications that come with OS X (Safari, Mail, iChat, Preview, iCal and so on), as well as the Utilities folder (/Applications/Utilities/), where more advanced users get to know Activity Monitor, Terminal, Network Utility, Airport Utility, Spaces and more.
If you have run an installer -- such as for iLife, iWork, Microsoft Office and so on‚ more than likely that installer has added programs to the /Applications/ folder. The /Applications/ folder is also usually linked to when you mount a disk image (those files that end with .dmg), encouraging you to drag applications to the /Applications/ directory. The Applications folder is also in the sidebar of the Finder.
But some other users use a second Applications directory in their Home folder: ~/Applications/. Why keep two separate sets of Applications?
Maybe it would help to think of /Applications/ like a public library: everyone gets to use what's in there; whereas apps stored in ~/Applications/ are just for you. Some people prefer to keep their /Applications/ directory as clean as possible, and having a separate directory for your personal apps lets you do that easily.
Now, of course you don't need to store all your applications in /Applications/ or in ~/Applications/. You can keep them anywhere you want, but just like it's easier to keep all your pictures in ~/Pictures/ and all your music in ~/Music/, it's easier to keep all your applications in one (or two) places. It helps to make sure that you don't end up with two versions of the same application. Not to mention that when you need to find an app, you'll know where it is.
Finally, here's a tip that falls a little bit outside the Mac 101 purview, but that you might find useful anyway. Personally, I use /Applications only for the default applications, apps which come with installers and Dropbox. My ~/Applications/ folder is actually a link to ~/Dropbox/Applications/ where I keep all my other applications, which don't require installers. That way I have all of my apps on all of my folders, and when an application needs to be updated, I only have to update it once. I've been doing this for more than six months and have not had any problems. That said, if you decide to do that, do so at your own risk. When you are done working on one computer, quit all apps, logout or shut down the computer to minimize the chance of running the same app at the same time on different computers.
The Applications folder is a powerful element of the OS X interface -- there's a lot of benefit and possibility in keeping all of the Applications on your Mac in just one or two places.
Below: the ~/Applications/ directory, which is in the Home directory for your user account.
And here, below, we see the /Applications/ directory, nestled just inside your Macintosh HD itself, or at the "top" of the hierarchy. These are applications available to all users.