If you're switching from Windows you may be familiar with an old friend, the Task Manager. Apple provides a similar tool, with an easy way to force unruly applications to quit. I'll get to why you'd want to do this in a moment. To access Force Quit, you can either go to the Apple menu in the top left corner of your screen, and choose Force Quit... Alternately, you can press three keys at once: Command + Option + Esc.* It's sort of the Mac version of Ctrl-Alt-Delete.
Why would you want to force an application to quit? Sometimes applications get "stuck" for whatever reason and can't recover. If you notice the "rainbow wheel" spinning when you try to access the application and it just won't do anything, check the Force Quit tool and see if the name of the application is red and there's a parenthetical warning "not responding." This is your cue to use Force Quit -- just be aware that sometimes an application might not be responding, but might still be "alive." Typically what I do is go grab a cup of coffee, or stretch, and give the app 2-5 minutes to get itself together. If it is still stuck, I force quit.
For more on Force Quit, check out this Apple tips article on "Knowing When to (Force) Quit." And knowing is half the battle.
*Fixed, thanks to the handy commenters!