Let's start with the elephant in the room: Many of the coolest accessories are made for GoPro cameras, and they have a proprietary connector. So, the first thing you'll want if you have a different brand of camera is an adapter like this one, available in stores, or on eBay for a few dollars. Conversely, if you have a GoPro, pick up the tripod mount, and hey presto, (nearly) all the accessories work on all the cameras.
As I suggested in the introduction, the second basic item is a monopod. Or, if I must use the word, a "selfie stick." Unlike the cheap-looking ones people insist on using with phones, your action camera will need something a little more robust. For GoPros, I highly recommend the company's own "3-Way" ($70). It articulates into a number of positions, and comes with a mini tripod inside the handle. It can also serve as a regular handgrip. If you just want a simple, durable stick, the Rhino Pole ($30) is one of our favorites. Made from carbon fiber, this 20-inch pole is light and strong. For an extra $20, you can get the extending "EXT" version, which lets you get a wider shot (i.e., more of the scenery), or reach places a shorter pole cannot. Both versions have a regular tripod (1/4-20) connection in the base of the handle, which means you can attach it to other accessories for even more creative options.
If you plan on using your action camera all day (or at least would like to), consider the PowerPole from Polar Pro ($100). The aluminum pole is extendable between 17 and 30 inches, and at 13 ounces, it's built like a tank. That's not the talking point, though; as the name suggests, the PowerPole has a 5,200mAh battery inside, and two USB ports near the tip. With this pole, you can charge your camera as you use it, for an additional 10 hours use, according to the company. For my part, I never did fully exhaust it; let's put it that way. If your activity needs the camera in waterproof housing, you won't be able to charge while you film (you can still use the pole, as the USB ports have rubber plug/covers to protect them), but you've still got a great monopod with a battery you can use between takes.
If you prefer to be behind the camera, Polar Pro has something for you, too -- a phone case with a GoPro connector ($30 -- available for iPhone 5/6 and Galaxy S5). At first, this might seem a little odd, but it makes a lot of sense. The case turns your phone into a rudimentary grip mount, but it makes even more sense when you use it with your camera's companion app. Now, you've got a grip that doubles as a real-time viewfinder. Great for easy access to camera settings, too, and no need to spend out on a dedicated LCD screen accessory. Simple.
We can't explore everything here (there really are some niche accessories), but there are some mounts that lend themselves to a wide variety of uses. The Capture POV by Peak Design ($70) is one example. The belt clip mounts a camera on anything you can fit between its jaws (most likely belts or backpack straps). In many ways, it's like a more-versatile "Chesty" mount. The metal finish gives it a high-quality, rugged feel, and an adjustable pin lets you pivot the camera on one axis. There's a quick-release button, too, if you need to pop the camera out to change settings on the go, or take a shot from a different angle.
Water sports deserve a special mention, given that waterproofing is a key selling point for many action camera owners. One of our favorites is Octomask ($80). The idea is simple: Make a diving mask, and add a GoPro mount for instant hands-free POV filming. The silicone skirt on the goggles is soft -- great for comfort -- and the choice of three goggle styles means there's one for your preference. The tempered-glass lenses are strong and won't scratch up like a cheap seaside pair -- you can even get prescription lenses in them if you want. If you're really brave, you could wear these for POV video on dry land, too. No judgment here.
If you've been adding the above items to your cart as you go, you're going to need to keep everything organized, and that means a case. There are many, but we found SP Gadgets' POV Case (starting at about $20) to be a good choice for the budding accessory collection. It's available in three sizes, with the largest one ($50) having enough space for two cameras plus batteries, remote controls, grips, cables and more. We also like that it's available in (almost, if you squint) Engadget blue.
The next level