That fancy iPhone 3G won't feel much speedier than your first gen iPhone if you aren't in an area that is covered by AT&T's high-speed 3G network. Luckily, AT&T has a list of cities (yes, if you're not in a metropolitan area chances are you'll be using EDGE more often than not) that have 3G coverage.
AT&T doesn't offer up a map of the US with their 3G coverage highlighted, so I took the liberty of using their 'Coverage Viewer' and my copy of Acorn to whip up a full US map with AT&T's 3G coverage in deep blue. Check it out here (the map isn't perfect, I know, but it does give you a good idea of the extent of AT&T's 3G network).
Update: I've added another map that only shows AT&T's data coverage. 3G is the deepest blue, while EDGE is slightly lighter (and more widespread).