To illustrate the functionality, Jobs used a game, Tap Tap Revenge. "Now I want to go to eBay -- I go right to where I left off in the apps," Jobs said. "Now I want to play a game, Tap Tap Revenge -- it keeps me where I was. Again, the game stops -- let me go back and look at mail." But it's not all about games, of course -- Apple also demoed music-streaming app Pandora playing in the background regardless of app (take note, Last.FM on Xbox 360!) and Skype, available to take calls in the background.
Update: And, just as some of you feared, this particular enhancement will not work on iPhone 3G or earlier. "We are releasing it for end users for iPhone and iPod Touch ... for the 3GS and iPod Touch 3rd Gen," Jobs said. "And, for iPhone 3G and iPod Touch 2nd Gen, they will run many things ... but there are some things they won't run, like multitasking." So there you have it. While your iPhone 3G or 2nd-gen Touch may run many other components of OS 4 (Game Center?), it won't be running them at the same time.
Apple iOS 4
Apple iPhone 6