The (unofficial) PlaystationTeam posts this list of media formats that are compatible with PlayStation 3:
Blu-Ray (ROM, -R, -RE)
DVD (ROM, -R, -RW, R, RW)
CD (ROM, -R, -RW, Super Audio)
MPEG-2 (PS, TS)
Aside from obvious disc differences, PS3 and PSP accept a near-identical list of formats (PSP is compatible with WMA files). It is disappointing that PS3 won't support some of the popular video codecs (e.g., XviD, DivX) out of the box, but if this Linux proposal comes together, then presumably, the compatibility list would grow.
Media formats compatible with PSP (for your reference):