Although I'm not a lawyer, I was able to find some information courtesy of the ACLU and the EFF. (Note that this information pertains to US citizens specifically.)
If you're at the border, police are permitted to search your computer and portable devices, regardless of whether they have a warrant. Other times law enforcement doesn't need a warrant to search your phone: If you consent to a search, if they've asked a roommate or spouse for access to the device, or if you've just been arrested. If you're arrested, police are permitted to search your phone under limited circumstances but you're not required to turn over encryption keys or passwords.
The Supreme Court has ruled that police cannot search data on a cell phone unless they believe there is evidence on the phone that is likely to be destroyed. However, they can remove a case or battery. The Fifth Amendment, which protects citizens from being forced to give self-incriminating testimony, has generally been taken to extend to passwords or encryption keys. Law enforcement cannot threaten or force you into giving up a password or unlocking a phone or computer. (However, a judge or a grand jury may be able to compel you unlock devices.) The EFF recommends contacting them and seeking legal help if you find yourself in circumstances where law enforcement or someone in the justice system is pressing you to provide a password or encryption keys.
According to the ACLU, if an officer has asked for your phone while you're recording video or taking photographs or a protest or police activity, you are within your rights to refuse to comply, as the request is unlawful. Police officers "may not confiscate or demand to view your photographs or video without a warrant," and they are never permitted to delete any photographs or video off of your phone. Taking photographs and video in public is covered under your First Amendment rights, but there's still a chance you could be (unlawfully) arrested if you refuse to comply with the request.