Most people who buy Google smart speakers use them for direct streaming, but that hasn't been entirely voluntary for the past year. Ever since late 2018, people have complained that any Bluetooth connections (from phones or to other speakers) would drop after a few minutes or less. Not much of a feature, is it? However, relief appears to be in sight. Google staff have confirmed that they're "investigating" a fix. There's no mention of when it might be ready, but it's a welcome relief after many months of silence.
There is a workaround if you're piping audio from your phone. Android Police noted that you can use the Google Home app to cast your phone's audio over WiFi (through the speaker button). You'll need an active internet connection, but it appears to be much more stable. You're stuck if you use Bluetooth to send audio to other speakers, though.