If you're a YouTube Premium or Music Premium subscriber, switching between a song and its music video should be smoother as of today. You can hop back and forth between the two, and playback will continue from the same point with no pauses or interruptions, YouTube says.
When you listen to a song in YouTube Music, the app will display an icon at the top of the screen if there's a music video for it, and tapping that will take you to said video. The time-matching trick will work for more than five million official music videos out of the gate.
If you're not interested in watching, you can turn off the option to play music videos entirely from the settings. As for all those music videos with annoyingly long interruptions or pauses, switching from those to a song means you'll be able to listen to a tune as it was intended.