Valve is expanding Steam's to let just about anyone play games with you, even if they don't have a Steam account. Your friend will need to install the Steam Link app but they won't have to sign up.
For now, you'll need to be running the to use Invite Anyone. You can grab an invite link from your friends list on the Steam overlay. Send it to whoever you like on Windows, iOS, Android or Raspberry Pi and they'll be able to join you as long as they have a decent internet connection. If they don't already have the Steam Link app, the link will prompt them to install it.
You can see which games in your library support Remote Play Together from the advanced filtering options. The games you can play with your buddies remotely include A Way Out, Brawlhalla, Stardew Valley, Overcooked 2, NBA 2K21 and Mortal Kombat 11.
There is one key drawback to Invite Anyone. You'll only be able to play with one person at a time through this method. If you want to group up with a bunch of friends, everyone else will need to be playing through Steam (you can right click their name on your friends list and select Remote Play Together).
The feature gives you a way to play local multiplayer games (i.e. shared-screen or split-screen titles) over the internet. Don't expect to hop into a game like Valheim with someone who doesn't have a Steam account and their own copy. Still, the Invite Anyone option should make it easier to play many games with friends and family who aren't in the same space as you.