Until now, the only way to access 4K gaming was by using a Chromecast Ultra with a television. If it's working on the web, you should see a noticeable bump in quality. You can confirm it by opening the in-game menu (hit shift-tab or use the Stadia button on your controller), where you should see a green 4K icon under "data usage."
4K streaming has been a thorny issue for Stadia. As Engadget noted last month, a key part of Google's pitch was that Stadia could hit 4K at 60 fps. However, that resolution isn't supported on key games like Red Dead Redemption 2 and Destiny 2, something that Google didn't make clear. Instead, players had to figure out which titles supported 4K for themselves. Also, games that do support 4K appear to be running at a lower resolution than native 4K games on a PC.
Google recently boosted the number of Stadia games by five and added key features like screenshot and clip downloading. Several key features are still missing, though, including support for wireless Stadia controllers on the web and expanded Assistant support. Google still hasn't made 4K Stadia play on the web official, but the fact that a lot of players are seeing the feature means an announcement could be imminent.