Google has confirmed the issue to Engadget and offered a workaround. "We've been working on an option for users to require their eyes to be open to unlock the phone, which will be delivered in a software update in the coming months," the company wrote in a statement. "In the meantime, if any Pixel 4 users are concerned that someone may take their phone and try to unlock it while their eyes are closed, they can activate a security feature that requires a pin, pattern or password for the next unlock."
According to a leaked image spotted by the Verge, it seemed that an earlier version of the Pixel 4 had a setting called "require eyes to be open," much as Face ID does. However, that feature was nowhere to be found on launch day, and won't be available when the phone goes on sale on October 24th.
Other than that issue, the Pixel 4's face unlock system is secure thanks to an infrared system that creates a depth map of your face so that it can't be tricked by a photo. "Pixel 4 face unlock meets the security requirements as a strong biometric, and can be used for payments and app authentication, including banking apps," Google said. "It is resilient against invalid unlock attempts via other means, like with masks."
If you're not comfortable with the eyes-closed-only setting for now, your best bet is to enable the "lockdown" option as shown here. It can be added directly to the power menu option for quick access, and each time you enable the feature, it will force the user to enter a PIN.