Resetting custom mode on VirtualBox

I tried getting a custom resolution in one of my vms and now I get this error:

Failed to open a session for the virtual machine Windows.

One of the custom modes was incorrect. The format or bit count of the custom mode value is invalid. (VERR_VGA_INVALID_CUSTOM_MODE).

Result Code: E_FAIL (0x80004005) Component: ConsoleWrap Interface: IConsole {872da645-4a9b-1727-bee2-5585105b9eed}

Could I fix this without any data loss?

(I'm running on Windows 10, and the vm is a Windows 7 one)