Firefox and Chrome headless mode not working on VPS (Ubuntu 18.04 LTS) with Python/Selenium

I am trying to get Chrome running in headless mode with Python, but for some reason I only get a blank page back no matter what website I try to read.

Google Chrome: v77

Chromedriver: latest version (tried older versions as well)

OS: Ubuntu 18.04.3 LTS (Virtual Server)

After doing some research for a couple of hours I found several similar issues and lots of different solutions, but none of those solutions helped me.

I tried the script with the same options on my local machine where it runs without any problems.

Python code:

options = Options()
options.add_argument('--remote-debugging-port=9555')
options.add_argument('--no-sandbox')
options.add_argument('--disable-dev-shm-usage')
options.add_argument('--headless')
options.add_argument('--disable-gpu')
options.add_argument('--ignore-certificate-errors')
options.add_argument('--ignore-ssl-errors')
options.add_argument('--allow-insecure-localhost')
options.add_argument('--allow-running-insecure-content')
options.add_argument('--disable-extensions')
options.add_argument('--start-maximized')
options.binary_location = "/usr/bin/google-chrome-beta"

driver = webdriver.Chrome(options=options)

driver.get("https://www.google.de")

driver.save_screenshot('picture.png')
print(driver.page_source.encode('utf-8'))

I also tried to switch to Firefox with geckodriver without any success as well.

Firefox: 68, 69, 70 (nightly)

Geckodriver: 0.24 and 0.23

(Tested with different Firefox/geckodriver version combinations)

Here the geckodriver log for it:

1566393297399   mozrunner::runner   INFO    Running command: "/usr/bin/firefox-trunk" "-marionette" "--headless" "--width=1920" "--height=1200" "--allow-insecure-localhost" "-foreground" "-no-remote" "-profile" "/tmp/rust_mozprofileFeIeHY"

 *** You are running in headless mode.

 1566393298482  addons.webextension.screenshots@mozilla.org WARN    Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid extension permission: mozillaAddons
 1566393298482  addons.webextension.screenshots@mozilla.org WARN    Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid extension permission: telemetry
 1566393298483  addons.webextension.screenshots@mozilla.org WARN    Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid extension permission: resource://pdf.js/
 1566393298483  addons.webextension.screenshots@mozilla.org WARN    Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid extension permission: about:reader*

 [Parent 2681, Gecko_IOThread] WARNING: pipe error: Datenübergabe unterbrochen (broken pipe): file /build/firefox-trunk-Wppa8r/firefox-trunk-70.0~a1~hg20190819r488674/ipc/chromium/src/chrome/common/ipc_channel_posix.cc, line 728
 [Parent 2681, Gecko_IOThread] WARNING: FileDescriptorSet destroyed with unconsumed descriptors: file /build/firefox-trunk-Wppa8r/firefox-trunk-70.0~a1~hg20190819r488674/ipc/chromium/src/chrome/common/file_descriptor_set_posix.cc, line 23
 [Parent 2681, Gecko_IOThread] WARNING: FileDescriptorSet destroyed with unconsumed descriptors: file /build/firefox-trunk-Wppa8r/firefox-trunk-70.0~a1~hg20190819r488674/ipc/chromium/src/chrome/common/file_descriptor_set_posix.cc, line 23
 [Parent 2681, Gecko_IOThread] WARNING: FileDescriptorSet destroyed with unconsumed descriptors: file /build/firefox-trunk-Wppa8r/firefox-trunk-70.0~a1~hg20190819r488674/ipc/chromium/src/chrome/common/file_descriptor_set_posix.cc, line 23
 [Parent 2681, Gecko_IOThread] WARNING: FileDescriptorSet destroyed with unconsumed descriptors: file /build/firefox-trunk-Wppa8r/firefox-trunk-70.0~a1~hg20190819r488674/ipc/chromium/src/chrome/common/file_descriptor_set_posix.cc, line 23
 [Parent 2681, Gecko_IOThread] WARNING: FileDescriptorSet destroyed with unconsumed descriptors: file /build/firefox-trunk-Wppa8r/firefox-trunk-70.0~a1~hg20190819r488674/ipc/chromium/src/chrome/common/file_descriptor_set_posix.cc, line 23
 [Parent 2681, Gecko_IOThread] WARNING: FileDescriptorSet destroyed with unconsumed descriptors: file /build/firefox-trunk-Wppa8r/firefox-trunk-70.0~a1~hg20190819r488674/ipc/chromium/src/chrome/common/file_descriptor_set_posix.cc, line 23
 [Parent 2681, Gecko_IOThread] WARNING: pipe error (119): Die Verbindung wurde vom Kommunikationspartner zurückgesetzt: file /build/firefox-trunk-Wppa8r/firefox-trunk-70.0~a1~hg20190819r488674/ipc/chromium/src/chrome/common/ipc_channel_posix.cc, line 358
 [Parent 2681, Gecko_IOThread] WARNING: pipe error (121): Die Verbindung wurde vom Kommunikationspartner zurückgesetzt: file /build/firefox-trunk-Wppa8r/firefox-trunk-70.0~a1~hg20190819r488674/ipc/chromium/src/chrome/common/ipc_channel_posix.cc, line 358
 [Parent 2681, Gecko_IOThread] WARNING: pipe error (123): Die Verbindung wurde vom Kommunikationspartner zurückgesetzt: file /build/firefox-trunk-Wppa8r/firefox-trunk-70.0~a1~hg20190819r488674/ipc/chromium/src/chrome/common/ipc_channel_posix.cc, line 358
 [Parent 2681, Gecko_IOThread] WARNING: pipe error (124): Die Verbindung wurde vom Kommunikationspartner zurückgesetzt: file /build/firefox-trunk-Wppa8r/firefox-trunk-70.0~a1~hg20190819r488674/ipc/chromium/src/chrome/common/ipc_channel_posix.cc, line 358
 [Parent 2681, Gecko_IOThread] WARNING: pipe error (130): Die Verbindung wurde vom Kommunikationspartner zurückgesetzt: file /build/firefox-trunk-Wppa8r/firefox-trunk-70.0~a1~hg20190819r488674/ipc/chromium/src/chrome/common/ipc_channel_posix.cc, line 358

 ###!!! [Parent][MessageChannel] Error: (msgtype=0x37001A,name=PContent::Msg_RegisterChromeItem) Channel error: cannot send/recv

 A content process crashed and MOZ_CRASHREPORTER_SHUTDOWN is set, shutting down

 JavaScript error: resource://activity-stream/lib/ActivityStreamPrefs.jsm, line 27: NS_ERROR_ILLEGAL_VALUE: Component returned failure code: 0x80070057 (NS_ERROR_ILLEGAL_VALUE) [nsIPrefBranch.removeObserver]
 ERROR 2019-08-21T13:15:04Z: xulstore::persist: error creating XULStore thread: NS_ERROR_OUT_OF_MEMORY
 JavaScript error: resource://gre/modules/XULStore.jsm, line 70: NS_ERROR_NOT_AVAILABLE: Component returned failure code: 0x80040111 (NS_ERROR_NOT_AVAILABLE) [nsIXULStore.setValue]
 1566393304745  addons.xpi-utils    WARN    Error: XPI database modified after shutdown began(resource://gre/modules/addons/XPIDatabase.jsm:1483:17) JS Stack trace: saveChan

 ges@XPIDatabase.jsm:1483:17
 makeAddonVisible@XPIDatabase.jsm:2285:10
 addToDatabase@XPIDatabase.jsm:2213:12
 install@XPIInstall.jsm:4344:27
 _activateAddon@XPIInstall.jsm:4371:14
 async*installBuiltinAddon@XPIInstall.jsm:4297:16
 async*XPIProvider[meth]@XPIProvider.jsm:3110:28
 installBuiltinAddon@AddonManager.jsm:2588:7
 installBuiltinAddon@AddonManager.jsm:4071:33
 ensureBuiltinExtension@SearchService.jsm:1151:28
 _loadEngines@SearchService.jsm:1110:20
 async*_init@SearchService.jsm:743:18
 async*init@SearchService.jsm:2010:18
 getDefault@SearchService.jsm:2633:16
 delayedStartupInit@browser.js:4442:21
 _delayedStartup@browser.js:2042:19
 EventListener.handleEvent*onLoad@browser.js:1898:12
 EventHandlerNonNull*@browser.xhtml:118:39
 1566393304746  addons.xpi-utils    WARN    Error: XPI database modified after shutdown began(resource://gre/modules/addons/XPIDatabase.jsm:1483:17) JS        Stack trace: saveChanges@XPIDatabase.jsm:1483:17
 addToDatabase@XPIDatabase.jsm:2216:10
 install@XPIInstall.jsm:4344:27
 _activateAddon@XPIInstall.jsm:4371:14
 async*installBuiltinAddon@XPIInstall.jsm:4297:16
 async*XPIProvider[meth]@XPIProvider.jsm:3110:28
 installBuiltinAddon@AddonManager.jsm:2588:7
 installBuiltinAddon@AddonManager.jsm:4071:33
 ensureBuiltinExtension@SearchService.jsm:1151:28
 _loadEngines@SearchService.jsm:1110:20
 async*_init@SearchService.jsm:743:18
 async*init@SearchService.jsm:2010:18
 getDefault@SearchService.jsm:2633:16
 delayedStartupInit@browser.js:4442:21
 _delayedStartup@browser.js:2042:19

 JavaScript error: resource://gre/modules/UrlClassifierListManager.jsm, line 602: NS_ERROR_NOT_AVAILABLE: Component returned failure code: 0x80040111 (NS_ERROR_NOT_AVAILABLE) [nsIUrlClassifierStreamUpdater.downloadUpdates

 1566393304895  Marionette  INFO    Listening on port 37073
 JavaScript error: resource://gre/modules/JSONFile.jsm, line 173: Error: Data is not ready.
 console.log: "RemoteSettingsWorker error: UnknownError: The operation failed for reasons unrelated to the database itself and not covered by any other error code."

 ###!!! [Parent][DispatchAsyncMessage] Error: PBackgroundIDBFactory::Msg_PBackgroundIDBFactoryRequestConstructor Value error: message was deserialized, but contained an illegal value

 JavaScript error: chrome://marionette/content/server.js, line 105: NS_ERROR_NOT_INITIALIZED: Component returned failure code: 0xc1f30001 (NS_ERROR_NOT_INITIALIZED) [nsIServerSocket.close]

Tested it on my local machine (Arch Linux) as well where it is working without any issues. Almost sure I tried everything I could find online. Any idea or recommendation (Firefox or Chrome) would be really appreciated.

I am already thinking about switching from Ubuntu to CentOS on the server, but want to give it another try first.