I am trying to get Chrome running in headless mode with Python, but for some reason I only get a blank page back no matter what website I try to read.
Google Chrome: v77
Chromedriver: latest version (tried older versions as well)
OS: Ubuntu 18.04.3 LTS (Virtual Server)
After doing some research for a couple of hours I found several similar issues and lots of different solutions, but none of those solutions helped me.
I tried the script with the same options on my local machine where it runs without any problems.
Python code:
options = Options()
options.add_argument('--remote-debugging-port=9555')
options.add_argument('--no-sandbox')
options.add_argument('--disable-dev-shm-usage')
options.add_argument('--headless')
options.add_argument('--disable-gpu')
options.add_argument('--ignore-certificate-errors')
options.add_argument('--ignore-ssl-errors')
options.add_argument('--allow-insecure-localhost')
options.add_argument('--allow-running-insecure-content')
options.add_argument('--disable-extensions')
options.add_argument('--start-maximized')
options.binary_location = "/usr/bin/google-chrome-beta"
driver = webdriver.Chrome(options=options)
driver.get("https://www.google.de")
driver.save_screenshot('picture.png')
print(driver.page_source.encode('utf-8'))
I also tried to switch to Firefox with geckodriver without any success as well.
Firefox: 68, 69, 70 (nightly)
Geckodriver: 0.24 and 0.23
(Tested with different Firefox/geckodriver version combinations)
Here the geckodriver log for it:
1566393297399 mozrunner::runner INFO Running command: "/usr/bin/firefox-trunk" "-marionette" "--headless" "--width=1920" "--height=1200" "--allow-insecure-localhost" "-foreground" "-no-remote" "-profile" "/tmp/rust_mozprofileFeIeHY"
*** You are running in headless mode.
1566393298482 addons.webextension.screenshots@mozilla.org WARN Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid extension permission: mozillaAddons
1566393298482 addons.webextension.screenshots@mozilla.org WARN Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid extension permission: telemetry
1566393298483 addons.webextension.screenshots@mozilla.org WARN Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid extension permission: resource://pdf.js/
1566393298483 addons.webextension.screenshots@mozilla.org WARN Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid extension permission: about:reader*
[Parent 2681, Gecko_IOThread] WARNING: pipe error: Datenübergabe unterbrochen (broken pipe): file /build/firefox-trunk-Wppa8r/firefox-trunk-70.0~a1~hg20190819r488674/ipc/chromium/src/chrome/common/ipc_channel_posix.cc, line 728
[Parent 2681, Gecko_IOThread] WARNING: FileDescriptorSet destroyed with unconsumed descriptors: file /build/firefox-trunk-Wppa8r/firefox-trunk-70.0~a1~hg20190819r488674/ipc/chromium/src/chrome/common/file_descriptor_set_posix.cc, line 23
[Parent 2681, Gecko_IOThread] WARNING: FileDescriptorSet destroyed with unconsumed descriptors: file /build/firefox-trunk-Wppa8r/firefox-trunk-70.0~a1~hg20190819r488674/ipc/chromium/src/chrome/common/file_descriptor_set_posix.cc, line 23
[Parent 2681, Gecko_IOThread] WARNING: FileDescriptorSet destroyed with unconsumed descriptors: file /build/firefox-trunk-Wppa8r/firefox-trunk-70.0~a1~hg20190819r488674/ipc/chromium/src/chrome/common/file_descriptor_set_posix.cc, line 23
[Parent 2681, Gecko_IOThread] WARNING: FileDescriptorSet destroyed with unconsumed descriptors: file /build/firefox-trunk-Wppa8r/firefox-trunk-70.0~a1~hg20190819r488674/ipc/chromium/src/chrome/common/file_descriptor_set_posix.cc, line 23
[Parent 2681, Gecko_IOThread] WARNING: FileDescriptorSet destroyed with unconsumed descriptors: file /build/firefox-trunk-Wppa8r/firefox-trunk-70.0~a1~hg20190819r488674/ipc/chromium/src/chrome/common/file_descriptor_set_posix.cc, line 23
[Parent 2681, Gecko_IOThread] WARNING: FileDescriptorSet destroyed with unconsumed descriptors: file /build/firefox-trunk-Wppa8r/firefox-trunk-70.0~a1~hg20190819r488674/ipc/chromium/src/chrome/common/file_descriptor_set_posix.cc, line 23
[Parent 2681, Gecko_IOThread] WARNING: pipe error (119): Die Verbindung wurde vom Kommunikationspartner zurückgesetzt: file /build/firefox-trunk-Wppa8r/firefox-trunk-70.0~a1~hg20190819r488674/ipc/chromium/src/chrome/common/ipc_channel_posix.cc, line 358
[Parent 2681, Gecko_IOThread] WARNING: pipe error (121): Die Verbindung wurde vom Kommunikationspartner zurückgesetzt: file /build/firefox-trunk-Wppa8r/firefox-trunk-70.0~a1~hg20190819r488674/ipc/chromium/src/chrome/common/ipc_channel_posix.cc, line 358
[Parent 2681, Gecko_IOThread] WARNING: pipe error (123): Die Verbindung wurde vom Kommunikationspartner zurückgesetzt: file /build/firefox-trunk-Wppa8r/firefox-trunk-70.0~a1~hg20190819r488674/ipc/chromium/src/chrome/common/ipc_channel_posix.cc, line 358
[Parent 2681, Gecko_IOThread] WARNING: pipe error (124): Die Verbindung wurde vom Kommunikationspartner zurückgesetzt: file /build/firefox-trunk-Wppa8r/firefox-trunk-70.0~a1~hg20190819r488674/ipc/chromium/src/chrome/common/ipc_channel_posix.cc, line 358
[Parent 2681, Gecko_IOThread] WARNING: pipe error (130): Die Verbindung wurde vom Kommunikationspartner zurückgesetzt: file /build/firefox-trunk-Wppa8r/firefox-trunk-70.0~a1~hg20190819r488674/ipc/chromium/src/chrome/common/ipc_channel_posix.cc, line 358
###!!! [Parent][MessageChannel] Error: (msgtype=0x37001A,name=PContent::Msg_RegisterChromeItem) Channel error: cannot send/recv
A content process crashed and MOZ_CRASHREPORTER_SHUTDOWN is set, shutting down
JavaScript error: resource://activity-stream/lib/ActivityStreamPrefs.jsm, line 27: NS_ERROR_ILLEGAL_VALUE: Component returned failure code: 0x80070057 (NS_ERROR_ILLEGAL_VALUE) [nsIPrefBranch.removeObserver]
ERROR 2019-08-21T13:15:04Z: xulstore::persist: error creating XULStore thread: NS_ERROR_OUT_OF_MEMORY
JavaScript error: resource://gre/modules/XULStore.jsm, line 70: NS_ERROR_NOT_AVAILABLE: Component returned failure code: 0x80040111 (NS_ERROR_NOT_AVAILABLE) [nsIXULStore.setValue]
1566393304745 addons.xpi-utils WARN Error: XPI database modified after shutdown began(resource://gre/modules/addons/XPIDatabase.jsm:1483:17) JS Stack trace: saveChan
ges@XPIDatabase.jsm:1483:17
makeAddonVisible@XPIDatabase.jsm:2285:10
addToDatabase@XPIDatabase.jsm:2213:12
install@XPIInstall.jsm:4344:27
_activateAddon@XPIInstall.jsm:4371:14
async*installBuiltinAddon@XPIInstall.jsm:4297:16
async*XPIProvider[meth]@XPIProvider.jsm:3110:28
installBuiltinAddon@AddonManager.jsm:2588:7
installBuiltinAddon@AddonManager.jsm:4071:33
ensureBuiltinExtension@SearchService.jsm:1151:28
_loadEngines@SearchService.jsm:1110:20
async*_init@SearchService.jsm:743:18
async*init@SearchService.jsm:2010:18
getDefault@SearchService.jsm:2633:16
delayedStartupInit@browser.js:4442:21
_delayedStartup@browser.js:2042:19
EventListener.handleEvent*onLoad@browser.js:1898:12
EventHandlerNonNull*@browser.xhtml:118:39
1566393304746 addons.xpi-utils WARN Error: XPI database modified after shutdown began(resource://gre/modules/addons/XPIDatabase.jsm:1483:17) JS Stack trace: saveChanges@XPIDatabase.jsm:1483:17
addToDatabase@XPIDatabase.jsm:2216:10
install@XPIInstall.jsm:4344:27
_activateAddon@XPIInstall.jsm:4371:14
async*installBuiltinAddon@XPIInstall.jsm:4297:16
async*XPIProvider[meth]@XPIProvider.jsm:3110:28
installBuiltinAddon@AddonManager.jsm:2588:7
installBuiltinAddon@AddonManager.jsm:4071:33
ensureBuiltinExtension@SearchService.jsm:1151:28
_loadEngines@SearchService.jsm:1110:20
async*_init@SearchService.jsm:743:18
async*init@SearchService.jsm:2010:18
getDefault@SearchService.jsm:2633:16
delayedStartupInit@browser.js:4442:21
_delayedStartup@browser.js:2042:19
JavaScript error: resource://gre/modules/UrlClassifierListManager.jsm, line 602: NS_ERROR_NOT_AVAILABLE: Component returned failure code: 0x80040111 (NS_ERROR_NOT_AVAILABLE) [nsIUrlClassifierStreamUpdater.downloadUpdates
1566393304895 Marionette INFO Listening on port 37073
JavaScript error: resource://gre/modules/JSONFile.jsm, line 173: Error: Data is not ready.
console.log: "RemoteSettingsWorker error: UnknownError: The operation failed for reasons unrelated to the database itself and not covered by any other error code."
###!!! [Parent][DispatchAsyncMessage] Error: PBackgroundIDBFactory::Msg_PBackgroundIDBFactoryRequestConstructor Value error: message was deserialized, but contained an illegal value
JavaScript error: chrome://marionette/content/server.js, line 105: NS_ERROR_NOT_INITIALIZED: Component returned failure code: 0xc1f30001 (NS_ERROR_NOT_INITIALIZED) [nsIServerSocket.close]
Tested it on my local machine (Arch Linux) as well where it is working without any issues. Almost sure I tried everything I could find online. Any idea or recommendation (Firefox or Chrome) would be really appreciated.
I am already thinking about switching from Ubuntu to CentOS on the server, but want to give it another try first.
User contributions licensed under CC BY-SA 3.0