Nathan Wailes - Blog - GitHub - LinkedIn - Patreon - Reddit - Stack Overflow - Twitter - YouTube
Using PhantomJS on PythonAnywhere
- Nathan Wailes
Errors you may encounter
- selenium.common.exceptions.WebDriverException: Message: Unable to start phantomjs with ghostdriver: [Errno 2] No such file or directory
- Solution:
- https://www.pythonanywhere.com/forums/topic/1320/
A note for anyone trying to get phantomjs running. Here is how I did it.
Let's say I am currently working in /home/conrad/my-site
First run npm install phantomjs, which will create a node_modules folder.
Then, from a python script, something like this should work:
from selenium import webdriver
driver = webdriver.PhantomJS(executable_path='/home/conrad/my-site/node_modules/phantomjs/bin/phantomjs')
- https://www.pythonanywhere.com/forums/topic/1320/
- Solution:
- PermissionError: [Errno 13] Permission denied: '/usr/lib/python3.4/site-packages'
- Solution:
- https://www.pythonanywhere.com/forums/topic/1673/
Ah! I was just playing around to see what's going on and your second post gave me the clue I needed. pip3 is not installed into the virtualenv, it's the version that is installed on PythonAnywhere, so it's trying to downgrade the Django that is installed for everyone. The pip executable that you run when you've activated a virtualenv is specific to the version of Python that you specified when you created the virtualenv (with the --python argument), so you don't need to specify pip3, because that's already built-in (assuming you created the virtualenv correctly).
- Basically, when I had created the virtualenv I hadn't specified a version and so it created it with 2.7 by default, but then I realized that my program on my local machine was assuming Python3.4, and so I tried to just run Python3.4 from my virtualenv and install packages to that version, but it turns out (according to the link above) that I was actually trying to modify the non-virtualenv-version of Python3.4, and that's what was causing the permissions error.
- https://www.pythonanywhere.com/forums/topic/1673/
- urllib.error.URLError: <urlopen error [Errno 111] Connection refused>
- Solution:
- This happened because I had just hit another error, and after that first error I tried to access a second website, but I didn't realize that the first error had canceled / ended my browser session. So you need to call 'driver = webdriver.PhantomJS' again.
- http.client.BadStatusLine: ''
- Cause:
- It is probably some kind of incompatibility between PhantomJS and whatever selenium thing they have installed.
- Sources:
- https://www.pythonanywhere.com/forums/topic/1605/
We've carefully chosen a version of Firefox (17) that works nicely with our default selenium (v 2.35) and with more recent ones (I just tested the latest selenium in a virtualenv). Later versions of Firefox had compatibility problems with our platform.
- https://www.pythonanywhere.com/forums/topic/1884/
OK let's strip it back down to a minimal repro. Create a new Python file, with the following code:
from pyvirtualdisplay import Display from selenium import webdriver import traceback browser = None try: display = Display().start() browser = webdriver.Firefox() browser.get('http://www.google.com') print browser.title except Exception as e: traceback.print_exc() finally: if browser is not None: browser.quit() display.stop()
If it works, then it must be something that's different about your specific code, and then we can try and track that down. If it doesn't work, then it must be something to do with your user account (because it works fine in mine), and we can try and track that down...
- https://www.pythonanywhere.com/forums/topic/1605/
- Sources:
- It is probably some kind of incompatibility between PhantomJS and whatever selenium thing they have installed.
- Solution:
- Basically just take the code that harry provided in that snippet above and make your code look like it.
- Install the 'pyvirtualdisplay' module.
- Insert 'display = Display().start()' at the beginning of your function and 'display.stop()' at the end.
- Use 'browser = webdriver.Firefox()' instead of PhantomJS.
- Basically just take the code that harry provided in that snippet above and make your code look like it.
- Things that didn't end up solving the issue:
- http://stackoverflow.com/questions/30767045/selenium-webdriver-http-client-badstatusline-error
Latest firefox version is 38, have you tried upgrading to any stable release of firefox?
- http://stackoverflow.com/questions/29981182/python-3-urllib-exception-http-client-badstatusline
- No answer given :/
- http://stackoverflow.com/questions/30767045/selenium-webdriver-http-client-badstatusline-error
- Cause: