Category: selenium

selenium python tutorial w3schools

Selenium is a web automation framework. It starts a web browser and any task that can be done typically on the web, Selenium+Python can do for you.

Related course

Browser Automation with Python Selenium

Articles

Note
To use Selenium you need either Python version:


Python 2.6, 2.7
Python 3.3+

To check your version use:

python --version

Installation of a web driver is mandatory, without installing one the browser wont start even though one is installed on your computer.

pip install selenium

Selenium is a web automation tool. A web browser can be controlled using Python code, any task you would normally do on the web can be done using the selenium module.

To use use selenium, you need both the selenium module and the web driver installed. This can be quite tricky to get right, if you are new to selenium I recommend the course below.

Install selenium

To get started, first you should setup a virtual environment. Once that’s setup and activated, you want to install the selenium module inside it. You can do that by typing the command:


pip install -U selenium

This will install the selenium module, but that’s not all yet. You need to install the driver.

Then get the web driver from https://docs.seleniumhq.org/projects/webdriver. There are all kind of webdrivers including:

ChromeDriver
FirefoxDriver
RemoteWebDriver
EdgeDriver
IEDriver
SafariDriver
OperaDriver

Selenium Example

Depending on which driver you install, you can load a different browser. If you use Chrome, you could do this:

from selenium.webdriver import Chrome
driver = Chrome()

For the Firefox driver, initialize like this:

from selenium.webdriver import Firefox
driver = Firefox()

This also works for Edge

from selenium.webdriver import Edge
driver = Edge()

After installation of the web driver, we can make Python start the browser using the code below:


from selenium import webdriver
import time

options = webdriver.ChromeOptions()
options.add_argument('--ignore-certificate-errors')
options.add_argument("--test-type")
options.binary_location = "/usr/bin/chromium"
driver = webdriver.Chrome(chrome_options=options)
driver.get('https://python.org')

Save the program as example.py and run it with

python3 example.py

If everything went right, it will start the Chromium browser and open the python site.

Selenium will start the chromium browser automatically

Download Selenium Examples

Given a webpage that features a text area or text field, it’s possible to automate the process of entering text using Python. Through Python, it’s feasible to initiate a web browser, navigate to the desired page, and input text. This level of automation can be achieved using the selenium framework.

Related Course: Browser Automation with Python Selenium

Setting Up Selenium

In this segment, we will showcase this automation using a succinct code example.

To begin, it’s essential to import the selenium module. Note that this module isn’t part of Python’s standard library. Therefore, if it isn’t already present in your environment, you can easily install it using Python’s package manager, pip.

1	pip install selenium

Along with selenium, it’s crucial to have the appropriate driver and web browser installed.

Implementing Text Input with Selenium

Let’s dive into the code. The initial step involves utilizing the webdriver to initiate the web browser. After that, the target website can be accessed using the method provided below:

1	driver.get(url)

Here, the url parameter specifies the web address of the target site.

To interact with an HTML element on the web page, one can use the find_element_by_id method. However, ensure that the HTML element in question possesses an ID. If it doesn’t, you might have to resort to using xpath.

1	driver.find_element_by_id(id)

Once the desired element is selected, text can be automatically entered into it using the send_keys method. This method accepts a string as its parameter:

1	send_keys(str)

The comprehensive code example is provided below:

from selenium import webdriver
import time

options = webdriver.ChromeOptions()
options.add_argument('--ignore-certificate-errors')
options.add_argument("--test-type")
options.binary_location = "/usr/bin/chromium"
driver = webdriver.Chrome(chrome_options=options)
driver.get('http://codepad.org')

text_area = driver.find_element_by_id('textarea')
text_area.send_keys("This text is automated input via Python code.")

Upon execution, the browser will autonomously launch and begin inputting the specified text into the textbox.

Selenium textbox automation

Download Selenium Code Examples

selenium get link

selenium click button

Selenium can automatically click on buttons that appear on a webpage. In this example we will open a site and click on a radio button and submit button.

Related course
Browser Automation with Python Selenium

Selenium button click
Start by importing the selenium module and creating a web driver object. We then use the method:


drivers.find_elements_by_xpath(path)

to find the html element. To get the path, we can use chrome development tools (press F12). We take the pointer in devtools and select the html button we are interested in. The path will then be shown, as the example screenshot:

Find element by xpath, using chrome dev tools
After we have the html object, we use the click() method to make the final click.
Full code:


from selenium import webdriver
import time

options = webdriver.ChromeOptions()
options.add_argument('--ignore-certificate-errors')
options.add_argument("--test-type")
options.binary_location = "/usr/bin/chromium"
driver = webdriver.Chrome(chrome_options=options)
driver.get('http://codepad.org')

# click radio button
python_button = driver.find_elements_by_xpath("//input[@name='lang' and @value='Python']")[0]
python_button.click()

# type text
text_area = driver.find_element_by_id('textarea')
text_area.send_keys("print('Hello World')")

# click submit button
submit_button = driver.find_elements_by_xpath('//*[@id="editor"]/table/tbody/tr[3]/td/table/tbody/tr/td/div/table/tbody/tr/td[3]/input')[0]
submit_button.click()

Download Selenium Examples

phantomjs python

PhantomJS is a headless browser that can be used with the Selenium web automation module. Unlike the FirefoxDriver or ChromeDriver, the browser stays totally invisible during the process.

It behaves just like the other browsers. To design the process, you can change the webdriver to ChromeDriver or FirefoxDriver and change it to PhantomJS once its working.

Related course
Browser Automation with Python Selenium

Install PhantomJS

Like the other web drivers, the PhantomJS driver needs to be installed.
On Apple Mac you can install PhantomJS with the line:

brew install phantomjs

On Microsoft Windows or Linux, get PhantomJS from the official site.

Selenium PhantomJS

To use the PhantomJS webdriver, all you need to do is change it to PhantomJS().
Example PhantomJS code:

from selenium import webdriver
import time

driver = webdriver.PhantomJS()
driver.get('https://python.org')

html = driver.page_source
print(html)

After importing the selenium module, we create the webdriver object using


driver = webdriver.PhantomJS()

Then will This will work with both Python 2.7 and Python 3.

If you are on a Windows machine you can specify the path to the phantomjs executable:


driver = webdriver.PhantomJS("C://phantomjs.exe")
driver.get("https://python.org/")

On Windows, the path should be changed to the location of your phantomjs installation.

On Mac or Linux, simply create the driver using:


driver = webdriver.PhantomJS()
driver.get("https://python.org/")

Download Selenium Examples

selenium screenshot python

selenium download image

Selenium,a web automation framework, can be used to get the all of the image links from a webpage.In this article we’ll given an example of that.

Related course
Browser Automation with Python Selenium

Get image links
At first we import the selenium module and start the web driver object.


driver = webdriver.Chrome(chrome_options=options)

The webpage consists of html code, defined by tags. To show an image a webpage has a code:


<img src=".../image.jpg">

Then we find all the elements using the img tag (from html):


images = driver.find_elements_by_tag_name('img')

Finally we print the link of each image using:


for image in images:
    print(image.get_attribute('src'))

Full code:


from selenium import webdriver

options = webdriver.ChromeOptions()
options.add_argument('--ignore-certificate-errors')
options.add_argument("--test-type")
options.binary_location = "/usr/bin/chromium"
driver = webdriver.Chrome(chrome_options=options)

driver.get('https://imgur.com/')

images = driver.find_elements_by_tag_name('img')
for image in images:
    print(image.get_attribute('src'))

driver.close()

That will return all the image urls on the webpage.
To download import urllib and use the line:


urllib.urlretrieve(src, "filename.png")

You could also use wget, by using os.system();

Download Selenium Examples

Category: selenium

selenium python tutorial w3schools

pip install selenium

Install selenium

Selenium Example

selenium python

python selenium get html

selenium input text python

Setting Up Selenium

Implementing Text Input with Selenium

selenium get link

selenium click button

phantomjs python

Install PhantomJS

Selenium PhantomJS

selenium screenshot python

selenium download image