Selenium is a web automation module that can be used to get a webpages html code. In this article we will show how to achieve that.

You may need to set the path to chromium:

export PATH=$PATH:/usr/lib/chromium/

Related course
Practice Python with interactive exercises

Get html source We import webdriver from the selenium module. A webdriver object is created (chromium) and we can optionally specify if we want to ignore certificate errors. Once the web browser started we navigate it to a webpage URL using the get() module. Then we get the page source.

from selenium import webdriver
import time
options = webdriver.ChromeOptions()
options.add_argument('--ignore-certificate-errors')
options.add_argument("--test-type")
options.binary_location = "/usr/bin/chromium"
driver = webdriver.Chrome(chrome_options=options)
driver.get('https://python.org')
html = driver.page_source
print(html)

It will output the webpage source, which is stored in the variable html.

Selenium will start the chromium browser automatically