How it works...

Step 1 in the How to do it… section shows how to create a Selenium page and go to a particular URL.

Selenium works in a similar way to Beautiful Soup. Select the adequate element, and then manipulate it. The selectors in Selenium work in a similar way to those in Beautiful Soup, with the most common ones being find_element_by_id, find_element_by_class_name, find_element_by_name, find_element_by_tag_name, and find_element_by_css_selector. There are equivalent find_elements_by_X  that return a list instead of the first found element ( find_elements_by_tag_name, find_elements_by_name, and more). This is also useful when checking whether the element is there or not. If there's no elements, find_element will raise an error while find_elements will return an empty list.

The data on the elements can be obtained through .get_attribute() for HTML attributes (such as the values on the form elements) or .text.

The elements can be manipulated by simulating sending keystrokes to input text, with the method .send_keys(), clicked with .click() or submitted with .submit() if they accept that. .submit() will search on a form for the proper submission, and .click() will select/deselect in the same way that a click of the mouse will do.

Finally, step 6 closes the browser.