ABOUT THE PROJECT

In this project I created a custom python script that utilized the Selenium library to automate a web browsers interactions and scrape all job listings using XPATH to parse specific DOM elements. It used a Chrome Driver to provide a platform to launch and perform tasks in and ran off a Ubuntu server that used Virtualenv to create an isolated python environment to avoid any possible conflicts. The script would automatically paginate through multiple pages and build a full dataset of all available job postings. After this, it used a MySQL connector to insert all content into a database. In order to retrieve this data, I designed a RESTful API utilizing the Flask Micro Web Framework. This allowed me to create reusable endpoints to grab all the content and even allowed for ordering and filtering of all the elements.

FEATURES

• Flask Micro Web Framework for full RESTful API endpoints
• Selenium Browser Automation Library for data mining
• Chrome Driver to emulate a real browser
• XPATH to target specfic DOM elements
• MySQL connector to perform SQL queries on the database
• Virtualenv to avoid any possible conflicts with native python

MADE FOR

Web Team