organization. We are looking for a senior software engineer with the right
experience to take full ownership of all the web scraping data missions.
In this role, you will have the sole authority regarding web scraping. This is
the only function in the company – you will lead the strategic decisions,
build your team, work with the project manager to develop new crawlers,
maintenance and take full ownership for the scraping product end to end.
Web scraping TL duties and Responsibilities:
* Develop new web crawlers
* Provide technical support, deploy updates and fixes for running crawlers
* Develop and maintain the full data flow from the crawlers to our data cloud.
* Design and implement build, deployment, and configuration management
* Investigate and resolve technical issues
* Design procedures for system troubleshooting and maintenance
* Maintain day-to-day management and administration of projects
*
Requirements and Qualifications:
* Outstanding personal & communication skills
* At least 2 years experience of working with Python – must
* Experience with scarpy (python package) – very big advantage
* Experience with ScrapingHub platform (advantage)
* Experience with overcoming antibots while scraping – (advantage)
* Deep knowledge with HTML,CSS and XPATH – very big advantage
* 1-2 years of experience as a SW team lead
* Experience and deep understanding of web page scraping concepts and methods
* Experience with AWS cloud environment
