Deep Future is a data intelligence company specialising in large-scale web data acquisition and analysis. We build robust, production-grade data collection systems that power real-world intelligence products across modern web platforms.
We are seeking a Python Web Scraping Engineer to design, build, and operate reliable web data collection systems. This role focuses on production-ready Python scrapers and browser automation, working with dynamic and JavaScript-heavy websites. You will contribute to the reliability, scalability, and quality of Deep Future's data pipelines.
-----------------------------------
Key Responsibilities
* Design, develop, and maintain Python-based web scrapers
* Build and maintain browser automation workflows using Playwright and Selenium
* Extract data from dynamic, JavaScript-heavy websites
* Maintain and improve scraping systems to handle routine target-site changes
* Containerise scraping services using Docker
* Deploy and operate scraping workloads on AWS
* Monitor scraper performance, reliability, and data quality
* Troubleshoot common failures and performance issues
-----------------------------------
Required Skills & Experience
* Commercial experience using Python in production environments
* Experience building and maintaining web scrapers
* Practical experience with Playwright and/or Selenium
* Experience containerising applications with Docker
* Working knowledge of AWS services (e.g. EC2, ECS, Lambda, S3, SQS)
* Understanding of:
* HTTP, cookies, headers, and basic session management
* Proxies, rate limiting, and request optimisation fundamentals
* Headless and headful browser automation
* Ability to write clear documentation and communicate effectively within a team
* Experience collaborating using tools such as Slack, Google Workspace, Jira, and Confluence
-----------------------------------
Desirable (Nice to Have)
* Exposure to sites protected by Cloudflare or similar WAFs
* Familiarity with CAPTCHA and bot-detection challenges
* Basic understanding of browser fingerprinting concepts
* Experience with higher-volume or distributed scraping systems
* Familiarity with monitoring and logging using AWS CloudWatch
-----------------------------------
What Deep Future Offers
* Ownership of meaningful parts of core data acquisition systems
* Opportunity to work on real-world data intelligence problems
* Support and mentorship from experienced engineers
* Flexible working arrangements
* Competitive remuneration aligned with experience