Description

Design and build distributed scraping architectures that scale across thousands of domains. You will own the full lifecycle of data extraction, from reverse engineering APIs and mobile apps to managing proxy infrastructure and ensuring high availability.

This role is on-site.

Responsibilities

• Reverse engineer APIs and mobile applications using Frida, mitmproxy, Charles Proxy, and Burp Suite.
• Build human-like crawlers that bypass anti-bot measures, including JavaScript checks and browser fingerprinting.
• Manage proxy infrastructure, including residential, datacenter, and rotating proxies, while handling session management, cookie injection, and TLS tweaking.
• Set up anomaly detection and monitoring systems to identify blockers or failures early.
• Debug complex issues across browser, HTTP traffic, and device-level interactions to maintain optimal scraping performance.

Required Skills

• 5+ years of experience with Python and large-scale scraping projects.
• Strong proficiency in Scrapy, Selenium, and Playwright.
• Experience with NoSQL databases, specifically MongoDB and Redis.
• Deep understanding of Anti-bot Technologies and how to circumvent them.
• Competent in JavaScript for browser-based scraping tasks.
• Experience with Proxy Management strategies and implementation.

Preferred Skills

• Experience with real-device simulation for mobile scraping.
• Knowledge of distributed task queues (Celery, Kafka) and cloud environments (AWS, GCP, Kubernetes).

Key Skills

Education

Any Graduate