Description

Key Skills: Python, Web Scraping, Scrapy, Selenium, Playwright, Anti-bot Technologies, MongoDB, Redis, Proxy Management, JavaScript

Good to Have Skills: Experience with real-device simulation for mobile scraping, building and maintaining distributed task queues (Celery, Kafka), scraping high-security platforms, mobile-first environments, Linux or cloud environments (AWS, GCP), Kubernetes, Cloud Run, Cloud Functions, structured storage formats (Parquet, Avro), real-time stream processing, Frida, mitmproxy, Charles Proxy, Burp Suite, httpx, Requests, InfluxDB.

Roles & Responsibilities:

• Design and build distributed scraping architectures that can scale across thousands of domains.
• Reverse engineer APIs and mobile apps using tools like Frida, mitmproxy, Charles Proxy, and burp suite.
• Detect common anti-bot measures like JavaScript checks or browser behavior detection to build human-like crawlers.
• Lead technical innovation in scraping by identifying edge cases, prototyping new approaches, and sharing knowledge with team.
• Set up anomaly detection and monitoring systems to detect blockers or failures early in the process.
• Handle proxies including residential, datacenter, rotating proxies along with session management, cookie injection, and TLS tweaking.
• Debug complex issues across browser, HTTP traffic, and device-level interactions to ensure optimal scraping performance.
• Collaborate closely with Customer Success, Operations, and Data teams to extract high-quality, actionable data at scale.

Experience Required: 2 to 10 years of experience with Python and large-scale scraping projects

Key Skills

Education

Any Graduate