Description

You will design and implement large-scale data extraction systems and automated scraping workflows.

Responsibilities

• Build and maintain web scrapers using Python and modern extraction frameworks.
• Develop strategies to bypass anti-bot mechanisms and manage proxy rotation.
• Extract structured data from complex web environments using XPath and JSON Path.
• Schedule and orchestrate data collection tasks using job scheduling tools.
• Debug and optimize scraping scripts to ensure data integrity and reliability.

Required Skills

• 3+ years of experience in web scraping and data extraction.
• Proficiency in Python and libraries including Scrapy, BeautifulSoup, Selenium, or Playwright.
• Deep understanding of HTML, CSS, JavaScript, and HTTP protocols.
• Hands-on experience with task scheduling tools such as Celery, Apache Airflow, Cron, or Redis Queue.
• Expertise in handling large-scale scraping and anti-bot bypass techniques.
• Strong command of SQL and NoSQL databases.
• Competency with cloud platforms including AWS, Google Cloud, or Azure.
• Degree in Computer Science, Software Engineering, Data Science, or equivalent experience.

Preferred Skills

• Experience with Docker, Kubernetes, and containerized scraping environments.
• Familiarity with CI/CD pipelines for automated script deployment.
• Knowledge of AI/ML applications in data extraction.

Key Skills

Education

Any Graduate