Description

Build and maintain scalable web scraping solutions to extract data from static and dynamic sources.

This role is remote.

Responsibilities

• Develop and maintain scalable web scraping infrastructure using Python.
• Extract data from static and dynamic websites, handling pagination, authentication, and anti-bot mechanisms.
• Clean, structure, and store scraped data efficiently in required formats.
• Collaborate with internal teams to define and meet specific data requirements.
• Manage IP blocking, CAPTCHA challenges, rate limits, and proxy rotation.

Required Skills

• 3+ years of professional software engineering experience.
• Strong proficiency in Python programming.
• Hands-on experience with Beautiful Soup, Requests, Scrapy, Selenium, and Playwright.
• Solid understanding of HTML, CSS, and XPath for DOM parsing.
• Experience managing IP blocking, CAPTCHA, rate limits, and proxies.
• Familiarity with data formats including CSV, JSON, and Excel.
• Ability to interact with databases for efficient data storage.

Preferred Skills

• Experience with any graduate degree in Computer Science or related field.
• Familiarity with anti-bot bypass techniques beyond standard proxy rotation.

Key Skills

Education

Any Gradute