Description

You will build and maintain Python-based web scrapers to extract structured and unstructured data from diverse web sources.

Responsibilities

• Develop Python scripts to automate scraping tasks and schedule jobs using cron or Airflow.
• Manage scraped data within SQL/NoSQL databases or cloud storage solutions.
• Implement techniques to bypass CAPTCHAs, IP blocking, and other scraping obstacles.
• Optimize scrapers for performance and large-scale data extraction.
• Transform raw data into structured formats like JSON or CSV through cleaning and validation processes.

Required Skills

• 3+ years of professional software engineering experience with a focus on Python development.
• 2+ years of experience with web technologies including JavaScript, HTML, CSS, and XML.
• 2+ years of experience using Scrapy and BeautifulSoup for HTML parsing.
• Hands-on experience with Selenium or Playwright for handling dynamic, JavaScript-rendered content.
• Proficiency in designing, querying, and managing SQL or NoSQL databases.
• Experience handling and validating data formats such as JSON and CSV.
• Bachelor’s degree in Computer Science, Software Engineering, IT, or a related technical field.
• English proficiency of B2 level or higher.

Preferred Skills

• Experience with cloud platforms including AWS, Google Cloud, or Azure.
• Understanding of network traffic and web scraping legal standards.
• Experience with SDLC, testing, Git, and CI/CD pipelines.

Key Skills

Education

Any Graduate