Description

You will build and maintain scalable data pipelines and infrastructure to support enterprise data warehousing and analytics.

This role is on-site.

Responsibilities

• Create and optimize data pipelines using PySpark, Python, Scala, Java, SQL, and Databricks.
• Develop and execute data processes directly within the Databricks platform.
• Build and maintain ETL processes using Python and PySpark for data movement and transformation.
• Collaborate with cross-functional teams to design and implement data solutions.
• Identify and resolve data issues within pipelines and the enterprise data warehouse.

Required Skills

• 2-3 years of experience in data engineering.
• Proficiency in PySpark, Python, Scala, Java, SQL, and Databricks.
• Experience designing pipelines to ingest and transform data from databases, APIs, and sensors.
• Hands-on experience with cloud platforms such as AWS, Azure, or Google Cloud Platform.
• Working knowledge of DevOps and CI/CD pipeline automation.
• Strong understanding of database management systems including SQL and NoSQL.
• Experience with Agile methodology.
• Ability to troubleshoot data pipeline issues and identify data anomalies.

Preferred Skills

• Experience with batch and streaming processing techniques.
• Knowledge of microservices architecture and RESTful Web Services.
• Familiarity with data visualization tools like Tableau or Power BI and production log analysis using Splunk.

Key Skills

Education

Any graduate