Description

You will design, build, and maintain data pipelines and workflows to support analytics and reporting. You own the end-to-end data engineering lifecycle from ingestion to transformation.

Responsibilities

• Design and develop scalable data workflows and notebooks using Databricks and Spark.
• Build ETL/ELT processes to ingest and transform data into a data lake using Delta Lake.
• Develop data models and schemas that meet reporting and analytics requirements.
• Implement data quality checks, integrity controls, and security measures.
• Monitor pipeline performance, identify bottlenecks, and resolve technical issues.

Required Skills

• 3+ years of experience in data engineering.
• Proficiency with PySpark or Spark SQL.
• Hands-on experience with Python and SQL.
• Experience with Delta Lake and ETL/ELT processes.
• Experience working within AWS cloud platforms.
• Experience with Azure environments.
• Understanding of DevOps principles and practices.
• Bachelor's degree in a relevant field.

Key Skills

Education

Bachelor's degree