Description

You will lead the design and implementation of scalable data architectures and end-to-end data pipelines for large-scale structured and unstructured datasets.

Responsibilities

• Lead the design and implementation of scalable, efficient data architectures to meet analytical requirements.
• Develop and optimize data pipelines, ETL processes, and data integration solutions for high performance and reliability.
• Oversee the management of databases, data warehouses, and data lakes to ensure integrity and security.
• Provide technical leadership through code reviews, mentorship of junior engineers, and architectural decision-making.
• Establish data quality standards, validation rules, and governance practices to ensure compliance and accuracy.
• Implement monitoring systems to track pipeline performance and identify bottlenecks.

Required Skills

• 10+ years of experience in data engineering or related roles.
• Extensive experience with AWS, PySpark, Spark, and Hadoop.
• Proficiency in Python, Scala, Java, and SQL.
• Hands-on experience with NoSQL and PostgreSQL.
• Strong understanding of ETL frameworks and data integration.
• Experience with data architecture, data governance, and data security.
• Ability to work within Agile methodologies.
• Bachelor's or Master's degree in Computer Science, Data Science, or a related field.

Preferred Skills

• Experience with Databricks and CI/CD practices.
• AWS Certified Big Data - Specialty certification.
• Databricks Certified Professional Data Engineer certification.

Key Skills

Education

Bachelor's or Master's degrees