Description

You will design, build, and maintain scalable data infrastructure and pipelines as a senior individual contributor.

Responsibilities

• Lead the design, development, and maintenance of scalable data pipelines, ETL/ELT processes, and data models.
• Build and manage data ingestion frameworks using big data technologies to ensure efficiency and reliability.
• Develop and optimize data storage solutions across on-prem data centers, AWS, Azure, or Google Cloud.
• Design and maintain workflows using Airflow and Kubeflow for reliable data processing and storage.
• Implement best practices for data quality, governance, and security through robust monitoring systems.

Required Skills

• 7+ years of experience in data engineering, data warehousing, or ETL/ELT development.
• Proficiency in Python, Shell scripting, Java, or Scala.
• Hands-on experience with Kafka, Spark, HDFS, HBase, Flink, DBT, and SQL Mesh.
• Experience managing data pipelines in multi-cloud environments (AWS, Azure, or Google Cloud).
• Strong SQL skills and experience with both relational and NoSQL databases.
• Proficiency with containerization and orchestration using Docker and Kubernetes.
• Experience with hybrid cloud data governance and data quality frameworks.
• Knowledge of data formats including Parquet and Avro.
• Bachelor’s degree in Computer Science, Engineering, or a related field.

Preferred Skills

• Master’s degree in a technical field.
• Familiarity with data visualization tools such as Tableau, Power BI, or Looker.
• Experience mentoring or leading teams of data engineers.

Key Skills

Education

Bachelor's degree