Description

Key Skills: Data Engineer, Airflow, ETL, Bigquery, Python, GCP, MySQL, GIT, DevOps, Data Modelling, Data quality

Roles and Responsibilities:

  • Design, develop, and maintain scalable data pipelines for extracting, transforming, and loading data from multiple sources to ensure seamless data availability and accessibility.
  • Collaborate with cross-functional teams to integrate data from disparate systems while ensuring consistency, reliability, and accuracy of enterprise data.
  • Optimize data processing workflows, storage solutions, and pipeline performance for scalability, efficiency, and cost-effectiveness.
  • Implement data validation, quality checks, and monitoring processes to ensure completeness, consistency, and integrity of data across the lifecycle.
  • Monitor and troubleshoot data pipelines, infrastructure, and processing systems to ensure stability, reliability, and high availability.

Skills Required:

Must-Have:

(Ideally should not be more than 3-5)

  • Expertise in Google Cloud Data Engineering services such as BigQuery, Dataproc, and related GCP data services.
  • Strong hands-on experience in Python and PySpark for data manipulation, transformation, and engineering tasks.
  • Expertise in SQL and NoSQL databases with strong query optimization skills.
  • In-depth understanding of data warehousing concepts and modern data warehousing solutions.
  • Proficiency in designing, developing, and maintaining ETL/ELT processes using tools such as Apache Airflow, Talend, or Informatica.

Good-to-Have:

(Ideally should not be more than 3-5)

  • Understanding of DevOps concepts and implementation within GCP environments.
  • Strong knowledge of data modeling concepts and scalable data architecture design.
  • Experience with version control systems such as Git for collaboration and code management.
  • Knowledge of data governance, data quality standards, and data security best practices.
  • Familiarity with performance optimization and cost management strategies in cloud data platforms.

Education: Bachelor's or Master's degree in Engineering

Key Skills
Education

Any Graduate

Back To Jobs