Description

You will own the design, development, and maintenance of scalable ETL data pipelines.

Responsibilities

• Design, develop, test, and maintain scalable ETL data pipelines using Python.
• Architect enterprise solutions incorporating Kafka, multi-cloud services, GKE auto-scaling, Load balancers, and APIGEE proxy API management.
• Perform data ingestion from diverse sources, applying transformation and cleansing logic to ensure data quality.
• Implement and enforce data quality checks, validation rules, and monitoring across pipelines.
• Collaborate with data science and analytics teams to deliver efficient data solutions and manage CI/CD deployments via GitHub.

Required Skills

• 10 to 12 years of hands-on experience in Python for backend or data engineering projects.
• Strong working experience with GCP cloud services, including Dataflow, BigQuery, and Cloud Composer.
• Solid understanding of data pipeline architecture, data integration, and transformation techniques.
• Experience with Apache Spark, Kafka, FastAPI, Airflow, and GCP Composer DAGs.
• Strong SQL proficiency with enterprise databases such as SQL Server, Oracle, or PostgreSQL.
• Experience managing data migrations from on-premise sources to cloud platforms.
• Familiarity with storage solutions like MongoDB and Redis/Bigtable.
• Experience deploying applications in GKE and Cloud Run.
• Knowledge of version control systems like GitHub and CI/CD practices.

Key Skills

Education

Any Gradute