Description

Design, build, and maintain scalable ETL/ELT data pipelines for analytics and GenAI use cases. Own the full data lifecycle from ingestion to model optimization.

Responsibilities

• Develop scalable ETL/ELT pipelines using Python (Pandas, PySpark) and SQL, ingesting data from APIs, cloud storage, databases, and streaming platforms.
• Prepare structured and unstructured datasets for reporting, ML models, and GenAI applications including vector embeddings and RAG pipelines.
• Implement data quality checks, validation, monitoring, and lineage to ensure accuracy and reliability.
• Optimize data models and pipeline performance for cost efficiency and scalability.
• Document pipelines, data models, and operational processes clearly.

Required Skills

• 5+ years of experience in Data Engineering or hybrid Data Engineering/Data Analytics.
• Strong proficiency in Python (Pandas, PySpark) and advanced SQL.
• Experience with cloud data warehouses: Snowflake, BigQuery, Redshift, or Synapse.
• Hands-on experience designing ETL/ELT pipelines and data models.
• Exposure to orchestration and transformation tools such as Airflow, DBT, ADF, Glue, Dataflow, or Databricks.
• Familiarity with Git, CI/CD, and basic DevOps practices.
• Bachelor’s degree in Computer Science, Engineering, Data, or a related field (or equivalent experience).

Preferred Skills

• Experience with GenAI data workflows including vector databases and document ingestion.
• Deep expertise in specific cloud platforms: AWS (Redshift, Glue, S3, Athena), Azure (ADF, Synapse, Databricks), or GCP (BigQuery, Dataflow, Pub/Sub).

Key Skills

Education

Bachelor's degree