Description
You will design, develop, and optimize ETL workflows and data pipelines.
Responsibilities
- Implement data ingestion from APIs, databases, and flat files.
- Ensure data quality, integrity, and consistency across all ETL processes.
- Collaborate with data architects, analysts, and business stakeholders on data requirements.
- Monitor and troubleshoot ETL job performance issues.
- Automate data workflows and implement CI/CD practices for data pipeline deployment.
Required Skills
- 12+ years of experience in ETL development and data engineering.
- Hands-on experience with Snowflake, including data modeling, performance tuning, and SQL scripting.
- Proficiency in AWS services (S3, Lambda, Glue, Redshift, CloudWatch).
- Strong programming skills in Python or Scala for data processing.
- Experience with orchestration tools like Apache Airflow or AWS Step Functions.
- Experience working with APIs, databases, and flat files.
- Familiarity with version control systems (e.g., Git) and CI/CD pipelines.
Preferred Skills