Description

You will design, build, and maintain scalable data pipelines and processing workflows.

Responsibilities

• Develop, implement, and deploy ETL solutions to process large datasets.
• Collaborate with software engineers and stakeholders to translate business requirements into technical features.
• Build and optimize complex SQL queries, stored procedures, and data models for performance and scalability.
• Monitor and maintain production data pipelines and models, identifying opportunities for optimization.
• Document development processes and results to facilitate knowledge sharing.

Required Skills

• 10+ years of experience in data engineering roles.
• Strong programming skills in Python, PySpark, and SQL.
• Experience building and optimizing ETL workflows using Spark, Snowflake, Airflow, Azure Data Factory, Glue, or Redshift.
• Proficiency using Snowpark for data processing within Snowflake.
• Experience with cloud infrastructure including Azure, AWS, or GCP.
• Hands-on experience implementing CI/CD pipelines through DevOps platforms.
• Knowledge of API integrations to facilitate data workflows.
• Bachelor's or Master's degree in Computer Science, Engineering, or a related field.

Preferred Skills

• Experience with Docker and Kubernetes.
• Exposure to HTML, CSS, JavaScript/jQuery, Node.js, and Angular/React.
• Experience in API development using Flask or Django.

Key Skills

Education

Bachelor's or Master's degrees