Description

You own the design, development, and operation of data pipelines and platforms.

Responsibilities

• Ingest and model data from APIs, files/SFTP, and relational sources, implementing layered architectures using PySpark/SQL and dbt.
• Design and operate data pipelines using Prefect (or Airflow), managing scheduling, parameterization, SLAs, and documentation.
• Build on cloud data platforms, leveraging cloud storage (S3/ADLS/GCS) and Spark compute platforms, managing jobs and secrets.
• Enforce data quality and governance via data contracts, validations, lineage, and proactive alerting.
• Optimize platform performance and cost through partitioning, clustering, query tuning, and workload management.

Required Skills

• 5+ years of professional data engineering experience.
• Strong proficiency in Python, SQL, and Spark (PySpark).
• Hands-on experience building ETL/ELT workflows with Prefect (or Airflow) and dbt.
• Experience with cloud data platforms, including storage and compute services on AWS, Azure, or GCP.
• Familiarity with data warehousing solutions such as Snowflake or Databricks.
• Experience managing data lifecycle and security, including encryption and PII handling.
• Skills in data visualization tools like Tableau.
• Working knowledge of Git-based workflows and CI/CD principles.

Key Skills

Education

Any Graduate