Description

You will build and maintain scalable data ingestion pipelines and ETL/ELT processes to support the data platform.

This role is remote.

Responsibilities

• Design and develop ingestion pipelines to extract data from REST and SOAP APIs.
• Implement data transformation and loading processes using Databricks and PySpark.
• Manage and optimize data schemas and storage within Snowflake and Postgres.
• Collaborate with BI developers to optimize data models for Tableau performance.
• Monitor pipelines for errors and discrepancies, implementing fixes and technical documentation.

Required Skills

• 5+ years of experience in data engineering with a focus on API integration.
• Proficiency in Python and PySpark.
• Hands-on experience with Databricks, notebooks, and pipeline orchestration.
• Strong SQL skills for query optimization and database design in Postgres.
• Experience managing cloud data warehouses using Snowflake.
• Working knowledge of cloud platforms such as AWS or Azure.
• Experience building ingestion logic for authentication, error handling, and data parsing.

Preferred Skills

• Familiarity with containerization using Docker.
• Experience with ETL/orchestration tools like Airflow, Fivetran, or dbt.

Key Skills

Education

Any Gradute