Description

Design and maintain scalable data pipelines, ETL/ELT processes, and cloud-native data solutions for enterprise data lakes, warehouses, and lakehouse platforms.

This role is on-site.

Responsibilities

• Build and optimize batch and real-time data ingestion, transformation, and processing frameworks using Python, PySpark, and Snowpark.
• Develop modular, reusable SQL-based data workflows and implement data transformation using dbt within Snowflake or Databricks.
• Implement and support Master Data Management (MDM) initiatives, ensuring data quality, metadata management, and lineage tracking.
• Orchestrate pipeline scheduling and automation using tools like Airflow, Databricks Workflows, or Snowflake Tasks.
• Integrate diverse data sources including APIs, microservices, file-based batch, and streaming data via Kafka.

Required Skills

• 5+ years of hands-on experience in data engineering and enterprise-scale data integration.
• Strong SQL expertise with programming/scripting in Python, PySpark, or Snowpark.
• Proficiency with dbt (Data Build Tool) for ELT pipeline development, testing, and documentation.
• Experience with cloud platforms (Azure, AWS, or GCP) and integration with Snowflake and Databricks.
• Solid understanding of data lake, warehouse, and lakehouse architectures.
• Experience with orchestration tools (Airflow, Databricks Workflows, Snowflake Tasks).
• Familiarity with CI/CD processes, Agile methodologies, and DevOps environments.
• Knowledge of data security, governance, RBAC, and compliance standards.

Preferred Skills

• Financial Services or Banking industry experience.
• Experience with real-time data processing and streaming technologies.

Bachelor’s degree in Computer Science, Information Systems, Engineering, or related field.

Key Skills

Education

Bachelor's degree