Description
You will build and maintain data pipelines to support analytical workloads.
Responsibilities
- Design and implement data ingestion workflows using batch and incremental processing.
- Transform and validate data using advanced SQL techniques.
- Implement data lakehouse patterns, managing Bronze, Silver, and Gold layers.
- Manage data processing jobs leveraging Spark and Databricks.
- Maintain CI/CD workflows for data engineering assets.
Required Skills
- 5+ years of experience in Data Engineering.
- Strong hands-on experience with Apache Spark and Databricks.
- Expertise in SQL for complex data transformation and tuning.
- Practical experience with at least one major cloud platform (Azure, AWS, or GCP).
- Experience with batch ingestion and structured data transformations.
- Familiarity with Git-based version control and CI/CD pipelines.
- Knowledge of distributed data processing frameworks.
Preferred Skills
- Experience with streaming technologies like Kafka, Event Hubs, or Kinesis.
- Familiarity with Terraform or Infrastructure as Code principles.