Description

You will build and maintain scalable data pipelines and architectures to support complex data processing requirements.

Responsibilities

• Develop data pipelines using batch, micro-batching, and streaming methodologies.
• Design and implement data models using techniques such as Dimensional, Data Vault, Kimball, or Inmon.
• Manage data lifecycle across relational stores, NoSQL databases, and cloud lakehouse platforms.
• Execute development within Agile frameworks, including PI planning and roadmap development.
• Apply TDD or BDD practices and manage deployments via CI/CD tools.

Required Skills

• 5+ years of experience in data engineering.
• Proficiency in Python, Scala, or Java.
• Experience with cluster computing frameworks like Spark, Flink, or Storm.
• Hands-on with cloud data lakehouse platforms such as AWS data lake services, Databricks, or Hadoop.
• Experience with relational databases like Postgres or Oracle.
• Proficiency with NoSQL stores including Cassandra, Dynamo, or MongoDB.
• Experience orchestrating workflows with Airflow, AWS Step Functions, or similar tools.
• Knowledge of data serialization formats like JSON, Avro, or Protobuf.
• Expertise in big-data storage formats such as Parquet or Iceberg.
• Competency with Jenkins and Git for CI/CD.

Preferred Skills

• Experience using AWS Bedrock APIs and Generative AI concepts like RAG, vector embeddings, or model fine-tuning.
• Experience with Infrastructure as Code using Terraform or AWS CloudFormation.

Key Skills

Education

Any Gradute