Description

You will design and implement scalable data architectures, including data lakes, warehouses, and pipelines.

Responsibilities

• Build and maintain ETL processes to ingest, process, and transform large volumes of data from databases, APIs, and streaming platforms.
• Design data models and schemas for efficient storage, retrieval, and analysis.
• Implement data transformation and enrichment processes to support analytics, reporting, and machine learning.
• Optimize data pipelines and queries using partitioning, indexing, and query optimization techniques.
• Establish data governance policies to ensure integrity, security, and regulatory compliance.

Required Skills

• 10+ years of experience as a Big Data Engineer or in a similar role.
• Proficiency with Apache Spark and Apache Hadoop.
• Strong experience with AWS technologies and AWS CloudFormation.
• Extensive knowledge of Database Management and Big Data tools.
• Proven experience in API Development.
• Hands-on experience with A/B Testing.
• Ability to integrate data from diverse sources including files and streaming platforms.

Preferred Skills

• Experience with complex data lake architectures.
• Background in machine learning data pipelines.

Education: Any Graduate

Key Skills

Education

Any Gradute