Description

You will design and implement data architectures, migrations, and data lake formations.

Responsibilities

• Produce architecture documentation for small to large-scale solution implementations.
• Design capacity planning, partition planning, and read/write strategies using Kafka.
• Manage high-volume streaming data in formats including JSON, XML, Avro, and Snappy.
• Architect data pipelines utilizing Spark and AWS services.

Required Skills

• 12+ years of experience in Data Engineering, Data Quality, and Data Architecture.
• 5+ years of hands-on experience with AWS services including S3, EMR, VPC, EC2, IAM, EBS, RDS, Glue, Lambda, and Lake Formation.
• In-depth knowledge of Spark architecture, including Spark Code, Spark SQL, DataFrames, Spark Streaming, and Spark MLlib.
• Experience with Big Data technologies, MapReduce, and Spark.
• Proficiency with diverse database types: RDBMS, NoSQL (Columnar, Document, Distributed, Cloud, and In-memory).
• Experience handling high-volume streaming data.

Preferred Skills

• Python exposure.

Key Skills

Education

Any Graduate