Description

You will design, build, and maintain data platforms and pipelines.

Responsibilities

• Design and deploy data lakes, data warehouses, and streaming solutions.
• Implement ETL processes, ensuring proper partitioning, error handling, and data validation.
• Automate infrastructure using IaC tools and manage pipeline deployment via CI/CD.
• Model data structures and tune SQL queries for optimal performance.

Required Skills

• 5–15 years of hands-on experience in data engineering or data platform development.
• Expert proficiency in Python (pandas, PySpark, boto3, SQLAlchemy).
• Advanced experience with AWS Data Services: Glue, Lambda, EMR, Step Functions, DynamoDB, Redshift, Athena, S3, Kinesis, QuickSight.
• Strong command of SQL, data modeling, and performance tuning.
• Experience with infrastructure automation tools like CloudFormation or Terraform.
• Proficiency in version control (Git) and implementing CI/CD for data pipelines.
• Solid understanding of IAM and CloudWatch for monitoring and security.
• Knowledge of containerization (Docker/Kubernetes) and core DevOps concepts.

Preferred Skills

• Familiarity with workflow orchestration tools like Airflow, dbt, or Dagster.
• Exposure to real-time data streaming technologies (Kafka, Kinesis Data Streams, or Firehose).

Key Skills

Education

Any Graduate