Description

You will build and manage data integration pipelines and warehousing solutions within an AWS ecosystem.

Responsibilities

• Design, develop, test, deploy, and maintain data integration pipelines.
• Optimize complex database queries, user-defined functions, views, and indexes.
• Improve pipeline performance through tuning and debugging.
• Manage continuous integration and deployment workflows using Git, Bitbucket, and Jenkins.

Required Skills

• 5+ years of experience in data integration and pipeline development.
• Proficiency in Python development, specifically using PySpark.
• Hands-on experience with Databricks, Apache Spark, EMR, Glue, Kafka, and Kinesis.
• Experience managing data in S3, Redshift, RDS, and MongoDB/DynamoDB.
• Strong command of Perl and Unix scripting.
• Expertise in SQL and database management with Oracle, SQL Server, PostgreSQL, Redshift, or MySQL.
• Experience with CI/CD, Airflow, and Jenkins.
• Working knowledge of Git and Bitbucket.

Preferred Skills

• Exposure to ETL tools such as Informatica.
• Degree in CS, CE, or EE.

Key Skills

Education

Any Graduate