Description

Design and build scalable data pipelines and architectures to support enterprise analytics, reporting, and data governance.

This role is on-site.

Responsibilities

• Design, develop, and maintain scalable batch and streaming data workflows for business-critical functions.
• Optimize data processing job performance, latency, and throughput through profiling and tuning.
• Implement security, privacy, and regulatory standards within data pipelines and repositories.
• Collaborate with data scientists, BI, and application teams to ensure data quality and availability.
• Automate data ingestion, transformation, and deployment processes to improve operational efficiency.

Required Skills

• 4+ years of experience designing and implementing enterprise data pipelines in cloud environments.
• Hands-on proficiency with Python, PySpark, and Scala for building scalable data processing jobs.
• Expertise in big data platforms including Spark, Hadoop, and Hive.
• Proven experience in end-to-end ETL pipeline development, covering ingestion, transformation, and validation.
• Strong familiarity with AWS cloud data ecosystems, specifically S3, Glue, EMR, and Data Pipeline.
• Experience with relational databases such as PostgreSQL, Oracle, and SQL Server.
• Knowledge of containerization using Docker and Kubernetes for scalable deployment.
• Understanding of data governance, security standards, and regulatory compliance.

Key Skills

Education

Bachelor's degree