Description

Own the design and delivery of scalable data solutions and high-performance ETL pipelines.

This role is remote.

Responsibilities

• Develop production-grade ETL pipelines using Spark and PySpark on Databricks.
• Design scalable data models and document technical specifications for data lake and warehouse implementations.
• Write optimized SQL queries using Teradata SQL, Hive SQL, and Spark SQL.
• Implement CI/CD pipelines and orchestrate Databricks Workflows for deployment to AWS.
• Monitor production jobs, troubleshoot issues, and ensure system reliability.

Required Skills

• 12+ years of experience in application development or data engineering.
• Strong hands-on expertise with Apache Spark and PySpark.
• Proficiency in Teradata and Databricks ecosystems.
• Advanced SQL skills, including stored procedures and complex query optimization.
• Experience with Unix/Linux shell scripting (KSH) and server administration.
• Hands-on experience with AWS services, specifically S3, SQS, EC2, Lambda, Glue, IAM, and CloudWatch.
• Familiarity with DevOps tools such as Git, Jenkins, and Artifactory.

Key Skills

Education

Any Graduate