Description

Focus on data engineering and automation to ensure the integrity of complex data applications and pipelines.

Responsibilities

• Build automation frameworks using Python to support data workflows and product deployment.
• Design, develop, and manage reports and dashboards for data quality analysis.
• Analyze data sources, volumes, and business rules to ensure accurate data integration.
• Test database applications to validate source-to-destination data movement and transformations.
• Troubleshoot data issues and anomalies to determine effective resolutions.

Required Skills

• 10+ years of experience in data engineering and automation.
• Extensive experience with Python for ETL automation and pipeline development.
• Deep knowledge of AWS components including S3, Athena, EMR, Glue, Redshift, Kinesis, and SageMaker.
• Proficiency in SQL, Unix, and Linux scripting.
• Experience with ETL technologies such as AWS Glue, Ab Initio, Informatica, or Alteryx.
• Strong understanding of data architecture, data modeling, and data warehousing concepts.
• Experience in DevOps and DataOps environments.
• Proven ability to test large-scale applications including Data Lakes and Data Warehouses.

Preferred Skills

• Experience with Jenkins, GitLab, and HDFS.
• Experience with data migration from on-premise systems to the cloud.
• Familiarity with Data Science platforms like H2O or Machine Learning Studio.

Key Skills

Education

Any Graduate