Description

You will build and maintain ETL processes to support privacy-related applications and state-level regulatory compliance.

Responsibilities

• Develop ETL processes for data extraction, transformation, and loading from sources including OLEDB, Excel, XML, and flat files.
• Implement data conversions and aggregations using merge, join, union, split, sort, and lookup transformations.
• Write UNIX scripts to move data from source servers to Teradata and validate file integrity across servers.
• Create and manage temporary and fact tables using Teradata and Spark SQL.
• Optimize ETL objects, indexing, and partitioning, and tune SQL, HQL, and Spark SQL queries for performance.
• Promote components to production via CI/CD pipelines using GitHub.
• Execute testing phases including Unit, User Acceptance, Regression, Parallel, and System testing.

Required Skills

• 3+ years of experience in data engineering.
• Proficiency in SQL, HQL, and Spark SQL.
• Experience writing UNIX scripts for data movement and validation.
• Hands-on experience with GitHub and CI/CD processes.
• Ability to work with OLEDB, Excel, and XML data sources.
• Experience managing data flows into SQL data warehouses.
• Knowledge of ETL optimization, indexing, and partitioning.
• Bachelor’s degree in Computer Science, Computer Information Systems, Technology Management, or equivalent experience.

Key Skills

Education

Any Graduate