Description

You will design, develop, and maintain data flows and architectures within a DataLakehouse environment.

Responsibilities

• Design and implement DataLakehouse layers for ingestion, harmonization, and curation.
• Develop data integration mappings using transformations such as Joiner, Lookup, Aggregator, and XML Parser.
• Convert Linux-based Perl DIF Framework scripts to Windows-based code.
• Build Snowflake stored procedures to load data from AWS S3 via Snowpipe.
• Optimize resources and perform performance tuning on data pipelines.

Required Skills

• 6+ years of experience in data engineering roles.
• Proficiency in Python, Spark, and Databricks.
• Hands-on experience with Informatica and Streamsets for real-time data ingestion.
• Strong knowledge of AWS Cloud, Data Warehousing, and Dimensional Modelling.
• Experience with UDM Model and NW applications.
• Technical expertise in Perl, Unix, Linux, and Shell Scripting.
• Experience with Kubernetes, Docker, and cloud-native technologies.
• Proficiency in SQL Server, Stored Procedures, and AutoSys Jils.
• Experience using CI/CD tools such as Concourse, CircleCI, Azure DevOps, Bitbucket, Bamboo, XLRelease, or Git/GitHub.

Key Skills

Education

Any Graduate