Description

You will own the technical execution of complex data migration and transformation projects.

Responsibilities

• Convert existing SAS EG code into PySpark, SQL, or R logic.
• Access and migrate data from DB2 and mainframe systems into the cloud data lake.
• Optimize PySpark code for performance and scalability within the Databricks platform.
• Implement automated testing and validation frameworks to ensure data parity.
• Document conversion logic and integrate version control into CI/CD pipelines.

Required Skills

• 5+ years of professional experience in data engineering.
• Expertise in Python and SQL.
• Proven experience converting SAS code to PySpark.
• Hands-on experience with Databricks notebooks and Delta Lake.
• Familiarity with mainframe systems (JCL, COBOL, IMS).
• Experience accessing DB2 data via ODBC/JDBC or ETL tools.
• Proficiency with PySpark and Spark SQL.
• Understanding of schema mapping and data lineage tracking.
• Bachelor's degree required.

Key Skills

Education

Bachelor's degree