Description

Experience: 5+ Years
The selected candidate will execute priority workload migrations across Lift & Shift, Refactor, and New Build tracks. The role involves building production-grade pipelines using Databricks and Snowflake, performing dual-run validations, and addressing technical debt across schemas, naming conventions, and data lineage.

𝐊𝐞𝐲 𝐑𝐞𝐬𝐩𝐨𝐧𝐬𝐢𝐛𝐢𝐥𝐢𝐭𝐢𝐞𝐬
Execute automated Lift & Shift migrations, including schema mapping, Iceberg table creation, and Bronze/Silver pipeline generation.
Develop and refactor ETL/ELT pipelines using modular PySpark and Snowpark patterns.
Review and validate AI-assisted code conversions.
Execute dual-run validations covering row counts, hash-based integrity, and business rule comparisons.
Optimize schemas, naming conventions, and data lineage.
Develop and deploy Apache Airflow DAGs to replace Control-M orchestration.
Measure complexity, runtime, and cost improvements.
Create operational runbooks, SOPs, and migration documentation.
Support knowledge transfer, shadowing, and reverse-shadowing activities.

Must-Have Skills
Strong proficiency in SQL, Python, PySpark, and Apache Spark.
Hands-on experience with Databricks (Delta, DLT, Autoloader, Unity Catalog) and/or Snowflake (Snowpark, Snowpipe, Streams, Tasks).
AWS services: S3, Redshift, Athena, Glue, EMR, and Lambda.
Apache Airflow DAG development.
GitLab version control and CI/CD.
SQL performance tuning and optimization.
Data quality validation and reconciliation.
Nice-to-Have Skills
Migration experience from Redshift/Athena to Databricks or Snowflake.
Familiarity with Control-M and Control-M to Airflow migration.
Qlik Replication / AWS DMS.
Linux/Unix shell scripting.
AWS or Databricks certifications.

Key Skills

Education

Any Graduate