Description

You will design, build, and maintain data architecture, models, and pipelines for Data Lake and Data Warehouse projects on AWS.

Responsibilities

• Design, build, and deploy data pipelines using EMR, Databricks, Spark, PySpark, and Shell Scripting.
• Create high-level solution architectures and present technical strategies to stakeholders.
• Develop POCs and conduct technical show-and-tell sessions.
• Automate data ingestion, cleansing, and transformation using DevOps methodologies.
• Design data security frameworks, replication systems, and lineage tracking.

Required Skills

• 7+ years of IT experience with a focus on Data Engineering.
• 5+ years with EMR, Databricks, Spark, PySpark, Jupyter notebooks, and Shell Scripting.
• 3+ years in Data Warehouse and ETL projects.
• End-to-end implementation with Redshift, Postgres, or Snowflake.
• Hands-on expertise with Postgres SQL, Redshift Spectrum, AWS Glue, Athena, and Snowflake utilities (SnowSQL, SnowPipe).
• Proficiency in Python and AWS Lambda modeling.
• Experience migrating data from RDBMS to Redshift or Snowflake.
• Advanced Snowflake skills: RBAC, virtual warehouse sizing, query tuning, zero-copy cloning, and time travel.
• Deep understanding of Star and Snowflake schemas and Dimensional Modeling.

Preferred Skills

• Experience with AWS or Azure storage technologies (S3, ADLS) and Platform as a Service (PaaS).
• Expertise in RDBMS, complex SQL, PL/SQL, and Unix Shell Scripting.
• Experience working within Agile development methodologies.

Key Skills

Education

Any Graduate