Description

Required Qualifications

• Bachelor’s degree in computer science, Information Systems, Engineering, or related field.
• 10+ years of IT experience with 6+ years in Data Engineering and Big Data development.
• Strong hands-on experience with PySpark and Spark based data processing.
• Experience developing data pipelines on Cloudera Data Platform (CDP).
• Strong knowledge of Apache Iceberg table architecture and data lake concepts.
• Experience designing and managing workflows using Apache Airflow.
• Experience with SQL and data modeling techniques.
• Strong understanding of ETL/ELT development and data transformation frameworks.
• Experience working with large scale structured and unstructured datasets.
• Knowledge of Git, CI/CD pipelines, and DevOps practices.
• Strong analytical, troubleshooting, and problem solving skills.
• Excellent communication and collaboration skills.

Preferred Qualifications

• Experience with AWS services including S3, Glue, Lambda, ECS/EKS, EMR, RDS, IAM, and CloudWatch.
• Experience building cloud native data lake and Lakehouse solutions.
• Strong knowledge of data partitioning, performance tuning, and query optimization.
• Experience with Kafka or event driven data processing frameworks.
• Experience with Java or Python application development.
• Experience implementing data quality, metadata, and data governance solutions.
• Healthcare, Medicaid, Claims Processing, or Insurance industry experience.
• Experience working in Agile development environments.
• AWS Certification and/or Cloudera Certification preferred.
• Experience supporting large scale cloud migration and modernization initiatives.

Experience: 10 12+ years overall with strong expertise in PySpark, Cloudera CDP, Apache Iceberg, Airflow, and AWS Data Engineering

Key Skills

Education

Bachelor's degree