Description

You will apply data engineering and machine learning skills to build applications for model development and advanced reporting.

Responsibilities

• Cleanse, manipulate, and analyze large structured and unstructured datasets including XML, JSON, and PDF files using the Hadoop platform.
• Develop Python, PySpark, and Spark scripts to filter, cleanse, map, and aggregate data.
• Manage and implement data processes and data quality reports.
• Develop data profiling, deduplication, and matching logic for analysis.
• Present technical recommendations regarding Hadoop and related technologies to management.

Required Skills

• 10+ years of experience in data engineering or related roles.
• Proficiency in Python, PySpark, and Spark for data ingestion and processing.
• Hands-on experience with the Hadoop platform.
• Expertise in handling JSON and XML formats.
• Strong SQL skills for data manipulation.
• Experience with Snowflake.
• Ability to perform exploratory data analysis.
• Strong grasp of data architectures.
• Analytical, quantitative, and problem-solving skills.

Preferred Skills

• Domain experience in Mortgage.

Key Skills

Education

ANY GRADUATE