Description
You will design and construct scalable data pipelines and manage data flows.
Responsibilities
- Design and build efficient, scalable data pipelines.
- Manage and build complex data flows using Apache NiFi.
- Perform data transformations and build data models using DBT.
- Execute large-scale data processing tasks with PySpark.
- Use Snowflake for data storage and analysis.
Required Skills
- 7-10 years of experience in data engineering and pipeline building.
- Proficiency in SQL for querying and manipulating data.
- Hands-on experience with Snowflake.
- Experience with Apache NiFi.
- Experience with PySpark for large-scale processing.
- Experience with DBT (Data Build Tool).
- Familiarity with cloud platforms including AWS, Azure, or GCP.
- Strong problem-solving skills.
Preferred Skills