Description

Key Skills: PySpark, Cloudera, Big Data, Hadoop Ecosystem, Python, Airflow, Spark, SQL

Roles & Responsibilities:

• Design and develop scalable data processing systems using PySpark and Hadoop ecosystem tools.
• Build, manage, and optimize data pipelines and workflows using Apache Airflow.
• Work with Cloudera platforms for efficient data storage, processing, and retrieval.
• Collaborate with data scientists and analysts to understand requirements and deliver data solutions.
• Implement data quality checks, validation, and governance best practices.
• Optimize data workflows for performance, scalability, and reliability.
• Develop and maintain ETL processes for large-scale data processing.
• Monitor and troubleshoot data pipelines to ensure seamless operations.
• Ensure adherence to data security, compliance, and governance standards.
• Stay updated with emerging big data technologies and continuously improve data architecture.

Experience Required:

• 5 - 9 years of experience in big data or data engineering roles.
• Strong hands-on experience with PySpark, Hadoop ecosystem, and Cloudera.
• Experience in workflow orchestration using Apache Airflow.
• Proficiency in Python and data processing frameworks.
• Knowledge of Spark and SQL for data transformation and querying.
• Experience building and optimizing large-scale data pipelines.
• Understanding of data governance, quality, and security practices.
• Strong analytical, problem-solving, and communication skills.

Education: Any Graduation

Key Skills

Education

Any Graduate