Description

Key Responsibilities:

Data Pipeline Development: Design, construct, test, and maintain highly scalable data management systems. Develop and implement architectures that support the extraction, transformation, and loading ETL of data from various sources.

• Data Integration: Integrate structured and unstructured data from multiple data sources into a unified data system, ensuring data quality and consistency.

• Data Warehousing: Build and maintain data warehouses and data lakes to store and retrieve vast amounts of data efficiently. Optimize the performance of databases and queries to meet business needs.

• Data Processing: Implement data processing frameworks e.g., Hadoop, Spark to process large datasets in real-time or batch processing.

• Automation and Monitoring: Automate manual processes, optimize data delivery, and develop data monitoring systems to ensure data integrity and accuracy.

• Collaboration: Work closely with data scientists, analysts, and other stakeholders to understand data needs and provide technical solutions that meet business requirements.

• Data Governance: Ensure data governance policies are followed, including data security, data privacy, and compliance with regulations

Key Skills

Education

Bachelor’s degree