Description

Key Skills Python, PySpark, SQL, ETL, Data Pipelines, Machine Learning, Big Data, AWS, Hive, Data Engineering

Good to Have Skills Experience with data lakehouse architecture such as AWS Cloud/Airflow/Starburst/Iceberg. Strong proficiency in scripting languages like Bash, UNIX Shell scripting. Experience with data ingestion tools such as Talend or Ab Initio. Working with Bigdata technologies like Impala/S3/HDFS. Data pipeline and workflow management tools experience.

Roles & Responsibilities

• Build and maintain batch or real-time data pipelines in data platform environments.
• Maintain and optimize the data infrastructure required for accurate extraction, transformation, and loading of data from a wide variety of data sources.
• Develop ETL (extract, transform, load) processes to help extract and manipulate data from multiple sources.
• Monitor and control all phases of development process and analysis, design, construction, testing, and implementation as well as provide user and operational support on applications to business users.
• Automate data workflows such as data ingestion, aggregation, and ETL processing for improved efficiency.
• Analyze large datasets to identify trends, anomalies, and behavioral patterns using advanced analytical techniques.
• Apply machine learning and AI concepts (supervised / unsupervised learning) to support predictive and exploratory analysis.
• Perform feature engineering and data transformations to enable machine learning models for business insights.
• Support trend analysis, segmentation, clustering, and forecasting use cases for business decision making.
• Prepare raw data in Data Warehouses into a consumable dataset for both technical and non-technical stakeholders.
• Build, maintain, and deploy data products for analytics and data science teams on data platform.
• Ensure data accuracy, integrity, privacy, security, and compliance through quality control procedures and monitoring.
• Monitor data systems performance and implement optimization solutions to improve system efficiency.
• Act as subject matter expert to senior stakeholders and serve as advisor or coach to new or lower level analysts.

Experience Required 8 to 14 years of relevant experience in Data engineering role with strong proficiency in object-oriented languages and data analysis.

Education Bachelor's degree/University degree or equivalent experience

Key Skills

Education

Any Graduate