Design and develop scalable ETL/ELT data pipelines using AWS services such as AWS Glue, Amazon S3, Lambda, Step Functions, and Amazon Redshift.
Build data ingestion frameworks for structured and semi-structured data from enterprise applications, databases, APIs, files, and streaming sources.
Develop data transformation and processing solutions using Python, PySpark, and SQL.
Design and optimize cloud-based data lake and data warehouse solutions on AWS.
Develop performant data models and datasets for downstream analytics, reporting, machine learning, and AI use cases.
Work closely with business stakeholders, solution architects, data analysts, and application teams to translate business requirements into scalable technical solutions.
Optimize AWS data workloads for performance, scalability, reliability, and cloud cost.
Implement appropriate security controls including AWS IAM, encryption, secrets management, and role-based access.
Support CI/CD, automated deployment, production support, troubleshooting, and operational monitoring of data pipelines.
Required Skills:
Strong hands-on experience with AWS Glue, Amazon S3, Amazon Redshift, Lambda, IAM, CloudWatch, and Step Functions.
Strong programming experience in Python and PySpark.
Advanced SQL and experience working with large-scale relational and analytical datasets.
Experience implementing ETL/ELT pipelines, data lakes, data warehouses, and dimensional/data modeling.
Experience with Apache Spark and distributed data processing.
Good understanding of AWS security, networking, encryption, and access-control concepts.
Experience with Git and CI/CD practices for data engineering deployments.
Strong troubleshooting, performance optimization, and production-support skills.
Preferred Qualifications:
Experience with Amazon EMR, Kinesis, Athena, Lake Formation, MWAA/Airflow, or DynamoDB.
Experience with automotive, manufacturing, supply-chain, connected-vehicle, sales, dealer, or customer datasets is highly desirable.
Exposure to Terraform or AWS CloudFormation for Infrastructure as Code.
AWS certification such as AWS Certified Data Engineer – Associate is preferred.
Experience supporting enterprise data platforms with strong data governance and data-quality requirements