Description
You will design, develop, and maintain data pipelines supporting enterprise analytics and data migration.
Responsibilities
- Design, develop, and optimize ETL/ELT workflows for enterprise analytics and data migration.
- Build and maintain resilient big data architectures in cloud environments supporting large data volumes.
- Collaborate with data scientists, BI teams, and business analysts to translate requirements into scalable data solutions.
- Conduct data validation, reconciliation, and security assessments supporting compliance standards.
- Monitor data pipeline performance, troubleshoot issues, and optimize for reliability and security.
Required Skills
- 6+ years of experience with big data ecosystems supporting large-scale data pipelines.
- 6+ years experience with SQL (MySQL, PostgreSQL, Oracle) for data validation and performance tuning.
- 6+ years experience building ETL/ELT pipelines using Informatica, Talend, or similar tools.
- Expertise in big data tools: Apache Spark, Hadoop, and Hive for large data processing.
- Demonstrated ability to troubleshoot, optimize, and automate large data processes.
- Experience supporting data security policies, encryption, and compliance frameworks (GDPR, HIPAA).
- Familiarity with cloud data platforms like AWS (Glue, S3, Redshift), Azure, or GCP.
- Experience with relational databases including MySQL, Oracle, and PostgreSQL.
Preferred Skills
- Knowledge of NoSQL databases like MongoDB or Cassandra.
- Familiarity with Python or Scala for automation and data scripting.