Description
Key Skills: Apache Iceberg, Oracle, Starburst, PySpark, ETL/ELT, Data Pipelines, SQL, Database Development, Performance Tuning, Data Modeling
Good to Have Skills: Experience in Financial Services industry, Trino, Data Warehousing, Big Data Technologies, Data Governance, Data Quality Management, S3, ADLS, Machine Learning support, Analytics and Reporting, Distributed Data Processing, Data Virtualization, Schema Design, Stored Procedures.
Roles & Responsibilities:
- Utilize knowledge of applications development procedures and concepts, and basic knowledge of other technical areas to identify and define necessary system enhancements.
- Consult with users, clients, and other technology groups on issues, and recommend programming solutions, install, and support customer exposure systems.
- Apply fundamental knowledge of programming languages for design specifications and analyze applications to identify vulnerabilities and security issues.
- Design, develop, and maintain robust and scalable data solutions within data lakehouse architecture, leveraging Apache Iceberg for open table format management.
- Develop and optimize data ingestion, transformation, and curation processes using technologies compatible with Iceberg, Oracle, and Starburst.
- Utilize Starburst for high-performance querying and data virtualization across various data sources, including Iceberg tables and Oracle databases.
- Work extensively with Oracle databases for enterprise data management, including schema design, stored procedures, and performance tuning.
- Collaborate with data architects, data scientists, and business intelligence teams to translate business requirements into technical data models.
- Implement and manage ETL/ELT pipelines to move data efficiently between Oracle, data lake storage, and Iceberg tables.
- Ensure data quality, integrity, and security across all data platforms, adhering to data governance standards.
Experience Required: 5-8 years of relevant experience in the Financial Service industry with intermediate level experience in Applications Development role. Strong understanding of big data technologies, data warehousing concepts, and ability to work with large datasets.
Education: Bachelor's degree/University degree or equivalent experience