Design, develop, test, deploy and maintain data pipelines using Airflow to integrate various data sources into a centralized Data Hub.
Collaborate with cross-functional teams to identify business requirements and design solutions that meet those needs.
Develop CI/CD pipelines for streaming applications using Flyway to ensure efficient deployment and monitoring of services.
Troubleshoot issues related to data flow, processing, and quality in real-time.
Job Requirements :
4-7 years of experience in software engineering with expertise in Airflow, CI/CD, Continuous Integration, Data Pipeline development.
Strong understanding of Streaming technologies such as Flyway.
Experience working on large-scale projects involving complex data processing workflows.
Job Classification
Industry: IT Services & ConsultingFunctional Area / Department: Data Science & AnalyticsRole Category: Data Science & Analytics - OtherRole: Data Science & Analytics - OtherEmployement Type: Full time