Lead and senior GCP Data Engineer
- Innova Solutions
- 10 - 15 years
- Noida, Gurugram
- 11 days ago
- Email to a friend
- Report this job
Job Description
Hiring a GCP Lead Data Engineer for Hyderabad, Chennai, Noida, Pune, and Bangalore locations
Role & responsibilities
- 10+ years of hands-on database experience, data engineering experience including relational databases, Bigquery and document databases
- 5+ years of cloud experience, preferably on Google Cloud Platform.
- Strong experience with RDBMS technologies, including PostgreSQL, MySQL, SQL Server, Oracle, or equivalent relational database platforms.
- Strong experience designing and implementing data pipelines, including batch, streaming, event-driven, and API-based processing patterns.
- Strong programming experience, especially with Python, including object-oriented programming, modular application design, error handling, logging, unit testing, package management, and production-grade backend development.
- Experience with Dataflow, Apache Beam, or equivalent data pipeline frameworks.
- Experience integrating with external APIs, including authentication, retries, rate limits, error handling, and monitoring.
- Docker and CI/CD experience.
- Production monitoring, logging, alerting, and troubleshooting experience.
- Experience with search systems such as ElasticSearch, OpenSearch, vector databases, or hybrid keyword/vector search systems.
- Strong system design and architecture skills.
- Ability to mentor mid-level and junior engineers.
- Ability to create clear technical documentation, architecture diagrams, and implementation plans.
Preferred candidate profile
- Design, build and support scalable healthcare data pipelines on GCP.
- Process healthcare files from GCS and route them through Pub/Sub, Eventarc, Cloud Run, Dataflow, BigQuery, FHIR Store, and downstream systems.
- Design relational and cloud database schemas for operational, analytical, and document-oriented workloads.
- Build APIs to monitor pipeline status, file processing, Pub/Sub messages, Dataflow jobs, Cloud Run services, BigQuery loads, and FHIR data validation.
- Design database models for pipeline metadata, orchestration configuration, processing history, audit logs, document metadata, and search indexes.
- Implement batch and streaming data pipelines.
- Build data validation, reconciliation, retry, and dead-letter handling processes.
- Integrate structured, semi-structured, and unstructured healthcare data.
- Support semantic search, NER, embeddings, and search result snippets for healthcare documents.
- Optimize database performance, query cost, indexing strategy, and storage design.
- Implement monitoring, logging, alerting, and operational traceability.
- Ensure secure handling of healthcare data, including PHI-aware design and least-privilege access.
- Mentor mid-level and junior engineers and define engineering best practices.
Job Classification
Industry: IT Services & ConsultingFunctional Area / Department: Engineering - Software & QARole Category: Software DevelopmentRole: Data EngineerEmployement Type: Full timeContact Details:
Company: Innova SolutionsLocation(s): Noida, Gurugram+ View Contact
Fraud Alert to job seekers!
₹ Not Disclosed
Similar positions
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in