Your browser does not support javascript! Please enable it, otherwise web will not work for you.

Engineering Expert - AI & Data Products, Runtime Platform @ Sapital Recruitments

Home > DBA / Data warehousing

 Engineering Expert - AI & Data Products, Runtime Platform

Job Description

Software-Design and Development Engineering Expert - AI Data Products, Runtime Platform
As an Engineering Expert, you will be instrumental in the expansion of our Foundation Services team in Bangalore. Your profound expertise in distributed data processing, Spark optimization, data transformation pipelines, data engineering architecture principles, AI Agent development, and LLM-driven engineering will drive scalable data solutions and intelligent automation, reinforcing SAP Data Analytics' pivotal role in SAP's Data AI strategy. This is a product expert role requiring deep hands-on expertise across the modern cloud-native stack.
What you'll build
  • Lead the design and execution of optimized data processing solutions at scale.
  • Define and enforce data engineering architecture principles including data lineage, schema evolution, idempotency, fault tolerance, data partitioning strategies, and separation of concerns across processing layers.
  • Spearhead Spark optimization strategies using PySpark to maximize performance and scalability in data processing.
  • Architect and refine pluggable data transformation pipelines for efficient processing of large datasets.
  • Establish architectural standards for data lakehouse patterns, batch vs. streaming paradigms, data contracts, and metadata-driven pipeline design.
  • Design and develop AI Agents, MCP (Model Context Protocol) Servers, and MCP Tools to enable intelligent, autonomous workflows.
  • Craft and optimize advanced prompts using prompt engineering techniques (chain-of-thought, few-shot, Zero-shot prompting) to drive reliable and high-quality LLM outputs in product workflows.
  • Contribute to LLM fine-tuning and training efforts - including data curation, training dataset preparation, RLHF (Reinforcement Learning from Human Feedback), evaluation benchmarking, and model alignment for domain-specific use cases.
  • Build and maintain microservices architectures that support modular, scalable product capabilities.
  • Implement GitOps practices using Argo CD and Helm to advance CI/CD pipelines and operational efficiency on Kubernetes.
  • Apply advanced SQL skills to refine data transformations across vast datasets.
  • Integrate AI ML technologies into high-performance data solutions, staying informed on emerging trends in agentic AI and LLM-based tooling.
  • Utilize SAP HANA Spark to drive innovation in data engineering processes.
  • Drive architectural decisions around data quality, observability, scalability, and reliability across the platform.
  • Mentor junior engineers, fostering a culture of continuous improvement and technical excellence.
  • Collaborate effectively with global stakeholders to achieve successful project outcomes.
What you bring
  • Extensive experience in data engineering, distributed data processing, and expertise in SAP HANA or similar databases.
  • Deep understanding of data engineering architecture principles - including data pipeline architecture, ELT/ETL patterns, Delta tables and their optimization (Z-ordering, compaction, vacuum, partitioning, liquid clustering), schema-on-read vs. schema-on-write, and pipeline orchestration best practices.
  • Proficiency in Python (PySpark) is essential.
  • Hands-on experience in AI Agent development, MCP Server development, and MCP Tool development.
  • Strong prompt engineering skills - experience designing, testing, and iterating on prompts for LLMs (GPT-4, Claude, Gemini, or similar) across diverse use cases including code generation, data transformation, and agentic reasoning.
  • Strong expertise in microservices architecture and development.
  • Deep knowledge of Kubernetes, Helm, and Argo CD for container orchestration and GitOps-driven deployments.
  • Advanced understanding of Spark optimization and scalable data processing techniques.
  • Proven experience in architecting data transformation pipelines with a focus on reusability, extensibility, and maintainability.
  • Strong understanding of AI ML technologies, LLMs, and industry trends in agentic systems.
  • Knowledge of data governance, data quality frameworks, and observability in distributed data systems.
  • Familiarity with LLM frameworks and tooling such as LangChain, LlamaIndex, and LangGraph.
  • Effective communication skills within a global, multi-cultural environment.
  • Proven track record of leadership in data processing, platform initiatives, and product engineering.

Job Classification

Industry: IT Services & Consulting
Functional Area / Department: Engineering - Software & QA
Role Category: DBA / Data warehousing
Role: Database Architect / Designer
Employement Type: Full time

Contact Details:

Company: SAP
Location(s): Bengaluru

+ View Contactajax loader


Keyskills:   kubernetes cd continuous integration python data processing ci/cd pyspark engineering data engineering helm artificial intelligence microservices sql spark container orchestration data transformation sap hana etl communication skills architecture

 Fraud Alert to job seekers!

₹ Not Disclosed

Similar positions

Forward Deployed Data Engineering Expert

  • Sapital Recruitments
  • 3 - 6 years
  • Bengaluru
  • 23 days ago
₹ Not Disclosed

Engineering Expert - AI & Data Products, Runtime Platform

  • Sapital Recruitments
  • 10 - 17 years
  • Bengaluru
  • 1 month ago
₹ Not Disclosed

Sapital Recruitments

Sapital Recruitments specializes in IT and Healthcare hiring. We take a practical, transparent approach to connect companies with right talent. Founded on 6th February 2025, Sapital Recruitments is a dynamic and performance-driven placement and HR solutions firm dedicated to building strong workfor...

Job Listings