Senior Data Engineer
- Cargill
- 8 - 13 years
- Bengaluru
- 15 days ago
- Email to a friend
- Report this job
Job Description
- The Senior Data Engineer designs, builds, and operates scalable, reliable data products and platforms that power analytics, reporting, and downstream applications. This role owns endtoend delivery of batch and streaming data pipelines on a modern AWSbased cloud data platform, applying strong engineering patterns to ensure performance, security, observability, and cost efficiency.With minimal supervision, the role partners closely with product, analytics, and platform teams to translate business requirements into robust technical solutions across a Lakehouse (Iceberg) and approved warehousing platforms (e.g., Snowflake). The Senior Data Engineer also mentors other engineers, drives code quality, and raises the engineering bar across the organization.
- Data Analytical Solutions
- Designs and delivers scalable data products using standard cloud and data engineering architectures.
- Owns technical decisions (batch vs. streaming, Lakehouse vs. warehouse) and ensures solutions meet reliability, security, governance, latency, and cost requirements.
- Reviews designs and contributes reusable components, templates, and standards.
Data Pipelines- Builds and operates endtoend batch and streaming pipelines.
- Implements transformations using SQL/dbt and PySpark as needed.
- Integrates realtime or eventdriven ingestion using Kafka.
- Orchestrates workflows with Airflow; establishes SLAs/SLOs and CI/CDbased deployments.
Data Systems Architecture- Optimizes data architectures for performance, scalability, and cost.
- Applies best practices for Iceberg table design, incremental processing, and query optimization across Hive, Impala, Snowflake, and RDBMS.
- Diagnoses systemic issues and drives remediation with platform teams.
Data Infrastructure (AWS)- Leads technical readiness across dev/test/prod environments.
- Works handson with AWS services including S3, Glue, Lambda, IAM, and SageMaker.
- Partners with governance and platform teams on access control, tagging, and operational support.
Data Modeling Formats- Leads modeling across RAW, CURATED, and SERVING layers.
- Applies dimensional or normalized models for correctness, performance, and usability.
- Implements efficient formats (Parquet + Iceberg) with clear schema evolution strategies.
DevOps CI/CD- Designs and improves Gitbased CI/CD pipelines and infrastructureascode using Terraform.
- Ensures quality gates, auditability, and compliance with governance requirements.
Stakeholder Engineering Leadership- Partners with product, analytics, and platform teams to align on requirements, data contracts, and SLAs.
- Communicates complex technical topics clearly and leads technical discussions.
- Coaches engineers and raises engineering standards through reviews and documentation.
AIFirst Product Mindset- Uses GenAIassisted development responsibly to accelerate delivery.
- Builds products, not just pipelines, focusing on usability, adoption, reliability, and lifecycle ownership.
- Designs systems endtoend and continuously optimizes costperformance tradeoffs using metrics.
- 8+ years of total experience with 6+ years ofData Engineering experience.
- Strong expertise inAWSbased data engineering and scalable cloud architectures
- Proven experience building endtoend batch and streaming pipelines, including Kafka
- Advanced proficiency in SQL, Hive, Impala, and PostgreSQL / RDBMS
- Strong programming skills in Python and PySpark
- Handson experience with AWS Glue, Lambda, S3, IAM, and SageMaker
- Experience with Snowflake and modern data warehousing
- Expertise in CI/CD, Terraform, and DevOps practices
- Proficiency in Airflow for workflow orchestration
- Experience with Power BI for data visualization and reporting
- Strong foundation in data modeling, performance optimization, and largescale data systems
Job Classification
Industry: BankingFunctional Area / Department: Engineering - Software & QARole Category: Software DevelopmentRole: Data EngineerEmployement Type: Full timeContact Details:
Company: CargillLocation(s): Bengaluru
Keyskills: Data Engineer Airflow S3 Query Optimization Performance Optimization Compliance Architecture Power Bi Postgresql Kafka Streaming Data Devops Sql Hive Iam Terraform Snowflake Data Warehousing Data Modeling Aws Lambda Python
Similar positions
Cargill
Cargill Business Services India Private Limited Cargill Business Services (CBS) provides high-quality shared services to Cargill. Since its inception in January 2015, CBS in India has grown steadily to establish itself as a global multi-function shared services hub providing services across mult...
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in