Senior Data Engineer
- Iris Software
- 3 - 8 years
- Noida, Gurugram
- 24 days ago
- Email to a friend
- Report this job
Job Description
Senior Data EngineerWe are seeking a skilled and dedicated hands-on Data Engineer to join our team. The successful candidate will have a strong background in data architecture, database management, and data pipeline development, working closely with our business stakeholders, clients, vendors, analysts and data scientists to ensure the smooth flow of data across the organization. You will play a key role in designing, developing, and maintaining robust and scalable data solutions that support critical business insights.
Responsibilities: - Data Infrastructure Management: Design, implement, and maintain efficient and scalable data infrastructure, including data lakes, data warehouses, and relational and NoSQL databases.
- Reporting: Maintain existing reports and data feeds. Optimize existing ones for resilience and spend. Build new reports and feeds into our internal and external customers.
- Data Pipeline Development: Develop, test, and maintain ETL (Extract, Transform, Load) pipelines to collect, process, and transfer data across different sources and destinations.
- Data Quality and Governance: Ensure data quality, consistency, and accuracy through data validation, monitoring, and management of data governance standards.
- Optimization: Optimize database queries, data storage, and retrieval for performance improvements.
- Collaboration: Work closely with data scientists, analysts, and product teams to understand data requirements and translate them into efficient technical solutions.
- Data Security: Implement and enforce data security best practices to ensure compliance with regulatory standards.
- Documentation: Document data architecture, processes, and infrastructure changes.
- Education: Bachelors or Masters degree in Computer Science, Data Science, Engineering, or a related field.
- Experience: 3+ years of experience in a data engineering role or similar.
- Technical Skills:
- Proficiency in SQL and experience with relational databases (e.g., MySQL, PostgreSQL).
- Strong programming skills in Java, Python and Spring Boot.
- Hands-on experience with big data processing tools (e.g., Apache Spark, Hadoop).
- Experience with cloud platforms (e.g., AWS, Google Cloud, Azure) and their data services.
- Familiarity with ETL tools (e.g., Apache NiFi, Airflow, Talend).
- Knowledge of data warehousing concepts (e.g., Redshift, Snowflake).
- Soft Skills: Strong analytical skills, problem-solving abilities, and excellent communication and teamwork skills.
- Experience with data modeling and database schema design.
- Familiarity with real-time data streaming (e.g., Kafka).
- Knowledge of machine learning infrastructure (e.g., MLflow, TensorFlow Extended).
- Experience working in the PC Insurance industry.
- Big Data - Big Data - Pyspark
- Data AI - ETL OTHERS - Snowflake
- Data Science and Machine Learning - Data Science and Machine Learning - Databricks
- Data AI - Data Engineering - Apache Kafka
- Programming Language - Python - Apache Airflow
- Database - Database Programming - SQL
- Data Science and Machine Learning - Data Science and Machine Learning - Apache Spark
- Data AI - Data Engineering - Data Quality Validation
- Data AI - Data Engineering - Stream Processing
- Beh - Communication and collaboration
Job Classification
Industry: IT Services & ConsultingFunctional Area / Department: Engineering - Software & QARole Category: Software DevelopmentRole: Data EngineerEmployement Type: Full timeContact Details:
Company: Iris SoftwareLocation(s): Noida, Gurugram+ View Contact
Keyskills: Data Engineer Airflow Tensorflow Compliance Ai Kafka Spring Boot Machine Learning Data Science Aws Python Java Azure Architecture Postgresql Hadoop Big Data Sql Apache Spark Nosql Mysql Business Insights Spark Databricks Etl
Fraud Alert to job seekers!
₹ Not Disclosed
Similar positions
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in