Databricks - Lead
- Iris Software
- 5 - 7 years
- Noida, Gurugram
- 1 month ago
- Email to a friend
- Report this job
Job Description
Job DescriptionKey ResponsibilitiesPlatform Administration Configuration
- Manage and maintain the AWS Databricks platform, including cluster configuration, workspace setup, and compute resources.
- Implement and optimize Databricks configurations based on business and security requirements.
- Work closely with architects to understand system requirements and make necessary configuration adjustments.
- Monitor performance, availability, and cost of Databricks resources, optimizing where necessary.
User Access Management Security
- Develop and manage user access controls, ensuring least privilege principles.
- Define and enforce Identity and Access Management (IAM) policies in AWS and Databricks.
- Work with security teams to audit user access, permissions, and data security policies.
- Manage integration with Azure AD, Okta, or other authentication providers for Single Sign-On (SSO).
User Persona Development
- Define and manage user personas for different types of Databricks users, including data engineers, data scientists, analysts, and administrators.
- Assign appropriate permissions and roles based on business needs and compliance policies.
- Develop governance frameworks to ensure proper access provisioning and de-provisioning.
Collaboration Support
- Work closely with architects, data engineers, and security teams to ensure best practices are followed.
- Support troubleshooting, debugging, and issue resolution related to Databricks performance, security, and configuration.
- Document platform configurations, best practices, and user guidelines.
- Provide training and support to Databricks users across the organization.
Monitoring Compliance
- Set up monitoring and alerting mechanisms for performance, security, and cost management.
- Ensure compliance with industry regulations (e.g., GDPR, HIPAA) and internal security policies.
- Perform regular audits and reviews to maintain platform security and compliance.
Required Skills Qualifications
- 5-7 years of experience as a Databricks Administrator or similar role.
- Strong experience with AWS services (IAM, S3, EC2, Lambda, Glue, etc.).
- Expertise in Databricks administration, workspace management, and security configurations.
- Hands-on experience with AD groups, user access management, RBAC, and IAM policies.
- Experience in developing and managing user personas within enterprise environments.
- Strong understanding of network security, authentication, and data governance.
- Proficiency in Python, SQL, and Spark for troubleshooting and automation.
- Familiarity with Terraform, CloudFormation, or Infrastructure as Code (IaC) is a plus.
- Knowledge of CI/CD pipelines and DevOps best practices is desirable.
- Excellent communication and documentation skills.
Preferred Certifications
- AWS Certified Solutions Architect Associate / Professional
- Databricks Certified Data Engineer / Administrator
- Certified Information Systems Security Professional (CISSP) Nice to have
Mandatory Competencies
- Data Science and Machine Learning - Databricks
- Big Data - PySpark
- Data AI - Data Engineering - Apache Kafka
- Cloud - AWS - AWS S3, S3 glacier, AWS EBS
- Beh - Communication and collaboration
Job Classification
Industry: IT Services & ConsultingFunctional Area / Department: Engineering - Software & QARole Category: Software DevelopmentRole: Technical LeadEmployement Type: Full timeContact Details:
Company: Iris SoftwareLocation(s): Noida, Gurugram
Keyskills: databricks s3 python issue resolution data engineer ai machine learning sql identity and access management lambda data science iam spark ec2 compliance devops kafka debugging big data aws cloudformation azure
Similar positions
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in