LTIMindtree

Associate Principal - Data Engineering

  • Sarasota, FL
  • Posted 2 days ago
  • Staffing / consulting
Apply on LTIMindtree's site ↗Checked today on the employer's career site. We never take applications.

Experience 11to15 years

RoleTechnical Lead

Role Summary

We are looking for an experienced Databricks Technical Lead with 11 to15 years of overall experience and strong expertise in designing and delivering enterprisescale data engineering solutions The candidate should have handson experience with Databricks Lakehouse architecture Delta Lake PySpark Unity Catalog and cloudnative data services with the ability to build secure scalable and productionready data platforms

Primary Skills

Databricks

PySpark Spark

Delta Lake and Lakehouse Architecture

Bronze Silver and Gold data architecture

Databricks Workflows

Unity Catalog

Databricks Serverless Compute

SQL and Data Modeling

Cloud Data Engineering preferably AzureAWS

Data Pipeline Development and Optimization

Data Quality Monitoring and Observability

CICD and DevOps for Data Engineering

Key Responsibilities

Design develop test deploy and support enterprisescale data products data pipelines and integration solutions using Databricks and cloudnative technologies

Build and maintain Bronze Silver and Gold data layers within a Lakehouse architecture using Delta Lake PySpark Databricks Workflows Unity Catalog and serverless compute

Develop scalable data transformation validation and enrichment pipelines supporting analytics reporting operational and AIML workloads

Implement robust data quality monitoring reconciliation and observability controls to ensure trusted and reliable data products

Design and maintain enterprise data models business rules and reusable data engineering frameworks aligned with architectural standards

Work closely with solutiondata architects product owners business stakeholders and engineering teams to translate requirements into scalable technical solutions

Conduct and participate in technical design reviews code reviews unitintegration testing and CICD processes

Troubleshoot complex pipeline platform and performance issues and optimize Spark workloads for performance reliability and cost

Implement data governance security lineage and rolebased access controls using Unity Catalog and cloudnative security capabilities

Automate deployment testing dataquality validation monitoring and operationalsupport activities

Create and maintain technical designs data mappings operational procedures engineering standards and support documentation

Mentor engineers and provide technical leadership around Databricks Spark data engineering and software engineering best practices

Continuously evaluate newer Databricks and cloud capabilities to improve scalability automation performance and maintainability

Secondary Skills

Cloud Microsoft Azure or AWS

Azure ADLS Gen2 Azure Data Factory Key Vault Entra ID

AWS S3 IAM Glue and related data services

Programming Python SQL

DevOps Azure DevOps GitHub Actions Jenkins

Infrastructure as Code Terraform

Data Modeling Dimensional modeling StarSnowflake schemas

Streaming Apache Kafka Spark Structured Streaming

Orchestration Databricks Workflows ADF or Airflow

Data Governance Data lineage RBAC data classification and metadata management

Engineering Git automated testing CICD and AgileScrum

Exposure to MLflow Databricks SQL and AIML workloads would be advantageous

Qualifying Questions

1 Databricks Architecture

Have you designed and implemented an enterprise Databricks Lakehouse solution using Bronze Silver and Gold layers Please briefly explain how you used Delta Lake PySpark and Unity Catalog in the solution

Qualify if Candidate demonstrates handson ownership of Medallion architecture Delta tables transformation pipelines and Unity Catalog rather than only theoretical knowledge

2 Performance Production Troubleshooting

Describe a production DatabricksSpark pipeline performance issue you resolved What was the root cause and what specific optimization techniques did you use

Qualify if Candidate can discuss practical techniques such as partitioning shuffle optimization skew handling Spark execution plans join strategies caching file sizingcompaction Photon or clusterserverless optimization

3 Governance CICD

How have you implemented enterprise data governance and automated deployment in Databricks using Unity Catalog and CICD

Qualify if Candidate demonstrates experience with catalogsschemas RBAC or privileges lineage service principalsmanaged identities environment promotion Gitbased version control and automated deployment

Mandatory Skills : Databricks

Variable Compensation: This position includes a bonus or variable payout component based on individual and organizational performance. Actual compensation within the range will be dependent upon the individual's skills, experience, performance and internal equity.

Benefits/perks listed below may vary depending on the nature of your employment with LTIMindtree (“LTIM”):

Benefits and Perks:

  • Comprehensive Medical Plan Covering Medical, Dental, Vision
  • Short Term and Long-Term Disability Coverage
  • 401(k) Plan with Company match
  • Life Insurance
  • Vacation Time, Sick Leave, Paid Holidays
  • Paid Paternity and Maternity Leave

The range displayed on each job posting reflects the minimum and maximum salary target for the position across all US locations. Within the range, individual pay is determined by work location and job level and additional factors including job-related skills, experience, and relevant education or training. Depending on the position offered, other forms of compensation may be provided as part of overall compensation like an annual performance-based bonus, sales incentive pay and other forms of bonus or variable compensation.

Disclaimer

this posting. LTIMindtree

information, atypical hereditary cellular or blood trai where such considerations are bona fide occupational qualifications permitted by law.

Similar open jobs

All H-1B data engineer jobs in Florida →