# Lead Data Engineer – Data Engineering, Analytics & AI — Carlyle

- Location: New York, NY
- Posted pay: $180k–$200k/yr
- Posted: Nov 16, 2026
- Status: open (checked Nov 20, 2026)
- Apply (employer's site): https://carlyle.wd1.myworkdayjobs.com/Carlyle/job/New-York-NY/Associate-Vice-President_R-00190
- Employer record: https://visajobs.io/company/carlyle-group-employee.md

## Sponsorship history
- Tier: Good history
- Occupation (SOC): Software Developers (15-1252)
- Employer's certified LCAs for this occupation in NY, FY2024–FY2026: 4
- …in the whole US: 5
- With this exact title: 0
- Median DOL prevailing wage, level I / II: $95k / $124k

## Description
Position Summary
We are seeking a highly experienced, hands-on Data Engineering Lead to design, develop, and own end-to-end enterprise data solutions supporting business applications, analytics, reporting, and AI initiatives.
The ideal candidate brings strong technical expertise in modern data engineering, particularly Snowflake, dbt, Python, SQL, and orchestration frameworks, along with the ability to independently architect and deliver scalable production solutions.
This is a hands-on technical leadership role requiring active involvement in solution design, coding, troubleshooting, engineering standards, and delivery. The individual will work closely with business, analytics, AI, and technology teams to build reliable, reusable, and future-ready data products.
A technically strong, delivery-oriented engineering lead who can take a complex business problem, independently design the end-to-end data solution, develop critical components, guide implementation, and own the solution through production.
The candidate should be equally comfortable writing advanced SQL and Python, designing enterprise data models, troubleshooting production pipelines, collaborating with analytics teams, and enabling emerging AI use cases.
This is not a purely managerial or architecture-only role. Continued hands-on engineering involvement is essential.
In-Office Requirement: 4 days a week
Primary Responsibilities
End-to-End Solution Ownership: Lead the architecture, design, development, deployment, and ongoing support of enterprise data engineering solutions, from source-system integration through data consumption.
Data Ingestion & Integration: Design and implement scalable ingestion frameworks supporting databases, enterprise applications, APIs, SaaS platforms, files, and third-party data providers.
Modern Data Pipelines: Build and optimize batch and incremental ETL/ELT pipelines, including change data capture (CDC), with the ability to evaluate and implement near-real-time and event-driven ingestion patterns as business needs evolve.
Data Transformation & Modeling: Develop reusable dbt transformations, dimensional models, fact and dimension tables, bridge tables, and curated data marts supporting enterprise reporting and analytics.
Snowflake Engineering: Design and optimize Snowflake data solutions, including complex SQL transformations, query tuning, workload performance, data sharing, and cost optimization.
Workflow Orchestration: Develop, manage, and troubleshoot data workflows using managed Apache Airflow or equivalent orchestration frameworks, including dependencies, scheduling, retries, failure recovery, and operational monitoring.
Data Quality & Reliability: Establish automated data validation, reconciliation, freshness monitoring, alerting, observability, lineage, and production support standards.
Analytics Enablement: Partner with BI and analytics teams to deliver performant, governed datasets and reusable business metrics for dashboards, reporting, and self-service analytics.
AI Enablement: Collaborate with AI engineering teams to build trusted, secure, AI-ready data products, including structured and unstructured data integration, reusable data-access interfaces, and data services for AI applications.
Engineering Excellence: Implement Git-based development, CI/CD, automated testing, code reviews, documentation, and consistent engineering standards across data pipelines and models.
Technical Leadership: Lead technical design discussions, mentor engineers, resolve complex engineering challenges, and take direct accountability for successful production delivery while remaining actively involved in coding and implementation.
Requirements
Education & Certificates
Bachelor’s Degree, required
Degree in Computer Science, Data Science, Finance, Business, or a related field, preferred
Professional Experience
Minimum 8+ years of professional experience in data engineering, data warehousing, or related software engineering roles, with significant hands-on technical leadership experience.
Proven experience independently designing, building, deploying, and owning end-to-end data engineering solutions.
Hands-on experience developing and maintaining ETL/ELT pipelines, including batch ingestion, incremental processing, source-to-target mapping, and data reconciliation.
Experience with Apache Airflow, including managed Airflow environments, or equivalent production orchestration frameworks.
Strong experience integrating multiple data sources through databases, REST APIs, enterprise applications, and file-based ingestion.
Experience designing solutions for large datasets, complex transformations, schema changes, historical tracking, and efficient data processing.
Demonstrated ability to lead engineers technically while remaining hands-on with development, code reviews, debugging, and problem resolution.
Experience with Fivetran, SnapLogic, or similar enterprise data integration platforms.
Experience designing near-real-time or streaming ingestion solutions, including CDC, event-driven architectures, Kafka, Snowpipe Streaming, or equivalent technologies.
Experience with Power BI, including semantic models, DAX, Import versus DirectQuery, refresh strategies, and performance optimization.
Experience using AI-assisted development tools, such as Cursor, GitHub Copilot, or equivalent tools, to improve engineering productivity.
Experience with Snowflake Cortex, Snowpark, Dynamic Tables, Streams, Tasks, or Openflow.
Experience with data observability, cataloging, lineage, and governance platforms such as Atlan, Monte Carlo, or equivalent solutions.
Experience integrating Salesforce, CRM systems, financial applications, or other enterprise SaaS platforms.
Financial services or investment management industry experience.
Competencies & Attributes
Strong hands-on expertise in Snowflake, advanced SQL, Python, and dbt, including production development, performance optimization, and troubleshooting.
Strong understanding of enterpr

> History, not a promise: past H-1B filings do not mean an employer will sponsor you or a given role. Data: U.S. DOL OFLC LCA disclosure (FY2024–FY2026 for role counts, certified LCAs) and the USCIS H-1B Employer Data Hub. Not legal advice. Methodology: https://visajobs.io/methodology
