Virtusa

Lead Software Engineer

  • New York
  • full time
  • Posted 2 days ago
  • Staffing / consulting
Apply on Virtusa's site ↗Checked today on the employer's career site. We never take applications.

Key Responsibilities

Data Platform Architecture & Pipeline Engineering

Design & Build: Develop robust, scalable, and automated Lambda/Kappa data pipelines for both batch and real-time streaming data using GCP Dataflow (Apache Beam) and Cloud Data Fusion. Data Warehousing: Optimize and manage enterprise data warehouses in Google BigQuery, ensuring efficient storage, partitioning, clustering, and query performance. Orchestration: Work with orchestration tools like Cloud Composer (Apache Airflow) to manage complex data workflows and dependencies. CI/CD & DevOps: Implement Infrastructure as Code (IaC) using tools like Terraform and maintain CI/CD pipelines for seamless data deployment.

Business Intelligence & Enablement

Semantic Modeling: Collaborate with analytics teams to model data and develop robust LookML models within Looker. Performance Tuning: Optimize Looker queries and BigQuery costs by implementing caching strategies, materialized views, and efficient BI Engine configurations.

Data Governance & Quality (Preferred)

Compliance & Security: Implement data governance frameworks on GCP using Dataplex, Cloud Data Loss Prevention (DLP), and IAM policies to ensure data privacy and compliance. Data Lineage & Cataloging: Establish and maintain data catalogs, metadata management, and end-to-end data lineage. Data Quality: Build automated testing frameworks to monitor data quality, anomalies, and pipeline health.

Qualifications & Skills

Required Technical Skills : GCP Expertise: Minimum of 3–5 years of hands-on experience building production-grade data solutions on Google Cloud Platform. Core Tools: Deep, practical knowledge of BigQuery (SQL optimization, slot management) and Dataflow (Apache Beam in Python or Java).

Similar open jobs

All H-1B software engineer jobs in New York →