JAVA + APACHE SPARK DEVELOPER
Snowflake / Databricks | Enterprise Data Engineering
Experience
5-9 years
Location
New York
Work model
Hybrid: 3 days in office per week
Employment
Full-time / Project deployment
Role Overview
We are seeking a hands-on Java and Apache Spark Developer to build, enhance and support scalable data-processing solutions on modern enterprise data platforms. The role requires strong Java engineering, distributed data-processing expertise and practical experience with either Snowflake or Databricks. The engineer will work across design, development, testing, optimisation, deployment and production support in an Agile environment.
Key Responsibilities
• Design, develop, test and maintain high-volume batch and near-real-time data-processing applications using Java, Apache Spark and Spark SQL.
• Build resilient, reusable and maintainable data pipelines for ingestion, transformation, enrichment and distribution across enterprise systems.
• Develop Java-based services, utilities and integration components that support data workflows and downstream consumers.
• Implement data solutions on Snowflake and/or Databricks, including transformations, curated datasets and consumption-ready data structures.
• Optimise Spark jobs through appropriate partitioning, caching, broadcast strategies, join selection, shuffle reduction, resource tuning and data-skew handling.
• Write complex SQL for transformation, validation, reconciliation and performance-sensitive workloads.
• Partner with architects, business analysts, product owners and engineering teams to translate requirements into practical technical designs.
• Implement data-quality controls, logging, monitoring, ing, lineage and operational support features.
• Create unit, integration, regression and performance tests; support defect resolution, release validation and production deployment.
• Contribute to code reviews, CI/CD pipelines, technical documentation, runbooks and engineering standards.
• Troubleshoot production issues across application code, Spark execution, SQL workloads, data pipelines and platform integrations.
Mandatory Skills and Experience
• 5-7 years of relevant experience in data engineering, backend development or large-scale data-processing applications.
• Strong hands-on programming experience in Java, including object-oriented design, collections, exception handling, multithreading and performance-conscious coding.
• Strong practical experience with Apache Spark using Java, Spark SQL and DataFrame/Dataset APIs.
• Hands-on experience designing and tuning distributed data-processing jobs for scalability, reliability and efficient resource utilisation.
• Hands-on experience with at least one modern data platform: Snowflake or Databricks.
• Strong SQL skills, including complex joins, CTEs, window functions, aggregations, query optimisation and data validation.
• Experience developing ETL/ELT pipelines and working with structured and semi-structured data.
• Experience with Git-based development, code reviews, build tools and CI/CD practices.
• Good understanding of software development lifecycle, Agile delivery, testing and production-support practices.
• Strong analytical, debugging, communication and stakeholder-collaboration skills.
Platform-Specific Expectations
Development
SQL transformations, views, stages, streams/tasks or equivalent platform objects
Notebooks/jobs, Spark workloads, Delta Lake tables and platform workflows
Performance
Warehouse/query optimisation, clustering/partition awareness and workload efficiency
Spark job tuning, partitioning, caching, shuffle optimisation and cluster efficiency
Operations
Loading, monitoring, validation, access controls and production support
Job orchestration, monitoring, data-quality checks, access controls and production support
Preferred Skills
• Experience with Spring Boot, REST APIs or Java-based microservices.
• Exposure to Kafka or another enterprise messaging/streaming platform.
• Experience with cloud services on Microsoft Azure, AWS or Google Cloud Platform.
• Knowledge of Python or PySpark for data engineering and automation.
• Experience with orchestration tools such as Apache Airflow, Azure Data Factory or equivalent.
• Familiarity with data modelling, data governance, lineage, security and regulated-industry controls.
• Exposure to financial services, risk, regulatory or capital-markets data is advantageous.
Technology Landscape
Programming
Java; optional Python
Data processing
Apache Spark, Spark SQL, DataFrame/Dataset APIs
Data platforms
Snowflake and/or Databricks; Delta Lake where applicable
Integration
REST APIs, Kafka or equivalent messaging tools
Engineering toolchain
Git, Maven/Gradle, Jenkins/Azure DevOps or equivalent CI/CD tools
Cloud
Microsoft Azure, AWS or Google Cloud Platform
Ways of working
Agile/Scrum, code reviews, automated testing and production support
Education and Work Arrangement
• Bachelor's or Master's degree in Computer Science, Information Technology, Engineering or a related discipline.
• Must be willing to work from the Mumbai East - Goregaon office at least 3 days per week.
• Must be comfortable collaborating with distributed teams and supporting business-critical applications as required.
Equal Opportunity Statement
We are an equal opportunity employer and are committed to an inclusive workplace. Employment decisions are based on role requirements, skills, experience and business needs.
Salary range $63000 to $134500