Java Spark developer
ChennaiFull-time3–6 yrsPosted 1 week ago
Mid-level data science role at Match Me Resume Match | Citi in Chennai. Needs 3+ years and a B.Tech, B.E. or B.Sc.
About the role
We are seeking an experienced and motivated Java Spark Developer to design, develop, and maintain high-performance, large-scale data processing pipelines and distributed applications. In this role, you will leverage Core Java and Apache Spark to build resilient batch and real-time streaming data architectures, optimize distributed data workloads, and collaborate with cross-functional teams including Data Scientists, Cloud Engineers, and Solution Architects.
What you'll do
- Leverage Core Java and Apache Spark to build resilient batch and real-time streaming data architectures, optimize distributed data workloads
- Design, implement, and maintain robust, scalable data ingestion and ETL/ELT pipelines using Apache Spark (Core, SQL
- Develop performant, low-latency microservices and distributed processing modules integrated with messaging platforms (e.g., Apache Kafka)
- Build and maintain interfaces to relational databases, distributed data lakes, and NoSQL stores (e.g., Hive, Cassandra, HBase, MongoDB, Delta Lake, Snowflake)
- Analyze query execution plans, DAGs, and Spark UI metrics to eliminate data skew, reduce shuffle overhead, and minimize bottleneck latencies
What you need
- Spark
- Java
- SQL
- AWS
- Kafka
- Hadoop
Good to have
- Python
- Databricks
- BigQuery
- Airflow
- NoSQL
- Unit testing