Big Data Engineer Chennai | Pune
Chennai, PuneFull-time6–11 yrsPosted open since sept 2024
Senior data science role at Panasia HR Solutions in Chennai and Pune. Needs 6+ years.
About the role
We are looking for a core team member to join our Agile development team , where the primary responsibility will be to build backend data applications and services that support our Analytics platforms . The role will require hands-on expertise in technologies such as SCALA, SPARK, PYTHON, and SHELL SCRIPTING , as well as a basic understanding of AWS cloud services for ongoing cloud migration efforts .
What you'll do
- As a core member of our Agile team , you will contribute to the design and development of backend data applications
- These applications and services will be the foundation for our Analytics platforms , ensuring that the data processing systems are reliable, scalable
- A key part of your role will be applying your hands-on expertise in SCALA, SPARK, PYTHON , and SHELL SCRIPTING to develop and maintain data processing pipelines
- Be tasked with building robust back-end solutions and implementing best practices for the technologies mentioned
- Your experience will enable you to address complex data challenges, process large datasets, and optimize data flows for faster, more efficient processing
What you need
- Python
- AWS
- Spark
- ETL
- Shell scripting
- Jenkins
Good to have
- CI/CD