Programming Languages: Spark 3.x on YARN or Kubernetes, Python, Java, sqoop, Scala, Map Reduce, shell scripting and HIVE and Hive on Tez.
Platform/Environment: Hadoop
Database Management System: HBASE, SQL
Application Packages, etc.:
Years of experience on each of the Technical must have skills: 6+ Yrs
Any Certifications required: No
Detailed Job Description:
Participate in the entire lifecycle, from design through the development and test processes to production support.
Tasked with testing and maintaining highly scalable data processing pipelines using BigData technologies and SOA architecture.
Plan, create, articulate, execute, review and refactor manual and automated test cases.
Lead root cause analysis and track defect resolution to completion.
Write utilities and automation tools using Java, Hive, and Shell scripts.
Participate in cross-group and internal customer feature demos.
Implement continuous integration, testing, deployment practices and automation frameworks to increase productivity.
Responsible for identifying opportunities for further enhancements and refinements on standards, processes, application functionalities, and operation excellence.
Experience in large volume data / ETL testing, writing complex SQL queries for data analysis, hands on experience navigating Hadoop, testing and troubleshooting programming constructs in Spark 3.x on YARN or Kubernetes, Python, Java, sqoop, Scala, Map Reduce, shell scripting and HIVE and Hive on Tez.
Experience on Unix and shell scripting, test automation and writing test tools and test data creation & maintenance
Experience in Drools rules engine, JIRA, Jenkins CI/CD is a plus
