Big Data/hadoop Developer Resume
NY
SUMMARY:
- Around 10 years of extensive Experience in IT Industry in the field of web applications, distributed application development with a zest for Big Data technologies.
- Cloudera Certified Hadoop Developer with more than two Year of experience in BigData & HADOOP ecosystem (HDFS, MapReduce MRV1,MRV2, HBase, Pig, Hive, Flume, Sqoop, MRUnit).
- Deep Experience in Hadoop architecture and various components such as HDFS, Job Tracker, Task Tracker, Name Node, Data Node & MapReduce programming paradigm.
- In depth knowledge of HDFS file system and customizing it by writing custom data types, input and output formats.
- Good experience in optimizing MapReduce algorithm using combiners and practitioners to deliver the best results. Experience in Hadoop unit Testing Tools MRUnit.
- Good Experience in data loading from RDBS system to HDFS system using SQOOP and Flume.
- Experience in analyzing data using HIVEQL, PIG Latin and custom MapReduce programs in JAVA. Extending HIVE and PIG core functionality by using custom UDF’s.
- Good Experience on HBase (NoSQL databases). Worked on Java API to handle Hbase Timestamp for segmentation Data. Performance tuning of HDFS DFS block size based on Hbase Region Servers. Worked on MongoDB.
- Has very good working experience with AWS (Amazon Web Services). Managed multiple instances in EC2 with load balancing and auto scaling features. Worked on managing the data in S3 on AWS.
- Good Experience in AGILE methodology & SCRUM model to deliver software services. Experience in all phases of SDLC including analysis, Design, Development, Coding and testing.
- Good Experience in Developing JAVA and J2EE technologies using JSP, JDBC and open source like Struts, Spring, Hibernate and JUnit.
- Experience in developing RESTful WebServices.
- Research, evaluate and utilize new technologies/tools/frameworks around Hadoop Eco system.
- Experience in Configuration Management tools like GitHub, Mercurial & Subversion.
- Experience in continuous Integration process and build management tools like Jenkins.
- Experience in using tools Log4J, Unit Testing Tools JUnit, and Build Tools such as Apache Maven.
- Good Implementation Knowledge in J2EE Design Patterns.
- Experience in Onsite and Offshore development model with business Analysis, Requirement Gathering and Client Interaction.
- Strong analytical and problem - solving skills with good interpersonal and communication skills.
TECHNICAL SKILLS:
Hadoop Core: HDFS, MapReduce (MRV1 & MRV2), YARN 2.5.1
Hadoop Ecosystem: Hive, Pig, Hbase, Sqoop, Flume, ZooKeeper, Oozie, MRUnit, Spark
Hadoop Cluster: Cloudera CDH3/4, Hortonworks HDP 2.1, Amazon EC2
DataBase: HBase, MongoDB, NOSQL, Sql Server, MySQL
JAVA / J2EE: JAVA 5.0/ 6.0/ 7.0, JDBC, Servlets, JSP.
Frameworks: Apache Struts 1.2, 2, Spring 3.
Markup languages: HTML, XML (SCHEMA, DTD, SAX, DOM, XSLT) and Ajax.
Application/Web Servers: Jetty 7, JBoss 4, Apache Tomcat 5-7, WebLogic 8.1, Apache 2.2.x.
Versioning tools: GitHub, Mercurial, SVN, CVS.
Others: JUnit, Maven, Ant, Sonar, Yourkit profiler, Jmeter, MarkLogic, XQuery,Python, Eclipse (IDE)
PROFESSIONAL EXPERIENCE:
Confidential
Big Data/Hadoop Developer
Environment: Process: Processes of Agile Methodology & Scrum model. /Tech Used: Hadoop & Yarn 2.5.1, Hive 0.13.0, Pig 0.12.1, Sqoop 1.4.4, HBase 0.98.0 Oozie 4.0.0, Zookeeper 3.4.5, MRUnit 1.1.0, Hortonworks HDP 2.1Cloudera CDH 4 Distribution, JDK 1.7, JUnit 4.11, Ambari, Tez, HDFSEclipse, Maven, Mercurial, Jenkins, ReviewBoard, RedMine
Responsibilities:
- Developed various Map Reduce Programs for Segmentation Ingestion & Retrieval Job, those are running on the cluster.
- Developed UDFs for Pig Data Analysis. Developed custom Java code to call the Pig Job who runs the custom Pig UDF on runtime through REST WebServices. Installed and configured Pig and also written PigLatin scripts.
- Worked on Java API to handle HBase Timestamp for segmentation Data. Created HBase tables and wrote multiple queries to get the data with the latest timestamp for data analysis to meet the business requirements.
- Performance Optimization of Hadoop cluster based on job requirements. Runtime optimization based on HDFS Block Size to handle the millions of rows in multiple Hbase Region Servers.
- Involved in performance tuning of MapReduce code, Pig scripts and HBase Queries.
- Involved in installing Hadoop developer machine and Ecosystem components.
- Developed Scripts and Batch Job to schedule various Hadoop Program.
- Used MRUnit & JUnit for Hadoop/Java unit testing.
- Installed Oozie workflow engine to run multiple Hive and Pig jobs.
- Installed and configured Hive and also written Hive UDFs. Created Hive tables and working on them using Hive QL.
- Utilized Agile Scrum Methodology to help manage and organize a team of 6 developers with regular code review sessions.
- Used Mercurial & GIT for version control and created automated build scripts. Use Maven for continuous Integration. Used SONAR as code standard tool. Used ReviewBoard for code review tools.
Confidential, NY
Big Data/Hadoop Developer
Environment: Process: Processes of Agile Methodology & Scrum model. /Tech Used: Hadoop 2.0, MapReduce, HDFS, Hive 0.9, Pig 0.10, Sqoop 1.4, Oozie 3.2.0 Cloudera CDH3/4 Distribution, MRUnit, JUnit, Eclipse, Maven, GIT, Jenkins
R esponsibilities:
- Involved in moving all log files generated from various sources to HDFS for further processing. Responsible to manage data coming from different sources.
- Handled 2 TB of data volume and implemented the same in Production.
- Developed and Supported Map Reduce Programs those are running on the cluster.
- Installed and configured Pig and also written PigLatin scripts.
- Developed UDFs for Pig Data Analysis.
- Involved in installing Hadoop developer machine and Ecosystem components.
- Performance Optimization of Hadoop cluster based on job requirements.
- Installed and configured Hive and also written Hive UDFs. Created Hive tables and working on them using Hive QL.
- Wrote Hive queries for data analysis to meet the business requirements.
- Developed Scripts and Batch Job to schedule various Hadoop Program.
- Utilized fully Agile Scrum Methodology, Involved in distributed Scrum and also participate as Scrum master.
- Used MRUnit & JUnit for Hadoop/Java unit testing.
- Analyzed the data by performing Hive queries and running Pig scripts to know user behavior.
- Used GIT for version control and created automated build scripts. Use Maven for continuous Integration.
- Installed Oozie workflow engine to run multiple Hive and Pig jobs.
Confidential, NY
Tech Lead Developer
Environment: Struts 1.2, JDBC, JUnit4, Spring 3, JAX-RSDatabase: MSSQL, NOSQL - Mongo DB, MarkLogicApp Server: Jetty 7Others: Eclipse, Sonar, JIRA, Hudson (CI), GitHub, Selenium, cucumber, Yourkit profilerProcess: Processes of Agile Methodology.
Responsibilities:
- Responsible for coding various interfaces and classes, DAO, coding Junits, and doing the code review as per best practice.
- Developed various Business logic, Dynamic web pages, XQuery module for Marklogic and also involved in writing few python scripts.
- Requirement analysis, Client interaction, client calls, coordinating with Client and developer regarding the analysis of requirement and the possible way of implementing technically.
- Being senior developer, coordinating with developers to resolve any impediments (technical or requirements) to ensure that tasks assigned to them are delivered as per timelines.
- Following the execution of different releases and updating client with status. Making sure all deliveries comply with the checklist provided by client and are timely released.
- Responsible for releasing Build for QA on different testing servers and Staging. Coordination with QA team for the automation of projects and its integration with Hudson.
- Involved in Object Oriented Design/Object Oriented Analysis with usage of UML Sequence Diagrams and Class Diagrams.
- Implemented MVC architecture by developing struts framework.
- Struts Validator framework is used to do client side validation.
- Implemented the Spring dependency injection of the Database helper instance to the action objects.
- Wrote Action classes, Business Objects and Service classes.
- Configured Struts-config.xml with all the mappings required by the architecture.
- Involved in developing custom Velocity, Servlets, DB2 - stored procedures
- Created detail design documents which has the UML Design diagrams, table information, object model etc. which are required for the developer
- Worked with various version control tools like GIT and Subversion (SVN).
- Prepare Java/J2EE development structure for Maven.
Confidential
Senior Developer
Environment: JAX-RS, Java 1.6, Spring-Jdbc,JerseyFramework, Junit4, Google Guice, MSSQL Others: GitHub, Sonar, JIRA, Hudson (CI), Jetty 7
Responsibilities:
- Analysis and study of the requirement specifications.
- Worked on this product development from scratch, developed Business logic using Core Java and writing SQL script. Implemented as per JAX-RS specification.
- Coding & development of DAO, and their implementations With Spring JDBC.
- Unit testing (J-Unit), Integration testing and functional testing.
- Involved in the design of the project and represented the requirements through the diagrams using UML.
- Involved in the Database design, creation, Querying. Involved in creation of Stored Procedures.
- Prepared proposal for implementing content based manuscript creation for Emerald’s site.
Confidential
Senior Developer
Environment: Struts 1.2, Java 1.6, JPA with Hibernate, WRAP, Jetty 6, SQL-Server, JUnit4
Responsibilities:
- Coding & development of DAO, Implemented Hibernate.
- Unit testing (J-Unit), Integration testing and functional testing.
- Prepared proposal for implementing TBA for nature’s site.
- Developed pom.xml for the build of the application using Maven
- Code review and Performance tuning.
Confidential
Senior Team Member
Environment: Velocity, Servlets, MSSql, Struts, JBoss, Mercurial, JIRA, Bugzilla
Responsibilities:
- Responsible for developing new module using Struts Action classes and Velocity.
- Responsible for implementing delegate classes and facades
- Responsible for writing the Service classes.
- Designed and developed a new database for the project.
- Implemented MVC architecture by developing struts framework.
- Struts Validator framework is used to do client side validation.
- Implemented the Spring dependency injection of the Database helper instance to the action objects.
- Wrote Action classes, Business Objects and Service classes.
Confidential
Senior Team Member
Environment: Velocity, Servlets, MSSql, Struts, JBoss,Mercurial, JIRA, Bugzilla, EJB
Responsibilities:
- Responsible for developing new module using Struts Action classes and Velocity.
- Responsible for maintenance and enhancement of the WebAdmin module.
- Responsible for implementing delegate classes, facades and Service classes.
- Implemented MVC architecture by developing struts framework.
- Struts Validator framework is used to do client side validation.
- Implemented the Spring dependency injection of the Database helper instance to the action objects.
- Wrote Action classes, Business Objects and Service classes.
- Configured Struts-config.xml with all the mappings required by the architecture.
- Involved in developing custom JSP, Servlets, EJB’s and DB2 - stored procedures
- Writing Python script to generate various User reports in CSV format.
Confidential
Team Member
Environment: Win 2000, JSP, EJB, Jdbc, Servlet, Mercurial, WebLogic, Struts and Oracle
Responsibilities:
- Responsible for writing Struts Action classes and JSPs.
- Involved in writing presentation layer using jsp’s.
- Responsible for writing the Enterprise Beans.
- Responsible for implementing delegate classes and facades.
- Attended 2 months Java provided by company.