Sr.hadoop Developer Resume
New, JerseY
PROFESSIONAL SUMMARY:
- Hadoop Developer with almost 7+ years of IT experience in developing, delivering of software using wide variety of technologies in all phases of the development life cycle.
- Expertise in Java /Big data technologies as an engineer, proven ability in project based leadership, teamwork and good communication skills.
- Excellent knowledge in Hadoop Architecture, HDFS and its eco system like MapReduce (MR), Hive, Zookeeper, Pig, HBase, Sqoop, Oozie, Flume, Spark for data extraction, storage and analysis.
- Great knowledge in using Map Reduce programming modelfor analyzing the data stored in HDFS and experience inwriting Map Reduce codesin Java as per business requirements.
- Well experienced in Shuffling and Reducer and Combiner process along with custom Partitioning for efficient processing.
- Experience in analyzing data using HiveQL, Pig Latin, HBase and custom MapReduce programs in Java.
- Solid understanding of Hadoop MR V1 and MR V2 (or) YARN Architecture.
- Extensively worked on Hive for ETL Transformations and optimized Hive Queries
- Worked with relational database systems (RDBMS) such as MySQL, MS SQL, Oracle and NoSQL database systems like HBase and Cassandra. Extensively experience in working on IDEs like Eclipse, Net Beans and Edit Plus.
- Working knowledge of Agile and waterfall development models.
- Experienced in Application Development using Hadoop, Java, J2SE, J2EE, JSP, Servlets, Struts, RDBMS, Tag Libraries, JDBC, Hibernate, XML and Linux shell scripting.
- Implemented Java/J2EE Design Patterns like Business Delegate and Data Transfer Object (DTO), Data Access Object (DAO) etc.
- Worked with different software version control, bug tracking and code review systems like CVS, Source Depot, Bugger, TFS and Code Flow.
- Expertise in Object Oriented Analysis and Programming like UML and use of various Design Patterns.
- Worked onHadoopframework with Horton works platform (HDP).
- Well - disciplined hands-on coder in writinghigh quality, reusable and concise code compliant tocoding guidelines.
- An individual with excellent interpersonal and communication skills, strong business acumen, creative problem solving skills, technical competency, team-player spirit, and leadership skills.
TECHNICAL SKILLS:
Big Data: Hadoop, Map Reduce, HDFS, Hive, HBase, Pig, Sqoop, Oozie, Zookeeper, Flume, YARN, Storm, Spark, Mongo DB, Kafka, Cassandra and Impala.
Java/J2EE: Java, J2EE, JSP, JavaScript, Servlets, JDBC, Struts, Java Beans, JMS, EJB.
Frameworks: Apache Struts, Hibernate, Spring, MVC.
Languages: Core Java, J2EE, C, SQL, PL/SQL, UML.
WebService/ Technologies: REST, SOAP, JSP, JavaScript, XML, HTML, CSS, AJAX and JSON.
Databases: NoSQL, SQL Server, MySQL, Oracle, DB2, PL/SQL.
Web/Application Servers: Apache Tomcat, Web Logic, Web Sphere, JBOSS.
Tools: Eclipse, NetBeans and Edit Plus.
Build Tools: UML, Design Patterns, Maven, Ant.
Operating System: Linux, Windows XP/Vista/7/8.
Hadoop Distributions: Cloudera, Horton works, MapR.
PROFESSIONAL EXPERIENCE:
Confidential, New Jersey
Sr.Hadoop Developer
Responsibilities:
- Developed multiple MapReduce jobs in java for data cleaning and preprocessing.
- Analyzed large data sets by running Hivequeries and Shell scripts.
- Worked with the Data Science team to gather requirements for various data mining projects.
- Involved in creating Hivetables, loading with data and writinghive queries which will run internally in map reduce way.
- Involved in running Hadoop jobs for processing data coming from different sources.
- Involved in development and design of a Hadoop cluster using Apache Hadoop for POC and sample data analysis.
- Used Cassandra for high speed data ingestion.
- Importing and exporting data into HDFS and Hive.
- Implemented modules using Core Java APIs, Java collection, Threads and integrating the modules.
- Analyzed large and critical datasets using Cloudera, HDFS, Hbase, Map Reduce, Hive, Pig, Sqoop, and Zookeeper.
- Used HBase in accordance with Pig/Hive as per requirements
- Primary responsibilities include building scalable distributed data solutions using Hadoop ecosystem.
- Developed Simple to complex Map reduce streaming jobs using Java language that are implemented using Hive and Pig.
- Optimized Map Reduce Jobs to use HDFS efficiently by using various compression mechanisms.
- Analyzed the data by performing Hive queries (HiveQL) and running Pig scripts (Pig Latin) to study customer behavior.
- Used complex data types like bags, tuples and maps in Pig for handling data.
Environment: Shell, Hive, Map Reduce, Sqoop, LINUX, Cloudera, Java APIs, Java collection, JDBC, spring, REST, WSDL, WebLogic, Avro, HBase, Cassandra, Zookeeper and MySQL.
Confidential, Jacksonville, Florida
Big Data / Hadoop Developer
Responsibilities:
- Involved in discussions for setting up the64 node clusterand configuring the Hadoop platform.
- Migrating the needed data from MySQL in to HDFS using Sqoopand importing various formats of flat files into HDFS.
- Mainly worked onHive queriesto categorize data of different claims
- Integrated thehivewarehouse with HBase for information sharing among teams.
- Written customized Hive UDFsin Java where the functionality is too complex.
- Designed and created Hive external tablesusing shared meta-store and supported partitioning, dynamic partitioning forfaster data retrieval.
- HiveQLscripts to create, load, and query tables forextracting the summarized information
- Loaded unstructured data into hive tables.
- Used Avro’s thefile storage formatto save disk storage space.
- Supported Map Reduce Programs that are running on the cluster.
- Maintained System integrity of sub components primarily HDFS,MR,HBase, andHive.
- Monitored System health and logsand respond accordingly to any warning or failure conditions.
Environment: Apache Hadoop, HDFS, Hive, Map Reduce, HBase, Java, Pig, Sqoop, Cloudera CDH4, MYSQL, Tableau, Avro.
Confidential, Tulsa, OK
Hadoop Developer
Responsibilities:
- Installed and configured Hadoop MapReduce, HDFS and developed multiple MapReduce jobs in Java for data cleansing and preprocessing.
- Involved in loading data from UNIX file system to HDFS.
- Analyzed large amounts of data sets to determine optimal way to aggregate and report on it.
- Responsible for building scalable distributed data solutions using Hadoop.
- Responsible for cluster maintenance, adding and removing cluster nodes, cluster monitoring and troubleshooting, manage and review data backups, manage and review Hadoop log files.
- Spring IOC being used to inject the parameter values for the Dynamic parameters.
- EJB Session Beans being used to interact with Database using the JPA.
- Handled importing of data from various data sources, performed transformations using Hive, MapReduce, and loaded data into HDFS.
- Installed Oozie workflow engine to run multiple Hive.
Environment: Java 6, Eclipse, Oracle 10g, Sub Version, Hadoop, Hive, HBase, Oozie, Linux, MapReduce, HDFS, Hive, Hadoop Distribution of Cloudera, MapReduce, SQL server, Toad 9.6.
Confidential
Java/ J2EE Developer
Responsibilities:
- Involved in various phases of Software Development Life Cycle (SDLC) as design development and unit testing.
- Developed and deployed UI layer logics of sites using JSP, XML, JavaScript, HTML/DHTML, Ajax and CSS. CSS and JavaScript were used to build rich internet pages.
- Agile Scrum Methodology been followed for the development process.
- Designed different design specifications for application development that includes front-end, back-end using design patterns, UML.
- Developed proto-type test screens in HTML and JavaScript. Involved in developing JSP for client data presentation and, data validation on the client side with in the forms.
- Developed the application by using the Spring MVC framework.
- Developed data mapping to create a communication bridge between various application interfaces using XML, XSL, and XSLT.
- Developed JUnit testing framework for Unit level testing.
- Actively involved in code review and bug fixing for improving the performance.
- Created connection through JDBC. Creating JDBC statements to call stored procedures.
Environment: Struts, Hibernate, Servlets, UML JSP, XML, MVC, JavaScript, HTML/DHTML, Ajax, CSS, SQL server 2008R2 and Oracle 11g.
Confidential
Java Programmer
Responsibilities:
- Involved in Development of Servlets and Java Server Pages (JSP).
- Involved in writing Pseudo-code for Stored Procedures.
- Developed PL/SQL queries to generate reports based on client requirements.
- Enhancement of the System according to the customer requirements.
- Created test case scenarios for Functional Testing.
- Used Java Script validation in JSP pages.
- Helped design the database tables for optimal storage of data.
- Coded JDBC calls in the servlets to access the Oracle database tables.
- Responsible for Integration, unit testing, system testing and stress testing for all the phases of project.
- Prepared final guideline document that would serve as a tutorial for the users of this application.
Environment: Java, Servlets, J2EE, JDBC, Oracle, PL/SQL, HTML, JSP, Eclipse, UNIX.