Hadoop Developer Resume
San Jose, CA
PROFESSIONAL SUMMARY:
- Having 7+ years ofexperience in IT / Software Development acquiring rich experience in working diverse domain projects withJava/J2EE and Big Data technologies using Hadoop Ecosystem
- 3+ years of extensive hands on experience in architecture, design and development of Big data solutions using Hadoop, HDFS, Map Reduce, HBase, Pig, Hive, Sqoop, Oozie,Flume,YARN and Zoo Keeper.
- Capable of processing large volumes of structured,semi structured and unstructured data and supporting systems application architecture
- Able to assess business rules, collaborate with stakeholders and perform source - to-target data mapping, design and review.
- Experienced in monitoring cluster set up, logs, jobs using Web based UI and Cloudera Manager.
- Experiencewriting complex Map Reduce programs to process large data sets for finding and generatingTop K records, Inner product of two sparse data sets, Time Series processing, multiplication of large number of product values, etc.
- Experience working with different standard data and input / output file formats such asText file, Mahout, XML, JSON, Avro, Sequence files
- Implemented Hive queries to process and analyze the data over HBase, RDBMS table or Cassandra column family
- Involved in creating Hive tables, loading with data and writing Hive queries that will run internally in map reduce way.
- Created Pig Latin transformation statements for data cleaning and pre processing
- Created multiple UDF functions to handle multiple data formats
- Experienced with handling real time analytical queries on Big data using HBase.
- Used Flume to collect, aggregate, and store the web log data from different sources like web servers, mobile and network devices and pushed to HDFS..
- Used Oozie to automate data loading into the HDFS and Pig to pre-process the data.
- Experienced in working with Hadoop/Big Data storage and analytical frameworks over Amazon Web services (AWS).
- Experience in design and development of applications using Java, J2EE, Spring Framework, Hibernate, JMS, EJB, Struts, JSF XML, SOAP and REST Services.
- Expert knowledge over J2EE Design Patterns like MVC Architecture, Singleton, Factory Pattern, Front Controller, Session Facade, Business Delegate and Data Access Object for building J2EE Applications.
- Expert in Query writing using SQL, Procedures, Functions and Packages using PL/SQL.
- Expertise with Build tools like Maven, Ant and continuous integration with Jenkins.
- Expertise with Application servers and web server’s likeOracle Web Logic, IBM Web Sphere, Apache Tomcat, JBOSS 4.0.3 and VMware.
- Proficient in using RDBMS concepts with Oracle 9i/10g, DB2, MySQL and PostgreSQL.
- Excellent verbal and written communication skills.
TECHNICAL SKILLS:
Programming Languages/Web Technologies/Scripting Languages/Others Tools: C, Java JSP, Html, java script, jquery, Ajax, Shell Script, Python Oracle, PostgreSQL, HBase, Cassandra Hadoop, Map Reduce, Hive, Pig, Oozie, Sqoop, Flume, Zoo keeper, YARN Amazon Web Services (AWS) Spring, Hibernate, Ejb, Jms, Struts, Web Services Cloudera, Horton works. Agile, waterfall
PROFESSIONAL EXPERIENCE:
Confidential, San Jose CA
Hadoop Developer
Responsibilities:
- Developed PIG scripts to gain helpful information from unstructured information.
- Implemented Inverted Index's to be used when quick search responses are required and move result into data base using Map Reduce programs.
- Experienced with Map Reduce design patterns to implement solutions for complex problems using Map Reduce programs.
- Worked on map-reduce Joins in querying multiple semi-structured data as per analytic needs
- Experienced with Map Reduce frame works counter utility to calculate global sum entirely on Map side.
- Implemented generic filter look at each record and decided whether keep this record or delete it at Map Reduce level.
- End-to-end performance tuning of Hadoop clusters and Hadoop Map/Reduce routines against very large data sets
- Experienced with handling different files like Xml, Json, Avro data files, Text files and Custom formats in Map Reduce programs.
- Developed multiple Map Reduce jobs in java for data cleaning and preprocessing.
- Implemented Custom Input formats to implement fixed size splits.
- Experienced with transforming row based data to hierarchical data like xml in Map Reduce.
- Experienced with handling Avro files using SerDe's at Hive.
- Implemented Complex business logic using Hive UDF's to apply on structured data.
- Experienced with implementing bucket map join that is used to optimize join process.
- Implemented dash boards for data scientists used to query on structured data using HiveQL.
- Experienced with handling different compression techniques to optimize data both at Map Reduce and Hive level.
- Implemented utility system to move data from RDBMS to HDFS and Hive using Sqoop import/export options.
- Created HIVE partitioned tables to store information as tables on top of HDFS.
- Configured scheduler jobs using Oozie work flow that integrates different kind of Hadoop actions like Map Reduce, Hive, Pig and Sqoop.
- Accessed Rest web services using Curl to get JSON response and process in Map Reduce.
- Involved in installing, configuring, monitoring POC, Production cluster using Cloudera manager.
- Experienced in writing unit test cases using JUnit, MRUnit and other mock frame works.
- Experienced in convert high level design documents to low level design documents using Rational Rose.
- Moved records from local file system to HDFS and vice versa using UNIX commands.
- Completely included being developed and investigation stage.
Environment: HDFS, Map Reduce, Hive, Oozie,Sqoop, Cloudera manger, MySQL,SQL,Rest Web Services, Java, LINUX, Eclipse,Maven
Confidential, Dallas TX
Hadoop Developer
Responsibilities:
- Analyze the current Healthcare application to make a POC for Hadoop framework.
- Implemented unified system to analyze claims data at real time by using HBase.
- Integrated Map Reduce with HBase to move bulk amount of data using Table Input format.
- Implemented java programs to access HBase data using Java API to perform business logic.
- Supported Map Reduce programs those are running on the cluster
- Implemented Observers, Co-processors that execute as kind of event handling on HBase data.
- Implemented POC to find out best technology for business requirements on HBase and Cassandra.
- Registering different UDF functions in piggybank
- Involved in loading data from UNIX file system to HDFS
- Experience with performing analysis on unstructured data using Pig Latin operations.
- Responsible to manage data coming from different sources
- Join different data sets using join operations in pig and used Pig transformations to analyze data.
- Implemented Secondary Sorting to get Top K results using Map Reduce programs.
- Injected Members and Claims information to HDFS/HBase utilizing Sqoop.
- Write Map Reduce programs to process the Claims information into staging tables.
- Created Hive queries to create member and claims tables at Data Acquisition and ensuing layers.
- Involved in cluster configuration alongside Hadoop Admin in choosing Node prerequisite, Block size, and so forth.
- Performed Hive query enhancement & Project Health Information Exchange
- Used SVN for version control
Environment: Hadoop, HDFS, Map Reduce, Pig, Hive, HBase, Sqoop,SVN,Oracle 10g, MR Unit, Unix, Flat files
Confidential, New York NY
Java Developer
Responsibilities:
- Involved in all SDLC (Software Development Life Cycle) phases of the project.
- Developed and designed the front end using HTML, CSS and Java Scriptwith JSF Ajax and tag libraries.
- Handled events and runtime errors using JSF event listeners and validations.
- Used Spring Core Annotations for Dependency Injection.
- Convention Based Coding is Designed and developed by utilizing Hibernates persistence framework and OR Mapping capability to enable dynamic fetching and displaying of various table data with JSF Tag libraries.
- Designed and developed Hibernate configuration and session-per-request design pattern for making database connectivity and accessing the session for database transactions respectively.
- Used HQLandSQL for fetching and storing data in databases.
- Designed and developed the Validators, Controller Classes and Java bean components.
- Participated in the design and development of database schema and Entity-Relationship diagrams of the backend Oracle database tables for the application.
- Implemented Web Services with ApacheAxis.
- Designed and Developed Stored Procedures, Triggers in Oracle to cater the needs for the entire application.
- Developed complex SQL queries for extracting data from the database.
- Designed and built SOAPWeb Servicesinterfaces implemented in Java.
- Used Apache Ant for the build process.
Environment: Spring, Spring MVC, Hibernate, SOAP Web Service, Jms, Multithreading, JPAJBoss, Oracle, Maven, JUnit
Confidential
Java Developer
Responsibilities:
- Implemented reusable JSP templates using Struts Tiles framework.
- Involved in user interface design and development using Struts tags, JSP, HTML, CSS and JavaScript.
- Implemented Spring MVC using Spring Controllers.
- Integration of PHP, XML, JSP, java scripting, JSTL Tags done on UNIX using ColdFusion Extension Tags (CFXes).
- Responsible for implementing Action classes and Action Forms using Struts.
- Responsible for implementing Validation criteria using Struts.
- Involved in working with java concepts like collections, multi-threading.
- Developed custom exception classes for Exception Handling.
- Developed functions using java script for front-end validation.
- Responsible for implementing Hibernate mapping files.
- Implemented Spring Dependency Injection and Aspect Oriented Programming.
- Designed and developed DAO interfaces & implementations for database related operations.
- Implemented SQL, PL/SQL Stored Procedures, Triggers and Cursors.
- Used Web Services to exchange data between different applications and different platforms.
- Involved in implementing Class and Sequence Diagrams using Visio.
- Utilized JUnit for unit testing of the modules.
- Implemented asynchronous communication using AJAX.
- Used XML for Storing data and DOM to process XML documents using JAXP.
- Analyzed the requirements and prepared requirement specifications.
- Involved in preparation of the design document and functional specification for the project.
- Responsible for implementing Enhancements.
- Used ANT to write build scripts as well as deployment scripts.
- Extensively used log4j for logging the log files.
- Involved in implementing test cases.
Environment: Java, spring, Hibernate, Soap Web Services, Jms, Ejb, JSF, Shell script, SQL, Ajax, Jquery, Web sphere.
Confidential
Java Developer
Responsibilities:
- Involved in the design and development phases of Rational Unified Process (RUP).
- Designed Use Case Diagrams, Class Diagrams and Sequence Diagrams and Object Diagrams, to model the detail design of the application.
- Application was based on MVC architecture with JSP 2.0 serving as presentation layer, Servlets as controller and EJB,Hibernate in business layer to access to Oracle 9i Database.
- Developed the application using Jakarta Struts Framework, developed action classes, form beans andJava Server Pages, Struts-Tiles using WSAD.
- Developed the presentation Tier using JSP, Struts tags, XML, HTML, and JavaScript.
- Used RESTful Web services for transferring/retrieving data between different Applications.
- Developed Session beans which encapsulate the workflow logic.
- Designed and implemented Business Delegate, Session Facade and DTO Design Patterns.
- Involved in implementing the DAO pattern for database access.
- Used JAXB API to bind XML Schema tojava classes.
- Used Hibernate extensively to store the persistence data into the Oracle9i database and written HQL to access the data from the database.
- Tested the application functionality with JUnit Test Cases and Suites.
- Developed logging module-using Log4J to create log files to debug as well as trace application.
- Extensively used ANT 1.5 as a build tool, and used CVS for version control.
- Deployed the applications on Web logic Server.
Environment: Hibernate, Struts, EJB, Servlets, JSP, JDBC, SQL, JavaScript, Oracle 9i, JUnit, CVS, ANT and Log4J, Web logic Server.