Senior Hadoop Developer Resume
Kansas City, KS
SUMMARY
- 7 years of total IT experience in Java Development, Web application Development, Database Management.
- 3 years of Experience in Installation, Configuration and Managing Hadoop cluster using Cloudera Managerand Apache Ambari.
- Experience with Hadoop ecosystem (Apache MapReduce, Pig, Hive, Oozie, HBase, Sqoop and Flume).
- Expertise in writing Hadoop Jobs for analyzing data using Hive and Pigand integrating HBase
- Worked on developing ETL processes to load data from multiple data sources to HDFS using Flume and Sqoop, perform structural modifications using MapReduce, Hive and analyze data using visualization/reporting tools
- Well versed with writing workflows using Apache Oozie with job controllers like MapReduce, Hive.
- Experience in writing topologies using Apache Storm and stream using Apache Kafka.
- Experience in configuring Zookeeper to coordinate the servers in clusters to maintain the data consistency in configuring High Availability
- Expertise in designing and implementing of enterprise applications using Java/J2EE, JDBC, JSP, HTML, Struts, JavaScript, Servlets, JavaBeans, Oracle and Hibernate.
- Experience in writing DAO using Hibernate, JDBC
- Experience in Object Relational Mapping Frameworks such as Hibernate
- Experience implementing MVC Architecture using Struts Framework.
- RDBMS experience includes Oracle, PostgreSQL, MySQL and programming using PL/SQL, SQL.
- Experience in a public cloud like Amazon(AWS EC2,EMR,S3)
- Developed search engine using Lucene indexer across millions of SEC filings on multiple Amazon EC2 cluster.
- Experience in writing various test cases using JUNIT framework.
- Experience in various phases of Software Development Life Cycle (Analysis, Requirements gathering, Designing) with expertise in documenting various requirement specifications, functional specifications, Test Plans, Source to Target mappings, SQL Joins.
- Experience with version controllers SVN, CVS, GitHub
TECHNICAL SKILLS
Hadoop Ecosystem: HDFS, MapReduce, Hive, Pig, Oozie, Sqoop, flume, Zookeeper
Scripting Languages: Shell Scripting, Perl
Relational Databases: Oracle 10g/11g, MySQL
Languages: C++, Java, JavaScript
IDE: Visual Studio, Eclipse, OpenPerl
Web Tools: HTML, XML, JDBC, EJB, JSON, JSP, Servlets, JMS,Spring and Hibernate
Tools: Jetkins, HP Quality Center, Selenium, Junit
PROFESSIONAL EXPERIENCE
Confidential, Kansas City, KS
Senior Hadoop Developer
Responsibilities:
- Setup and benchmarkedHadoop /HBase clusters for internal use.
- Developed Java MapReduce programs for the analysis of sample log file stored in cluster.
- Developed MapReduce programs to cleanse the data in HDFS obtained from heterogeneous data sources to make it suitable for ingestion into Hive schema for analysis
- Developed multiplescripts for analyzing data using Hive and Pigand integrating withHBase.
- Used Sqoop to import data into HDFS and Hive from other data systems.
- Migration of ETL processes from Oracle to Hive to test the easy data manipulation.
- Data was pre - processed and fact tables were created using HIVE.
- Experienced in writing topologies using Apache Storm and log aggregators.
- The resulting data set was exported to SQL server for further analysis.
- Create Hive scripts to extract, transform, load (ETL) and store the data.
- Created Hadoop Work Flows and scheduled jobs using oozie
- Automated all the jobs from pulling data from databases to loading data into SQL server using shell scripts.
Environment: Apache Hadoop, HDFS, Cloudera Manager, Java, MapReduce, Eclipse Indigo, Hive, PIG, Sqoop, SQL,PL/SQL.
Confidential, Kansas City, MO
Java /Hadoop developer
Responsibilities:
- Developed Java MapReduce programs to extract the required Information from the semi-structured price claim information.
- Loading data into HIVE data warehouse and extracting data based on requirements.
- Implemented Partitioning, Dynamic Partitions, and bucketing in HIVE for Decomposing Datasets.
- Created Normal table, Externals tables in HIVE and used various HIVE joins to merge the data sets.
- Stored and Processed sparse data for real-time access in HBase.
- Worked with flume to import the log data from the reaper logs, syslog’s into the Hadoop cluster.
- Created Sqoop scripts to capture structured claims and rate sheet data.
- Created Hadoop Work Flows and scheduled jobs using oozie.
- Responsible for building scalable Distributed data solution Using Cassandra.
- Migration of ETL processes from Oracle to Hive to test the easy data manipulation.
- Hive framework is used for generating reports from previous accidents information.
- Designed table architecture and developed DAO layer using Cassandra NoSQL database.
- Developed Web services using SOAP, JAX-WS and WSDL.
- Developed persistence layer using ORM hibernate for transparently store objects into database.
- Used Hibernate to map the database tables using hbm.xml files.
- Used both SAX and DOM parser for XML related development.
- Used Web services - WSDL and SOAP for getting credit card information from third part
- Implemented Multithread concepts in java classes to avoid deadlocking.
- Developed Data Access Classes using the Hibernate.
- Developed request/response paradigm by using Spring Controllers, Inversion of Control and Dependency.
Environment: Apache Hadoop, Pig, Hive, Sqoop, Cassandra, NoSQL, Big Data, HBase, ZooKeeper, Cloudera, Ajax, JavaScript, Java 6, Spring, Hibernate, JMS, Weblogic Application Server, Eclipse, Ant, REST, Web Services, Ant, HP Service Manager, Project Server, Unix, Windows
Confidential, KS
Java Developer
Responsibilities:
- Developed web application to search financial and non-financial information and to discover the knowledge using data mining tools.
- Implemented features such as frequency distribution of words, calculation of financial and fraud risks.
- Incorporated Spring MVC architecture in the application architecture to develop the front end of the system
- UsedSpring ORMto integrate Spring Framework with Hibernate
- Developed Perl scripts that extract and store specified reports on Amazon EC2 instances.
- Wrote multiple scripts to automate report extraction process for large database of companies.
- Maintain and pre-process stored data on the Amazon EC2 instances.
- Developed Simple to complex Map/reduce Jobs using Hiveand Pig to store extracted data.
- Create Hive scripts to extract, transform, load (ETL) and store the data.
- Loading data into HIVE data warehouse and extracting data based on requirements.
- Evaluating performance of big datasets using Machine Learning algorithms.
Environment: Perl, Java, JSP, AJAX, JDBC, Spring, EjB, Hibernate, Amazon EC2, S3 storage, Lucene, Tomcat7, Weka, SPM, Machine Learning.
Confidential
Java Developer
Responsibilities:
- Involved in all phases of application development like Requirement Gathering, Architecture design using MVC pattern, Decision Making, Development and Deployment on production.
- Developed DB access module using JDBC API to build bridge between application & backend
- Implemented Logging facility using Log4J to log Fatal, Debug, Info, Error, Warning messages
- Responsible for Database Modeling, Design and developed DB objects like Tables, constraints, triggers etc.
- Responsible for implementing coding convention according to client coding specifications.
- Responsible for configuring, building and deployment of application using Apache ANT.
- Extensively used IOC and AOP concepts of Spring Framework as part of development.
- Developed application service components and configured beans using spring, creation of Hibernate mapping files and generation of database schema.
- Developed Data Access Classes using the Hibernate.
- Used Log4J logging framework. Log messages with various levels are written in all the Java code.
- Used the MVC model to create the Object Model.
- Used XML Technologies like DOM for transferring data.
- Used Ant for compiling and creating war files, CVS for Version Control, and Weblogic as Application Server.
Environment: JavaScript, Java 7, Spring, Hibernate, JMS, Weblogic Application Server, Eclipse, Ant, CVS
Confidential
Java Developer
Responsibilities:
- Actively involved in UI design for application.
- Developed entire JSP pages for the application.
- Developed the User interface screens using HTML, JSP and AJAX.
- Worked on GUI with Java Script frameworks JQuery, XML.
- Developed server side code using Struts and Servlets.
- Created struts-config.xml file for the Action Servlet to extract the data from specified Action form so as to send it to specified instance of action class.
- Used JSTL and custom tag library with frameworks such as Ajax and JQuery, to build interactive and attractive user interfaces.
- Used JSON objects to transfer data between controllers and web services.
- Wrote complete Test Coverage Matrix (TCM) from the Business Requirements.
- Wrote multiple test cases and executed them on HP-Quality Center.
- Involved in various stages of Testing including System testing, Integration testing, Regressing Testing and User Acceptance Testing (UAT).
- Extracted daily test progress reports as deliverables
Environment: Java, JSP, HTML, CSS, JDBC, Struts, Ajax, JQuery,Eclipse, HP Quality Center, Testing, SQL, SVN