SUMMARY:

• Over 9 + years of IT experience in analysis, design and development using Hadoop, HDFS, Map Reduce and Hadoop Ecosystem (Pig, Hive, Impala and Spark), Java and J2EE.
• Extensive experience in installing, configuring and using ecosystem components like Hadoop MapReduce, HDFS, Sqoop, Pig, Hive, Impala & Spark
• Extensive experience in Java and J2EE technologies like Servlets, JSP, JSF, JDBC, JavaScript, Ext JS, hibernate, and Junit testing.
• Expertise in using J2EE application servers such as IBM WebSphere, JBoss and web servers like Apache Tomcat.
• Experience in different Hadoop distributions like Cloudera (CDH3 & CDH4) and Horton Works Distributions (HDP).
• Good Knowledge in Amazon AWS concepts like EMR and EC2 web services which provides fast and efficient processing
• Experience in analyzing data using HIVEQL, PIG Latin and custom MapReduce programs in JAVA. Extending HIVE and PIG core functionality by using custom UDF's.
• Experienced in configuring and administering the Hadoop Cluster using major Hadoop Distributions like Apache Hadoop and Cloudera.
• Diverse experience utilizing Java tools in business, Web, and client - server environments including Java Platform, J2EE, EJB, JSP, Java Servlets, Struts, and Java database Connectivity (JDBC) technologies.
• Good understanding in integration of various data sources like RDBMS, Spreadsheets, Text files, JSON and XML files.
• Experience in working in environments using Agile(SCRUM), RUP and Test Driven development methodologies.
• Good working experience on using Sqoop to import data into HDFS from RDBMS and vice-versa.
• Implemented Service Oriented Architecture (SOA) using Web Services and JMS (Java Messaging Service).
• Implemented J2EE Design Patterns such as MVC, Session Façade, DAO, DTO, Singleton Pattern, Front Controller and Business Delegate.
• Experienced in developing web services with XML based protocols such as SOAP, Axis, UDDI and WSDL.
• Experienced in MVC (Model View Controller) architecture and various J2EE design patterns like singleton and factory design patterns.
• Extensive experience in loading and analyzing large datasets with Hadoop framework (MapReduce, HDFS, PIG, HIVE, Flume, Sqoop, SPARK, Impala), NoSQL databases like MongoDB, HBase, Cassandra.
• Solid understanding of Hadoop MRV1 and Hadoop MRV2 (or) YARN Architecture.
• Hands on experience in configuring and administering the Hadoop Cluster using major Hadoop Distributions like Apache Hadoop and Cloudera.
• Good knowledge in SQL and PL/SQL to write Stored Procedures and Functions and writing unit test cases using JUnit.
• Extensive experience in Extraction, Transformation and Loading (ETL) of data from multiple sources into Data Warehouse and Data Mart.
• Strong knowledge in Object oriented design/analysis, UML modeling, Classic design patterns, and J2EE patterns.
• Hands on experience working with databases like Oracle 12g, SQL Server 2010 and MySQL.
• Hands on experience on all the latest UI stack including HTML, CSS, mobile friendly, responsive design, user-centric design etc.
• Expertise in using XML related technologies such as XML, DTD, XSD, XPATH, XSLT, DOM, SAX, JAXP, JSON and JAXB.
• Experience in using ANT and Maven for building and deploying the projects in servers and also using Junit and log4j for debugging.

TECHNICAL SKILLS:

Hadoop Eco System:: Hadoop 2.7/2.5, HDFS1.2.4, Spark 2.0/2.0.2, Hive, Pig, Sqoop, Map Reduce/YARN, Impala, Oozie.

Big Data Platforms: Hortonworks, Cloudera, Amazon AWS

Web Technology:: JSP, JavaScript, Ajax, Html 5/4, CSS 3/2, XML, XHTML, JSTL, JQuery, XSLT, Xpath, XQuery, SOAP, Restful

Programming Languages:: C, C++, Core Java, J2EE, AWS

Web servers:: JBoss 6, IBM WebSphere 7, Apache Tomcat 7, Oracle Weblogic 10g, Oracle Application Server

NoSQL Databases:: HBase, MongoDB & Cassandra

Development Methodologies:: Agile, Scrum, Waterfall.

Operating Systems:: Linux, windows, Mac, Unix

Infrastructure Design:: Microsoft Visio, MYSQL Workbench

ETL Design Tools:: Teradata Load Utilities (BTEQ, Fast, Multi load), SSIS, SSRS

Programming Languages:: Java, Shell Scripts, Scala, Python

Web IDEs: Eclipse, Netbeans, RAD, SQL Developer

Version Control:: IBM ClearCase, Visual Source Safe, SVN, CVS, Git Hub

PROFESSIONAL EXPERIENCE:

Confidential,Warren,NJ

Sr. Big Data/Hadoop Developer

Responsibilities:

• Worked closely with the business analysts to convert the Business Requirements into Technical Requirements and prepared low and high level documentation.
• Build Big Data Analytics and Visualization platform for handling high-volume batch-oriented and real-time data streams.
• Utilized Agile Scrum Methodology to help manage and organize a team with regular code review sessions.
• Loaded data from diff servers to AWS S3 bucket and setting appropriate bucket permissions. implements complex big data with a focus on collecting, parsing, managing, analyzing, and visualizing large sets of data to turn information into business insights using multiple platforms in Hadoop ecosystem.
• Worked on analyzing different big data analytic tools including Hive, Impala and Sqoop in importing data from RDBMS to HDFS.
• Developed and execute data pipeline testing processes and validate business rules and policies
• Reduced the overall EMR production cluster's cost (Amazon Web Services) by obtaining the best configuration for running data.
• Spinning up of Hadoop Cluster in AWS using Cloudera Director.
• Built code for real time data ingestion using Java, MapR-Streams (Kafka) and STORM.
• Designed high level ETL architecture for overall data transfer from the OLTP to OLAP.
• Created various Documents such as Source-To-Target Data mapping, Unit Test, Cases and Data Migration Document.
• Integrated NoSQL database like HBase with MapReduce to move bulk amount of data into HBase.
• Developed batch data flow using Spark code in python, Scala and Java
• Imported data from structured data source into HDFS using Sqoop incremental imports.
• Created Hive tables, partitions and implemented incremental imports to perform ad-hoc queries on structured data.
• Created Hive Generic UDF's to process business logic with Hive QL.
• Build Hive tables using list partitioning and hash partitioning.
• Developed story telling dashboards in Tableau Desktop and published them on to Tableau Server
• Created Rich dashboards using Tableau Dashboard and prepared user stories to create compelling dashboards to deliver actionable insights.
• Developed SQL scripts using Spark for handling different data sets and verifying the performance over Map Reduce jobs.
• Involved in converting Map Reduce programs into Spark transformations using Spark RDD's using Scala and Python.
• Extensively used Apache Sqoop for efficiently transferring bulk data between Apache Hadoop and relational databases (Oracle, MySQL) for predictive analytics
• Supported MapReduce Programs those are running on the cluster and also Wrote MapReduce jobs using Java API.
• Supported Map Reduce Programs those are running on the cluster and also Wrote MapReduce jobs using Java API.
• Imported data from mainframe dataset to HDFS using Sqoop.
• Wrote Hive queries for data analysis to meet the business requirements.
• Wrote Pig Latin scripts and also developed UDFs for Pig Data Analysis.
• Developed Scripts and Batch Job to schedule various Hadoop Program.
• Upgraded the Hadoop Cluster from CDH4 to CDH5 and setup High availability Cluster to Integrate the HIVE with existing applications
• Worked with the raw data, cleanses it and finally polishes it to the format where it can be consumed by Data Scientists to create critical insights.
• Optimized the mappings using various optimization techniques and also debugged some existing mappings using the Debugger to test and fix the mappings.
• Updated maps, sessions and workflows as a part of ETL change and also modified existing ETL Code and document the changes.
• Installed Oozie workflow engine to run multiple Hive and Pig jobs.

Environment: Hadoop, Java, MapReduce, HDFS, AWS, Amazon S3, Hive, Pig, Linux, XML, Eclipse, Cloudera, CDH4/5 Distribution, DB2, SQL Server, Oracle 12c, MySQL

Confidential,Juno Beach,FL

Sr. Big Data/Hadoop Developer

Responsibilities:

• Involved in the design and development phases of Scrum Agile Software development.
• Maintain Hadoop, Hadoop ecosystems, and database with updates/upgrades, performance tuning and monitoring
• Prepared data for analytics processing, and data egress for availability of analytics results to visualization systems, applications, or external data stores
• Worked with the Data Science team to gather requirements for various data mining projects. builds large-scale data processing systems in data warehousing solutions, and work with unstructured data mining on NoSQL database.
• Provisioning of Cloudera Director AWS instance and adding Cloudera manager repository to scale up Hadoop Cluster in AWS.
• Involved in creating Hive tables, and loading and analyzing data using hive queries.
• Developed Simple to complex MapReduce Jobs using Hive and Pig.
• Involved in running Hadoop jobs for processing millions of records of text data.
• Worked with application teams to install operating system, Hadoop updates, patches, version upgrades as required.
• Responsible for writing Hive Queries for analyzing data in Hive warehouse using Hive Query Language (HQL).
• Worked in Agile environment this uses JIRA to maintain the story points and Kanban model .
• Developed and maintained batch data flow using Sqoop, HiveQL, Pig and Unix scripting
• Developed multiple MapReduce jobs in java for data cleaning and preprocessing.
• Extracted files from MySQL through Sqoop and placed in HDFS and processed.
• Created and maintained Technical documentation for launching Hadoop Clusters and for executing Hive queries and Pig Scripts.
• Used Oozie and Zookeeper operational services for coordinating cluster and scheduling workflows.
• Implementing MR programs to analyze large datasets in warehouse for business intelligence purpose
• Developed customized Hive UDFs and UDAFs in Java , JDBC connectivity with hive development and execution of Pig scripts and Pig UDF's.
• Maintaining different cluster security settings and involving in creation and termination of multiple cluster environment.
• Specifying the cluster size, allocating Resource pool, Distribution of Hadoop by writing the specification texts in JSON File format.
• Created and Implemented highly scalable and reliable highly scalable and reliable distributed data design using NoSQL / Cassandra technology.
• Wrote MapReduce jobs using Java API and Pig Latin .
• Loaded the data from Teradata to HDFS using Teradata Hadoop connectors.
• Wrote Pig scripts to run ETL jobs on the data in HDFS.
• Used Hive to do analysis on the data and identify different correlations.
• Responsible for defining the data flow within Hadoop eco system and direct the team in implement them.
• Worked on importing and exporting data from Oracle and DB2 into HDFS and HIVE using Sqoop .

Environment: Hadoop, HDFS, MapReduce, Unix, REST, Python, Pig, Hive, HBase, Storm, NoSQL, Flume, Zookeeper, Cloudera, SAS, AWS, Kafka, Cassandra, Informatica, Teradata, Scala

Confidential,Charlotte,NC

Sr. Hadoop Developer

Responsibilities:

• Worked on analyzing Hadoop cluster and different big data analytic tools including Map Reduce, Hive and Spark.
• Involved in loading data from LINUX file system, servers, Java web services using Kafka Producers, partitions.
• Implemented Kafka Custom encoders for custom input format to load data into Kafka Partitions.
• Implemented Kafka High level consumers to get data from Kafka partitions and move into HDFS
• Migrated complex map reduce programs into Spark RDD transformations, actions.
• Utilized Agile Scrum Methodology to help manage and organize code review sessions.
• Implemented Spark RDD transformations to map business analysis and apply actions on top of transformations.
• Automated all the jobs from pulling data from Storage to loading data into MySQL using Shell Scripts
• Rendered and delivered reports in desired formats by using reporting tools such as Tableau.
• Used Kafka to stream the data with twitter4j from source to Hadoop.
• Responsible for troubleshooting issues in the execution of MapReduce jobs by inspecting and reviewing log files.
• Design and tested the data ingestion to handle data from multiple sources into the Enterprise Data lake.
• Used Pig to perform data validation on the data ingested using Sqoop and Flume and the cleansed data set is pushed into HBase.
• Offline Analysis was performed on HDFS and sent the results to MongoDB databases to update the information on the existing table, From Hadoop to MongoDB move was done using MapReduce, Hive/ Pig scripts by connecting with Mongo-Hadoop connectors.
• Developed the Map Reduce programs to parse the raw data and store the pre Aggregated data in the portioned tables.
• Created partitioned tables in Hive, mentored analyst and test team for writing Hive Queries.
• Developed Pig Latin scripts to extract the data from the web server output files to load into HDFS.
• Involved in Installing, Configuring Hadoop Eco System, and Cloudera Manager using CDH4 Distribution.
• Automation of all the jobs starting from pulling the Data from different Data Sources like MySQL and pushing the result dataset to Hadoop Distributed File System and running MR, PIG, and Hive jobs using Kettle and Oozie (Work Flow management)
• Worked with NoSQL databases like HBase in creating tables to load large sets of semi structured data coming from various sources.
• Performed ETL process with Python-SQL Server pipelines/framework to perform data analytics and visualization in Python, Pandas and MATLAB stack.
• Handled importing of data from various data sources, performed transformations using Hive, MapReduce, loaded data into HDFS and extracted the data from MySQL into HDFS using Sqoop
• Migrated the code into QA (Testing) and supported QA team and UAT (User).
• Configured Nagios for receiving alerts on critical failures in the cluster by integrating with custom Shell Scripts.
• Worked on implementing Flume to import streaming data logs and aggregating the data to HDFS through Flume.
• Used Spark API/Scala over Hadoop YARN to perform analytics on data in Hive.
• Worked with application teams to install operating system, Hadoop updates, patches, version upgrades as required

Environment: Hadoop, HBase, HDFS, Map Reduce, Kafka, Pig Latin, Sqoop, Hive, pig, MySQL, Oozie, Zookeeper, Python.

Confidential,Mason,OH

Sr. Java/J2EE Developer

Responsibilities:

• Used Agile Scrum to manage the full life cycle development of the project.
• Used JSF and provided a rich architecture for managing component state, processing component data, validating user input, and handling events.
• Developed application using Spring MVC, JSP and AJAX on the presentation layer, the business layer is built using spring and the persistent layer uses Hibernate with Jboss Application Server.
• Used spring configuration files and its application context object for creating various factory objects.
• Used Hibernate 3.x with JPA annotation and created several persistent classes.
• Developed DAO classes with Junit using Spring DAO package for data access and data manipulation.
• Worked on Restful API and invoked web services (consumed) that are based on EJBs.
• Responsible for using Rest API, and JAXP to deal with XML data. Called Restful API to update task to different examiner.
• Created different XML documents using XML, XSD and XSLT.
• Used Light DOM parser, and SAX parser for transforming the XML data.
• Used Oracle 11g as backend, Worked on SQL queries for persistence of Business Objects using JDO as ORM.
• Involved in creating different SQL scripts, worked on some of the core query operations using DB2.
• Used Maven to build application and deploy on the server.
• Configured and created application log files using Log4j required to trace application and GIT for version controlling.
• Testing is done through Test Driven Development and JUnit.

Environment: Java/J2EE, JSP, JQuery, Servlets, HTML, Spring, SOA, AJAX, JDK 1.4/1.5, Hibernate 3.x, Apache CXF, JavaScript, Oracle 11g, Windows 7/UNIX, My Eclipse 4.1, Apache-Maven-3.0.2, GIT

Confidential

Java Developer

Responsibilities:

• Responsible for requirement gathering and analysis through interaction with end users.
• Involved in coding for JSP pages, Form Beans, and Action Classes in Struts
• Designed and developed the application using various design patterns, such as session facade, business delegate and service locator.
• Used HQL (Hibernate Query Language) to query the Database System and used JDBC Thin Driver to connect to the database.
• Developed Hibernate entities, mappings and customized criterion queries for interacting with database.
• Involved in developing JSP pages using Struts custom tags, JQuery and Tiles Framework.
• Created Custom Tag Libraries to support the Struts framework.
• Developed using Eclipse and used SVN Version Control System for providing a common platform for all the developers.
• Deployed the application on Tomcat application server
• Involved in Database Connectivity through JDBC.
• Developed JUnit Test cases and performed application testing for QC team.
• Used Servlets, Struts, JSP and Java Beans for developing the Performance module using Legacy Code.
• Used JavaScript to perform client side validations and Struts-Validator Framework for server-side validation.
• Involved in creating Database SQL and PL/SQL queries and stored Procedures.
• Debugged and developed applications using Rational Application Developer (RAD).
• Developed a Web service to communicate with the database using SOAP.
• Developed DAO (data access objects) using Spring Framework.
• Deployed the components in to WebSphere Application server
• Used Asynchronous JavaScript and XML (AJAX) for better and faster interactive Front-End.
• Developed Unit Test Cases and JUnit for unit testing of the application.

Environment: Java 1.6, Servlets, JSP, Struts1.2, IBM Rational Application Developer (RAD) 6, Web sphere 6.0, AJAX, Rational Clear case, Rational Rose, Oracle 9i, log4j.