SUMMARY

• Over 8+ years of IT experience in analysis, design and development using Hadoop, Java J2EE, SQL.
• 3+ years of experience in dealing with Apache Hadoop components such as HDFS, Map reduce, Hive, HBase, PIG, Scala, Spark, Impala, OOZIE, Flume, HCatalog and Sqoop.
• Good knowledge on YARN (MRV2) architecture.
• Worked on HDFS, NAME NODE, JOB TRACKER, DATA NODE, TASK TRAKER and the Map reduce concepts.
• Experience in writing UNIX shell scripting.
• Worked on analyzing the data using HiveQL, Pig Latin and the Map Reduce programs in Java.
• Worked on writing custom UDF'S in extending Hive and Pig core functionality.
• Worked on managing and reviewing Hadoop log files.
• Worked on Sqoop, in moving the data from a relational database into Hadoop and used FLUME in collecting the data and populate hadoop.
• Worked on HBase in conducting the quick look ups such as updates, inserts and deletes in Hadoop.
• Experience in Data modeling, complex data structures, Data processing, Data quality, Data life cycle.
• Experience in running MapReduce and Spark jobs over YARN
• Hands - on experience in complete project life cycle (design, development, testing and implementation) of Client Server and Web applications
• Participated in design reviews, code reviews, unit testing and integration testing.
• A very good experience in developing and deploying the applications using Web logic, Apache Tomcat and JBOSS.
• Strong Experience on SQL, PL/SQL and the database concepts.
• Experience on NoSQL Databases such as Hbase and Casandra.
• A very good understanding of job workflow scheduling and monitoring tools like Oozie and Control M.
• Knowledge on administrative tasks such as installing Hadoop and its ecosystem components such as Hive and Pig

TECHNICAL SKILLS

Hadoop/BigData ecosystems: HDFS, MapReduce, Pig, Hive, Impala, Sqoop, Oozie, Zookeeper, Flume, Spark and Scala

No SQL Database: Hbase, Cassandra, MongoDB

IDE Tools: Eclipse, NetBeans

Languages: C, C++, Java, J2EE, PL/SQL, MR, Pig Latin, HiveQL, Unix shell scripting and Scala

Databases,Datawarehousing: Tera data, Oracle, SQL server, MySQL, DB2, PostgreSQL

ETL tools: Data stage, teradata

Operating Systems: Windows 95/98/2000/XP/Vista/7, Unix, Linux

PROFESSIONAL EXPERIENCE

Confidential, Thomasville, NC

Hadoop Developer

Responsibilities:

• Developed transformations using custom MapReduce, Pig and Hive
• Developed Pig Latin scripts in extracting and filtering the relevant data from the web server output files to load into HDFS.
• Created MapReduce jobs using Pig Latin and Hive Queries.
• Performed Map side joins in both Pig and Hive
• Used Sqoop in loading the data from RDBMS into HDFS.
• Knowledge on handling hive queries using Spark SQL that integrates with Spark environment.
• Optimized joins in Hive using techniques such as Sort-Merge join and Map side join
• Used JSON to represent complex data structure within a map reduce job
• Reviewed and managed the Hadoop Log files.
• By using Flume, log data is loaded into HDFS. Focused on creating the MapReduce jobs to power the data for search and aggregation.
• Used Sqoop in importing the data and metadata from Oracle.
• Involved in creating Hive tables, loading the data and writing Hivequeries.
• Used Pig Latin to apply transaction on systems of record.
• Developed Pig scripts and UDFs extensively for Value Added Processing (VAPs).
• Actively involved in the design analysis, coding and strategy development.
• Developed SQOOP commands to pull data from Teradata and push to HDFS.
• Developed Hive scripts for implementing dynamic partitions and buckets for retail history data.
• Developed Pig scripts to convert the data from Avro to text file format.
• Designed and developed read lock capability in HDFS.
• Dataware house is designed by using Hive.
• Developed workflow in Oozie to automate the tasks of loading the data into HDFS and pre-processing with Pig.

Environment: Hadoop, MapReduce, HDFS, Pig, Hive, HBase, Java, Oracle 10g, MySQL, Ubuntu.

Confidential, Milwaukee, WI

Hadoop Developer

Responsibilities:

• Developed MapReduce jobs in both PIG and Hive for cleaning and pre-processing
• Converted various database objects like packages, procedures and functions written in PL/SQL to scala language.
• Developed Sqoop scripts for loading data into HDFS from DB2 and preprocessed with PIG
• Automated the tasks of loading the data into HDFS and pre-processing with Pig by developing workflows using Oozie
• Worked on streaming to collect this data from Flume and performed real time batch processing
• Read from Flume and involved in pushing batches of data to HDFS and HBase for real time processing of the files
• Implemented partitioning and bucketing techniques in Hive
• Developed Hive scripts for implementing dynamic partitions
• Loaded data from UNIX file system to HDFS and written Hive User Defined Functions
• Developed code to pre-process large sets of various types of file formats such as Text, Avro, Sequence files, Xml, JSON and Parquet
• Created multi-stage Map Reduce jobs in Java for ad-hoc purposes
• Used Sqoop to load data from DB2 to Hbase for faster querying and performance optimization
• Developed suit of Unit Test Cases for Mapper, Reducer and Driver classes using testing library
• Collected the logs data from web servers and integrated into HDFS using Flume
• Worked on developing ETL Workflows on the data obtained using Python for processing it in HDFS and Hbase using Oozie
• Performed POCs on Spark test environment

Environment: Hadoop, HDFS, Hive, Pig, Flume, Hbase, Sqoop, Oozie, Linux, Hortonworks Distribution, Relational Databases, PL/SQL, Scala, Hive, Impala.

Confidential, NY

Hadoop Developer

Responsibilities:

• Identified data source systems integration issues and proposing feasible integration solutions.
• Partnered with Business Users and DW Designers to understand the processes of Development Methodology, and then implement the ideas in Development accordingly.
• Worked with Data modeler in developing STAR Schemas and Snowflake schemas.
• Created Oracle PL/SQL queries and Stored Procedures, Packages, Triggers, Cursors and backup-recovery for the various tables.
• Identifying and tracking the slowly changing dimensions (SCD)
• Extracting data from Oracle and Flat file, Excel files sources and performed complex joiner, Expression, Aggregate, Lookup, Stored procedure, Filter, Router transformations and Update strategy transformations to extract and load data into the target systems.
• Created reusable Mailing alerts, events, Tasks, Sessions, reusable worklets and workflows in Workflow manager.
• Scheduled the workflows at specified frequency according to the business requirements and monitored the workflows using Workflow Monitor.
• Extensively used TOAD for source and target database activities
• Generated simple reports from the data marts using Business Objects.

Environment: Business Objects XI, Oracle 9i, SQL/PL, SQL, UNIX Shell Programming, UNIX, and Windows NT.

Confidential, NY

Java Developer

Responsibilities:

• Enhancements and new development of various flow services and enhancements /Maintenance of various Java based services in Web Methods.
• Worked extensively on Web Methods JDBC Adapters, MQ Adapters, Flow Services and notifications.
• Is wholly responsible for the new enhancements and design of MVC web application.
• Extensively used Web Services using Apache AXIS.
• Implemented various Design Patterns like MVC, Factory, DAO and Façade.
• Development/Enhancements of various Oracle stored procedures.
• Interact with various project participants from different teams to understand the interfaces and impact.
• Responsible for Developing JUnit Test Case to Unit Test the Services outside the server.
• Worked with Web Services, XML/XS, AJAX for Live Link Content Management, WSDL, SOAP.
• Simultaneously support the production system by responding and fixing various queries and issues raised by customer support. Worked on Web Logic as well as AXIS upgrades. Re-designed the EMCST application with WRIA components in the frontend, Restful web services implemented in Jersey returning JSON in the backend.

Environment: Eclipse 3.2, WebLogic 8.1, WebLogic 10.2, Apache Ant, Rational Clear Case, Servlets, MVC, Apache AXIS1.2, AXIS 1.4, JMS, MQ Adapters, Web Methods 6.5, Anthill, Oracle 10g, Oracle SQL Developer 1.2, Jersey and JSON, Waterfall method.

Confidential

Java Developer

Responsibilities:

• Designed the application using the J2EE design patterns such as Session Façade, Business Delegate, Service Locator, Value Object and Singleton.
• Developed presentation tier as HTML, JSPs using Struts 1.1 Framework. Used AJAX for faster page rendering.
• Developed the middle tier using EJBs Stateless Session Bean, Java Servlets.
• Entity Beans used for accessing data from the Oracle 8i database.
• Worked on Hibernate for data persistence.
• Prepared high and low level design documents for the business modules for future references and updates.
• Deployed the application in JBoss Application Server in development and production environment.
• Implemented CVS as Version control system.
• Code Walkthrough/ Test cases and Test Plans.
• Used ANT as build tool. Used Junit for writing Unit tests

Environment: Eclipse, HTML, Java Script, CoreJava, JUnit, JSP, Servlets, JDBC, Oracle 8i, AJAX, CVS and JBoss Application Server.

Confidential

SQL Server Developer/DBA

Responsibilities:

• Responsible for creating Databases, Tables, Indexes, Constraints, Views, Stored Procedures
• Was involved in writing Triggers and Stored Procedures.
• Actively involved in Normalization &De-normalization of database.
• Was involved in performance tuning of the Database
• Developed custom reports using Microsoft reporting Services.

Environment: SQL Server 2005, Enterprise Manager, UML, MS Project, Windows 2008 Server, .NET, C#, ASP.NET, DTS, Microsoft reporting Services.