Objective

A results-driven, customer-focused, articulate and analytical certified Hadoop Engineer in building complex technical solutions with 6 years of IT experience in Big Data ecosystem related technologies.

Professional summary

• Expertise in designing and architecting large-scale Hadoop deployments for production, development and testing environments, which include planning, configuration, installation, performance tuning.
• Excellent understanding of Hadoop YARN architecture
• Hands on experience in installing, configuring, and using Hadoop ecosystem components like Hadoop Map Reduce, HDFS, HBase, Oozie, Hive, Sqoop, Pig, and Flume.
• Experience in planning, designing and developing applications spanning full life cycle of software development from writing functional specification, designing, implementing, documentation, unit testing and support.
• Experience in analyzing data using HiveQL, Pig Latin, custom Map Reduce programs in Java and No SQL data stores like HBase.
• Experience in installation, configuration, supporting and managing - Cloudera's Hadoop platform along with CDH3 4 clusters.
• Experience in working with cloud environments such as Amazon Web Services AWS EC2/EMR/S3.
• Worked on developing ETL processes to load data from multiple data sources to HDFS using FLUME and SQOOP, perform structural modifications using MapReduce, HIVE and analyze data using visualization/reporting tools.
• Hands on experience in application development using RDBMS, and Linux shell scripting.
• Experience in developing pipelines and processing data from various sources, processing them with Hive and Pig.
• Extensively worked on database applications using DB2 UDB, Oracle, SQL Plus, PL/SQL, SQL Loader.
• Solid understanding of the high volume, high performance systems.
• Experience in developing pipelines and processing data from various sources, processing them with Hive and Pig.
• Good Understanding on various SDLC's including Agile Engineering.
• Self-starter and ability to adapt by learning new things quickly which will make me more effective and efficient towards reaching the company's goals.

Skill synopsis

Professional experience

Confidential

Hadoop Developer

Responsibilities:

• Worked on the Log Analytics platform, which collects usage data from CRM users and determine popularity of features, adoption of newly added features and retire unused features.
• Involved in extracting customer's Big data from various data sources into Hadoop HDFS. This included data from Excel, ERP systems, databases and also log data from servers.
• Worked on data ingestion pipelines that bring log data, via Flume, from app servers into HDFS.
• Developed Pig scripts to process raw log data and load them into Hive tables.
• Used Hive data warehouse tool to analyze the unified historic data in HDFS to identify issues and behavioral patterns.
• Benchmarked and improved performance using de-normalization, partitioning and bucketing as appropriate.
• The Hive tables created as per requirement were internal or external tables defined with appropriate static and dynamic partitions, intended for efficiency.
• Improved performance of business analysts HiveQL queries.
• Summarized / aggregated data using HiveQL, exported and loaded the data into existing Oracle DW/BI environment using Sqoop.
• Used Oozie workflow engine to manage interdependent Hadoop jobs and to automate several types of Hadoop jobs such as Flume, Hive, Pig and Sqoop.
• Worked with BI teams in generating the reports and designing ETL workflows on Pentaho, Qlikview BI tool.
• Involved in Analyzing system failures, identifying root causes, and recommended course of actions.
• Shared responsibility for administration of Hadoop, Hive and Pig.

Environment: Hadoop 2.2, Hive 0.13.0, Pig 0.12.0, Sqoop 1.4, Hue 2.4, Flume 1.4.0, RHEL 6.4, Oozie 4.0.0, Oracle 11g

Confidential

Hadoop Developer

Responsibilities:

• Worked on the Campaign and User Data Warehouse components of the Real Time Bidding platform.
• Designed and developed data flow and processing pipelines for ingestion of data from Data Management Platforms DMPs such as Nielsen, BlueKai, Datalogix and Polk.
• Data was obtained from DMPs via SFTP / WebServices push / pull methods throughout the day.
• Responsible for sourcing data from additional DMPs on a need basis.
• Migration of queries from MySQL Database to Hive using Sqoop.
• Processed CSV, XML and JSON data, transformed it into a common CSV structure and loaded it into Hive tables.
• Used Pig as ETL tool to do transformations, event joins and some pre-aggregations.
• Performed analytics using HiveQL, exported and loaded the data into existing MySQL DB for reporting using Sqoop.
• Created HBase tables to load large sets of structured, semi-structured and unstructured data coming from a variety of portfolios.
• Responsible for reviewing end-user HiveQL queries, rewriting them to improve performance
• Developed Oozie workflows to automate the process of ingesting to HDFS Performed a POC on Sqoop imports from heterogeneous data sources to HDFS.

Environment: CDH4, Hadoop 2.0, Hive 0.12.0, Pig 0.12.0, Sqoop 1.4, Oozie 4.0.0, HBASE 0.94,RHEL 6.4, MySQL 5.1

Confidential

Role: Java Developer

Roles Responsibilities

• Modeling conceptual design using Use Case, UML Class, and Activity diagrams using Rational Rose
• Used JSP, JavaScript, HTML, AJAX, CSS, and XSLT to create the user interface views
• Involved in the development of Enterprise Java Beans EJB's to implement Business logic for maintaining state information and database operations
• Deployed the application using WebLogic Application Server
• Experienced in writing SQL and PL/SQL scripts including Stored Procedures, functions, packages and triggers
• Implemented Database access through JDBC at Server end with Oracle.
• Used Spring Aspect Oriented Programming AOP for addressing cross cutting concerns.
• Developed request/response paradigm by using Spring Controllers, Inversion of Control and Dependency Injection with Spring MVC.
• Object relational mapping and Persistence mechanism is executed using Hibernate ORM.
• Used Web Services like SOAP and WSDL to communicate over internet.
• Involved in the JMS Connection Pool and the implementation of publish and subscribe using Spring JMS.
• Used CVS for version control and Log4j for logging.
• Used JProbe and JConsole to profile application for memory leaks and resource utilization.
• Developed test classes in JUnit for implementing unit testing.

Environment: Java, JDK1.5, J2EE, JDBC, Servlets, JSP, JSTL, EJB, HTML, AJAX, Spring Framework, Java Script, CSS, XSLT, XML. JUnit, WebServices, SOAP, WSDL, Weblogic, CVS, Rational Application Developer RAD, Hibernate, Rational Rose, JMS.

Confidential

Role: Java Developer

Roles Responsibilities:

• Interacting with the business and user group to ascertain requirements to carry out Business Reengineering and Business Process Managements processes with detailed functional and technical analysis with OOAD concepts and Agile RUP Methodology.
• Extensively used Structs component classes for developing applications for handling user requests.
• Involved in High Level Design and prepared Logical view of the Application.
• Involved in designing and developing of Object Oriented methodologies using UML and Created User Case, Class, Sequences diagrams.
• Developed Data Access Classes using the Hibernate.
• Implemented Database access through SQL and HQL at Server end with DB2.
• Involving in writing Stored Procedures, Functional, and Triggers.
• Implemented EJB Components using State less Session Bean and State full session beans.
• Created java Interfaces and Abstract classes for different functionalities.
• Implemented Multithread concepts in java classes to avoid deadlocking.
• Implemented java design patterns like Singleton, Factory, Command Patterns.
• Communicated with external applications JMS messages using MQ Series.
• Written independent JavaScript, CSS files and reused in UI pages.
• Developed persistence layer using ORM hibernate for transparently store objects into database.
• Used JUnit to write repeatable tests mainly for unit testing.

Environment: Java, Servlets, Structs, JSF, Swing, JSTL, XML, ClearCase, XSLT, JMS, UML, Log4J, BD2, Web services, WSDL,, EJB, JavaScript, JQuery, SOAP, Maven, Quality Center, PL/SQL Developer, JUnit.