SUMMARY:

• Hadoop Developer having 8+ Years of professional IT experience which includes 4 years of experience in Big data ecosystem related technologies like Hadoop HDFS, Map Reduce, Apache Pig, Hive, Sqoop, HBASE, Kafka,Flume , Oozie, Zookeeper and Spark.
• Experienced in installing, configuring and monitoring Hadoop clusters.
• Accomplished Big - Data Hadoop Stack using Cloudera, Hortonworks, MapR Platforms.
• Superior background in object oriented development including C, C++, Java, Scala, Python and shell Scripting.
• Acquaintance on NOSQL databases like HBASE, Cassandra, and MONGODB.
• Written Database Queries, creating stored Procedures, Views, Indexes, Triggers, Functions, code optimization and performance.
• Trained in manipulating the streaming data to clusters through Flume .
• Optimized streaming log files with no time latency using Flume and more importantly operating the data down stream flow to Hadoop ecosystems and it analysis segments.
• Worked on Oozie and Zookeeper to automate the flow of jobs and coordination in the cluster respectively.
• Successfully integrated Kafka and Spark for Real Time processing.
• Experience on Kafka Mirror Maker setup, data replication across data centers, Kafka topics, replication and partitions creations.
• Experience on Bash/Shell scripts for automation and testing jobs.
• Developed Spark scripts by using Scala Shell commands as per the requirement.
• Good knowledge on creating RDD's, Pair RDD's, Transformations and Actions in Spark.
• Responsible for creating Data Frames and DStreams for Relational data processing in Spark SQL.
• Managed both Waterfall and Agile methodologies.
• Facilitate in reviewing and analyzing required documents, understanding business process flow diagrams and use cases.
• Familiar in creation of reusable components using multiple technologies.
• Knowledge on ORC, RC, Sequence File, AVRO and Parquet file formats.
• Good working skills on cloud platforms like AWS, Azure.
• Java unit and integration test experience with frameworks such as Junit, Mockito, Testing.
• Involved in the development of interfaces for the application using JSP, Servlets, and JavaScript.
• Experience in developing Shell scripts and Python Scripts for system management.
• Comprehensive knowledge of Software Development Life Cycle coupled with excellent communication skills.
• Highly motivated team player with the ability to work independently and adapt quickly to new and emerging technologies.
• Proficient in verbal and written communication and pragmatic team player.

TECHNICAL SKILLS:

Hadoop Technologies: MapReduce, HDFS, Hive, Pig, Impala, Sqoop, Flume, Oozie, Zookeeper, Spark, KafkaNOSQL databases: HBASE, Cassandra, MongoDB

Hadoop Distributions: Cloudera, Hortonworks, MapR

Programming Languages: C, C++, Java, Python, Scala, Shell scripting

Frame Works: Spring, Hibernate, Struts, EJB, JMS, JSF

Databases: Oracle, MySQL, Teradata, DB2, SQL Server

IDE/Tools: Maven, Version One, GIT HUB, Eclipse, Clear Case, Clear Quest

Platforms: Windows XP, 7, 8.1, Unix, Linux Centos, RHEL

Web/Application Servers: Web Sphere, Web logic, Helix, Tomcat

PROFESSIONAL EXPERIENCE:

Sr.Hadoop Developer

Confidential, MN

Responsibilities:

• Proactively monitored systems and services, architecture design and implementation of Hadoop deployment configuration management, backup, and disaster recovery systems and procedures.
• Skilled in writing MapReduce Programs and using Apache Hadoop API for analyzing the structured and unstructured data.
• Created Sqoop jobs for both Historical and Incremental data migration from legacy systems.
• Involved in transferring Streaming data from different data sources into HDFS and NoSQL databases using Apache Flume.
• Used Hive to analyse the partitioned and bucketed data and compute various metrics for reporting on the dashboard.
• Automated all the jobs for pulling data from FTP server to load data into Hive tables using Oozie workflows.
• Created and maintained Technical documentation for launching HADOOP Clusters and for executing Hive queries and Pig Scripts.
• Developed Pig Custom UDF's for custom input formats for performing various levels of optimization.
• Optimized Map reduce algorithm using combiners and partitions to deliver the best results and worked on Application performance optimization for a HDFS cluster.
• Developed Kafka producer and consumer components for real time data processing.
• Loading Data into Hbase using Bulk Load and Non-bulk load.
• Used Spark for fast processing of data in Hive and HDFS.
• Done batch processing of data sources using Apache Spark, Elastic search.
• Created Spark SQL jobs with various transformations and actions for data processing.
• Import the data from different sources like HDFS/Hbase into Spark RDD.
• Created Spark jobs with RDD's, Pair RDD's, Transformations and Actions, Data Frames for Data transformation from relational stores.
• Involved in converting Map Reduce programs into Spark transformations using Spark RDD's on Scala.
• Configured Tableau to Hive data and also used Spark as execution engine for Tableau instead of MapReduce.
• Responsible for creating Hive tables using Partitions, Buckets, UDF's, HQL Scripts in landing layers for analytics.
• Implemented Spark Streaming to read real-time data from Kafka in parallel and processed in parallel and save the result as parquet format in Hive.
• Responsible for user stories creations, tracking and delivering as per sprint.

Environment: Cloudera Manager, Hadoop, HDFS, Spark, MapReduce, Pig, Hive, HBASE, Sqoop, Oozie, Flume, Scala, Python, Java, XML, SQL Server.

Hadoop Developer

Confidential, Chicago

Responsibilities:

• Installed and benchmarked Hadoop HBase clusters for internal use.
• Provided a solution using HIVE, SQOOP (to export/ import data), for faster data load by replacing the traditional ETL process with HDFS for loading data to target tables.
• Implemented MapReduce Custom File Formats, Custom Writable and Custom Practitioners.
• Involved in collecting, aggregating and moving data from servers to HDFS using Apache Flume and Talend.
• Performed custom interceptors for flume to filter data as per requirement.
• Maintaining and monitoring Hive data warehouse tool-creating tables, data distribution by implementing partitioning and bucketing, writing and optimizing the Hive QL queries.
• Designed Pig Latin scripts to sort, group, join and filter the data as part of data transformation as per the business requirements.
• Data files were merged and loaded into HDFS using java code and tracking history related to merge files are maintained in HBASE.
• Implemented a script to transmit information from Oracle to Hbase using Sqoop.
• Created data ingestion plans for loading the data from external sources using Sqoop.
• Extensively worked setting up workflow using Apache Oozie workflow engine for managing and scheduling Hadoop jobs using Oozie-Coordinator.
• Performed transformations, cleaning and filtering on imported data using Hive, Map Reduce, and loaded final data into HDFS.
• Developed Spark code using scala and Spark-SQL/Streaming for faster testing and processing of data.
• Worked collaboratively with different teams to smoothly slide the project to production.

Environment: Hortonworks HDP-2.2 YARN cluster, HDFS, Map Reduce, Apache Hive, Apache Pig, HBase, Talend Studio, Apache Spark, Sqoop, Oozie, XML.

Hadoop Developer

Confidential, NC

Responsibilities:

• Excellent understanding of Hadoop architecture and different demons of Hadoop clusters which include Resource Manager, Node Manager, Name Node and Data Node.
• Involved in defining job flows, managing and reviewing log files.
• Imported and exported data using Sqoop from HDFS to relational database system and vice-versa.
• Familiar in configuring and working with Flume to load the data from multiple sources directly into MongoDB.
• Configured streaming log data into MongoDB from web servers using Flume.
• Analyzed the Big Data business requirements and transformed it into Hadoop centric technologies.
• Responsible to write MapReduce Programs and using Hadoop API for analyzing the structured and unstructured data.
• Proficient on various daemon processes like Job Tracker, Task Tracker, Name Node and Data Node.
• Worked with application teams to install operating system, Hadoop updates, patches, version upgrades as required.
• Developed Pig Latin scripts using operators such as LOAD, STORE, DUMP, FILTER, DISTINCT, FOREACH, GENERATE, GROUP, ORDER, LIMIT, UNION, to extract data from data files to load into HDFS.
• Written HBASE Client program in Java and web services.
• Extending Hive and Pig core functionality by writing custom UDFs.
• Used Pig Latin scripts to extract the data from the output files, process it and load into HDFS.
• Used Kafka for conjunction with Zookeeper for deployment management, which necessitates monitoring its metrics alongside Kafka clusters.

Environment: Hadoop, MapReduce, Hive, Pig, Shell Scripting, Eclipse, MongoDB, Sqoop, Kafka, Flume, XML.

Java Developer

Confidential

Responsibilities:

• Implemented project using Agile, scrum methodology. Involved in daily standup meeting, spring showcase and spring retrospective.
• Developed Spring MVC framework which includes writing Controller classes for handling requests, processing form submissions and also performed validations.
• Designed and developed User Interface using HTML, JSP, JSP Custom Tag and AJAX .
• Took stand for client side validation using JaveScript and Jquery.
• Optimized J2EE design patterns like Data Access Object and Business Delegate to handle the business flow.
• Extensively worked on Spring AOP and integrated the application using Spring dependency injection.
• Coded JUnit test cases to test the application and performed random checks to analysis the portability, reliability and flexibility of the project.
• Used Log4J for tracking errors and debugging the code.
• Validated user interface Using JSP , JavaScript, HTML, CSS mapped to Spring Framework controller, to simplify the Application complexity and easy usage to User.
• Involved in writing Multi-Threading environment in Java to improve the performance of merging operations.
• Coded Junit, Pit coverage for the application in Enterprise level. Resolved the defects and made some changes to code to get App worked in the entire environment.
• Worked on HPSM (HP service management) tool for incident management and resolving the Bugs in the production phase.
• Experience in developing service components using JDBC .
• Used various J2EE design patterns, like DTO, DAO, and Business Delegate, Service Locator, Session Facade, Singleton and Factory patterns.
• Extensive usage of Rest full web services throughout modules to communicate with all external system.
• Developed Java beans to be used by JSP pages for presenting web content.
• Worked with multithreaded custom framework similar to Spring-Batch using OSGI and caching (Dynamic and non Dynamic) to develop end of the day batch loaders.

Environment: Spring MVC, Oracle 10g, Hibernate, Restful Web services, My Eclipse IDE, Websphere Application Server, Java, JSP, HTML, JMS, Java script, JQuery, SQl, UNIX, DB2, JIRA, and Log4j.

Java Developer

Confidential

Responsibilities:

• Used Web Sphere for developing use cases, sequence diagrams and preliminary class diagrams for the system in UML.
• Extensively used Web Sphere Studio Application Developer for building, testing, and deploying applications.
• Developed the presentation layer and GUI framework in HTML, JSP and Client-Side validations were done.
• Involved in Java code, which generated XML document, which in turn used XSLT to translate the content into HTML to present to GUI.
• Implemented XQuery and XPath for querying and node selection based on the client input XML files to create Java Objects.
• Used Web Sphere to develop the Entity Beans where transaction persistence is required and JDBC was used to connect to the MySQL database.
• Developed the user interface using the JSP pages and DHTML to design the dynamic HTML pages.
• Developed Session Beans on Web Sphere for the transactions in the application.
• Utilized WSAD to create JSP, Servlets, and EJB that pulled information from a DB2 database and sent to a front end GUI for end users.
• Used Struts framework for presentation layer.
• Implemented UI Layer with Struts, Struts validation, Struts Tiles, JSP, XML, CSS, HTML, JavaScript, AJAX and Angular JS.
• In the database major responsibilities includes creation of tables, triggers, stored procedures, sub-queries, joins, integrity constraints and views.
• Performed requirements analysis as per the client requirements.
• Proven ability to work with senior technical managers and staff to provide expert-level support for the installation, maintenance, upgrading, and administration of full-featured database management systems.

Environment: Java, WebSphere, Versata, Clear Case, Clear Quest, HTML, JAVA, J2EE, EJB, JSP, JavaScript, Servlets, Websphere, WSAD, Oracle, SQL, Web Services, SOAP, SDLC, Ant, Struts, Eclips, XML, RAD.

SQL Developer

Confidential

Responsibilities:

• Converted Data Transformation Services (DTS) application to SQL server Integration services (SSIS) as assigned.
• Developed DTS packages to extract, transform and load into the Campaign database from OLTP database using of SQL Server Integration Services (SSIS).
• Designed dynamic SSIS Packages to transfer data crossing different platforms, validate data during transferring, and archived data files for different DBMS.
• Extracted, Transformed and Loaded (ETL) data from Excel, Flat file, Oracle to MS SQL Server by using BCP utility, DTS and SSIS services.
• Perform unit testing at various levels of the ETL.
• Worked on Migration of packages from DTS using SQL Server IntegrationService.
• Generating reports using SQL Reporting Services (SSRS) for customized and ad-hoc Queries.
• Involved in Report writing according to business requirement using SSRS.
• Generated database monitoring and data validation reports in SQL Server Reporting Service (SSRS).
• Wrote Triggers and Stored Procedures to capture updated and deleted data from OLTP systems.
• Designed and created the table schema and stored procedures that will be used in Data QC.
• Daily support of system wide replication tasks including monitoring, alerting, and problem resolution.
• Responsible for documentation of system related activities.
• Setup replication within the sql databases.
• Provides technical documentation of the system
• Setup the database with suitable memory and hard disk space allocation, creating different user and security profiles for the team.

Environment: SQLServer2008, MS Access, Query Analyzer, DTS, BCP, MS Access, Load Runner, QTP and Win Runner, SQL Profiler, Import & Export Data, XML, Windows XP.