SUMMARY

• Over 8+ years of IT experience with 2+ year of experience in development and maintenance of Big Data Hadoop ecosystem's.
• Experienced in all phases of the software development lifecycle, Python, Java and Strong working experience in Big Data Hadoop - HDFS, MapReduce, Pig, Hive, Spark SQL, Sqoop, Oozie, Hue.
• Hands on experience in writing custom UDF for improving Pig and Hive scripts with object oriented programming using Java or Python.
• Experience using Sqoop to import/export of data from HDFS to Relational Database Management systems.
• Experience in back-end procedure development for RDBMS, writing Sub Queries, Stored Procedures, Triggers, Cursors and Functions on MySQL and DB2 database.
• Strong project management experience, skilled in requirements elicitation and project documentation within small and large organizations and across the enterprise
• Support wide variety of adhoc data needs and build high-volume real-time data processing applications.
• Strong business orientation and experience in development and testing on IBM Mainframe with Cobol, DB2, VSAM, JCL, CICS, Endevor, File-Aid, SPUFI, CA7 Scheduler, Jobtrac, SyncSort
• Worked closely with users and project managers and Business Analyst to understand system requirements to develop different testing for Client server.
• Familiar with data architecture including data ingestion pipeline design, Hadoop information architecture, data modeling and data mining and advanced data processing.
• Efficient and effective facilitator. Excellent organization and planning skills. Exceptional oral and written communication skills and interpersonal skills
• Have worked in an onsite/offshore model, possess excellent client interaction skills and have experience communicating at all levels within an organization.
• Highly motivated, detailed oriented programmer with experience on working independently as well as lead a team effectively.

TECHNICAL SKILLS

Big Data Ecosystem: Hadoop, HDFS, MapReduce, PIG, HIVE, Sqoop, Oozie, Spark SQL

Language: Java, Python

Mainframe: COBOL, DB2, VSAM, JCL, CICS, Endevor, File-Aid, SPUFI, CA7 Scheduler, JOBTRAC, SYNCSORT

Database: MySQL, DB2, MS Access, SQL

Operating Systems: LINUX, Windows, Z/OS

Microsoft tools: MS Word, MS Excel, Visio, POWERPOINT & MS Project management

PROFESSIONAL EXPERIENCE

Hadoop Developer/Team Lead

Confidential, Melville, NY

Responsibilities:

• Involved in Requirement gathering, analysis, design, coding and implementation of Big Data Hadoop ecosystem.
• On-hand experience in Sqoop to Import/export data from Hadoop Distributed File System (HDFS) to RDBMS.
• Developed Spark SQL to improve the Hive performance queries in Hadoop
• Written MapReduce programs to parse the raw data, written PIG scripts to transforming the data from different sources into forming baseline data.
• Performed Data transformations in HIVE and used partitions, buckets for performance improvements.
• Developed UDF in Java or Python to support custom hive and pig capabilities.
• Solved performance issues in Hive and Pig scripts by understanding Joins, Group, aggregation and how does it translate to MapReduce jobs.
• Good understanding on Spark SQL queries for analyzing and processing of raw data.
• Have Good Understanding knowledge on NOSQL database (HBase)
• Used different data formats (Text, ORC & Avro format) while loading the data into HDFS.
• Assisted in creating and maintaining technical documentation to launching Hadoop Clusters and even for executing Hive queries and Pig Scripts.
• Proactively monitored systems and services, architecture design and implementation of Hadoop deployment, configuration management, backup, and disaster recovery systems and procedures.
• Integrated Hive, Pig scripts, Sqoop and shell with Oozie workflows for performing ETL transformations on the transactional and application specific data sources.
• Write SQL's to generate data required in the final reports and pass these SQL's to the developers who would then convert these SQL into MapReduce programs.
• Communicate deliverables status to user/stakeholders, client and drive periodic review meetings

Environment: Hadoop-2.4.1, Hive-0.14.0, Pig-0.13.0, Map Reduce, Sqoop-1.4.3, Spark 1.5.2, Oozie, Python 3.5, Java, DB2, MySQL, Linux and CDH4.X.

Hadoop Developer

Confidential, Melville, NY

Responsibilities:

• On-hand experience in Hadoop, HDFS, developed MapReduce jobs in Pig, HIVE and processing data cleaning.
• On-hand experience in Sqoop to Import/export data from Hadoop Distributed File System (HDFS) to RDBMS.
• Processed data into HDFS by developing solutions, analyzed the data using Map Reduce, Pig, Hive and produce summary results from Hadoop to downstream systems
• Involved in creating Hive tables, loading data and writing Hive queries, which will run internally
• Established custom Map Reduces programs to analyze data and used Pig Latin to clean unwanted data.
• Worked on various performance optimizations like using distributed cache for small datasets, Knowledge on Partition, Bucketing in Hive and Map Side joins.
• Involved in Unit testing and delivered Unit test plans and results documents
• Proactively monitored systems and services, architecture design and implementation of Hadoop deployment, configuration management, backup, and disaster recovery systems and procedures.
• Requirement Elicitation on multiple merchant applications to support new infrastructure - BI front end applications, ETL to provide financial reporting.
• Communicate deliverables status to user/stakeholders, client and drive periodic review meetings

Environment: Hadoop-2.4.1, MapReduce, Pig-0.11.1, Hive-0.10.0, Sqoop-1.4.3, Oozie, Zookeeper, Python, Java, MySQL, DB2, Linux and Windows.

Team Lead/ Programmer Analyst

Confidential, Charlotte, NY

Responsibilities:

• Lead store enhancement and maintenance activities with successfully implementation of codes with minimal or zero defects.
• Reviewed Business and Functional requirement and determine the functionality of the application
• Analyzed user requirements by interacting with developers and business analysts and created test cases.
• Played a key role in most of the enhancement as well as successfully implemented it with zero defects.
• Ensure the proper review and proper testing for the enhancement before deployed it into the production environment and ensure the same thing followed by the other teammates
• Ensure the proper Confidential standard was maintained in all the life cycle for all the teammates and ensure it’s done across other team
• As a senior resource provided right direction and right resolution in time to meet the enhancement deadlines.
• Schedule a JOB as per the requirement in Production and verified the complete execution of all the JOB’s
• Involved in performance tuning on SQL to reduce the CPU utilization
• Learned and used on demand ADHOC technologies/Utilities like REXX, DYL, and SORT and given the short training to the other teams.

Environment: JCL, COBOL, DB2, VSAM, CICS, MS-ACCESS, Windows XP, IE

Programmer Analyst

Confidential, Chicago, IL

Responsibilities:

• Reviewed Business and Functional requirement and determine the functionality of the application
• Ensure the proper review and proper testing for the application before deployed it into the production environment and ensure the same thing followed by the other teammates
• Preparation of package and review the package for implement in QA and PROD region

Environment: JCL, COBOL, DB2, SQL, MS Access, Ms-Excel