Senior Hadoop Developer Resume
New, JerseY
SUMMARY:
- Hadoop Developer wif around 9.5 years of experience in Information Technology out of which 2.5+ years in Hadoop Ecosystem and around 6 plus years wif Oracle E - Business Suit.
- Expertise in Hadoop Ecosystem components HDFS, Map Reduce, Hive, Pig, Sqoop, NoSQL, Hbase and Oozie.
- Capable of processing large sets of structured, semi-structured and unstructured data sets.
- Performed Extract, Transform and Load (ETL) using Hive on huge volume of Verizon Wireless data.
- Managed BIG DATA Ingestion, Data Transformation using PIG, HIVE and HBASE. Implemented Pig UDF, UDAF and Hive UDF, UDAF functions to handle custom business logic.
- Extensive knowledge of creating manage tables and external tables in Hive Eco system
- Worked extensively on Pig Latin and Hive Queries to generate daily/weekly/monthly and ad hoc reports on huge data sets. Integrated Tableau wif Hive tables to visualize reports.
- Experienced in writing MapReduce programs to process large sets of structured, semi-structured and unstructured data sets and store them in HDFS using Pig, Java and Python.
- Worked closely wif data scientists to find solutions to business problems using predictive analytics, pattern matching and data classifications.
- Extensively used ETL methodology for supporting Data Extraction, transformations and loading processing, using Sqoop.
- Skilled on migrating teh data from different databases to Hadoop HDFS and Hive using Sqoop.
- Experience in job workflow scheduling and monitoring tools like Oozie.
- Proficient in designing and querying teh NoSQL databases like HBase.
- Deep Knowledge in teh core concepts of MapReduce Framework and Hadoop ecosystem
- Extensive knowledge of creating manage tables and external tables in Hive Eco system.
- Worked extensively in design and development of business process using SQOOP, PIG, HIVE, HBASE
- Good knowledge in Software Development Life Cycle (SDLC) and Software Testing Life Cycle.
- Extensive experience in developing stored procedures, functions, Views and Triggers, Complex queries using SQL Server, Oracle PL/SQL and SQL*Loader.
- Excellent communication and inter-personal skills detail oriented, analytical, time bound, responsible team player and ability to coordinate in a team environment and possesses high degree of self-motivation and a quick learner.
TECHNICAL SKILLS:
Big Data Technologies: Hadoop Architecture, HDFS, Map Reduce, Hive, Pig, Hbase, NoSQL, Sqoop, Oozie
Databases: Oracle 9i/10g, Teradata
Coding Language: SQL, PL/SQL, C, Shell Script, Core Java, Python
Development Tools: Forms6i, Reports6i, XML Publisher, Oracle Alerts
Tools: Toad, SQL * Loader, SQL Developer, Filezilla, SVN, PVCS, HPPM
Operating Systems: Windows, Linux (Cent OS, Ubuntu)
ERP: Oracle Applications 11i & R12 (PO, AP, AR, GL, INV, AOL)
PROFESSIONAL EXPERIENCE:
Confidential, New Jersey
Senior Hadoop Developer
Responsibilities:
- Worked closely wif data scientists for data gathering enterprise level to predict consumer behavior, such as Call In Rate, Battery Drain, Call Drop, SMS Message Drop and made recommendations based on recognizing patterns.
- Involved in end to end data processing like ingestion, processing, quality checks and splitting
- Bringing teh data into Big Data Lake using Pig, Sqoop and Hive
- Wrote Pig scripts to process unstructured data and create structure data for use wif Hive
- Importing Large Data Sets from Oracle, Teradata to Hive Table using Sqoop.
- Created Hive Managed and External Tables as per teh requirements
- Designing and developing tables in HBase and storing aggregating data from Hive
- Developing Hive Scripts for data aggregating and processing as per teh Use Case.
- Writing Java, Python Custom UDF's for processing data in Hive and Pig.
- Developing and maintaining Workflow Scheduling Jobs in Oozie for importing data from RDBMS to Hive.
- Teh Hive tables created as per requirement were managed or external tables defined wif appropriate static and dynamic partitions, intended for efficiency.
- Implemented Partitioning, Bucketing in Hive for better organization of teh data
- Optimized Hive queries for performance tuning.
- Developed data pipeline using Sqoop, Pig and Java map reduce to ingest customer behavioral data and purchase histories into HDFS for analysis.
- Involved wif teh team of fetching live stream data from Oracle to Hive, Hbase table using Apache Kafka.
Environment: Hadoop v2.6.0, HDFS, Map Reduce, HBase, Sqoop, Core Java, Hive,Pig, Oozie,Oracle,Teradata,Linux
Confidential
Hadoop Developer/Oracle Apps Technical Consultant
Responsibilities:
- Working wif business users and business analyst for requirements gathering and business analysis.
- Converted business requirement into high level and low level design.
- Worked on teh proof-of-concept for Apache Hadoop 1.20.2 framework initiation.
- Implemented Proof of concepts on Hadoop stack and different bigdata analytic tools, migration from different databases(Oracle, MySQL) to Hadoop
- Installed and configured Hadoop clusters and eco-system.
- Monitored Hadoop cluster job performance and capacity planning
- Hands-on experience wif Hadoop technology stack (HDFS, MapReduce, Hive,Oozie)
- Automated all teh jobs starting from pulling teh Data from Oracle to pushing teh result set data to Hadoop Distributed File System using Sqoop.
- Distributing teh work between offshore team member and tracking teh development progress.
- Discussing teh technical way of development wif onsite and offshore team lead.
- Doing teh review of code developed from offshore.
- Review and Analysis of Functional Specifications.
- Interaction wif Functional Specification owner on various functional issues during development.
- Development of Technical Specifications.
- Developed XML reports as per client requirement.
- Registration of Reports, Concurrent Programs.
- Prepared MD120 and UTP for developed components.
- Interacting wif Functional Consultants for requirements gathering and analysis.
- Modification and enhancement of Oracle Reports and Interfaces.
- Providing all kinds of production support for Business Users.
- Working closely wif Functional Consultants, Sys-admin & DBA team for regular patch Applications, Outages, Data Fix, and Code Fix.
- Involved in testing teh modified reports and interface programs.
- Developed Pig Latin scripts for teh log data and stored teh cleansed data in teh Apache Hadoop.
Environment: Scoop, Hive, Oozie, Unix, Oracle9i, Oracle Applications11i & R12
Confidential
Oracle Apps Technical Consultant
Responsibilities:
- Gathering connectivity details from respective SME owners.
- Converting teh script from FTP to SFTP Connection.
- Involved in testing teh modified scripts.
- Interacting wif SME owners for their
Environment: Oracle Applications 11i, Unix, PVCS, HPPM
Confidential
Oracle Apps Technical Consultant
Responsibilities:
- Review and Analysis of Functional Specifications
- Interaction wif Functional Specification owner on various functional issues during development
- Development of Technical Specifications
- Developed Bank Statements Load Interface program
- Developed e-text template for payment process request program as per client required format.
- Created Notifications using Oracle Alerts as per client requirement.
- Developed XML reports as per client requirement.
- Registration of Reports, Concurrent Programs.
- Prepared MD120 and UTP for developed components.
- Developed Validation scripts in PL/SQL for importing teh Invoices from teh legacy system into Oracle Payables module.
- Developed GL Daily conversion rates program as per client requirements.
- Developed and Customization of Reports and Forms.
- Creation of Value Sets.
- Prepared test cases for developed components.
- Validation scripts in PL/SQL for importing teh Item Category Assignments from teh legacy system into Oracle Inventory module.
- Developed Item Outbound Interface as per client requirements.
- Validation scripts in PL/SQL for importing teh Invoices from teh legacy system into AP module
- Registration of concurrent programs for developed reports and Interfaces.
- Creation of request sets for developed interface.
- Building Technical Design documents MD 070 based on MD 050.
- Developed and Customized XML reports as per client requirement.
- Creation of Value Sets and Assigning to concurrent programs
- Developed interface to load data from legacy systems to oracle 11i system
- Registration of concurrent programs for developed reports and Interfaces
- Development of 6i and 10g reports.
- Creation of Value Sets and Assigning to concurrent programs
- Prepared Construction Work book and Unit Test plan.
Environment: Forms6i, Reports6i, XML Publisher, Oracle Alerts, Oracle Applications 11i & R12
Confidential
Oracle Apps Technical Consultant
Responsibilities:
- Developed interface to load Purchase Orders from legacy systems to oracle apps systems.
- Registration of concurrent programs for developed reports and Interfaces.
- Developing Code from Technical Design.
- Creation of Value Sets and Assigning to concurrent programs.
- Involved in Support, Enhancements / Bug fixing.
- Involved in development of GUI Screens using Forms 6i
- Involved in development of Reports using Reports 6i.
- Developed Procedures, Functions and Packages.
- Modifications are undertaken and done as a part of change management.
- Involved in testing.
Environment: Forms6i, Reports6i, Oracle9i, Oracle Applications 11i