Hadoop Big Data Consultant Resume
New, YorK
PROFESSIONAL SUMMARY:
- Big Data Consultant having around 9 years of experience in analysis, design, development, testing and implementation of Applications in Financial, Retail and Healthcare Industries.
- Technical expertise in Big data/HadoopHDFS, Map Reduce, Spark, HIVE, PIG, Sqoop, Flume, Oozie, NoSQL Data bases HBase, Cassandra, MongoDB, SAS/SQL, UE STUDIO, Unix Scripting.
- Extensively used ETL to transform and load data from Teradata database, XML files, flat files to Oracle.
- Good knowledge ofHadoopArchitecture and various components such as HDFS, Job Tracker, Task Tracker, Name Node, Data Node, and Map Reduce concepts.
- Experience in using Pig, Hive, Sqoop and Cloudera Manager.
- Experience in importing and exporting data using Sqoop from HDFS to Relational Database Systems and vice - versa.
- Hands on experience in RDBMS and Unix shell scripting.
- Extending Hive and Pig core functionality by writing custom UDFs.
- Experience in analyzing data using HiveQL, Pig and Map Reduce.
- Knowledge on NoSQL databases including HBase, Cassandra.
- Knowledge in job work-flow scheduling and monitoring tools like Oozie.
- Possess expertise in Data management, data extraction, manipulation, validation, and analyzing huge volume of data.
- Experience in extracting data from relational databases like Oracle, Sybase and Teradata 14.1.
- Experience in working with large technical teams to understand requirements and to develop code based on SDLC procedures.
- Detail oriented, organize and enthusiastic to work in a fast paced and team oriented environment
- Possess strong communication skills and problem solving skills.
TECHNICAL SKILLS:
BIg Data Skills: Hadoop, Map Reduce, Hive, Pig, Sqoop, HBase(NoSQL database) and UNIX Shell Scripting.
ETL & Reporting Tools: SAS/MACROS SAS/SQL, Ab- Initio (GDE 3.12.5.2, Operating system 3.12.3)
RDBMS: Teradata 14.1, Oracle 8i/9i/10g, SQL Server, .
DB Tool: SQL Advantage, TOAD, Teradata SQL Assistant
Platforms: Sun Solaris 7.0, HP-UX UNIX, Windows NT/XP/2000/98/95, MS- DOS.
WORK EXPERIENCE:
Confidential, New York
Hadoop Big Data Consultant
Responsibilities:
- Import/export data from Teradata database to/from HDFS using Sqoop and Hue.
- Implemented Custom Input formats that handles input files received from java applications to process in Map Reduce.
- Implemented Filter Mappers to eliminate un-necessary records.
- Created partitions, bucketing in Hive to handle structured data.
- Implemented Dash boards that handle Hive queries internally like arithmetic functions, basic hive operations, and different kinds of joins.
- Implemented business logic based on state in Hive using Generic UDF's.
- Managing and scheduling batch Jobs on aHadoopCluster using Oozie.
- Created production jobs using Oozie work flows that integrated different actions like Map Reduce, Sqoop, Hive.
- Experience in managing and reviewingHadoopLog files.
- Designing Test Plans, Test Cases and performed System Testing.
Environment: Hadoop, Map Reduce, Hive, Sqoop, Oozie, Cassandra, Unix, Teradata
Confidential, Newyork
Sr. SAS Developer
Responsibilities:
- Extracting the data from the source system (Unix Server) to the staging area.
- Tranforming the data using Abinitio Graphs. An Abinitio graph would be designed for this purpose.
- The transformed data would be converted into the load ready file using the load ready graph.
- The load ready file would then be loaded into the Teradata table.
- Any new customer or existing customers are identified by their customer id and account number as the keys in the Teradata table while loading or accessing data from the tables.
- Earlier SAS ETL was used in extracting, transforming and loading the data..
- The SAS coding had to be converted into Abinitio graphs to perform the ETL Tasks according to the customer request.
Environment: SAS 9.1.3, Ab Initio (GDE 3.12.5.2, Co>Op 3.12.3), Teradata V2R6, TOAD, UNIX.
Confidential
SAS Developer
Responsibilities:
- Tasks in Data Segmentation & Building a predictive Model (Score Card) using Data Integration Studio,
- Designing a Process in the repositories.
- Creating Metadata that defines Source, Target and Transformation.
- Report generation
- Deploy jobs for scheduling.
- Managing Resources throughout the Intelligence Value Chain
- Managing Job scheduler, Server and User manager.
- Application Monitoring & Management.
Environment: SAS 9.1.3, SAS Macros, SQL, Ab Initio (GDE 3.12.5.2, Co>Op 3.12.3), UNIX, Toad, Teradata, Remedy.
Confidential
SAS Developer
Responsibilities:
- Requirement Analysis, Design and Building of Reports through SAS Enterprise Guide.
- Scheduling of Reports to run every quarter.
- Setting up of Automatic Email reminders to the concerned individuals.
Environment: SAS 9.1.2 SAS Enterprise Guide 4, Windows 2000.
Confidential
SAS Developer
Responsibilities:
- Requirement Analysis, Design and building of various interfaces like Max Interface, EtrainInterface.
- People Information reporting Online Interface.
- Develop and maintain Business reports for People Capital Analysts and line Managers.
Environment: SAS 9.1.3, SAS AF, SAS Macros, Windows 2000
Confidential
SAS Developer
Responsibilities:
- Responsible for creation for designing and development.
- Used SAS and ASP to bring the features of SAS on to the web.
- Extensively used Base SAS and SAS AF for the ETL process.
- Implemented role level security features for the online reports.
Environment: Base SAS 9.12, SAS AF, SAS macros
Confidential
SAS Programmer
Responsibilities:
- Production Support for the Auto analyst Application.
- Enhancement Work like adding some code to include some functionality in the existing process.
- Maintenance Work that includes fixing the bugs.
- Unit Testing and Module Testing of the code before promoting it to SQA.
Environment: MAINFRAMES, BASE SAS, SAS MACROS, SAS SQL
Confidential
Statistical Programmer
Responsibilities:
- Importing data from Oracle database for Phase I clinical study data and employing PROC IMPORT to analyze data.
- Providing statistical SAS programming support, by producing analysis datasets, QC programs for pharmacokinetic (PK) studies
- Performing data validation on the Phase I clinical study and creating adhoc reports.
- Using SAS/MACRO with DATA step, SET statements and PROC SQL to bring data into processing programs, and drawing statistical information for reporting through PROC FREQ, PROC UNIVARIATE, and PROC REGRESSION.
- Using PROC FORMATS and PROC INFORMATS to change data types in accordance with protocol.
- Generating tables and listings for Patient Demography and Adverse Events data for PhaseI and PhaseII studies.
- Creating SAS datasets of clinical data from clinical databases
- Developing status and efficacy datasets, project specific macros and formats.
- Loading client data from other platforms and other software packages;.
- Preparing documentation describing all datasets and variables including derived variables.
- Performing data checks as needed, to ensure integrity and correctness of data displays.
- Programming customized data displays, (including data listings, summary tables and routine graphics) in accordance with approved statistical analysis plan and shell displays for clinical research studies.
- Coding using Base SAS programs, SAS procedures, or standardized macros.
- Performing data checks as needed to understand structure and content of data.
Environment: SAS 9.1.2, Oracle, SAS SQL, SAS Macros, MS Excel