SUMMARY:

• Around 12 years of experience in IT focusing on Big Data Technologies - Hadoop ecosystem/HDFS/ Map-Reduce Framework, Spark, Impala, Sqoop, Oozie, Storm, Scala, Kafka , Cassandra, Python, Zookeeper and HIVE data warehousing tool .
• 5+ years of experience in Hadoop technologies to provide the solutions for Banking and Audit process using Hive, Impala, Pig, MapReduce, HBase, Zookeeper, Flume, Sqoop and Oozie.\
• Expertise in Big Data technologies - Hadoop, HDFS, Hive, Oozie, Sqoop, Flume, Pig, HBase, Phoenix, NiFi & Kafka and Apache Spark.
• Building the data lake solutions on Hadoop to consolidate data from different sources to provide the single source of truth to business analytics.
• Good experience in implementing Hadoop solutions using Cloudera & Hortonworks Hadoop distributions.
• Very good understanding on full lifecycle of Hadoop solution, including requirement analysis, platform selection, technical architecture design, application design and development, testing, and deployment.
• Experience in loading raw files to HDFS and move the data to hive database for analytics. Developing Talend packages using HQL and creating Hive Data warehousing in Hadoop.
• Very good understanding of Partitions, Bucketing concepts in Hive and designed both Managed and External tables in Hive to optimize performance.
• Equal competence Integration Services, Analysis and Reporting Services for Design, Development, Integration & Implementation of Information Management Solutions
• Built interactive workload using Apache Phoenix on Hbase cluster for spotfire performance optimization.
• AWS cloud implementation using EC2 instance, S3 storage, redshift and different services
• Data ingestion into azure HDInsight cluster and building the Power BI reports pointing Azure Data Lake.
• Developing Data warehousing solution on snowflake and developing data ingestion pipe lines through AWS S3 using python.
• Experience in creating Jobs, Alerts, SQL Mail Agent, and scheduled SSIS Packages.
• Experience in Data Marts, Data warehousing, OLAP using the Dimensional Modeling techniques such as Star schema and Snow flake schema, FACT and Dimension Tables using MS Analysis Services (SSAS).
• Extensive Experience in Creating, Populating and maintaining Data marts. Thorough knowledge of Features, Structure, Attributes, Hierarchies, Star and Snow Flake Schemas of Data Marts.
• Highly proficient in the use of T-SQL for developing complex Stored Procedures, Triggers, Functions, Performance Tuning and Query Optimization.

TECHNICAL SKILLS:

Hadoop Distributions & Configuration Management: Hortonworks (HDP 3.1)

Hadoop Eco System: Hadoop3.1.1, HDFS, Sqoop: 1.4.6, Flume-ng-1.7.0, Oozie-4.3.1Nifi-1.1.2, Zookeeper-3.4.6

Distributed Processing: Hive3.1.0, Pig: 0.16.0

Real Time Processing: Spark: 2.3.2

Message Processing: Kafka: 2.0.0

NOSQL: HBase2.0.2, Phoenix: 4.5.2

Programming: PL/SQL, Python: 2.7.13

DevOps Tools: GitHub, Apache Maven, Atlassian JIRA

ETL Tools: DataStage 7.5/8.1/8.5/8.7/11.7

RDBMS: Oracle 11g, DB2 9.1, Teradata 12

Operating system: Linux, Centos, Windows

Development Tools: Toad 8.6.2, PL/SQL Developer

Cloud Platform: Google Cloud

Supporting tools: Putty, JIRA, Squirrel SQL Client

Scripting: Linux/Unix Shell Scripting/Python Scripting

PROFESSIONAL EXPERIENCE:

Confidential, Jacksonville, FL

Senior Bigdata Lead/ Architect

Responsibilities:

• Hadoop Developments Activities. Designed/Installed - Multi Node Cluster in Hadoop (HDP) Development Platform through Ambari. Installed all supporting Hadoop Components related to HDP 3.1 Repo’s. Designed/Installed - Work Flow Process - NiFi - Developed the Process Flow.
• Designed/Worked on Architecture Meta data Table Framework - Hive/HBase Table Design. Worked on Various Level of POC’s - Performance Tuning (Hive/Spark), Tool Selection for various Stages (Data Acquisition to Data Visualization) etc., Process documentation and revision as and when required.
• Handled name node fail over with the help of Standby name node. Monitoring the daily jobs scheduled in Oozie for data analytics team and for end user. Designed Hive repository with External Tables, Internal Tables, Buckets, Partitions.
• Experienced in developing Hive Queries on different data formats like Text file, CSV file, Log files and leveraging time-based partitioning yields improvement in performance using Hive. Implemented functionality-based data modelling on Hive Tables and stored the resultants record sets into Sql Server storage engine using Sqoop
• Developed Oozie Workflow & Nifi Workflow for Scheduling and Orchestrating the ETL process. Hands on with Job Monitoring, Tuning and Troubleshooting Hadoop Jobs. Worked in Capacity Planning, Resource Management, Release Deployments and Automating Application Builds.

Environment: Hadoop, Sqoop, Hive, Scala, HBASE, Phoenix, NIFI, Shell Scripting, Python.

Confidential, Dallas, TX

Bigdata Lead

Responsibilities:

• Mentor junior staff brought out of the box solution to various problems
• Helped the team to organize in Project management approach to the team functions and introduced ITIL methodology.
• Developed Data Conversion strategies from CAPP/AIRS system to Data warehousing, SAP BW systems.
• Developed SAS reporting in mainframe using DB2 tables.
• Developed strategy for Data cleansing, ETL using Confidential Datastage.
• Worked with Datastage Designer to create mainframe jobs.
• Worked with Datastage Director, Datastage Manager.
• Exposed to Datastage Administrator.
• Extensively used Eztrieve for synchronized processing, Data massaging and data conversion.
• Developed strategy to Data conversion to Views Portfolio Reporting system a Cognos based Web reporting environment.
• Design the necessary enhancements in Web Reporting with Cognos.
• Helped the team create Cognos frame work from transformed data in Datastage.
• Development of programs in COBOL, CICS, DB2, VSAM, and JCL
• Developed data conversion routines using mainframe SAS
• Implementation and production support
• Developing and adhering MS Project Plan
• Provide rotational On-Call Support for Production and other environments

Environment: Confidential S/390, COBOL 370, CICS, VSAM, IEBGENR, IEBCOPY, ICETOOL, SAS, Endevor, JCL, Xpediter, MS Project, VISIO

Confidential, Mooresville, NC

Sr. Bigdata / Hadoop Lead

Responsibilities:

• Driving requirements for Monitoring for Payment Services program with respect to PCI standards.
• Worked on Project plans and status reporting.
• Worked with Mainframe settlement systems (COBOL, CICS, DB2, and Zeke) to gather requirements.
• Worked with SAS programs in mainframe using DB2 tables.
• Payment services program is Lowe’s first tier-0, hot-hot application.
• Developed Project plan, Documents for Stage gate process.
• Worked with various stakeholders to develop a monitoring framework.
• Developed monitoring strategy in alignment with Confidential Vision.
• Managed the solution instrumentation from vendors like RSA and AJB.
• Managed the Solution delivery from Confidential engineers and contractors.

Environment: Hadoop, Hive, Pig, SQOOP, Kafka, Spark, OOZIE, Python, SCALA, UNIX, Shell scripting, Oracle PL/SQL, RDBMS, AWS, Oracle Golden Gate, Kyvos, Tableau/Qlik.

Confidential, Richardson, TX

Data Warehouse specialist - Teradata and Datastage

Responsibilities:

• Worked on Project plans and status reporting.
• Designed, developed and delivered EDW solution for history conversion of MMAI government program.
• Worked on Tableau data visualization as part of profiling.
• Designed handshake mechanism Axway (E-gateway) and Zeke, Zena Scheduling for automation of File pull and subsequently running of developed processes.
• Worked with Teradata from both Mainframe and UNIX Side.
• Helped optimize Queries from the old EDW system.
• Served as point person from EDW side for Teradata 14.1 upgrade and supported production for the compatibility and performance issues.
• Worked closely with QA with UAT and SIT using Quality Center.
• Designed, developed and delivered Lovelace historical claim conversion.
• Driven requirements for Montana conversion to Confidential Bluestar system.
• Integration analysis of conversion of QNXT to Bluestar.
• Design of new batch cycle to in corporate Montana feeds.
• Mentoring junior staff with technical challenges.
• Developed MLOAD, BTEQMAIN, FASTLOAD procedures for Lovelace and NMCC conversion projects.
• Helped ODS team with Datastage ETL development.

Environment: Confidential S/390,Mvs,Cobol 390, Cics, Vsam, DB2, JCL, Fileaid, Iebgenr, Iebcopy,Icetool, Zeke,Intertest,Endevor,Visio,Uss,Ftp,Sftp,Ms Project, Teradata, Mload, Bteqmain, Fastload, Datastage, Hp Qc, Rtc

Confidential

ETL Developer

Responsibilities:

• Requirement Analysis, Impact analysis, Design and Development of Modules.
• Creating procedures, function, packages, ETL Mappings, Sessions and workflows.
• Performance Tuning for the queries which are taking lot of time to execute.
• Developing test cases and do Unit testing and Integration testing
• Creation of Test data and Testing.
• Coordinating with Application Support Group for Deployments in different environments.
• Providing reports to the user.
• Support Data cleanup activity with reports.
• Coordinating with Client for managing change during the UAT and the post implementation phase.
• Assign tasks to the tester in JIRA.
• DataMart’s, OLAP, FACT & Dimensions tables, Physical & Logical data modeling.
• Extensively used database objects like tables, Index, views, Materialized Views triggers, Packages, functions and procedure.
• Programming: Well experienced in Oracle PL/SQL scripts and UNIX shell scripting.
• Well-versed with Configuration Management using version control systems and working under structured and controlled Change Management system.
• Expertise in preparing Functional & Technical Specifications besides User Guides.
• Experienced in Coordinating and leading off shore Development team.
• Expertise in Off-shore/On-site work culture, leading and mentoring Data Warehouse teams.

Environment: Informatica PowerCenter 8.6, Oracle 10g, PL/SQL, UNIX, Windows NT, SQL, PL/SQL, MS-VSS, PLSQL Developer, Toad, Putty.