PROFESSIONAL SUMMARY:

• In - depth understanding of SnowFlake cloud technology.
• In-Depth understanding of SnowFlake Multi-cluster Size and Credit Usage
• Played key role in Migrating Teradata objects into SnowFlake environment.
• Experience with Snowflake Multi-Cluster Warehouses .
• Experience with Snowflake Virtual Warehouses.
• Experience in building Snowpipe.
• In-depth knowledge of Data Sharing in Snowflake.
• In-depth knowledge of. Snowflake Database, Schema and Table structures.
• Experience in using Snowflake Clone and Time Travel.
• In-depth understanding of NiFi.
• Experience in building ETL pipelines using NiFi.
• Deep knowledge of various NiFi Processors .
• Experience in Splunk reporting system.
• U nderstanding of Spark Architecture including Spark Core, Spark SQL, Data Frames.
• Excellent knowledge of Hadoop architecture and various components such as HDFS, Job Tracker, Task Tracker, Name Node, Data Node and Map Reduce programing paradigm.
• Progressive experience in the field of Big Data Technologies, Software Programming and Developing, which also includes Design, Integration, Maintenance.
• Worked on Cloudera and Hortonworks distribution.
• Experience in analyzing data using HiveQL and Hive-llap
• Experience in Apache Druid.
• Experience in Sqoop ingesting data from relational to hive.
• Experience in Elastic Search, Kibana.
• Experience in various data ingestion patterns to hadoop.
• Have good Knowledge in ETL and hands on experience in ETL.
• Hands on experience in Hbase, Pig.
• Experience in various methodologies like Waterfall and Agile .
• Experience in working on Unix/Linux operating systems.

TECHNICAL SKILLS:

Cloud Technologies: Snowflake, AWS.

Spark, Hive: LLAP, Beeline, Hdfs,MapReduce,Pig,Sqoop,HBase,Oozie,Flume

Reporting Systems: Splunk

Hadoop Distributions: Cloudera,Hortonworks

Programming Languages: Scala, Python, Perl, Shell scripting.

Dashboard: Ambari, Elastic Search,Kibana.

DataWareHousing: Snowflake Teradata

DBMS: Oracle,SQL Server,MySql,Db2

Operating System: Windows,Linux,Solaris,Centos,OS X

IDEs: Eclipse,Netbeans.

Servers: Apache Tomcat

Data Integration Tool: NiFi, SSIS

WORK EXPERIENCE:

Confidential

Snowflake/NiFi Developer

Responsibilities:

• Involved in Migrating Objects from Teradata to Snowflake.
• Created Snowpipe for continuous data load.
• Used COPY to bulk load the data.
• Created internal and external stage and t ransformed data during load.
• Used FLATTEN table function to produce lateral view of VARIENT, OBECT and ARRAY column.
• Worked with both Maximized and Auto-scale functionality.
• Used Temporary and Transient tables on diff datasets.
• Cloned Production data for code modifications and testing.
• Shared sample data using grant access to customer for UAT.
• Time traveled to 56 days to recover missed data.
• Developed data warehouse model in snowflake for over 100 datasets using whereScape.
• Heavily involved in testing Snowflake to understand best possible way to use the cloud resources.
• Developed ELT workflows using NiFI to load data into Hive and Teradata.
• Worked on Migrating jobs from NiFi development to Pre-PROD and Production cluster.
• Scheduled different Snowflake jobs using NiFi.
• Used NiFi to ping snowflake to keep Client Session alive.

Confidential

Big Data Engineer

Responsibilities:

• Played key role in testing Hive LLAP and ACID properties to leverage row level transactions in hive.
• Volunteered in designing an architecture for a dataset in Hadoop with estimated data size of 2PT/day.
• Integrated Splunk reporting services with Hadoop eco system to monitor different datasets.
• Used Avro, Parquet and ORC data formats to store in to HDFS.
• Developed workflow in SSIS to automate the tasks of loading the data into HDFS and processing using hive.
• Develop alerts and timed reports Develop and manage Splunk applications.
• Provide leadership and key stakeholders with the information and venues to make effective, timely decisions.
• Establish and ensure adoption of best practices and development standards.
• Communicate with peers and supervisors routinely, document work, meetings, and decisions.
• Work with multiple data sources.
• Designed and Created Hive external tables using shared Meta-store instead of derby with partitioning, dynamic partitioning and buckets.
• Implemented Apache PIG scripts to load data to Hive.
• Worked with Various HDFS file formats like Avro, Sequence File and various compression formats like snappy, Gzip.
• Used spark-sql to create Schema RDD and loaded it into Hive Tables and handled structured data using Spark SQL.
• Analyzed the SQL scripts and designed the solution to implement using PySpark.
• Partner with Source teams to source the data to hadoop for supporting data science models.
• Used Avro, Parquet and ORC data formats to store in to HDFS.
• Involved in converting Hive/SQL quries into Spark transformation using Spark RDDs.
• Involved in creating Teradata FastLoad scripts.
• Provide assistance to business users for various reporting needs.
• Worked on data transfer mechanism from hive to Teradata.
• Worked with different platform teams to resolve cross dependency.
• Involved in Code Review Discussions, Demo’s to stakeholders.
• Worked on data ingestion from Oracle to hive.
• Managing and scheduling Jobs on a Hadoop Cluster using Active Batch and Crontab.
• Involved in different data migration activities.
• Involved in fixing various issues related to data quality, data availability and data stability.
• Worked in determining various strategies related to data security.
• Worked on Hue interface for Loading the data into HDFS and querying the data.
• Played a key role in Hadoop 2.5.3 Testing.

Confidential

Software Engineer

Responsibilities :

• Involved in creating and partitioning of hive tables for data loading and analyzing which runs internally in map reduce way.
• Worked with the business users to gather, define business requirements and analyze the possible technical solutions.
• Used real time streaming frameworks like Apache storm to load the data from messaging distribution systems like Apache kafka into hdfs.
• Involved in setting up 3 node storm and Kafka cluster in open stack servers using chef.
• Provide support to data analysts in running hive queries.
• Created Partitioned tables in hive for better performance and fast querying.
• Used Hive to compute various metrics for reporting.
• Implemented dynamic partitions in Hive .
• Involved in Hadoop jobs for processing billions of records of text data.
• Involved in importing data using Sqoop from traditional RDBMS like Db2, oracle, mysql including Teradata to hive.
• Involved in importing data of different formats like JSON, txt, csv, tsv formats to hdfs, hive.
• Monitoring the jobs to analyze performance statistics.
• Managing and scheduling batch Jobs on a Hadoop Cluster using Oozie.
• Trained the team members regarding different data ingestion patterns.
• Used Kibana for data analysis and product metric visualizations.

Graduate Assistant

Confidential, Indiana

Responsibilities:

• Worked on the maintenance of ISU web pages.
• Involved in requirements discussion with department heads.
• Design of web pages using html5, css, Jquery, Javascript.
• Performed POC on Drupal Framework.