PROFESSIONAL SUMMARY

• In - depth understanding of Snowflakes cloud Architect technology.
• In-Depth understanding of SnowFlakeMulti-cluster Size and Credit Usage
• Played key role in MigratingTeradataobjects intoSnowFlakeenvironment.
• Experience withSnowflake Multi-Cluster Warehouses.
• Experience withSnowflake Virtual Warehouses.
• Experience in buildingSnowpipe.
• In-depth knowledge ofData Sharingin Snowflake.
• In-depth knowledge of. SnowflakeDatabase, Schema and Tablestructures.
• Experience in using SnowflakeCloneandTime Travel.
• In-depth understanding of NiFi.
• Experience in building ETL pipelines using NiFi.
• Deep knowledge of variousNiFi Processors.
• Experience inSplunkreporting system.
• Understanding of Spark Architecture including Spark Core, Spark SQL, Data Frames.
• Excellent knowledge of Hadoop architecture and various components such as HDFS, Job Tracker, Task Tracker, Name Node, Data Node and Map Reduce programing paradigm.
• Progressive experience in the field of Big Data Technologies, Software Programming and Developing, which also includes Design, Integration, Maintenance.
• Worked on Matillion in Modern businesses seeking a competitive advantage to harness data to gain better business insights.
• Worked onClouderaandHortonworksdistribution.
• Experience in analyzing data usingHiveQL and Hive-llap
• Experience in Apache Druid.
• Experience in Sqoop ingesting data from relational to hive.
• Experience in Elastic Search, Kibana.
• Experience in various data ingestion patterns to hadoop.
• Have good Knowledge in ETL and hands on experience in ETL.
• Hands on experience in Hbase, Pig.
• Experience in various methodologies likeWaterfallandAgile.
• Experience in working onUnix/Linuxoperating systems.

TECHNICAL SKILLS

Cloud Technologies: Snowflake, AWS.

Reporting Systems: Splunk

Hadoop Distributions: Cloudera,Hortonworks

Programming Languages: Scala, Python, Perl, Shell scripting.

Dashboard: Ambari, Elastic Search,Kibana.

DataWareHousing: Snowflake Teradata

DBMS: Oracle,SQL Server,MySql,Db2

Operating System: Windows,Linux,Solaris,Centos,OS X

IDEs: Eclipse,Netbeans.

Servers: Apache Tomcat

Data Integration Tool: NiFi, SSIS

PROFESSIONAL EXPERIENCE

Confidential, NYC

Snowflake Cloud Architect

Responsibilities:

• Involved in Migrating Objects from Teradata to Snowflake.
• Created Snowpipe for continuous data load.
• Used COPY to bulk load the data.
• Created internal and external stage and t ransformed data during load.
• Used FLATTEN table function to produce lateral view of VARIENT, OBECT and ARRAY column.
• Worked with both Maximized and Auto-scale functionality.
• Used Temporary and Transient tables on diff datasets.
• Cloned Production data for code modifications and testing.
• Shared sample data using grant access to customer for UAT.
• Time traveled to 56 days to recover missed data.
• Developed data warehouse model in snowflake for over 100 datasets using whereScape.
• Heavily involved in testing Snowflake to understand best possible way to use the cloud resources.
• Developed ELT workflows using NiFI to load data into Hive and Teradata.
• Worked on Migrating jobs from NiFi development to Pre-PROD and Production cluster.
• Used Matillion data source connectors for on-premises and cloud databases, SaaS applications, documents, NoSQL sources, and more to quickly load data into your cloud data environment.
• Worked on Matillion ETL also to Create Own Connector to easily build custom connectors to any REST API source system.
• Scheduled different Snowflake jobs using NiFi.
• Used NiFi to ping snowflake to keep Client Session alive.

Confidential, Atlanta, GA

Snowflake Cloud Architect/

Responsibilities:

• Hands on experience in Amazon Web Services AWS provisioning and good knowledge of AWS services like EC2, Auto scaling, Elastic Load-balancers, Elastic Container service (Docker containers), S3, Elastic Beanstalk, Cloud Front, Elastic file system, VPC, Route 53, Cloud Watch, Cloud Formation, IAM.
• Involved in designing and deploying a large applications utilizing Snowflake almost all of the AWS stack (Including EC2, Route53, S3, RDS, Dynamo DB, SNS, SQS, IAM) focusing on high-availability, fault tolerance, and auto-scaling in Snowflake Cloud Formation.
• Managed multiple Snowflake Cloud accounts with multiple VPC's for both production and non-prod where primary objectives included automation, build out, integration and cost control.
• Developed Snowflake Cloud Formation scripts to automate entire CD pipeline.
• Configured Matillion to extracts data, loads it into your cloud data warehouse (CDW),
• Used the power and flexibility of CDW or Delta Lake to transform data intomeaningful business insights.
• Created and managed multiple Instances deployed for several test applications in those instances in QA environment.
• Configured a VPC and provisioned EC2 instances, EBS in different availability zones.
• Implemented and maintained the monitoring and alerting of production and corporate servers/storage using Cloud .
• Setup and build AWS infrastructure various resources, VPC EC2, S3, IAM, EBS, Security Group, Auto Scaling and RDS in Cloud Formation JSON templates.
• Creating Cloud Watch alerts for instances and using them in Auto Scaling launch configurations.
• Implementing good security compliance measures with MFA and authentication and Authorization with IAM roles
• Backing up the instances by taking snapshots of the required servers regularly.
• Setting up and administering DNS system in AWS using Route53.
• Written Ansible Playbooks from that can Provision several pre-prod environments, Deployment automation, instance mirroring, and several proprietary middleware installations.
• Big-data Hadoop Administration
• Provision Instances to support Big-data operations on a large scale enterprise environment with Autoscaling and Auto-configurations
• Experience on Hadoop cluster maintenance, including data and metadata backups, file system checks, commissioning and decommissioning nodes and upgrade hands on Experience in Installing, Configuring and using Hadoop Eco System Components like HDFS, Hadoop Map Reduce, Yarn, Zookeeper, Sentry, Sqoop, Flume, Hive, HBase, Pig, Oozie.
• Hands on Experience with Hadoop, HDFS, Map Reduce and Hive, Pig, Spark and Scala.
• Performed importing and exporting Data into HDFS and Hive using SQOOP.
• Imported the data from different sources like HDFS/Hbase into Spark RDD.
• Used Flume, Kafka to aggregate log data into HDFS.
• Created multiple Hive tables, implemented Partitioning, Dynamic Partitioning and Buckets in Hive for efficient data access.
• Excellent experience in ETL analysis, designing, developing, testing and implementing ETL processes including performance tuning and query optimizing of database.
• Implemented Spark RDD transformations, actions to migrate Map reduce algorithms.
• Implemented Spark using Scala and SparkSQL for faster testing and processing of data.
• Understanding and usage of Elastic Search and ELK stack .

Environment HDFS, CDH4, Kafka, Cassandra, Hive, Pig, Oozie, Map Reduce, Java, Sqoop, Oracle. Snowflake,Hadoop Distribution of Hortonworks, Cloudera

Confidential, Union, NJ

Big Data Engineer

Responsibilities:

• Played key role in testing Hive LLAP and ACID properties to leverage row level transactions in hive.
• Volunteered in designing an architecture for a dataset in Hadoop with estimated data size of 2PT/day.
• Integrated Splunk reporting services with Hadoop eco system to monitor different datasets.
• Used Avro, Parquet and ORC data formats to store in to HDFS.
• Developed workflow in SSIS to automate the tasks of loading the data into HDFS and processing using hive.
• Develop alerts and timed reports Develop and manage Splunk applications.
• Provide leadership and key stakeholders with the information and venues to make effective, timely decisions.
• Establish and ensure adoption of best practices and development standards.
• Communicate with peers and supervisors routinely, document work, meetings, and decisions.
• Work with multiple data sources.
• Designed and Created Hive external tables using shared Meta-store instead of derby with partitioning, dynamic partitioning and buckets.
• Implemented Apache PIG scripts to load data to Hive.
• Worked with Various HDFS file formats like Avro, Sequence File and various compression formats like snappy, Gzip.
• Used spark-sql to create Schema RDD and loaded it into Hive Tables and handled structured data using Spark SQL.
• Analyzed the SQL scripts and designed the solution to implement using PySpark.
• Partner with Source teams to source the data to hadoop for supporting data science models.
• Used Avro, Parquet and ORC data formats to store in to HDFS.
• Involved in converting Hive/SQL quries into Spark transformation using Spark RDDs.
• Involved in creating Teradata FastLoad scripts.
• Provide assistance to business users for various reporting needs.
• Worked on data transfer mechanism from hive to Teradata.
• Worked with different platform teams to resolve cross dependency.
• Involved in Code Review Discussions, Demo’s to stakeholders.
• Worked on data ingestion from Oracle to hive.
• Managing and scheduling Jobs on a Hadoop Cluster using Active Batch and Crontab.
• Involved in different data migration activities.
• Involved in fixing various issues related to data quality, data availability and data stability.
• Worked in determining various strategies related to data security.
• Worked on Hue interface for Loading the data into HDFS and querying the data.
• Played a key role in Hadoop 2.5.3 Testing.

Confidential, Jersey City, NJ

Software Engineer

Responsibilities:

• Involved in creating and partitioning of hive tables for data loading and analyzing which runs internally in map reduce way.
• Worked with the business users to gather, define business requirements and analyze the possible technical solutions.
• Used real time streaming frameworks like Apache storm to load the data from messaging distribution systems like Apache kafka into hdfs.
• Involved in setting up 3 node storm and Kafka cluster in open stack servers using chef.
• Provide support to data analysts in running hive queries.
• Created Partitioned tables in hive for better performance and fast querying.
• Used Hive to compute various metrics for reporting.
• Implemented dynamic partitions in Hive.
• Involved in Hadoop jobs for processing billions of records of text data.
• Involved in importing data using Sqoop from traditional RDBMS like Db2, oracle, mysql including Teradata to hive.
• Involved in importing data of different formats like JSON, txt, csv, tsv formats to hdfs, hive.
• Monitoring the jobs to analyze performance statistics.
• Managing and scheduling batch Jobs on a Hadoop Cluster using Oozie.
• Trained the team members regarding different data ingestion patterns.
• Used Kibana for data analysis and product metric visualizations.
• Worked on the maintenance of ISU web pages.
• Involved in requirements discussion with department heads.
• Design of web pages using html5, css, Jquery, Javascript.
• Performed POC on Drupal Framework.