SUMMARY

• Total 8+ hands on experience with building product ionized data ingestion and processing pipelines using Java, Spark, Scala etc and also experience in designing and implementing production grade data warehousing solutions on large scale data technologies
• Strong experience in migrating other databases to Snowflake.
• Work with domain experts, engineers, and other data scientists to develop, implement, and improve upon existing systems.
• Experience in analyzing data using HiveQL
• Participate in design meetings for creation of the Data Model and provide guidance on best data architecture practices
• Experience with Snowflake Multi - Cluster Warehouses.
• Experience in Splunk reporting system.
• Understanding of SnowFlake cloud technology.
• Experience with Snowflake cloud data warehouse and AWS S3 bucket for int egrating data from multiple source system which include loading nested JSON formatted data into snowflake table.
• Professional knowledge of AWS Redshift
• Experience in building Snowpipe.
• Experience in using Snowflake Clone and Time Travel.
• Experience in various data ingestion patterns to hadoop.
• Participates in the development improvement and maintenance of snowflake database applications
• Experience in various methodologies like Waterfall and Agile.
• Extensive experience in developing complex stored Procedures/BTEQ Queries.
• In-depth understanding of Data Warehouse/ODS, ETL concept and modeling structure principles
• Build the Logical and Physical data model for snowflake as per the changes required
• Define roles, privileges required to access different database objects.
• In-depth knowledge of Snowflake Database, Schema and Table structures.
• Define virtual warehouse sizing for Snowflake for different type of workloads.
• Worked with cloud architect to set up the environment
• Coding for Stored Procedures/ Triggers.
• Designs batch cycle procedures on major projects using scripting and Control
• Develop SQL queries SnowSQL
• Develop transformation logic using snowpipeline.
• Optimize and fine tune queries
• Performance tuning of Big Data workloads.
• Have good Knowledge in ETL and hands on experience in ETL.
• Experience on Migrating SQL database to Azure data Lake, Azure data lake Analytics, Azure SQL Database, Data Bricks and Azure SQL Data warehouse and Controlling and granting database access and Migrating On premise databases to Azure Data lake store using Azure Data factory.
• Analyze, design and build Modern data solutions using Azure PaaS service to support visualization of data. Understand current Production state of application and determine the impact of new implementation on existing business processes.
• Extract Transform and Load data from Sources Systems to Azure Data Storage services using a combination of Azure Data Factory, T-SQL, Spark SQL and U-SQL Azure Data Lake Analytics . Data Ingestion to one or more Azure Services - (Azure Data Lake, Azure Storage, Azure SQL, Azure DW) and processing the data in In Azure Databricks.
• Operationalize data ingestion, data transformation and data visualization for enterprise use.
• Mentor and train junior team members and ensure coding standard is followed across the project.
• Help talent acquisition team in hiring quality engineers.
• Experience in real time streaming frameworks like Apache Storm.
• Worked on Cloudera and Hortonworks distribution.
• Progressive experience in the field of Big Data Technologies, Software Programming and Developing, which also includes Design, Integration, Maintenance.
• Hands-on experience with Snowflake utilities, SnowSQL, SnowPipe, Big Data model techniques using Python / Java.

TECHNICAL SKILLS

Cloud Technologies: Snowflake, SnowSQL, SnowpipeAWS.

Data Warehousing: Snowflake, Redshift, Teradata

DBMS: Oracle, SQL Server, MySql, Db2

Operating System: Windows, Linux, Solaris, Centos, OS X

IDEs: Eclipse, Netbeans.

Servers: Apache Tomcat Spark, Hive LLAP, Beeline, Hdfs, MapReduce, Pig, Sqoop, HBase, Oozie, Flume

Reporting Systems: Splunk

Hadoop Distributions: Cloudera,Hortonworks

Programming Languages: Scala, Python, Perl, Shell scripting

PROFESSIONAL EXPERIENCE

Confidential - Livingston, New Jersey

Sr. Snowflake Data Engineer

Responsibilities:

• Worked on SnowSQL and Snowpipe
• Converted Talend Joblets to support the snowflake functionality.
• Created Snowpipe for continuous data load.
• Used COPY to bulk load the data.
• Created data sharing between two snowflake accounts.
• Created internal and external stage and transformed data during load.
• Redesigned the Views in snowflake to increase the performance.
• Unit tested the data between Redshift and Snowflake.
• Developed data warehouse model in snowflake for over 100 datasets using whereScape.
• Creating Reports in Looker based on Snowflake Connections
• Experience in working with AWS, Azure and Google data services
• Validation of Looker report with Redshift database.
• Good working knowledge of any ETL tool (Informatica or SSIS).
• Created Talend Mappings to populate the data into dimensions and fact tables.
• Created Pipelines in ADF using Linked Services/Datasets/Pipeline/ to Extract, Transform and load data from different sources like Azure SQL, Blob storage, Azure SQL Data warehouse, write-back tool and backwards.
• Developed JSON Scripts for deploying the Pipeline in Azure Data Factory (ADF) that process the data using the Sql Activity.
• Loaded the tables from the DWH to Azure data lake using azure data factory integration run time.
• Loaded the tables from the azure data lake to azure blob storage for pushing them to snowflake
• Wrote ETL jobs to read from web APIs using REST and HTTP calls and loaded into HDFS using java and Talend.
• Used Talend big data components like Hadoop and S3 Buckets and AWS Services for redshift.
• Validating the data from SQL Server to Snowflake to make sure it has Apple to Apple match.
• Consulting on Snowflake Data Platform Solution Architecture, Design, Development and deployment focused to bring the data driven culture across the enterprises
• Building solutions once for all with no band-aid approach.
• Implemented Change Data Capture technology in Talend in order to load deltas to a Data Warehouse.
• Develop stored procedures/views in Snowflake and use in Talend for loading Dimensions and Facts.
• Design, develop, test, implement and support of Data Warehousing ETL using Talend.
• Very good knowledge of RDBMS topics, ability to write complex SQL, PL/SQL

Confidential, Charlotte, NC

Sr. Snowflake Data Engineer

Responsibilities:

• Evaluate Snowflake Design considerations for any change in the application
• Build the Logical and Physical data model for snowflake as per the changes required
• Define roles, privileges required to access different database objects.
• Define virtual warehouse sizing for Snowflake for different type of workloads.
• Design and code required Database structures and components
• Build the Logical and Physical data model for snowflake as per the changes required
• Experience on working various distributions of Hadoop like CloudEra, HortonWorks and MapR.
• Worked with cloud architect to set up the environment
• Worked on Oracle Databases, RedShift and Snowflakes
• Define virtual warehouse sizing for Snowflake for different type of workloads.
• Major challenges of the system were to integrate many systems and access them which are spread across South America; creating a process to involve third party vendors and suppliers; creating authorization for various department users with different roles.
• Developed workflow in SSIS to automate the tasks of loading the data into HDFS and processing using hive.
• Develop alerts and timed reports Develop and manage Splunk applications.
• Involved in various Transformation and data cleansing activities using various Control flow and data flow tasks in SSIS packages during data migration
• Applied various data transformations like Lookup, Aggregate, Sort, Multicasting, Conditional Split, Derived column etc.
• Work with multiple data sources.
• Developed Mappings, Sessions, and Workflows to extract, validate, and transform data according to the business rules using Informatica.
• Worked with Various HDFS file formats like Avro, Sequence File and various compression formats like snappy, Gzip.
• Worked on data ingestion from Oracle to hive.
• Involved in fixing various issues related to data quality, data availability and data stability.
• Worked in determining various strategies related to data security.
• Performance monitoring and Optimizing Indexes tasks by using Performance Monitor, SQL Profiler, Database Tuning Advisor and Index tuning wizard.
• Worked on Hue interface for Loading the data into HDFS and querying the data.
• Designed and Created Hive external tables using shared Meta-store instead of derby with partitioning, dynamic partitioning and buckets.
• Wrote scripts and indexing strategy for a migration to Confidential Redshift from SQL Server and MySQL databases
• Used spark-sql to create Schema RDD and loaded it into Hive Tables and handled structured data using Spark SQL.
• Worked on AWS Data Pipeline to configure data loads from S3 to into Redshift
• Used JSON schema to define table and column mapping from S3 data to Redshift
• Involved in converting Hive/SQL quries into Spark transformation using Spark RDDs.
• Used Avro, Parquet and ORC data formats to store in to HDFS.

Confidential

Snowflake Data Engineer

Responsibilities:

• Developed Logical and Physical data models that capture current state/future state data elements and data flows using Erwin 4.5.
• Responsible for design and build data mart as per the requirements.
• Extensively worked on Views, Stored Procedures, Triggers and SQL queries and for loading the data (staging) to enhance and maintain the existing functionality.
• Done analysis of Source, Requirements, existing OLTP system and identification of required dimensions and facts from the Database.
• Created Data acquisition and Interface System Design Document.
• Designed the Dimensional Model of the Data Warehouse Confirmation of source data layouts and needs.
• Deploy various reports on SQL Server 2005 Reporting Server
• Installing and Configuring SQL Server 2005 on Virtual Machines
• Migrated hundreds of Physical Machines to Virtual Machines
• Conduct System Testing and functionality after virtualization
• Extensively involved in new systems development with Oracle 6i.
• USED SQLCODE returns the current error code from the error stack SQLERRM returns the error message from the current error code.
• Used Import/Export Utilities of Oracle.
• Created the External Tables in order to load data from flat files and PL/SQL scripts for monitoring.
• Writing Tuned SQL queries for data retrieval involving Complex Join Conditions.
• Extensively used Oracle ETL process for address data cleansing.
• Developed and tuned all the Affiliations received from data sources using Oracle and Informatica and tested with high volume of data.
• Responsible for developing, support and maintenance for the ETL (Extract, Transform and Load) processes using Oracle and Informatica PowerCenter.
• Created common reusable objects for the ETL team and overlook coding standards.
• Reviewed high-level design specification, ETL coding and mapping standards.
• Designed new database tables to meet business information needs. Designed Mapping document, which is a guideline to ETL Coding.
• Used ETL to extract files for the external vendors and coordinated that effort.
• Migrated mappings from Development to Testing and from Testing to Production.
• Performed Unit Testing and tuned for better performance.
• Created various Documents such as Source-to-Target Data mapping Document, and Unit Test Cases Document.
• Read data from flat files and load into Database using SQL Loader.

Confidential

Snowflake Data Engineer

Responsibilities:

• Served as the Snowflake Database Administrator responsible for leading the data model design and database migration deployment production releases to endure our database objects and corresponding metadata were successfully implemented to the production platform environments; (Dev, Qual and Prod) AWS Cloud (Snowflake).
• Performed day-to-day integration with the Database Administrators (DBA) DB2, SQL Server, Oracle and AWS Cloud teams to ensure the insertion of database tables, columns and its metadata have been successfully implemented out to the DEV, QUAL and PROD region environments in AWS Cloud - Aurora and Snowflake.
• Performed ETL data translation using informatica of functional requirements to Source to Target Data Mapping documents to support large datasets (Big Data) out to the AWS Cloud databases; Snowflake and Aurora.
• Performed logical and physical data structure designs and DDL generation to facilitate the implementation of database tables and columns out to the DB2, SQL Server, AWS Cloud (Snowflake) and Oracle DB schema environment using ERwin Data Modeler Model Mart Repository version 9.6.
• Assisted Project Managers and Developers in performing ETL solution design and development to produce reporting, dashboarding and data analytics deliverables.
• Technical Team Member of the T. Rowe Price Information Architect-Data Modeling Agile team; responsible for developing Enterprise Conceptual, Logical and Physical Data Models; Data Dictionary, supporting the three Business Units: Retirement Plan Services (RPS), Shared Support Platforms and Global Investment Services (GIS).