SUMMARY:

• 9 years of extensive IT Experience especially in Data Warehousing and Business Intelligence applications in Financial, Retail, Telecom, Insurance, HealthCare and Technology Solutions industries.
• Having experience on creating 2D drawings using Auto CAD.
• Experienced in creating Hive, Pig and custom map reduce programs for analyzing data.
• Experience in validating and analyzing Hadoop log files.
• Experience in loading multiple larger datasets into HDFS and processing the datasets by using the Hive and Pig.
• Familiar with data architecture including data ingestion pipeline design, Hadoop information architecture, data modelling and data mining, machine learning and advanced data processing.
• Intake happens through Sqoop and Ingestion happens through Map Reduce, HBASE.
• Deploying, managing and operating scalable, highly available, and fault tolerant systems on AWS.
• Worked on snowflake connector for developing python applications.
• Installing python connectors and python connector API
• Hands on experience on Hadoop technology stack (HDFS, Map - Reduce, Hive, HBase, Pig, Cassandra, Flume, Kafka and Spark)
• 3+ years of experience using Talend Data Integration/Big Data Integration (6.1/5.x) / Talend Data Quality.
• Performed research for lead architect on specifications, materials, building codes
• Well versed with Talend Big Data, Hadoop, Hive and used Talend Big data components like tHDFSInput, tHDFSOutput, tPigLoad, tPigFilterRow, tPigFilterColumn, tPigStoreResult, tHiveLoad, tHiveInput, tHbaseInput, tHbaseOutput, tSqoopImport and tSqoopExport.
• Strong experience in migrating other databases to Snowflake.
• Participate in design meetings for creation of the Data Model and provide guidance on best data architecture practices.
• Analyzing the source data to know the quality of data by using Talend Data Quality.
• Broad design, development and testing experience with Talend Integration Suite and knowledge in Performance Tuning of mappings.
• Developed jobs in Talend Enterprise edition from stage to source, intermediate, conversion and target
• Participates in the development improvement and maintenance of snowflake database applications
• Evaluate Snowflake Design considerations for any change in the application
• Build the Logical and Physical data model for snowflake as per the changes required
• Define roles, privileges required to access different database objects.
• Define virtual warehouse sizing for Snowflake for different type of workloads.
• Design and code required Database structures and components.
• Development of Jobs in Matillion redshift ETL tool
• Build the Logical and Physical data model for snowflake as per the changes required
• Worked with cloud architect to set up the environment
• Define virtual warehouse sizing for Snowflake for different type of workloads.
• Design and code required Database structures and components
• Ensure in corporation of best practices and lessons learned from prior projects
• Coding for Stored Procedures/ Triggers.
• Implement performance tuning where applicable.
• Designs batch cycle procedures on major projects using scripting and Control
• Develop SQL queries SnowSQL
• Develop transformation logic using snowpipeline.
• Optimize and fine tune queries
• Performance tuning of Big Data workloads.
• Have good Knowledge in ETL and hands on experience in ETL.
• Used ETL methodologies and best practices to create Talend ETL jobs. Followed and enhanced programing and naming standards.
• Write highly tuned and performant SQLs on various DB platform including MPPs.
• Develop highly scalable, fault tolerant, maintainable ETL data pipelines to handle vast amount of data.

TECHNICAL SKILLS:

Cloud Technologies: Snowflake, SnowSQL, SnowpipeAWS.

Spark, Hive: LLAP, Beeline, Hdfs, MapReduce, Pig, Sqoop, HBase, Oozie, Flume

Reporting Systems: Splunk

Hadoop Distributions: Cloudera, Hortonworks

Programming Languages: Scala, Python, Perl, Shell scripting.

Data Warehousing: Snowflake, Redshift, Teradata

DBMS: Oracle, SQL Server, MySQL, Db2

Operating System: Windows, Linux, Solaris, Centos, OS X

IDEs: Eclipse, Netbeans.

Servers: Apache Tomcat

Details: Bachelor's in Civil Engineering from JNTUK in 2013

PROFESSIONAL EXPERIENCE:

Confidential ; Phoenix, AZ

Sr. Snowflake Developer

Responsibilities:

• Perform unit and integration testing and document test strategy and results
• Created data sharing between two snowflake accounts (Prod—Dev).
• Migrate the database 500 + Tables and views from Redshift to Snowflake
• Redesigned the Views in snowflake to increase the performance.
• Unit tested the data between Redshift and Snowflake.
• Creating Reports in Looker based on Snowflake Connections
• Proactively support team building and on boarding efforts via mentoring contributions.
• White-boarding and planning.
• Import of JSON S3 data into redshift using matillion
• Offloading historical data to Redshift spectrum using Matillion, Python and AWS glue.
• Working in distributed computing.
• Performed daily admin tasks of user registration, security configuration and usage monitoring
• Worked on Letter generated programs using C and UNIX shell scripting.
• Validated the Map reduce, Pig, Hive Scripts by pulling the data from the Hadoop and validating it with the data in the files and reports.
• Utilized SQOOP, Kafka, Flume and Hadoop File System API’s for implementing data ingestion pipelines from heterogeneous data Sources.
• Experience with Data Analytics, Data Reporting, Ad-hoc Reporting, Graphs, Scales, PivotTables and OLAP reporting.
• As a Spotfire Admin, performed upgrades, hot fixes, server installations & webplayer installations.
• Created Talend jobs using the dynamic schema feature.
• Load and transform data into HDFS from large set of structured data /Oracle/Sql server using Talend Big data studio.
• Developed spark applications in python (PySpark) on distributed environment to load huge number of CSV files with different schema in to Hive ORC tables.
• Worked on reading and writing multiple data formats like JSON, ORC, Parquet on HDFS using PySpark.
• Involved in End to End migration of 800+ Object with 4TB Size from Sql server to Snowflake.
• Data moved from Sql Server Azure snowflake internal stage Snowflake with copy options.
• Created roles and access level privileges and taken care of Snowflake Admin Activity end to end.
• Converted 230 views query’s from Sql server snowflake compatibility.
• Publishing customized interactive reports and dashboards, report scheduling using Tableau server.
• Administered user, user groups, and scheduled instances for reports in Tableau.
• Involved in installation of Tableau desktop 8.1, Tableau server Application software
• Worked on snowflake connector for developing python applications.
• Installing python connectors and python connector API.
• Knowledge of Azure Site Recovery and Azure Backup Installed and Configured the Azure Backup agent and virtual machine backup, Enabled Azure Virtual machine backup from the Vault and configured the Azure Site Recovery (ASR).
• Implemented a CI/CD pipeline using Azure DevOps (VSTS, TFS) in both cloud and on-premises with GIT, MS Build, Docker, Maven along with Jenkins’s plugins.
• Experience in writing Infrastructure as a code (IaC) in Terraform, Azure resource management, AWS Cloud formation. Created reusable Terraform modules in both Azure and AWS cloud environments.
• Retrofitted 500 Talend jobs from SQL Server to Snowflake.
• Experience in validating map-reduce jobs to support distributed processing using java, hive and pig.
• Responsible to make sure that team members are maintaining technical skills, keeping track of risk and issues and other required documentations, on time delivery, coordination with other DBT teams, participate in all relevant meetings, prepare project report.
• Experience writing Hive Queries for analyzing data in Hive warehouse using Hive Query Language (HQL).
• Worked on SnowSQL and Snowpipe
• Utilize admin tools object manager, Command manager, Enterprise manager, System Manager, and operation manager in day-to-day BI administration operations.
• Expertise in all areas of data warehousing.
• Involved in DBT estimates, IQA, EQA.
• Participate in developing and documenting technical standards and best practices for the BI organization.
• Converted Talend Joblets to support the snowflake functionality.
• Data integration in performs debugging, troubleshooting, modifications and unit result of integration solutions.
• Analysed the SQL scripts and designed it by using PySpark SQL for faster performance.
• Knowledge of monitoring, logging and cost management tools that integrate with AWS.
• Validation of Looker report with Redshift database.
• Created data sharing out of snowflake with consumers.
• Validating the data from SQL Server to Snowflake to make sure it has Apple to Apple match.
• Consulting on Snowflake Data Platform Solution Architecture, Design, Development and deployment focused to bring the data driven culture across the enterprises
• Used different AWS Data Migration Services and Schema Conversion Tool along with Matillion ETL tool.
• Driving replacing every other data platform technology using Snowflake with lowest TCO with no compromise on performance, quality and scalability.
• Building solutions once for all with no band-aid approach.

Environment: Snowflake, Redshift, SQL server, BI Architect, AWS, AZURE, TALEND, JENKINS and SQL

Confidential

Snowflake Developer

Responsibilities:

• Participate in all Agile Ceremonies Pre-Planning, Planning, Backlog Refinement, Daily Stand up, Demo Presentation to Product Owner, Retrospective.
• To Design and Develop ETL pipelines in and out of Snowflake using Snow SQL and Snow pipe.
• Created Snowpipe for continuous data load from Azure blob to Snowflake staging.
• Implemented slowly changing dimensions methodology to keep track of historical data using snowflake streams.
• Used copy commands to load data from Azure Data Lake to snowflake staging.
• Good knowledge on Snowflake information metadata schema.
• Good understanding on Snowflake storage and processing cost model.
• Improved query performance using Snowflake Query profiler.
• Applied clustering on Snowflake table to improve performance.
• Ensure ETL/ELT’s succeeded and loaded data successfully in Snowflake DB.
• To Design and Develop ETL pipelines in and out of Snowflake using Snow SQL.
• Created data sharing between two snowflake accounts.
• To work on complex queries so that snowflake utilizes the Queries in optimized way.
• To do Performance tuning, application support, and user acceptance testing.
• Created internal and external stage and transformed data during load.
• To reduced Snowflake space used by adding transient tables where suitable and ensured optimum clustering of the columns.
• Redesigned the Views in snowflake to increase the performance.
• Unit tested the data between PNA/GPA/AIS and Snowflake.
• Experience in working with AWS, Azure.
• Good working knowledge of any ETL tool (ESP, SNOWFLAKE).
• Validating the data from PNA to Snowflake to make sure it has Apple to Apple match.
• Worked on DBT ( ELT tool) connection setup with Snowflake and have used DBT cloud to execute ELT pipelines in Snowflake.
• Implemented Snowflake Continuous Data Loading, Snowflake Time Travel & Fail-safe and secured Data Sharing

Environment: Snowflake, ETL, Redshift, SQL server, BI Architect, AWS, AZURE, TALEND, JENKINS and SQL

Confidential

Snowflake Data Engineer

Responsibilities:

• Created data pipeline for several events of ingestion, aggregation and load consumer response data from AWS S3 bucket into Hive external tables.
• Worked on Oracle Databases, RedShift and Snowflakes
• Core Technologies: End to end data platform with Snowflake, Matillion, Power BI, Qubole, Databricks, Tableau, Looker, Python, Data Iku & R
• Creating lambda SQS queues to create event based trigger for matillion
• Unloading data to AWS Athena using Glue and matillion
• Implemented a 'server less' architecture using API Gateway, Lambda, and Dynamo DB and deployed AWS Lambda code from Amazon S3 buckets. Created a Lambda Deployment function, and configured it to receive events from your S3 bucket.
• Create conceptual, logical and physical models for OLTP, Data Warehouse Data Vault and Data Mart Star/Snowflake schema implementations.
• Used Alteryx for Data Preparation and then Tableau for Visualization and Reporting.
• Processed data in Alteryx to create TDE for tableau reporting.
• Working experience with Kimball Methodology and Data Vault Modeling.
• Define virtual warehouse sizing for Snowflake for different type of workloads.
• Experience in Tableau Server administration tasks including Tableau server optimization and performance tuning.
• Extensively used Autosys for scheduling the UNIX jobs.
• Major challenges of the system were to integrate many systems and access them which are spread across South America; creating a process to involve third party vendors and suppliers; creating authorization for various department users with different roles.
• Experienced in Building a Talend job outside of a Talend studio as well as on TAC server.
• CSM certified, Worked in Scaled Agile (SAFE) environment as System/DBT QA, hands on experience with Rally, JIRA.
• Involved in design and development of GLS application developed in C/C++ on HP UNIX.
• Setup full CI/CD pipelines so that each commit a developer makes will go through standard process of software lifecycle and gets tested well enough before it can make it to the production.
• Validated the data load process for Hadoop using the HiveQL qurey’s.
• Evaluate Snowflake Design considerations for any change in the application
• Build the Logical and Physical data model for snowflake as per the changes required
• Define roles, privileges required to access different database objects.
• Define virtual warehouse sizing for Snowflake for different type of workloads.
• Design and code required Database structures and components
• Build the Logical and Physical data model for snowflake as per the changes required.
• Developed the Pysprk code for AWS Glue jobs and for EMR.
• Worked on scalable distributed data system using Hadoop ecosystem in AWS EMR, MapR distribution.
• Used more components in Talend and Few to be mentioned: tjava, toracle, txmlMap, tdelimited files, tlogrow, tlogback components etc. in many of my Jobs Design
• Worked on Joblets (reusable code) & Java routines in Talend
• Implemented Talend POC to extract data from Salesforce API as an XML Object & .csv files and load data into SQL Server Database.
• Worked with cloud architect to set up the environment
• Educate developers on how to commit their work and how can they make use of the CI/CD pipelines that are in place.

Environment: Snowflake, SQL server, AWS and SQL

Confidential ; Dallas, TX

Senior Data Engineer

Responsibilities:

• Develop ETL/ELT pipelines used to extract data from source systems (DB2) and load data into data warehouse (Snowflake) using Pentaho CE 9.0
• Performed data analysis, preparation, cleansing and enrichment.
• Implemented CDC process using ELT methodology in Snowflake warehouse.
• Converted legacy ETL code into new ELT.
• Good understanding and experience with Snowflake architecture.
• Hands on experience with loading data into snowflake using Copy command, create and use of internal/external stages, creating file formats (csv, JSON, Parquet), loading from AWS S3, SnowSQL, creating Snowpipes and Streams.
• Hands on experience loading unstructured data JSON and Parquet file formats.
• Knowledge on using Time travel, Fail safe, Types of tables (permanent, transient, and temporary), data sharing and dynamic data masking.
• Experience with performance optimization on using scaling up, scaling out policies, clustering.
• Test coded program routines and revise eliminate errors.
• Perform all necessary discovery and fact-finding to fully understand source and target data models.
• Perform administrative functions on ETL/ELT related systems.
• Experience migrating data from DB2 to Snowflake Data warehouse.
• Support and fix production issues and defects.
• Independently execute all assignments.
• Used GitHub as a central repository for developed code to move across environments.
• Used Agile Scrum through sprints in the project.

Environment: SQL Server 2016, SSRS, SSIS, T-SQL, Snowflake, ELT, Oracle 11g, SQL * Loader, Visual Studio 2015, XML, Excel.

Confidential

ETL Developer

Responsibilities:

• Extensively used Oracle ETL process for address data cleansing.
• Developed and tuned all the Affiliations received from data sources using Oracle and Informatica and tested with high volume of data.
• Responsible for developing, support and maintenance for the ETL (Extract, Transform and Load) processes using Oracle and Informatica Power Center.
• Created common reusable objects for the ETL team and overlook coding standards.
• Reviewed high-level design specification, ETL coding and mapping standards.
• Designed new database tables to meet business information needs. Designed Mapping document, which is a guideline to ETL Coding.
• Used ETL to extract files for the external vendors and coordinated that effort.
• Migrated mappings from Development to Testing and from Testing to Production.
• Performed Unit Testing and tuned for better performance.
• Created various Documents such as Source-to-Target Data mapping Document, and Unit Test Cases Document.
• Developed Logical and Physical data models that capture current state/future state data elements and data flows using Erwin 4.5.
• Responsible for design and build data mart as per the requirements.
• Extracted Data from various sources like Data Files, different customized tools like Meridian and Oracle.
• Extensively worked on Views, Stored Procedures, Triggers and SQL queries and for loading the data (staging) to enhance and maintain the existing functionality.
• Done analysis of Source, Requirements, existing OLTP system and identification of required dimensions and facts from the Database.
• Created Data acquisition and Interface System Design Document.
• Designed the Dimensional Model of the Data Warehouse Confirmation of source data layouts and needs.

Environment: MS SQL Server 2012/2008R2, SQL server integration services (SSIS) 2012, SQL server reporting services (SSRS) 2012, and TSQL, MS Office, Notepad++