SUMMARY

• Azure Data Engineer with 8+ years of experience in building data intensive applications, tackling challenging architectural and scalability problems.
• Experience in building data pipelines using Azure Data factory, Azure databricks and loading data to Azure data Lake, Azure SQL Database, Azure SQL Data warehouse and Controlling and granting database access.
• Experience in Developing Spark applications using Spark - SQL, Pyspark and Delta Lake in Databricks for data extraction, transformation, and aggregation from multiple file formats for analyzing & transforming the data to uncover insights into the customer usage patterns.
• Good understanding of Spark Architecture, MPP Architecture, including Spark Core, Spark SQL, Data Frames, Spark Streaming, Driver Node, Worker Node, Stages, Executors and Tasks.
• Productionize models in Cloud environment, which would include, automated processes, CI/CD pipelines, monitoring/alerting, and troubleshooting issues. Present the model and results to technical and non-technical audience
• Experience in Database Design and development with Business Intelligence using SQL Server 2014/2016, Integration Services (SSIS), DTS Packages, SQL Server Analysis Services (SSAS), DAX, OLAP Cubes, Star Schema and Snowflake Schema.
• Excellent communication skills with excellent work ethics and a proactive team player with a positive attitude.
• Strong skills in visualization tools Power BI, Microsoft Excel - formulas, Pivot Tables, Charts and DAX Commands.
• Highly skilled in various automation tools, continuous integration workflows, managing binary repositories and containerizing application deployments and test environments.
• Experienced working with configuration management tool chef by developing chef cookbooks to configure, deploy and manage web applications, config files, databases, mount points and packages of existing infrastructure.
• Experience in creating Docker containers leveraging existing Linux Containers and AMI's in addition to creating Docker containers from scratch.
• Experienced Application Security Researcher, thereby possess hands-on experience on wide
• Exceptional communication skills, business insights to comprehend and convey technical issues. Proven ability to collaborate, lead and work incessantly towards organizations growth.

TECHNICAL SKILLS

Cloud platform: Microsoft Azure, Amazon Web Services

CI/CD, DevOps Tools: Git,Jenkins,Chef,Docker

Languages/Scripts: C, C#, C++, SQL, XML, YAML, Ruby, Python, PowerShell, PowerShell DSC

Directory Services: Active Directory (2008/2012) Administration.

Databases: MySQL, SQL Server,SSIS, SSAS, SSRS,OLTP, OLAP

Operating Systems / Web servers: Windows 2012 R2/2008 R2 Servers, Windows Server 2000/2003 Windows 10/ 8/7/Vista/XP, Linux, RHEL, Ubuntu, CentOS, Apache, Nginx, IIS

Clustering Technologies: Windows 2012 R2/2008 R2 Clustering & High Availability

Virtualization: Microsoft Hyper-V, Citrix, VDI, VMware

PROFESSIONAL EXPERIENCE

Confidential

Azure Data Engineer

Responsibilities:

• Highly Involved into Data Architecture and Application Design using Cloud and Big Data solutions on AWS, Microsoft Azure.
• Leading the effort for migration of Legacy-system to Microsoft Azure cloud based solution. Re-designing the Legacy Application solutions with minimal changes to run on cloud platform.
• Built the data pipeline using Azure Service like Data Factory to load the data from Legacy SQL server to Azure Data Base using Data Factories, API Gateway Services, SSIS Packages, Talend Jobs, custom .Net and Python codes.
• Built Azure Web Job for Product Management teams to connect to different APIs and sources to extract the data and load into Azure Data Warehouse using Azure Web Job and Functions.
• Build various pipeline to integrate the Azure Cloud to AWS S3 to get the data into Azure DataBase.
• Set up the Hadoop and Spark cluster for the various POCs, specifically to load the Cookie level data and real-time streaming. Integrate with other ecosystems like Hive, Hbase, Spark, HDFS/DataLake/Blob Storage.
• Set up the Spark Cluster to process the more than 2 Tb of data and dumped into SQL Server. In addition, built various Spark jobs to run Data Transformations and Actions.
• Writing a different APIs to connect with the different Media Data feeds like, Prisma, Double Click Management, Twitter, Facebook, Instagram and Amnet to get the Data using Azure Web Job and Functions integration with Cosmos DB.
• Built the trigger based Mechanism to reduce the cost of different resources like Web Job and Data Factories using Azure Logic Apps and Functions.
• Extensively worked on Relational DataBase, Postgres SQL as well as MPP database like Redshift.
• Experience in Custom Process design of Transformation via Azure Data Factory & Automation Pipelines. Extensively used the Azure Service like Azure Data Factory and Logic App for ETL, to push in/out the data from DB to Blob storage, HDInsight - HDFS, Hive Tables.

Confidential, Pleasanton CA

Azure Data Engineer

Responsibilities:

• Developed mapping document to map columns from source to target.
• Created azure data factory (ADF pipelines) using Azure polybase and Azure blob.
• Performed ETL using Azure Data Bricks. Migrated on-premise Oracle ETL process to Azure Synapse Analytics.
• Worked on python scripting to automate generation of scripts. Data curation done using azure data bricks.
• Worked on azure data bricks, pyspark, hdinsight, u-sql, t-sql, spark sql, Azure ADW and hive used to load and transform data.
• Used Azure Data Lake as Source and pulled data using Azure Polybase.
• Used stored procedure, lookup, execute pipeline, data flow, copy data, azure function features in ADF.
• Experienced in azure Data Lake storage gen2 to store excel files, parquet files and retrieve using data using Blob API.
• Used azure data bricks to do ETL (workspace, hive, rdd, map, reduce so on).
• Worked on creating star schema for drilling data. Created pyspark procedures, functions, packages to load data.

Confidential, Chevy Chase MD

Azure Cloud Engineer

Responsibilities:

• Designed, deployed, maintained and lead the implementation of Cloud solutions using Microsoft Azure and underlying technologies
• Migrating Services from On-premise to Azure Cloud Environments. Collaborate with development and QA teams to maintain high-quality deployment
• Deployed the initial Azure components like Azure Virtual Networks, Azure Application Gateway, Azure Storage and Affinity groups.
• Conducted capacity planning and architecture of Storage Accounts.
• Configured Azure Traffic Manager to build routing for user traffic Infrastructure Migrations: Drive Operational efforts to migrate all legacy services to a fully Virtualized Infrastructure.
• Prepared capacity and architecture plan to create the Azure Cloud environment to host migrated IaaS VMs and PaaS role instances for refactored applications and databases.
• Installing the Applications Insights tool on the web services and configuring an Application Insight workspace in Azure. configured Application Insights to perform web tests and alerts.
• Handle deployment workflow including provisioning, load balancing, and health monitoring.
• Implemented HA deployment models with Azure Classic and Azure Resource Manager
• Configured Azure Active Directory and managed users and groups.
• Configure continuous integration from source control, setting up build definition within Visual Studio Team Services (VSTS) and configure continuous delivery to automate the deployment of ASP.NET MVC applications to Azure web apps
• Deployed DSC from Azure Automation to On-premises or cloud environments.
• Installing IIS and ASP.NET modules, configuring server and adding ASP.NET Applications via command line.
• Configured Azure Automation DSC configuration management to assign permissions through RBAC, assign nodes to proper automation accounts and DSC configurations, to get alerted on any changes made to nodes and their configuration.
• Configured stream analytics, Event hubs and worked to manage IoT solutions with Azure.
• Used PowerShell DSC to configure Vms, physical hosts.
• Used PowerShell DSC within Azure Automation runbooks to configure machines.
• Tested and deployed DSC configurations to all servers.
• Developed automation solution using PowerShell scripting for day to day operational tasks
• Modification of Certs, Password and Storage Accounts on the cloud platform Setting up and administer service accounts
• Worked on Configuration of Internal load balancer, load balanced sets and Azure Traffic manager.