Ms Azure Data Engineer Resume
2.00/5 (Submit Your Rating)
SUMMARY
- 11+ years in IT having extensive and diverse experience in Microsoft Azure Cloud Computing, SQL BI technologies.
- Hands on experience in Azure Cloud Services (PaaS & IaaS), Azure Synapse Analytics, SQL Azure, Data Factory, Azure Analysis services, Application Insights, Azure Monitoring, KeyVault, Azure Data Lake.
- Experience on Migrating SQL database toAzure data Lake, Azure data lake Analytics,Azure SQL Database, Data BricksandAzure SQL Data warehouseand Controlling and granting database accessandMigrating On premise databases toAzure Data lake storeusing Azure Data factory.
- Experience in Developingapplications usingSpark - SQLinDatabricksfor data extraction, transformation and aggregation from multiple file formats for analysing& transforming the data to uncover insights into the customer usage patterns.
- Have good experience designing cloud-based solutions in Azure by creating Azure SQL database, setting up Elastic pool jobs and design tabular models in Azure analysis services.Have extensive experience in creating pipeline jobs, schedule triggers using Azure data factory.
- Data Ingestion to one or more Azure Services - (Azure Data Lake, Azure Storage, Azure SQL, Azure DW) and processing the data in In Azure Databricks.
- Optimized Azure Data Factory (ADF) Pipelines with Various Partitioning Techniques
- Expert in Azure Integration Runtime (IR) Configurations and Data Integration Units DIUs
- Implemented Copy activity, Custom Azure Data Factory Pipeline Activities for On-cloud ETL processing.
- Expert in various On-Premise to Azure Cloud Migration Activities using DMA Tool.
- Proficient in Azure Data Factory to perform Incremental Loads from Azure SQL DB to Azure Synapse.
- Strong Experience in Extract Transform and Load data from Sources Systems to Azure Data Storage services using a combination of Azure Data Factory, T-SQL and U-SQL Azure Data Lake Analytics.
- Strong skills in visualization toolsPower BI,Confidential Excel - formulas, Pivot Tables, Charts.
- Expert in creating and developing Power BI Dashboards into rich look.Generated reports and charts by connecting to different data sources like excel, sq. server and implemented drill down functionality using Power BI tool.Implemented drill down functionality by building hierarchies in Power BI visualizations.
- Expert in Power BI, Power Pivot to develop data analysis prototype, and used power view and power map to visualize reports.Published Power BI Reports in the originations and made Power BI Dashboards available in web clients.
- Have extensive experience on Power BI Gateways to keep dashboards and reports up to date.
- Experience in Database Design and development with Business Intelligence usingSQL Server 2014/2016, Integration Services (SSIS), DTS Packages, SQL Server Reporting Services (SSRS),Star Schema and Snowflake Schema.
- Excellent in High Level Design of ETL DTS Packages &SSIS Packages for integrating data using OLE DB connection from heterogeneous sources like (Excel, CSV, Oracle, flat file, Text Format Data) by using multiple transformations provided by SSIS such as Data Conversion, Conditional Split, Bulk Insert, Merge and union all.
- Have Data warehousing experience in Business Intelligence Technologies and Database with Extensive Knowledge in Data analysis, TSQL queries, ETL & ELT Process, Reporting Services (using SSRS, Power BI) using SQL Server 2005 / 2008/2012/2014 /2016 SSIS, SSRS and SSAS, SQL Server Agent.
- Developing reports on SSRS on SQL Server (2005/2008/2012 ). Designed and developed various Service management reports, utilizing dynamic and cascading Prompts, sub-reports, charts, Formulas, parameterized reports, Navigation reports and dynamic reports using SSRS.
- Having good experience in ETL Concepts like Lookup Caches, Package Configurations and Variables.Extensive experience in T-SQL to write: stored procedures, triggers, functions, tables, views, indexes and relational database models.
TECHNICAL SKILLS
- Azure Data Lake
- Azure Data factory
- Azure Databricks
- Azure SQL database
- Azure SQL Datawarehouse
- Python
- SPARK
- Pyspark
- MSBI (SSIS&SSRS)
- Data Visualization
- Data Migration
- SQL Server programming
- Power BI
PROFESSIONAL EXPERIENCE
Confidential
MS Azure Data Engineer
Responsibilities:
- Extract Transform and Load data from Sources Systems to Azure Data Storage services using a combination of Azure Data Factory, T-SQL, Spark, Pyspark, SQL and U-SQL Azure Data Lake Analytics. Data Ingestion to one or more Azure Services - (Azure Data Lake, Azure Storage, Azure SQL, Azure DW) and processing the data in InAzure Databricks.
- Created Pipelines in ADF using Linked Services/Datasets/Pipeline/ to Extract, Transform and load data from different sources like Azure SQL, Blob storage, Azure SQL Data warehouse, write-back tool and backwards.
- Worked on migration of data from On-prem SQL server to Cloud databases(Azure Synapse Analytics (DW) & Azure SQL DB).
- Design Setup maintain Administrator the Azure SQL Database, Azure Analysis Service, Azure SQL Data warehouse, Azure Data Factory, Azure SQL Data warehouse.
- Build Complex distributed systems involving huge amount data handling, collecting metrics building data pipeline, and Analytics.
- Design and implement database solutions in Azure SQL Data Warehouse, Azure SQL.
- Architect & implement medium to large scale BI solutions on Azure using Azure Data Platform services (Azure Data Lake, Data Factory, Data Lake Analytics, Stream Analytics, Azure SQL DW)
- Design & implement migration strategies for traditional systems on Azure (Lift and shift/Azure Migrate, other third-party tools.
- Have good experience working with Azure BLOB andData lakestorage and loading data intoAzure SQL Synapse analytics (DW).
- Implemented Disaster Recovery and Failover servers in Cloud by replicating data across regions.
- Worked on tabular models onAzure analysis servicesfor meeting business reporting requirements.
- Have extensive experience in creating pipeline jobs, scheduling triggers, Pyspark, and Mapping data flows using Azure Data Factory (V2) and using Key Vaults to store credentials.
- Developed business intelligence solutions using SQL server data tools 2015 & 2017 versions and load data to SQL & Azure Cloud databases.
- Data Ingestion to one or more Azure Services - (Azure Data Lake, Azure Storage, Azure SQL, Azure DW) and processing the data in In Azure Databricks.
- Analyze, design and build Modern data solutions using Azure PaaS service to support visualization of data. Understand current Production state of application and determine the impact of new implementation on existing business processes.
Confidential
Power BI Developer
Responsibilities:
- Designed and documented the entire Architecture of Power bi Poc
- Integrated Custom Visuals based on business requirements using Power BI desktop.
- Used Power BI, Power Pivot to develop data analysis prototype, and used power view and power map to visualize reports.
- Published Power BI Reports in originations and made Power BI Dashboards available in web clients.
- Involved in creating data visualizations like Clustered Column Chart, Line Chart, Slicer, Gauge Visualization etc. with Power BI.
- Utilized Power BI gateway to keep dashboards and reports up to date with on premise data sources.
- Sharing of Power BI reports and managing workspaces in Power BI service.
- Data Ingestion to one or more Azure Services - (Azure Data Lake, Azure Storage, Azure SQL, Azure DW) and processing the data in In Azure Databricks.
- Imported data from SQL Server DB, Azure SQL DB to Power BI to generate reports.
- Provided continued maintenance and development of bug fixes for existing and new Power BI Reports.
- Used various sources to pull data intoPower BI such as Sql Server, SQL Azure.
- Installed and configured Enterprise gateway and Personal gateway in Power bi service.
- Created Workspace and content packs for business users to view the developed reports.
- Scheduled Automatic refresh and scheduling refresh in power bi service.
- Wrote calculated columns, Measures query’s in power bi desktop to show good data analysis techniques.
- Weekly presentation to the business users about the reports and their changes as required.
- Worked on all kind of reports such as Yearly, Quarterly, Monthly, and Daily.
- Worked on all types of transformations that are available in Power bi query editor.
- Developed tabular queries for efficient analysis of report usingPivot/Un pivotinT-SQL.
- Developed Excel PowerView, PowerQuery and PowerPivot dashboards for data analysis
- Developed confidential proprietary analytical tools and reports with Confidential Excel, and Power Pivot and Power Point
- Designed Dashboard Report,Linked reportandSub Report by Year, Quarter, Month, and Week.
Confidential
Sr. SQL Server BI Developer
Responsibilities:
- Highly responsible fordata transformationfrom various formats to company’s format on a weekly basis. Managed external client project contracts from technical consultation to delivery of projects
- Involved inPlanning,DefiningandDesigningdatabase based on business requirements and provided documentation.
- Transformed complex business logic into Database design and maintaining it by using SQL objects likeStored Procedures, User Defined Functions, Views, T-SQL Scripting and Jobs.
- Helped to designed all kind of production and sales reports for Sales and procurement departments usingSSRS
- DesignedSSIS Packagesto transfer data from flat files to SQL Server usingBusiness Intelligence Development Studio.
- Performed Index analysis for tables and came up with more efficient solutions to useClusteredandNon-Clustered Indexesfor significant performance boost using index tuning wizard.
- Filtered bad data usingDerived column, Lookups, Fuzzy Lookups,Conditional split.
- DesignedSSISpackages to import and transform data between databases and external data sources using various transformations likeData conversion, Pyspark,File system task, Row counts, OLE DB Source, OLE DB Destination, OLE DB Command, Merge Join, Look up, Sort, Data flow task.
- Generated ETL Packages tasks daily using SQL Server Agent jobs by scheduling and also manually configured.
- Used all types of container tasks such as for loop, foreach loop and sequence container in SSIS.
- Used Transformations such as Union All, Merge and Breakpoints in SSIS.
- Designed and implemented complexSSIS packageto migrate data from multiple data sources fordata analyzing,deployinganddynamicconfiguring of SSIS packages.
- Debugging SSIS packages, Performance tuning onSlowly running SSIS Packages.
Confidential
SQL Server BI Developer (SQL, SSIS/SSRS)
Responsibilities:
- Designed SSIS Packagesto import and export data fromMS Excel, SQL Server and Flat files.
- Used various Transformations in SSIS Dataflow, Control Flow using for loop Containers and Lookups.
- Designed SSIS packages for Data Conversion using data conversion transformation.
- Advanced extensible reporting skills using SQL Server 2008 Reporting Services (SSRS).
- Data Ingestion to one or more Azure Services - (Azure Data Lake, Azure Storage, Azure SQL, Azure DW) and processing the data in In Azure Databricks.
- Designed and created Report templates, bar graphs and pie charts based on the financial data.
- Developed various types of complex reports like Drill Down, Drill through
- Experienced in writing complex SQL Queries, Stored Procedures, Triggers, Views, Joins, Constraints, DDL, DML to implement the business logic
- Deployed the reports and SSIS packages in Dev and QA environment as well as created the deployment document for deploying the reports and the SQL code in staging and production environment.
- Installed, configured, managed and monitored SQL Server 2008 in development and QA environments.
- Involved in performance tuning and writing stored procedures,triggers based on the business requirement using SQL Server.
- Scheduled jobs daily through SQL Server Agent and once data are good then the data is sent to the production.
- DesignedSSRS reports based on the Customer’s specifications and created dashboards and visualized the data in such a way that the customer is satisfied and deployed the reports in the report server.