Aws Solution/data Architect /data Engineer- Enterprise Resume
3.00/5 (Submit Your Rating)
SUMMARY
- 16+ years of combined IT experience as a AWS Solutions/Data Architect/Data Engineer for (DWBI/Cloud/Integration/Migration specialist ) using different set of ETL tools including, Matillion for Redshift, Informatica Power Center/Cloud,Talend, AWS (EC2, S3, Glacier,EBS, Redshift, Redshift Spectrum ( Schema/Table creation, query), Athena, Dynamo DB, RDS, Postgres, SageMaker, EMR, Data pipeline, SNS, SQS,SWF,VPC, DMS, AWS SnowBall, Route53, AWS Glue, Catalog creation (Crawler), ELB,Cloud Watch, CLI Config, AMC, AWS Athena, Elastic Search, Auto Scaling,CFT, API Gateways, IAM,AWS billing & Cost Management,IoT Device management, Sqoop, MSBI, (SSIS, SSAS) and OBIA/OBIEE 10 & 11G, Oracle, MS SQL Server,Teradata and Repository Database.
- Extensive experience in Lambda Function to build scalable Serverless Archicture for application integrations, trigger jobs based on events, API gateways for web hosting for Dev,Stg, and Prod. Deployments.
- Expert in build S3 Datalake Architecture for different source feeds like (csv,JSON, Avro, OCR, Parquet)
- Hands on experience in automation of AWS services using Lambda, SQS, SNS, SES, RouteR3, AWS Glue, Cataloging tables Crawler, Glue manual database/external table creation, Glue ETL creation for Redshift fact and dimension tables loads for huge volume of data and unload data into S3 for Sagemaker/ML consumption for train the predictive models.
- Master in the creation of Cloud Architecture, including Private, Public and Hybrid architectures, IaaS, PaaS and SaaS models for any domain.
- Experience driving enterprise - grade Cloud services adoption across a medium-to-large organization with proven success.
- Expert in strategy consulting for AWS technical services and work with sales and Pre-sales team for business executive initiative for new clients and provide scalable solutions, implementation strategy for dev, prod deployments.
- Identify continuous services improvement, automation areas and prepare strategy script which helps in automating manual tasks and improve service quality and reliability
- Expertise in analysis, design and development of ETL programs as per business specifications and assist in troubleshooting for complex business performance
- Experience in architecting, deploying and managing cost-effective and secure AWS environments, across multi-zones and regions, leveraging AWS services such as EC2, S3, RDS, VPC.
- SME for Technical and Architectural Informatica DW/BI Arch and Domain experience in Healthcare, Logistic, Life Science, Pharmacy, Clinical Trials, Telecom, Finance/Investment, Operations, process improvement and re-Engineering.
- Hands on experience in using Informatica Cloud services like data synchronization, data replication, mapping configuration and object migration from Dev. to SIT, UAT and Prod Env.
- Hands on experience in creation of Global Delivery Framework model (GDF)
- Performed Customer Requirements Gathering, Requirements Analysis, Design, Development, Testing, End User Acceptance Presentations, Implementation and Post production support of BI / Cloud projects
- Hands-on experience performing migrations from On-Premise to Public Cloud solutions
- Managed and successfully developed/implemented various Cloud/DWBI projects related to end to end (HLD/LLD) development/maintenance, model optimization, workflow schedule automations and technical solutions delivery by using Tableau and OBIEE analytical tools.
- Hands on in advanced data model design/Data mart design, Dimensional data model /FACT by using Erwin and ER Studio and Vertablo.
- Collaborate with product development teams and senior designers to develop architectural requirements to ensure client satisfaction with product.
- Developed complex ETL graphs using various components such as Reformat, Join, Roll-up, and Filter by Expression, sort to facilitate cataloging flat files into EDS Data Hub. The components used rely on the various business requirements.
- Expertise in Technical architecture /Data modeling and re-design of existing model for strategic solutions
- Ability to build relationship and have both business-level discussions with the business and technical-level discussions with internal and external technical team members.
TECHNICAL SKILLS
- Bigdata
- Integration Matillion for Redshift
- AWS (S3
- Redshift
- EC2
- Dynamo DB
- Redshift Spectrum
- Athena
- Lambda Function
- API Gateways
- AWS Glue Sqoop
- EMR
- Data pipeline
- SQS
- SNS
- Cloud Watch
- Airflow)
- SQL Server
- Oracle
- UNIX
- Oracle Toad
- Jira
- SQL Server
- Veeva CRM
- IMS
- MDM
- CDIS
- Data one
- Route53.
PROFESSIONAL EXPERIENCE
Confidential, Atlanta, GA
AWS Solution/Data Architect /Data Engineer- Enterprise
Responsibilities:
- Responsible for configuring and build complete Cloud infrastructure for Public and Private network
- Creation of AWS Glue jobs, catalog tables creation with Crawler and manual database/table creation
- Creation of S3 data lake infrastructure and automate the entire process using AWS Lambda function and API Gateways and further ETL process to Cloud warehouse( Redshift) to support Advance Analytics ( Ml)
- Redshift administration, Cluster re-sizing (up gradation ), add compute notes to existing instance, build multi-region cluster replication, Cluster period backup(instance), snap shot of instance.
- Creation of catalog database tables for Athena for fast querying S3 data (more than TB) and setting up for query performance at Athena without moving data into Redshift .
- Hands on Python scripts ETL Glue implementation and writing /basic Sagemanker/ML algorithm for Meand avg, STD calculation on the fly and store back results into S3 for data visualization.
- Redshift spectrum external schema creation and tables creation for S3 data on Redshift upon running instance and query S3 data from Redshift and load data into other fact and dimension tables, rather than using COPY command if data volume are huge.
- Creation of Role based permission and creation of bucket policies to appropriate access to user
- Spin up and build scalable architecture based on client requirement and configure the instances
- Creation of Security group /Firewall groups, KMS Keys, attach them right instances
- Creation of public and private subnets, route tables, IGW, allocation of IP address and EIP and provisioning the web application, DB services, application on appropriate subnets and confiture the NAT gateway, inbound and outbound rules based on application for in/out traffic
- Creation of Matillion Jobs for Redshift to process multiple source systems data into S3, to process all S3 CSV files into redshift and unload the end results data set into S3 for further consumption of Sagemaker/ ML predictive modeling.
- Creation of Redshift data Model deign ( Fact/Dimension) and configure the WLM for query setting, and perform ANALYZE and VACUM operation
- Define the right distribution keys, Columns Encoding, Sort key, write Sql Queries in SQL WorkBench/J / Aginity to process data from S3 to Redshift by passing different parameters
- Process RDS data into Redshift model by using DMS service for one of the source system
- Analyzise and data exploration of source data, finding of data gaps and requirement gathering and responsible for Business stakeholders meetings for project progress on time to time for finalize the Enterprise level architecture and POC for meaching learning technologies/cloud services for on-prem to cloud migration project, and define the road map for POC as a Minimal Viable Product (MVP) at least services for production usage.
- Responsible for Data Engineering team and Technical interaction with client for Sign off on design docs and upfront provide tech. solutions if technical challenges for team
Confidential - Pittsburgh, PA
AWS Solution/Data Architect (AWS/DWBI)
Responsibilities:
- Designed and built cloud service models including Infrastructure-as-a-Service, Platform-as-a-Service, and Software-as-a-Service for 6 companies
- Redshift administration, Cluster re-sizing (up gradation ), add compute notes to existing instance, build multi-region cluster replication, Cluster period backup(instance), snap shot of instance.
- Build complete IaaS in VPC settings, subnets, route tables, IGW settings, setting Inbound/Outbound rules based on application, Setting security groups, apply KMS encryption Keys, Apply right Tags for Billings based on project.
- Responsible for cross region database replication, periodic instance snapshots and setting up cloud watch metrics and alarms.
- Redshift data model design for Sensor, event, leads and streaming data and define the Fact and Dimensions, Dist. Keys and compression column encoding and define the right sort keys for query performance.
- Matillion jobs creation of for Redshift, VACUUM and ANANYZE command for performance tuning for space and sort data for better query performance.
- Define the correct table encoding, Dist style, distribution key and sort key, Zones mapgs (see size) block sice, and periodic static collect.
- Creation of catalog database tables for Athena for fast querying S3 data (more than TB) and setting up for query performance at Athena without moving data into Redshift .
- Redshift spectrum external schema creation and tables creation for S3 data on Redshift upon running instance and query S3 data from Redshift and load data into other fact and dimension tables, rather than using COPY command if data volume are huge.
- Using Agility client tool load data for bulk operation to Redshift db rater than using ETL, because its way faster than any ETL for initial loads.
- Responsible for data extraction from different sources systems like Conduct systems design, feasibility and cost studies and recommend cost-effective cloud solutions Veeva CRM, IMS, MDM, CDIS, Data one and IRDA DB
- Closely working with all business stakeholders and set the right expectation and define the best strategic ways to increase target Revenue.
- Write reference architecture, best practices and thought leadership white papers, identify and enforce AWS architectural best practices
- Create and build the Cloud Infrastructure and responsible for continues data integration for heterogeneous sources
- Advise software development teams on architecting and designing web interfaces and infrastructures that safely and efficiently power the cloud environment
- Responsible ETL design, development and deployment for DEV, SIT, QA and Prod Env., using Talend Studio, TAC and SVN/Git and lead the entire ETL team of 30 resources which includes onshore/offshore.
- Responsible 06 countries (Germany, France, US, China, and Japan) Data model design (Dim/Fact) for Redshift model design to migrate on-premise data into AWS cloud from different sources like Veerva CRM, SQL Server, SFTP, Oracle for multiple marketing/sales channels like approved eMail, Detailing, eDetailing, Webcast, Promo, Events, Field, Education, Congress, Social Media, Web Analytics, Banner, Paid Search, Mobile apps, etc.
- Define the scope and responsible for AWS infrastructure and creation of AWS set of service instances based on user stories for cross country/regions.
- Responsible for BI Country specific and Global level report/Dashboard metrics and making sure data model design optimistic and Business Rule Engine (BRE) for common business rules for across the counties.
- Implement best practices to lead Global competitive market with best brand name and brand products (Medicine, pediatric and Hospital cleaning projects)
- Closely work with sales/pre-sales team to pitch the new customers for techno-functional end-to-end demos.
- Creation of Redshift warehouse design ( Fact/Dimension ), DIST. Keys, DIST. Style, and define the right sort keys for query performance.
- Creation of complex Matillion ETL jobs for Redshift initial/incremental loads and configure WLM for long and short run queries and define them into separate concurrency que limit setup.
- Configure the Data Migration service ( DMS) and Snowball for onprem to cloud migration and data transport to S3 loads .
- Build the S3 data lake ( Central Repository) Schema - Read approach and automate entire flow through AWS Lambda and API Gateways, also capture cloud logs for all API calls monitor and system performance.
- Responsible for frequent travel to 06 Must Win Countries to re-factoring the DW design solutions, provide strategic solutions and define the impact analysis of downstream systems to business stakeholders/executives.
- Responsible for Multi-channel marketing for Must-win-Countries and Must-Win -Brands all channel data integration from different source system for 20 countries data into Global data mart for Global reporting dashboard
- Creation of new RFP for new projects and provide the effort estimation for project cost
- Develop full SDLC project plans to implement ETL solution and identify resource requirements
- Manage build phase and quality assure code to ensure fulfilling requirements and adhering to ETL, SPARK (Scala) integration with AWS cloud services
- Extract, transform and loading data from multiple sources into Elastic Search, Dynamo DB and Redshift
- Responsible for the Configuration Management and Code Migration between various environments using Talend with Release Management and Versioning
- Project planning, effort estimation, deliverables, project costing and People management.
- Recommending and Implementing best practices for Cloud based application services Work flow configuration Task (WCT), performance Creation of the Business Requirements Document - BRD/Technical and Administrative for reporting/Dashboard abilities
- Modeling and Optimizing Elastic Search Indexes, Types based IMCM requirements
- Fixing the production bugs/job failures, collet the matrices (source and Target), uploading all resolution docs/fixes in JIRA tool.
Confidential, New York, NY
AWS Solution Architect (DWBI /AWS) - Techno /Functional
Responsibilities:
- Build scalable solutions secure and cost-effective AWS based solutions to customer requirements
- Work with customer's IT, Engineering, DevOps and Security teams in identifying and prioritizing business and technical requirements
- Configure end-to-end IaaS in VPC for different AWS services, determine the application categories web or db and provision the right instances/sizes on appropriate subnets, configure public and private IP address, NAT and IGW inbound and outbound rules, Security groups settings, Encryption keys /Serverside.
- Redshift query performance, WLM configuration, VACUUM and ANALYZE operation on tables,.
- Define the correct table encoding, Dist style, distribution key and sort key, Zones mapgs (see size) block sice, and periodic static collect.
- Creation of catalog database tables for Athena for fast querying S3 data (more than TB) and setting up for query performance at Athena without moving data into Redshift .
- Athena query performance and creation of metadata catalog tables and creation of Matillon ETL jobs.
- Create credibility and accelerate adoption of Managed Public Cloud and Drive cost reduction/efficiency initiatives (e.g. change EC2 from on-demand to spot instances)
- Requirement gathering, creation of Data Mart from DWH as a sources Oracle, Veeva system by using data extraction tool as Informatica Cloud.
- Develop full SDLC project plans to implement ETL solution and identify resource requirements
- Discuss options with customer considering business needs, security, cost, and operational requirements
- Setting /Accessing Dynamo DB Env and Creation of Tables, on Demand Backup, and access control etc
- Designed the Cloud infrastructure and recommend the on-demand software/instances
- Lead architect in the Governance and steering of the AWS public cloud at customer
- Define the tools and technologies for project scope, articulate the development framework and BRD design for dev. activities
- Work with others on the engineering team to manage the day-to-day development activities, participate in designs, design review, code review, and implementation
- Align proposed solutions with Client' public and private cloud offerings, and where needed identify alternative solutions to fill gaps
- Responsible for build and designthe AWS (PaaS, SaaS and AaaS) and strategic cloud solution to team
- Creation of complex Informatica mappings using set of Informatica transformation (Expression, Joiner, Mapplet, Filter, SQL and Lookup) and applied complex business rules
- Business Development Support & Solutions
- Manage build phase and quality assure code to ensure fulfilling requirements and adhering to ETL, SPARK (Scala) integration with AWS cloud services
- Extract, transform and loading data from multiple sources into Elastic Search and Redshift
- Conduct impact assessment and determine size of effort based on requirements
- Responsible in managing tasks and deadlines for the ETL teams both Onsite and Offshore
- Responsible for the Configuration Management and Code Migration between various environments using Informatica with Release Management and Versioning
- Analysis of knowledge business processes and functional requirement
- Responsible for demonstrating software to internal users
- Project planning, effort estimation, deliverables, project costing and People management.
- Recommending and Implementing best practices for Cloud based application services Work flow configuration Task (WCT), performance tuning by looking into Informatica activity log (DTM, source, trans, target) statistics.
- Modeling and Optimizing Elastic Search Indexes, Types based on eRoster Search Patterns and Requirements
- Fixing the production bugs/job failures, collet the matrices (source and Target), uploading all resolution docs/fixes in JIRA tool.
- Maintain current technical knowledge to support rapidly changing technology, always on a look out for new technologies and work with management and the development team in bringing in new technologies.
Confidential, Bridgewater, NJ
Enterprise Solution/Data Architect
Responsibilities:
- Responsible for design, development, and implementation of Cloud solutions
- Working with Business and System Analyst’s to transform business requirements into technical designs into EADW schema.
- Redshift tact and dimension table design, WLM queue settings, VACUUM and ANALYZE performance technique implementation and period backup for Redshift cluster snap shots and cross region replication
- Responsible for Security requirements and deployments, including the use of encryption, key-pairs, MFA, ... Database solutions EC2 on-demand, spot and reserved high-Availability
- Translate business requirements and operational strategy into a long-term, executable solution plan and roadmap
- Disaster Recovery solutions Storage and network solutions Cloud Formations Automation.
- Create credibility and accelerate adoption of Managed Public Cloud
- Assist with the product evaluation, selection and implementation for respective managed services that require 3rd party products to support application or business
- Write and implement the respective IAM policies as required per the respective services
- Participate in planning, implementation, and growth of our customer's Amazon Web Services (AWS) foundational footprint.
- Work with team to design, build, automate and document a multi-tiered managed services platform
- Understanding of security compliance, managed services in cloud based workloads
- Manage multiple projects simultaneously and be able to adapt to changing business needs
- Deployment and management of AWS services - including but not limited to: VPC, Route 53, ELB, EBS, EC2, S3, Dynamo Db, Redshift and Snowball
- Design and build Informatica mappings SCD-1,2 Solutions to meet customer specifications to maintain complete history
- I’ve been hands on experience in RDBMS /performance tuning and write custom SQL queries as business need.
- Task flow creation for xml upload/download files and map them run them in a schedule
- Lead and delivered multiple full lifecycle data warehousing projects with hands on experience in different phases such as requirements gathering, data mapping, dimensional data modeling, ETL and OLAP design and development, testing and overall implementation.
- Creative Ideas for Objects migration during the QA Traceability Matrixes and Deployment phases (End-to-End)
- Enhancing the existing EDW schema model and adding some additional table attributes and modifying attributes data types as business needs
- Creation of several task flows for combining multiple integration tasks
Confidential, Chicago, IL
Data Architect / Data Modular
Responsibilities:
- Requirement gathering, creation of Data Mart from DWH as a sources system by using data extraction tool as SSIS.
- Designing end to end solutions and collaborating with application teams in executing those solution blueprints and plans
- Develop full SDLC project plans to implement ETL solution and identify resource requirements
- Redshift Table Design (Dim/FACT) and define the sort and distributed key for data distribution.
- Setting /Accessing Dynamo DB Env and Creation of Tables, on Demand Backup, authendation and access control .
- Business Development Support & Suctioning
- Manage build phase and quality assure code to ensure fulfilling requirements and adhering to ETL architecture
- Conduct impact assessment and determine size of effort based on requirements.
- Perform as active, leading role in shaping and enhancing overall ETL Informatica architecture
- Manage build phase and quality assure code to ensure fulfilling requirements and adhering to ETL architecture.
- Played the role of a Program Technical Lead/ETL Architect and was responsible for designing the ETL Strategy & Architecture of the Project
- Responsible in managing tasks and deadlines for the ETL teams both Onsite and Offshore.
- Responsible for the Configuration Management and Code Migration between various environments using Informatica with Release Management and Versioning
- Involved in conducting the review of Informatica Code, Unit Test Cases & Results with the Developers
- Supporting as a project lead which included, onsite-offshore team coordination, reviewing reports/dashboards, requirement gathering, work assignment to team, Tableau Admin task etc. required Data Warehouse Consulting which include Data warehouse ETL Architect, Program Technical Lead, and Project/Team Lead
- QA and approval of logical, physical data models, QA and approval of ETL designs
- Lead design reviews with Client and participate in ETL and reporting requirements meetings
- Lead design IDW logical data models and Assist Client DBA’s in creation of IDW physical data models
- Lead the creation of detailed requirements to enable ETL design and creation of ETL integration test plans and ETL performance test plans (if needed)
- Lead design, development, unit-testing, and integration testing of ETL jobs and assist Support design of ETL reusable components
- Lead and ETL design development, unit-testing, and integration testing of ETL reusable components
- Provide oversight to ETL developers
- Provide necessary documentation to Client resources responsible for configuration of job scheduling
- Understand reporting requirements to enable design of reporting components (BOBJ Universe and BO reports)
- Lead creation of reporting migration, Tableau Server maintenance and integration test plans and reporting performance test plans (if needed)
Confidential, Charlotte, NC
Solution Architect/Data Modeler
Responsibilities:
- Identify and enforce AWS architectural best practices and full development life cycle
- Work with the customer success team, engineering team and professional service team to create a detailed project delivery plan and identify key delivery milestones and dependencies.
- Data modeling and transformation of large scale data sources using SQL, Redshift, Hadoop, Spark, Hive, EMR, or other Big Data technologies
- Requirement gathering from onsite, data model design finalization, re-design of existing model
- Design, build and own all the components of a high volume data-mart end to end in support of business reporting
- Architecting large-scale, modular and efficient data backend in database systems such as Redshift and Dynamo DB.
- Manage technical support teams and providing operational support in AWS Env.
- Running projects and delivery experience driving business requirements, identifying gaps and omissions, and designing solutions
- Provide blueprint, design and deploy scalable, HA and fault tolerant systems on AWS. responsible for code deployment in AWS deployment and management services
- Create POC to Migration of existing on-prem applications to AWS
- Understanding of network technologies as they relate to AWS and security features and tools that AWS provides and how they relate to traditional services
- Automate the code migration/code deployment from Dev to Prod and migration process to the maximum extent possible
- Involved in Project proposal and effort estimation for scope of project.
- Developing Web Service Framework for SIF real time data load and outbound services using Axis2 and Castor API.
- Monitoring of ETL Jobs on production system, Analysis of failed jobs, creation of tickets for failed jobs, deployment of ETL jobs from Dev. to Prod systems, creation of solution design for failed jobs, work with testing team and ETL design teams, deployment of production deliverables.
- Creation of complex ETL mappings, creation of ETL test cases, provide solution to ETL Analytics team.
- Strategic solutions for new business line, Staff augment ion onsite/offshore, Capability build for Client specific tools and technologies, Technical discussion with Maersk Like, Data Modeling, Design/ model architectural changes and part of solution design frame work.
- Solution Architect for Fastest MDM upgrade to 9.5.1 for 5 environments
- Upgrade of MDM from 9.1 to 9.5.1 HF4
- Understand current environment architecture and impact analysis
- Migration of configurations and data on upper environments
- Upgrade of Configuration and data from XU Version to 9.5 environment
- Handling Build and Migration of SIF and MDM Hub code
- Leading offshore team for upgrade.
Confidential, St. Louis, MI
Solution Architect / Project Manager
Responsibilities:
- DWBI Delivery Model Proposal, Define the DWBI strategic approach (ETL and Analytics), POC Creation for each business line, Staff augment ion onsite/offshore, Capability build for Client specific tools and technologies, Technical discussion with Florida Blue.
- Experience with Healthcare Data warehousing, Deep understanding of data warehousing concepts
- Good experience in physical/logical data modeling using Erwin
- Data modeling skills for data marts and data warehouse in a healthcare environment
- Deep understanding of analytics use cases in a provider scenario (especially US)
- Understanding of HL7 messaging and HL7 RIM and DICOM
- I have an extensive experience in Informatica Master Data Management to quantify of acquiring and retaining customers, improving efficiency, reducing costs and managing regulatory compliance to limit risk and enhance transparency.
- Developed and Implementation of extract, transformation, and load processes using SSIS as an ETL tool
- Participated in Mentoring of peers and higher Management seniors in Confidential and Informatica Data Director
- Analysis of requirements, create estimates, Supported ETL architect in designing & formulating load strategy’s
- Develop the mappings as per the logical data map, optimizing the mapping to load the data, creating and scheduling the sessions based on the mappings created to load the data into the data marts
- Responsible for component deliverable for client end and onsite offshore coordinator.
- Leading 15 -20 members @ onsite/offshore on day to day basis for smooth functioning of project.
- Defining the ETL strategy for SEDW/Dart Mart Layer as business required