Sr. Bigdata Architect Resume
5.00/5 (Submit Your Rating)
San Jose, CA
SUMMARY
- A seasoned Bigdata Architect with 20 plus year experience contributed as architect, Evangelist, and Technical Manager with very strong background in migration to big data platform, Cloud migration, system integration and data acquisition across companies, Creating POC, Database Modeling, Designing, Architecting, Developing, Implementing Enterprise business application, RDBMS, nosql, Data Warehouses, BI & Relational Systems and in managing large Projects with Hadoop, SAP HANA, SQL Server and Oracle.
- Highly experienced in Proposal Design, POC development, Pilot project implementation, Building Data Lake and Data Lake Management Platform, Bigdata Platform As A Service(PAAS), Reference Data Architecture building with Data Governance, Security, Master Data Management and Meta Data Management for batch processing, Streaming Data, Analytics, Data Quality Management, Data Modeling, Data Lineage, IOT Technologies, Cognitive Analytics, Service BI and Real Time sub - second Response applications using large volume of data with complex rules.
- Actively involved in various stages of building BigData Applications like plan, budget, architect, design, development, cluster setup, security setup, analysis and administration using HDFS, MapReduce, Hive, PySpark, Spark SQL, Spark Streaming, from Open Source to Commercial Distributions like Apache, Cloudera and Hortonworks
- Having perfect combination of BigData/Hadoop/Spark, Python, Database, Datawarehouse, Datamart and Business Intelligence Analytics technology experience
- Exposure to Data Lake Implementation using Apache Spark and developed Data pipelines and applied business logics using Spark and used Scala and Python to convert Hive/SQL queries into RDD transformations in Apache Spark.
- A long and successful career spanning analyzing business requirements and designing effective Data modeling, User Interface design for relations database / Data Warehouses / Systems, reporting solutions / BI Solutions that facilitates an organization to detect and respond immediately to opportunities and issues.
- Hands-on development experience in implementing more than 50 business application for OLAP and OLTP Systems, DW/BI Systems using Oracle, ETL, SQL, PL/SQL, Business Objects, Oracle Business Intelligence, ERWIN.
- Excellent Project Management skills using Agile, Waterfall methodologies; Managed medium to large size Web / Data Warehouse Projects involving Million+ lines of code, project team sizes varying from 10 to 75+.
- Outstanding communication and liaison skills proven by effective interaction with clients, business stakeholders, management, and team members. Involved in Architect, design development and implementation of various business applications and products for many industries.
- Have thorough understanding of Big Data ecosystems, Hadoop platform, application framework and architecture, performance tuning, HDFS, MapReduce and NoSQL.
- Recent experience working in the Cloud Computing Team for high performance and high-available servers, Amazon Web Services, Google Cloud Platform. Managed multiple teams for successful implementation of projects and products. Involved in creating Customer Master Data Management (MDM).
TECHNICAL SKILLS
- Windows 2000, HP/UX, Sun Solaris
- Oracle, MS SQL Server, SAP HANA
- Oracle Business Intelligence (OBIEE (BI Publisher, BI Answers, Dashboard),
- Business Objects
- Oracle Forms, Reports, PL/SQL (Stored Procedures, Triggers), OCI, Pro*C, SQL*Loader, Exp/Imp, SQL*Plus, Query Analyzer, Explain Plan
- C, PowerBuilder, Python
- Stored Procedures, Triggers
- Unix Shell Scripting
- MS project, ERWin, Oracle Designer, Visio, MS Office Applications
- OLAP - Kimball Life Cycle / Dimensional Star Schema, Inmon
- Project Management - Agile / Waterfall Methodology
PROFESSIONAL EXPERIENCE
Confidential, San Jose, CA
Sr. Bigdata Architect
Responsibilities:
- Worked with the Big Data Architecture Team which created the foundation of this Enterprise Analytics initiative in a Hadoop-based Data Lake.
- Data sizing Apache Hadoop clusters for application development and Hadoop tools in Hive.
- Designed, architect in migration process of Teradata Application to Hadoop.
- Automated reversed engineering process to code to create STM using Python
- Designed frame works using PySpark to process billions of rows.
- Designed and developed User Experience data to move from Teradata to Hadoop using Hive.
- Worked on database capacity planning and sizing, performed data quality check and profiled SSOT and Identity data.
- Build complete ERD for the entire project using Visio.
- Optimized queries to process billions of rows in a single table which is closure to petabyte
- Identify query duplication, complexity and dependency to minimize migration efforts.
- Created Hive External tables and loaded the data into tables and query data using HQL and worked with application teams to enable ACID properties, Hadoop updates, patches, version upgrades as required.
- Involved in Data Mapping, Data Governance, and Transformation and cleansing rules for the Master Data Management, product Architecture.
- Created POC for implementing streaming use case with Kafka and HBase services.
- Deployed Big Data solutions in the cloud. Built, configured, monitored and managed end to end Big Data applications
- Worked on Hbase to process the data where it required update in Hive tables
- Created data lake for user experience data from Hadoop to Snowflake DB
- Worked on Hive for exposing data for further analysis and for generating transforming files from different analytical formats to text files.
Environment: - Hortonworks, Hadoop, PySpark, Hive, Terradata, SQL, Python
Confidential, Woburn, MA
Sr. Enterprise Architect/ Manager
Responsibilities:
- Lead & managed a team of Big Data and mobile consultants to MDM applications
- Hadoop & Big Data architecture design, planning, installation, deployment
- Create data model in firebase for security apps for android and iOS devices
- Used Kafka to transform app change status data from mobile device to DataBase
Environment: FireBase, Hadoop, Python, Kafka
Confidential, CA
Sr. Enterprise Architect/ Manager
Responsibilities:
- Lead & manage a team of Big Data consultants
- Hadoop & Big Data architecture design, planning, installation, deployment
- Migration from relational databases (source systems), data warehouse (Teradata) to Big Data platforms on GCP.
- Reviewed and documented business logic in Informatica mapping, work flow to migrated to GCP.
- Build 360 view of Customer MDM (master data management) form 10 different customer source application
- Used Sqoop, Kafka to transform data from 85 different source system to cloud platform
- Responsible for creating and implementing the entire data architecture for moving 100 + source system to Hadoop big data platform in the GCP cloud
- Scope, plan and manage Big Data projects
- Direct the work and priorities of Big Data & Hadoop team
- Designed and utilized applications leveraging Google Cloud Platform
Environment: GCP, Hadoop, Python, Hive, Kafka, MS SQL Server, Oracle.
Confidential, Fremont, CA
Sr. Enterprise Data Architect
Responsibilities:
- Responsible in building enterprise data model, data mapping, creating data footprint for Lam.
- Responsible in setting of standards for EAM
- Responsible in creating CDM, LDM, PDM.
- Responsible in working with the team and review the models created and guiding the modeling and development team for successful implementation of various dash board in Qlik Sense.
- Directing new data development and ensuring consistency/integration with existing EDW structure. Maintaining data architecture, metadata, master data, data management standards, data dictionaries, and data element naming conventions.
- Migrating, System integration and data acquisition of data from across companies. Migrating data from SAP to SQL Server and migrating data from SQL Server to SAP HANA.
- Responsible for delivering as part of the comprehensive data model, the data dictionary and source-to-target mapping and other metadata as required.
- Participated in data analysis and data dictionary and metadata management - Collaborating with business analysts, ETL developers, data quality analysts and database administrators.
- Involved in analysis of source system files fields and identified master data fields and reference\metadata data fields and also identified the level of granularity of the source systems data.
- Created Efficient Metadata data model to know about the source systems, oracle staging and Mongo Systems metadata and programs.
- Mentoring team about provider and member metadata/master data/transaction data models and usage.
- Experience in SAP HANA data modeling using Attribute View, Analytic View and Calculation View and the onside and offshore team with respect to implication and performance of the front-end dash board in Qliks
- Hands on experience in HANA modeling. Created multiple schemas and table definitions. Experience in creating Attribute views, analytical view and calculated views.
- Load Flat file and Relational Database (SQL Server) to SAP BW Data Warehouse and SAP HANA database using BODS 4.2.
- Worked on creating Data profiling, identify cleanup of Equipment data in Information steward
- Design of analytic, calculation and attribute views in SAP HANA to provide data attributes for marketing dashboards, equipment for CSBG group
- Partnered with marketing business units to define functional data requirements that provided system/data functionality for new marketing platform
- Operational DataMart’s and replication using SLT from several SAP systems
- Led SAP Customer Address Cleaning Project
- Involved in POC for data lake and big data for real-time data for Machine processing data
- Identified the existing issues within business master data as well as technical master
- Created and implemented master data standards bases on SAP Best Practice for Data Governance model
Environment: SAP HANA, SAP ECC, SAP BW, MS SQL Server, Oracle.
Confidential, Bakersfield, CA
Sr. BI Manager / Enterprise Data Architect
Responsibilities:
- Worked on Creation of Master Data Management (MDM) for email customers.
- Influenced and managed change requests. (change management)
- Involved in creating E-R Diagram for FDS for various subject area (which has more than 1000 entities and 0.5 billon rows).
- Involved in architecting, POC for data lake and big data for real-time data for Wells pressure and temperature.
- Involved in writing SQL scripts to create source to target mapping from Oracle, MS SQL Server
- Directing new data development and ensuring consistency/integration with existing EDW structure. Maintaining data architecture, metadata, master data, data management standards, data dictionaries, and data element naming conventions.
- Migrating data from Oracle to SQL Server and migrating data from SQL Server to Oracle.
- Responsible for delivering as part of the comprehensive data model, the data dictionary and source-to-target mapping and other metadata as required.
- Participated in data analysis and data dictionary and metadata management - Collaborating with business analysts, ETL developers, data quality analysts and database administrators.
- Involved in provide a complete solution using (Service Oriented Architecture) SOA for enterprise data ware housing for entire Confidential and road map for transition 300’s of system which has 3000+ table in some of the system (Energy Component, LOWIS, Catalyst)
- Involved in consolidating 1200 to 300 reports for RBI (Reporting Business Intelligence) using SSAS, MDX, SSRS
- Involved in setting up standards for code and process
- Worked on building concept information Model (CIM) with Foundation Data Store
- Worked with BI and ETL lead to creating source to target mapping Review the code
- Profiled the source data for accuracy and consistency and to identified trusted source data
Environment: MS SQL Server, Oracle 10g, Erwin 7.1, Python