Lead Big Data Management (bdm)/sr.idq Developer Resume
San Francisco, CA
SUMMARY:
- Informatica Certified Developer with 12 + years of total IT experience in all aspects of project development life cycle including Analysis, Design, Development, Implementation, Modeling, Testing, Reconciling, and Supporting for Enterprise Data warehousing, Data marts, Data Conversion, Data Integration, Data management projects.
- 11+ years of extensive experience in Informatica Power Center (9.x/8.x/7.x/6.x) tools along with that Informatica Big Data Manager (BDM)/ IDQ/MDM.
- Hands on experience with Data Profiling/Data Quality using Informatica Developer, BDM and MDM toolset
- Expertise in Dimensional Data Modeling techniques such as Star and Snowflake schemas using ERWIN 7.x/4.x/3.x.
- Expertise in design and implementation of SCD - Slowly changing dimensions types (1, 2 and 3) and CDC - Change data capture.
- ETL best practices/methodologies (transformation usage, naming conversions, mapping standards, slowly changing dimensions, deployment groups, versioning, etc).
- Extensive experience with Data Extraction, Transformation, and Loading (ETL) from multiple data sources such as Oracle, DB2, SQL Server, Flat files, CSV files, VSAM files, COBOL files and XML files into a Reporting and Analytical Data Model.
- Have very good exposure on Change Data Capture using PowerExchange for Oracle.
- Expertise in Database Design, Development of SQL, PL/SQL and Query tuning for better performance.
- Knowledge in Hadoop, Good knowledge of database structures, theories, principles, and practices.
- Hands on experience in HiveQL, Analytical and problem solving skills, applied to Big Data domain
- Proven understanding with Hadoop, HBase, Hive, Pig, and HBase.
- Hands on experience on Pushdown optimizations.
- Experienced in writing Unix Shell Scripts for the automation of ETL processes and using schedulers like Autosys, Ctrl-M, UC-4, Maestro and ABC HP product.
- Good knowledge in Informatica Master Data Management (MDM) and Informatica Data Quality (IDQ).
- Knowledge on Greenplum, Big Data Analytics, Could systems.
- Exposure with OBIEE in designing Dashboards, Answers, Delivers and Analytics Administration tool.
- Excellent working knowledge on multiple platforms like HP-UX, Windows XP/200x/98, Unix-AIX, Win NT 4.0, Sun Solaris, Linux.
- Dedicated, self-motivated achiever who is committed to success and adept at handling multiple tasks in a high-pressured environment and Experience with working on a Scrum Agile team.
- Good communication skills and Ability to work in-groups as well as independently
TECHNICAL SKILLS:
ETL Technologies: Informatica Power Center 10.x/9.x/8.x/7.x./6.x, IDQ/IDE 10.x/9.x, Power Exchange, Sybase ETL Development, AbInitio (GDE 1.12, Co>Op 2.12), Data Stage 5.x/6.x.
OLAP: OBIEE 10.x, Siebel Analytics 7.8/7, Business Objects XI R2, Cognos 8.x/7.x/6.5
Databases: Oracle 11g/10g/9i/8x/, SQL Server 2000/2005, DB2, Teradata V2R6, MS Access 2000.
Applications: SAP, Oracle EBS, PeopleSoft, DELTEK, Tririga, LIMS(Lab ware), Open Lab, Empower2
Operating Systems: Windows 2000/NT, UNIX, DOS
Languages: SQL, PL/SQL, C, C++
DB Tools: SQL plus, SQL Loader, TOAD, Erwin 4.1
Data Modeling: E/R Modeling, Dimensional Modeling (Star, Snowflakes Schema Modeling).
PROFESSIONAL EXPERIENCE:
Confidential, San Francisco, CA
Lead Big Data Management (BDM)/Sr.IDQ Developer
Responsibilities:- Lead the development, implementation and take the responsibilities for setting up Informatica BDM tool & Hadoop cluster environment from inception to production
- Closely worked with Business users, Informatica Product support group and Hortonworks teams
- Implemented Informatica BDM mappings for extracting data from DWH to Data Lake
- Involved setting up Hadoop configuration (Hadoop cluster, Hive connection) using Informatica BDM.
- Design & development of BDM mappings in Hive mode for large volumes of INSERT/UPDATE
- Created BDM parameter sets & parameter files for daily & delta loads
- Proven understanding with Hadoop, Hive, Pig, and HBase.
- Medium to complex quality rule design, development and implementation patterns with cleanse, parse, standardization, global Address Validation, global phone number validation and format, Search Match, Match Merge.
- Used Siperian Web Services as physical object and call the API using Web Service Consumer transformation and handle the SOAP exception in case of failures, send e-mail notifications for success/failure with message return.
- Create & deployed IDQ applications and run the applications, workflows, mappings using Unix scripts.
- Develop numerous mappings using the various transformations including Address Doctor, Association, Case Converter, Classifier, Comparison, Consolidation, Match, Merge, Parser etc.
- Used IDQ Mapping variables/output and capture into workflow variables.
- Involved in designing Logical/Physical Data Model for IDQ custom metadata
- Data governance application for Informatica MDM Hub that enables business users to effectively create, manage, consume, and monitor master data
- Involved in monitoring of production ETL applications, Production Support through tickets, executing Scripts / managing scheduled activities based on business requirements.
- Configure the Sessions & Workflows for recovery and with High availability.
- Extensively worked on code migration from development box to various environments, create & manage Informatica reference data.
Environment: Informatica BDM 10.1.0, Hadoop 2.7.1, Hive 2.3.2, SOAP UI 5.2.1, Address Doctor 5.9, Oracle12c, SQL Server, Flat files, SQL, WSDL, PL/SQL, SQL Developer, SAP HANA, SURF, Windows XP, Mule soft, ActiveVOS, UNIX Shell Scripts, cron jobs.
Confidential, Akron, OH
Sr.BDE/IDQ Lead
Responsibilities:- Lead the development team and take the responsibilities for setting up Informatica BDM tool & Hadoop clutter endowment.
- Implemented Informatica mappings using Informatica Bid Data Edition for capturing change data from legacy systems
- Data profiling on legacy systems using IDQ tool prior to data loading into staging.
- Discussed with Business analysts and Data Modular with the profile results and involved in Gathering business requirements and Data Model.
- Medium to complex quality rule design, development and implementation patterns with cleanse, parse, standardization, Address validation, scorecard, configure thresholds, exception, and notification and reporting with Informatica BDE.
- Develop numerous mappings using the various transformations including Address Doctor, Association, Case Converter, Classifier, Comparison, Consolidation, Match, Merge, Parser etc.
- Used Session parameters, mapping variable/parameters and create parameter files for imparting flexible runs of workflows based on changing variable values.
- Created Complex ETL Mappings to load data using transformations like Source Qualifier, Sorter, Aggregator, Expression, Joiner, Dynamic Lookup, and Connected and unconnected lookups, Filters, Sequence, Router and Update Strategy
- Configure the Sessions & Workflows for recovery and with High Availability.
- Developed BDE mappings using Informatica Developer and created HDFS files in Hadoop system.
- Implemented SCD type1 mappings using BDE and load the data into Hadoop Hive tables using Push down mode.
- Involved in designing Physical Data Model for Hadoop Hive tables
- Wrote HiveQL queries validate HDFS files & Hive table data to make sure the data meet the requirements.
- Analytical and problem solving skills, applied to Big Data domain
- Proven understanding with Hadoop, Hive, Pig, and HBase.
Environment: Informatica BDE (Big Data Edition) 9.6.1 Grid, IDQ 9.6.1, Hadoop 2.7.1, Hive 2.3.2, Oracle11g/12c, SQL Server, Flat files, HDFS, SQL, PL/SQL, TOAD, Salesforce (SFDC), Windows XP, UNIX Shell Scripts, Maestro.
Confidential, Detroit, MI
Lead IDQ/Informatica Developer
Responsibilities:- Closely worked with Business users, Project manager & Stakeholder for gathering high-level requirements & project deadlines
- Implemented Informatica Data Quality application for both Inbound & Outbound files
- Creating reusable rules for profiling based on the business requirement and automated scorecards results to business user group.
- Medium to complex quality rule design, development and implementation patterns with cleanse, parse, standardization, Address validation, scorecard, configure thresholds, exception, and notification and reporting with Power Center.
- Developed Power Center custom mappings using IDQ metadata (Profile, Scorecards results) for end user reports
- Develop numerous mappings using the various transformations including Address Doctor, Association, Case Converter, Classifier, Comparison, Consolidation, Match, Merge, Parser etc.
- Used Session parameters, mapping variable/parameters and create Parameter files for imparting flexible runs of workflows based on changing variable values.
- Created Complex ETL Mappings to load data using transformations like Source Qualifier, Sorter, Aggregator, Expression, Joiner, Dynamic Lookup, and Connected and unconnected lookups, Filters, Sequence, Router and Update Strategy
- Involved in designing Logical/Physical Data Model for IDQ custom metadata
- Data governance application for Informatica MDM Hub that enables business users to effectively create, manage, consume, and monitor master data
- Involved in monitoring of production ETL applications, Production Support through tickets, executing Scripts / managing scheduled activities based on business requirements.
- Configure the Sessions & Workflows for recovery and with High availability.
- Implemented reusable mappings & sessions for Operational Audit Process
- Created files in Vendor requested format HL7 & EDI X12(810,832,835,850,855,856) using Informatica.
- Migrating code using deployment groups from development box to test
Environment: Informatica Power Center 9.5.1, IDQ 9.5.1, MDM 9.5.1, Facets 4.51, Oracle11/10g, SQL Server, Flat files, EDI X12, SQL, PL/SQL, TOAD, Erwin, Windows XP, UNIX Shell Scripts, Tivoli.
Confidential, Groton, CT
Sr.Informatica Developer
Responsibilities:- Involved in extracting data from Oracle EBS suite to SAP systems using Informatica Power Center
- Gathered application, source and target information details
- Used Informatica Designer to create complex mappings using different transformations like Filter, Router, Connected & Unconnected lookups, Stored Procedure, SQL Transformation, Joiner, Update Strategy, Expressions and Aggregator transformations to pipeline data to Data Mart
- Created IDQ plans using different components like RBA, Token Labeler, and Character Labeler for address & data standardization
- Involved in designing Logical/Physical Data Models (Star Schema), forward/reverse engineering using Erwin
- Involved in massive data profiling using IDQ (Analyst Tool) prior to data staging
- Used IDQ’s standardized plans for addresses and names clean ups.
- Worked on IDQ file configuration at user’s machines and resolved the issues.
- Used IDQ to complete initial data profiling and removing duplicate data
- Worked with different components in IDQ like Jaro distance, edit distance, Rule based analyzer, bigram, hamming distance.
- Wrote the SQL queries using analytical functions on source & target data to make sure the data meet the requirements.
- Involved in monitoring of production ETL applications, Production Support through tickets, Executing Scripts / managing scheduled activities based on business requirements
- Implemented reusable mappings & sessions for Operational Audit Process
- Configure the Sessions & Workflows for recovery and with High availability.
- Migrating code using deployment groups from development box to test
Environment: Informatica Power Center 9.5.1, IDQ 9.5.1, SAP, Oracle EBS 12.1, Oracle11, SQL Server, Flat files, SQL, PL/SQL, TOAD, Erwin, Pega Systems, LIMS, Windows XP, UNIX Shell Scripts, UC-4
Confidential, Buffalo, NY
Sr.Informatica Developer
Responsibilities:- Interacted with business community and gathered requirements based on changing needs. Incorporated identified factors into Informatica mappings to build Data Mart.
- Developed Informatica Mappings/Workflows to load domain data from Sybase Database, Flat Files into Db2
- Used IDQ to profile the project source data, define or confirm the definition of the metadata, cleanse and accuracy check the project data, check for duplicate or redundant records, and provide information on how to proceed with ETL processes.
- Profiled customer data and identified various patterns of the phone numbers to be included in IDQ plans
- Data extraction and data Transfer Sybase Database using utilities / tools like BCP, and BULK INSERT.
- Designed Informatica complex Mappings and Workflows to be re-startable after a failure
- Used Informatica Designer to create complex mappings using different transformations like Filter, Router, Connected & Unconnected lookups, Stored Procedure, SQL Transformation, Joiner, Update Strategy, Expressions and Aggregator transformations to pipeline data to Data Mart
- Designed tables required for the execution of the ETL processes using Erwin.
- Developed DB2 SQLs for the ETL and for populating run statistics
- Used scripts to import and export data to from the database.
- Developed Slowly Changing Dimensions for Type 1 SCD and Type 2 SCD.
- Developed UNIX Shell scripts for File Manipulation, FTP, Executing DB2 SQLs and archiving log files
- Created DB2 SQLs for Quality Management load that determines the accuracy of data loaded.
- Implemented various Performance Tuning techniques on Sources, Targets, Mappings, Sessions, Workflows and database
- Created files in Vendor requested format HL7 & EDI X12(810,832,835,850,855,856) using Informatica.
- Involved in designing Logical/Physical Data Models (Star Schema), forward/reverse engineering using Erwin
- Prepared Technical Documents related to design, development and testing as per CMM
- Implemented reusable mappings & sessions for Operational Audit Process
- Created mappings to keep historical data (SCD type 2) and load control for ETL extract.
- Prepared the technical specifications using use cases and maintained versioning & deployment groups’ functionality.
- Involved in monitoring of production ETL applications, Production Support through tickets, Executing Scripts / managing scheduled activities based on business requirements
Environment: Informatica Power Center 8.6.1, IDQ 8.6.1, CDC, IBM DB2 UDB 8, Facets 4.51, Sybase, Erwin 4.2, SQL Server 2005, Oracle11g, Excel, Flat files, SQL, PL/SQL, Win SQL, Windows XP, UNIX Shell Scripts, IBM AIX 5.3.4
Confidential, Groton, CT
Data Warehouse/Data Integration Specialist using Informatica
Responsibilities:- Gathered application, source and target information details
- Prepared data migration approach process and design document templates.
- Prepared source to target matrix templates
- Design, develop and implement Extract, Transformation and Load (ETL) processes, programs and scripts using structured software development method.
- Performance tuning of sources, targets, mappings and SQL queries in the transformations
- Analyzing the data to make sure, if the data can used for reporting purposes.
- Involved in monitoring of production ETL applications, Production Support through tickets, Executing Scripts / managing scheduled activities based on business requirements
- Created reports using Business Objects Desktop Intelligence
- Have very good knowledge LIMS, Empower2 applications.
Environment: Informatica Power Center 8.6.1, Business Objects XI R2, Teradata V2R6, Oracle11g, DB2, Excel, Flat files, SQL, PL/SQL, TOAD, Erwin, Autosys, LabWareLIMS 5.0/6.0,NuGenesis, Windows XP, UNIX Shell Scripts, Linux.
Confidential, Charlotte, NC
Application Developer/ IDQ- Enterprise Data Quality
Responsibilities:- Informatica Data Quality specialist for CRDS source systems
- Prepared the Technical Specifications to some extent gathering the requirements from the source systems, Maintained versioning & deployment groups’ functionality
- Design, develop and implement Extract, Transformation and Load (ETL) processes, and IDQ plans, created dictionaries for IDQ plans.
- Created stand-alone IDQ plans using different components like RBA, Token Labeler, Character Labeler and Search Replace
- Integrated IDQ plans (export as mapplets in PowerCenter) within mappings and implemented pushdown optimization for better performance.
- Performance tuning of sources, targets, mappings and SQL queries in the transformations
- Migrating code from development box to Production box
- Analyzing the data to make sure, if the data can used for reporting purposes.
- Wrote Unix Shell Scripts for the automation of ETL processes and using Ctrlol-M scheduler
- Created queries using analytical functions for reporting purpose and update the results for end users.
- Involved in monitoring of production ETL applications, Production Support through tickets, Executing Scripts / managing scheduled activities based on business requirements
Environment: Informatica Power Center 8.6.2, IDQ 8.6.2, IDE 8.6.2, Teradata V2R6, Oracle10g, DB2, MS Access, SQL Server 2005,MySQL, Excel, Flat files, HTML, SQL,PL/SQL, TOAD, UNIX (Sun Solaris), Control-M Windows XP, UNIX Shell Scripts.
Confidential, McLean, VA
Informatica Developer/Consultant
Responsibilities:- Translate Business Requirements into Informatica mappings.
- Using Microsoft Visio 2003 designed for graphical representation of Data flow from Source to Target and as well as used for Technical Detail Design specifications.
- Design, develop and implement Extract, Transformation and Load (ETL) processes, programs and scripts using structured software development method.
- Wrote stored procedures to populate staging tables
- Based on the logic, used various transformation like Source Qualifier, Normalizer, Expression, Filter, Router, Update strategy, Sorter, Lookup, Aggregator, Joiner transformation in the mapping.
- Created reusable transformations and Mapplets and used with various mappings.
- Created Connected, Unconnected and Dynamic lookup transformation for better performance and increased the cache file size based on the size of the lookup data.
- Assisted in designing Logical/Physical Data Models, forward/reverse engineering using Erwin.
- Migrating code from development box to Production box
- Wrote Unix Shell Scripts for the automation of ETL processes and using Ctrlol-M scheduler
- Involved in monitoring of production ETL applications, Production Support through tickets, Executing Scripts / managing scheduled activities based on business requirements
- Have very good knowledge on HR, PO, AR, AP, FA, Billing and GL modules.
Environment: Informatica Power Center 8.1.1, Oracle10g/9i, DB2, MS SQL Server, Excel, Flat files, XML, SQL,PL/SQL, TOAD, PeopleSoft 8.0, DELTEK Cost Point, DLETEK T&E, Mercury Quality Center 9.0, PVCS Tracker, Erwin 4.0, Windows XP, UNIX Shell Scripts.
Confidential, Peapack, NJ
ETL Consultant
Responsibilities:- Created mappings to load Slowly Changing Dimension tables based on the amount of historical dimension data wanted to keep.
- Tested the Informatica Mappings by creating test plans, test data, testing the maps, configuring both sessions and monitoring session streams.
- Developed Incremental and updateable loading through mappings
- Designed reusable transformations and shortcuts to share different mappings
- Created and edited multiple sessions and added them into batch component.
- Debug, and resolve the loading failures by verifying the log files.
- Validate the data in data marts after loading process balancing with source data.
- Resolve the performance issues by using different methods and techniques.
- Tuned Mappings for optimum performance, used various tuning principles to reduce the load time for sessions and workflows
Environment: Informatica Power Center 8.1.1., Oracle10g,MS-Access DB, SQL Server 2005, DataStream(EAMS) and Tririga (IWMS) Flat files, XML, SQL, PL/SQL, TOAD,UNIX Windows XP, Unix Shell Scripts.
Confidential, Denver, CO
Informatica Developer
Responsibilities:- Gathered requirements for SAP objects and profiling the source data for requirement analysis.
- Translate Business Requirement into Informatica mappings.
- Developing Complex mappings by extensively using Informatica Transformations like Lookup, Filter, Expression, Aggregator, Joiner, Stored Procedure, Router using Power Center designer
- Created IDQ mappings for used Developed various ETL transformations and PL/SQL program units for Star Schema Implementation
- Created mappings to load Slowly Changing Dimension tables based on the amount of historical dimension data wanted to keep.
- Tested the Informatica Mappings by creating test plans, test data, testing the maps, configuring both sessions and monitoring session streams.
- Developed Incremental and updateable loading through mappings
- Designed reusable transformations and shortcuts to share different mappings
- Created and edited multiple sessions and added them into batch component.
- Debug, and resolve the loading failures by verifying the log files.
- Validate the data in data marts after loading process balancing with source data.
- Resolve the performance issues by using different methods and techniques.
- Customized data by adding calculations, summaries and functions
- Tuning Mappings for optimum performance used various tuning principles to reduce the load time for sessions and workflows
Environment: Informatica Power Center 7.1.5, Oracle10g, SQL Server 2000, Flat files, XML, SQL, PL/SQL, TOAD, Erwin 4.1, UNIX (Sun Solaris 10) Windows XP, Unix Shell Scripts.
Confidential, Philadelphia, PA
Data Warehouse Project
Responsibilities:- Involved in upgrading the Informatica Power Center 7.1.0 to Informatica Power Center 8.1.1
- Responsible for reporting the existing mappings from 7.1.0 to 8.1.1
- Created shell scripts, which will substitute all the user and schema information in the SQL queries.
- Involved in extracting the data from different data sources like DB2 UDB, Oracle and flat files
- Involved in extracting the data from staging to Relational
- Involved in designing to different stages using complex mappings
- Automated the conditional execution of workflows in a specific sequence using shell scripts and working as a production support
- Modified the pre session and post session scripts
- Involved in moving from development to production environment
- Responsible for setting up the test environment
- Involved Unit test and System integration test.
- Completely tested all the workflows and Involved in configure Power Exchange with Mainframe Source.
- Debug, and resolve the loading failures by verifying the log files.
- Used debugger for error handling and to test the data flow
- Tuning Mappings for optimum performance used various tuning principles to reduce the load time for sessions Debug, and resolve the loading failures by verifying the log files.
- Used the Workflow Manager for creating, validating and running the workflows, sessions and scheduling them to run at specified times.
- Involved in Production Support, analyze and reports the production results.
Environment: Informatica Power Center 7.1.0/8.1.1, Power Exchange 5.2.2, Power Connect 6.2.2, Mainframe (VSAM), DB2, Oracle10g, SQL Server 2000, Erwin 4.0, Flat files, XML, SQL,PL/SQL, Query Analyzer, DTS, TOAD,UNIX, Unix Shell Scripts, Windows XP.