Sr. Etl Developer - Informatica Big Data Resume
Auburn Hill, MI
PROFESSIONAL SUMMARY:
- 8 years of experience in development of various projects using Data Warehousing tools like Informatica and Databases like Oracle, SQL Server, Teradata and Hadoop/Big Data technologies and Analysis, Design & Development of Data Warehouse Systems/Solutions using Informatica.
- Hands on Experience in Big data technologies and Hadoop ecosystem components like HDFS, Scala, Spark, Hive, Yarn, Oozie.
- Strong experience in designing and developing Business Intelligence solutions in data warehousing using ETL tools, Informatica PowerCenter 10.x/9.x/8.X, ETL, OLTP and Informatica Big Data Edition 10.2.1, 10.4.0. and Informatica Intelligent Cloud Service.
- Extensively worked on Informatica designer components - Source Analyzer, transformation developer, Mapplet and Mapping designer.
- Strong experience on workflow manager tools - Task developer, Workflow and worklet designer.
- Experience in debugging mappings. Identified bugs in existing mappings by analyzing the data flow and evaluating transformations.
- Worked extensively with complex mappings using different transformations like Source Qualifiers, Expressions, XML, Filters, Joiners, Routers, Union, Unconnected / Connected Lookups and Aggregators.
- Experience in using various database (Oracle 11g/10g/9i, SQL, PL/SQL).
- Worked on ETL process consisting of data Ingestion, transformation, mapping, conversion and loading to target using Spark, Blaze, Hive etc.
- Experience in importing & exporting data using Sqoop from HDFS to RDBMS and vice-versa.
- Used Informatica Client Tools - Designer, Source Analyzer, Warehouse Designer, Transformation Developer, Mapplet Designer, Mapping Designer, Workflow Manager and Workflow Monitor.
- Created ETL mappings using Informatica Power Center to move data from multiple sources such as XML, DB2, Teradata, MS SQL Server, flat files, and Oracle into a common target area such as data marts and data warehouse.
- Over 3+ years worked with Teradata utilities like BTEQ, Fast Export, Fast Load, Multi Load to export and load data to/from different source systems including flat files.
- Expertise in building Enterprise Data Warehouses (EDW), Operational Data Store (ODS), Data Marts, and Decision Support Systems (DSS) using Data modeling tool ERWIN and Dimensional modeling techniques (Kimball and Inmon), Star and Snowflake schema addressing Slowly Changing Dimensions(SCDs).
- Creating complex mappings in Power Center Designer using Aggregate, Expression, Filter, Sequence Generator, Update Strategy, Union, Lookup, Joiner, XML Source Qualifier, and stored procedure transformations.
- Knowledge in Informatica Data Quality (IDQ) 9.x/8.x, Informatica Big Data Edition (BDE) 9.x
- Involved in cleaning the data using Informatica Data Quality.
- Worked on Data Profiling using IDQ-Informatica Data Quality to examine different patterns of source data. Proficient in developing Informatica IDQ transformations like Parser, Classifier, Standardizer and Decision.
- Highly motivated and goal-oriented individual with a strong background in SDLC Project Management and Resource Planning using AGILE methodologies.
- Exported the data into from UNIX to Mainframe for backup.
- Extensive Knowledge on the development life cycle process from requirements gathering to deployment of code to production and good knowledge on the Dimensional modeling.
- Implementing performance tuning methods to optimize developed mappings.
- Extensive experience in Production support management involving Informatica, Oracle, UNIX and Mainframes.
- Hands on experience with DevOps tools like uDeploy, Jenkin, GitStash.
- Involved in Devops migration/automation processes for building and deploying systems
- Have significant experience in reporting using MS Excel and Business objects.
- Hands on Experience with the scheduling tools like Control M, Autosys scheduler.
- Good Understanding and Knowledge on the Business Intelligence Tools like Cognos.
- Extensive knowledge in DPI programming.
- Experienced in Erwin data modeling, Star-Schema design, Logical data modeling, Normalization, Physical design, Data Analysis, Data Modeling, Reports and Report Design.
- Involved in all phases of data warehouse project life cycle. Designed and developed ETL Architecture to load data from various sources like Oracle, Flat files, Db2UDB, XML files and MS SQL Server into Oracle, XML and SQL server targets.
- Developed UNIX Shell scripts and Perl script for running the informatica mappings and for moving the data in Cloudera tables which involved in capturing CDC logic.
- Excellent Analytical, problem solving, communication and interpersonal skills.
- Experience in Health and Banking domains.
PROFESSIONAL EXPERIENCE:
Confidential, Auburn Hill, MI
Sr. ETL Developer - Informatica Big Data
Responsibilities:
- Experience in doing enhancements to existing Informatica Big Data applications and mappings that move data between files, Oracle databases and within the Cloudera data lake environment
- Worked on Developing, testing, debugging, documenting and helping operationalize various applications leveraging industry best practices.
- Worked closely and collaborated with various cross functional IT specialists in implementing solutions.
- Lead the development, implementation and take the responsibilities for setting up Informatica BDE tool & Hadoop cluster environment from inception to production
- Implemented Informatica BDE mappings for extracting data from DWH to Data Lake.
- Developed BDE mappings using Informatica Developer and created HDFS files in Hadoop system.
- Design & development of BDE mappings in Hive mode for large volumes of INSERT/UPDATE.
- Implemented SCD type1 mappings using BDE and load the data into Hadoop Hive tables using Push down mode.
- Wrote HiveQL queries validate HDFS files & Hive table data to make sure the data meet the requirements.
- Extracting data from MS SQL server/Oracle data mart using Informatica and UNIX shell scripts.
- Extensively used Joiner, Lookup, Aggregator, web service and several other transformations before loading to final aggregated tables.
- Involved multiple sessions to bring data into Type-2 format following VWGofA ( Confidential Group of America) standards.
- Built a reusable mapplet to do Change data capture (CDC) that was used in several other mappings.
- Design, Development, Testing and Implementation of ETL processes using Informatica Cloud.
- Experience in migration of PowerCenter to IICS jobs.
- Developed IICS data integration mappings to facilitate inbound and outbound data movement from Salesforce to Data Lake.
- Created backfeed jobs using IICS and BDE to feed the data from Salesforce to Data Lake and Data Warehouse.
- Developed complex sequential and parallel Informatica IICS jobs.
- Developed and tested ETL Mappings in Informatica IICS.
- Involved in Design, Testing/Documentation and test plan review
- Designed the ETL jobs using Informatica Power Center to Extract, Transform and load the data into Staging and then into Oracle Database.
- Analytical and problem-solving skills applied to Big Data domain.
- Proven understanding with Hadoop, Hive, Pig, and HBase.
- Developed source-to-staging ETL jobs that perform data cleansing, data validation and bring into standard format.
- Design and development of Extract, Transform, and Load processes for extracting data SQL server, Flat Files etc. from various systems and loading into Oracle tables, Flat file.
- Developed UNIX shell scripts to invoke Informatica workflows.
- Prepared generic workflows and Unix scripts that perform Audit, Balance and Control for all the tables.
- Extensively defined & used Parameters and Variables in Mapping for many ETL Jobs.
- Involved in Performance tuning of all the ETL jobs and DB queries for bottleneck encounter during development and after migration to production environment.
- Created UNIX scripts for file transfers (FTP).
- Created Tivoli jobs and scheduling process to run the jobs in development and test environment.
- Worked to get the better performance of the ETL jobs where the file size was 5 gigs based on Performance Statistics
- Developed Source to Target mapping document for all the ETL jobs including but not limited to Unit Test Cases.
- Worked closely with ETL Architect in finalizing the requirements and in preparing the ETL Technical design document
- Worked with Toad and Toad for Data analyst to interact with Oracle 11g and used the tool for data validation
- Involved in fine tuning, trouble shooting, bug fixing, defect analysis and enhancement of the Informatica workflows.
- Worked with the SCM version control tool to move the code to Production.
Environment: Informatica BDE (Big Data Edition) 10.4.0/10.2.1/9.6.1 , Informatica 10.2.1/9.6.1/9.5.1 Power Center, Informatica Cloud, SQL Server 2008, Tivoli workload scheduler, Toad 12.10/10.1/8.5 , Informatica power exchange, Putty, WinSCP, CA Harvest SCM (Software change management)
Confidential, St Paul, MN
Sr ETL Developer/BDE/MDM
Responsibilities:
- Worked on doing enhancements to existing Informatica Big Data applications and mappings that move data between files, Oracle databases and within the Cloudera data lake environment
- Experience in doing enhancements to existing Informatica Big Data applications and mappings that move data between files, Oracle databases and within the Cloudera data lake environment
- Worked on Developing, testing, debugging, documenting and helping operationalize various applications leveraging industry best practices.
- Worked closely and collaborated with various cross functional IT specialists in implementing solutions.
- Participated in design activities leading to innovative solutions in meeting customer requirements
- Experience in data governance activities, particularly corresponding to data lineage, data catalog, data discovery and metadata management.
- Worked in dynamic cross-functional teams including business teams, solution architects and data engineers.
- Worked on extracting of unstructured and semi-structured data using data processor transformation.
- Worked on data ingestion, data profiling and data quality using various capabilities and transformations in Informatica BDM tool
- Developed UNIX Shell scripts and Perl script for running the informatica mappings and for moving the data in Cloudera tables which involved in capturing CDC logic.
- Designed the Data Model, Landing Table, & Staging Tables.
- Created Queries & Packages to view the data in IDD, Data Manager or Merge Manager.
- Configured batch jobs using batch group utility.
- Created the Mater Data Management (MDM) Hub Console configurations like Stage Process configuration, Load Process Configuration.
- Implemented cleansing functions such as pre-defined functions (STRING Functions, Logic Functions, Math Functions, Data Conversion Functions)
- Handled Master Data and Transactional Data and did the reporting of transactional data which is required by the Business Decision Support People.
- Worked on doing enhancements to existing Informatica Big Data applications and mappings that move data between files, Oracle databases and within the Cloudera data lake environment
- Experience in doing enhancements to existing Informatica Big Data applications and mappings that move data between files, Oracle databases and within the Cloudera data lake environment
- Worked on Developing, testing, debugging, documenting and helping operationalize various applications leveraging industry best practices.
- Worked closely and collaborated with various cross functional IT specialists in implementing solutions.
Environment: Informatica Power Center 10.2.1, Informatica Power Center 9.6.1, Informatica Big Data Edition 10.2.0, Informatica Big Data Edition 10.2.1, Oracle 11g, Oracle SQL* LOADER, Flat Files, UNIX Shell Scripting, TOAD, Teradata 14, Teradata SQL Assistant, SQL.
Confidential, Eden Prairie, MN
Informatica ETL Developer
Responsibilities:
- Worked with business analysts for requirement gathering (BRD), business analysis, and translated the business requirements into technical specifications to build the Enterprise data warehouse.
- Utilized Informatica Big Data (BDE)
- Extensive knowledge of Informatica Data Quality (IDQ) 9.x/8.x, Informatica Big Data Edition (BDE) 9.x
- Interacted with both Technical, functional and business audiences across different phases of the project life cycle.
- Developed mappings for fact and dimension tables using the various transformations to extract data from different source databases Oracle, Db2UDB, SQL Server and XML Files.
- Designed and developed complex ETL mappings by making use of transformations like Source Qualifier, Joiner, Update Strategy, Connected Lookup and unconnected Lookup, Rank, Expression, Router, Filter, Aggregator and Sequence Generator transformations.
- Worked on Migration of mappings from Data Stage to Informatica.
- Worked on Power Exchange bulk data movement processes by using Power Exchange Change Data Capture (CDC) method.
- Extracted Data from hadoop and Modified Data according to Business requirement and loaded into Hadoop.
- Designed Mappings using B2B Data Transformation Studio.
- Good knowledge in Unix Shell scripting.
- Excellent experience in ETL Tools like Informatica and on implementing Slowly Changing Dimensions (SCD).
- Extract and Load data using different tools in Teradata like Multiload, Fast Export, Fast load, OLE Load, BTEQ.
- Employing the Informatica Data Quality (IDQ) to help identify data issues and implementing the cleansing procedures with existing interfaces/migrations.
- Written and used UNIX shell scripts extensively for scheduling and pre/post session management.
- Created Workflows using various tasks like sessions, events raise, event wait, decision, e-mail, command, worklets, Assignment and worked on scheduling of the workflows.
- Created Informatica mappings with PL/SQL Procedures/Functions and triggers in T-SQL to build business rules to load data.
- Used mapping parameters and variables.
- Prepared mapping specification document, which gives the data flow and transformation logic for populating each column in the data warehouse table.
- Used debugger to analyze the data flow between source and target to fix the data issues.
- Developed PL/SQL procedures, functions to facilitate specific requirement.
- Optimized and Tuned SQL queries and PL/SQL blocks to eliminate Full Table scans to reduce Disk I/O and Sorts.
- Used SQL Loader to load huge flat files into Oracle database.
- Implemented audit and reconcile process to ensure Data warehouse is matching with the source systems in all reporting perspectives.
- Worked with PowerCenter parameter file which is a list of parameters and variables and their associated values, includes information for multiple services, service processes, workflows, worklets, and sessions in a single parameter file.
- Prepared the Standard Operating Procedure (Knowledge Transfer) document, which provides necessary information, required for the Maintenance and Operation of the application.
- Created the release requests for QA Builds to include all the release requirements and involved in the implementation of QA, UAT and Production releases.
- Extensive knowledge on Analyzing the Software Requirement Specifications and Software Development Life Cycle (SDLC) like water fall and agile methodologies.
- Developed a UNIX Shell scripts which will send the reports to client over the network by using file transfer protocol (FTP & SFTP) and generating the log file, which will keep the history for the FTP reports.
- Executed Informatica workflows using PMCMD utility thru UNIX scripts.
- Provided data loading, monitoring, System support and worked on data issues raised by end user during its production support phase.
- Wrote SQL Queries as part of Data validation in the Target tables.
Environment: Informatica Power Center 9.6.1, Informatica Power Center 10.2.0, Informatica Big Data Edition, Oracle 11g, Oracle SQL* LOADER, Flat Files, UNIX Shell Scripting, TOAD, Teradata 14, Teradata SQL Assistant, SQL.
Confidential, Detroit, MI
Informatica Developer
Responsibilities:
- Involved interactively with the business units to understand the requirements and gather the feedback during the design, development and testing of solutions.
- Worked on various data sources such as Oracle, DB2 UDB and flat files.
- Extracted the data from various data source systems into the Landing Zone area by creating the Informatica mappings using the Teradata fast Loader Connections.
- Wrote SQL Queries as part of Data validation in the Target tables
- Created the Scripts to process the business logic from the Landing Zone to the Common Staging Area(CSA)
- Created the Email task notifications to notify the error messages and also used the command task to run the BTEQ scripts in the workflow monitor.
- Created Dimension Tables and Fact Tables based on the warehouse design.
- Wrote Triggers and Stored Procedures using PL/SQL for Incremental updates.
- Integrated various sources in to the Staging area in Data warehouse to Integrate and for Cleansing data.
- Having brief practical knowledge on Power Exchange Navigator, Power Exchange Bulk Data movement, power exchange change data capture.
- Worked on utilities like FLOAD, MLOAD, FEXP of Teradata and created batch jobs using BTEQ.
- Created mapplets and reusable transformations.
- Creating sessions and batches to run with the logic embedded in the mappings using Informatica Power Center Workflow Manager.
- Developed PL/SQL stored procedures and triggers in T-SQL for various data cleansing activities.
- Used Informatica features to implement Type I, II changes in slowly changing dimension tables.
- Conducting debugging sessions and fixing invalid mappings.
- Designed and developed UNIX Scripts to automate the tasks.
- Used the different types of transformations in Informatica such as Source Qualifier, Look up, Router, Joiner, Union, Aggregator etc.
- Created necessary Repositories to handle the metadata in the ETL process.
- Created Data Breakpoints and Error Breakpoints for debugging the mappings using Debugger Wizard.
- Strong in UNIX Shell and PERL scripting. Developed UNIX scripts using PMCMD utility and scheduled ETL load using utilities like CRON tab, Maestro, Control-M.
- Done the Unit testing after successfully loading the data into Landing Zone (LZ) area.
Environment: Informatica Power Center 9.5.1, Oracle 11g, Toad, SQL, PL/SQL, UNIX Scripting, Teradata13.1, Teradata SQL Assistant, TPUMP .
Confidential
Informatica Developer
Responsibilities:
- Gathered the required information from the users.
- Interacted with different system groups for analysis of systems.
- Working with business units to identify mapping specifications.
- Created the ETL test plans and implemented the Code Review, Formal Testing, and Specification Testing using the Test Director.
- Implementing the standardized process to archive flat files and for loading indicator files
- Also used the Informatica Data Exchange to connect to the Main frames source and extract the data.
- Utilizing the MS Access database for the conversions that are (one time data conversions) for translations that are in Excel or CSV format.
- Created proper PI taking into consideration of both planned access of data and even distribution of data across all the available AMPS.
- Implemented slowly changing dimensions methodology to keep track of historical data.
- Expertise in performance tuning the user queries, execution of frequently used SQL operations and improves the performance.
- Interacted with people from various teams in the Project like Oracle/Teradata DBAs', Maestro Schedulers, Spec. Creators, Reporting and UNIX etc. to aid in the smooth functioning of the Project flow.
- Developed data marts using Cognos Power play and Transformer OLAP tools.
- Exported the data into from UNIX to Mainframe for backup.
- Knowledge transfer to the PS team.
- Developed the triggers to handle history and audit data for production environment.
- Created users, roles, profiles and Objects and system rights.
- Performance changes to allow fast handling of transaction processing related request (plan caching).
- Created Understanding documentation and Release Testing documentation on the tickets handled, and documented the issues found in a central repository.
- Involved in Performance testing of all the workflows/Mapping, which helped in analyzing its performance before moving into production.
- Extracted, Transformed and Loaded OLTP data into the Staging area and Data Warehouse using Informatica mapping and complex transformations (Aggregator, Joiner, Lookup, Normalizer, Filter).
- Involved in the code reviews prepared by the teams.
- Analyzing production support documents and finding the feasible time to run their jobs.
Environment : Informatica Power Centre 8.1.0, Oracle 10g, PL/SQL, Business Objects, UNIX, Windows XP.