Lead Talend Engineer Resume
Denver, CO
SUMMARY
- Over 11 years of experience in IT, with proficiency as Talend/Informatica developer and strong expertise in SQL queries, stored procedures & Macros.
- 3+ years of experience using Talend Integration Suite (5.0/5.5/6.1)/Talend Open Studio (5.0/5.5/6.1).
- With Strong Knowledge in all phases of Software Development Life Cycle (SDLC) such as requirement gathering, data analysis, design, development, testing and deployment of complex Data warehousing & B.I solutions.
- Reports, testing, Migrating, Administration, security management and production support.
- Extensive experience in Extraction, Transformation and Loading of data directly from different heterogeneous source systems like flat files, .xml, back end db’s, Oracle, SQL Server, Teradata.
- Developed mappings using Informatica transformations (i.e. Normalizer, Aggregator, Expression, Lookup, Joiner, filter, Router, sorter etc.)
- Involved in developing strategies for Extraction, Transformation and Loading ETL mechanism using DataStage tool. Talend Open studio, TALEND DI, TALEND DQ, Talend with Big data and TALEND with CLOUD ERA, TALEND with HDP, Talend with CLOUD(AWS).
- Implementing Data Integration process with Talend BigData Integration Suite
- Also Used Talend BidData Components like tHDFS, tHDFSRowCount, tHIVEConnection, tHIVERow.
- Extensively used ETL methodology for performing Data Profiling, Data Migration, Extraction, Transformation and loading using Talend/SSIS and designed data conversions from wide variety of source systems.
- Experience working with Data Warehousing Concepts like Ralph Kimball Methodology, Bill Inmon Methodology, OLAP, OLTP, Star Schema, Snow Flake Schema, Logical Data Modeling, Physical Modeling, Dimension Data Modeling.
- Performed the data profiling and analysis making use of Informatica Data Explorer (IDE) and Informatica Data Quality (IDQ).
- Experience in Informatica Metadata and Repository Management, was directly responsible for the Extraction, Transformation and Loading of data from Multiple Sources to Data Warehouse
- Experience in designing E - R diagrams, Logical and Physical database designs using Erwin and Visio.
- Demonstrated expertise utilizing ETL tools: Talend Data Integration, SQL Server Integration Services (SSIS), Developed slowly changing dimension (SCD) mappings using Type-I, Type-II, and Type-III methods. Have good experience to UNIX/Windows/Mainframe environments.
- Migration of Talend 5.x to Talend 6.x. Informatica upgrade from any version.
- Have built excellent knowledge on Talend for Design and development of ETL code and Mappings for Enterprise DWH ETL Talend Projects.
- Highly proficient in Extract, Transform and Load the data into target systems using Informatica power center Repository Manager, Designer, Workflow Manager and Workflow Monitor.
- Knowledge on Teradata Architecture and worked with utilities (fast load, fast export, bteq etc).
- Writing Teradata BTEQ scripts to implement the business logic.
- Creating Teradata database objects as per the requirements.
- Experienced in UNIX shell scripts to write wrapper scripts to call ETL Jobs.
- Excellent communication and interpersonal skills working within multi-diverse teams Strong analytical and communication skills and an eagerness to learn new technologies.
- Scheduling weekly meetings with the client to get their requirement specification and providing the same to the team and guiding them on the feasibility and attainments of the same.
- Managing / Leading teams and targets as and when assigned.
- Tracking the resource allocation of the team towards a project and handling the resource management towards the other projects
- Preparing shift schedules of the team to ensure capacity availability.
TECHNICAL SKILLS
Data Warehousing/ ETL: Informatica Power Center 9.6/9.5/9.1/8.6 , Informatica Power Mart 9/8.x, (Workflow Manager, Workflow Monitor, Source Analyzer, Warehouse Designer, Transformation Developer, Mapplet Designer, Mapping Designer, Repository manager), Metadata, Talend.
Data Modeling: Dimensional Data Modeling, 3NF Data Modeling, Star Schema Modeling, Snow-Flake Modeling, Physical and Logical Data Modeling, Erwin, ER Studio and Sybase Power Designer.
Databases & Tools: Teradata 12/13/13.10/14 , Oracle 11g/10g/9i/8i/8.x, DB2 UDB 8.5, SQL*Server, MS SQL Server 2005/2000/7.0/6.5 , SQL*Plus, SQL*Loader, TOAD, SQL Assistant, Natezza.
Scheduling Tools: Informatica Workflow Manager, Autosys, CTL-M.
Programming Languages: Unix Shell Scripting, SQL, PL/SQL, Perl, Teradata Procedure.
Platforms: UNIX, Win XP/NT 4.0, Sun Solaris 2.6/2.7, HP-UX10.20/9.0, IBM AIX 4.2/4.3.
PROFESSIONAL EXPERIENCE
Confidential, Denver, CO
Lead Talend Engineer
Responsibilities:
- Created ETL/Talend jobs both design and code to process data to target databases.
- Created Talend jobs to load data into various Oracle tables. Utilized Oracle stored procedures and wrote few Java codes to capture global map variables and use them in the job.
- Created Talend jobs to copy the files from one server to another and utilized Talend FTP components.
- Created Implicit, local and global Context variables in the job.
- Responsible for creating fact, lookup, dimension, staging tables and other database objects like views, stored procedure, function, indexes and constraints.
- Developed complex Talend ETL jobs to migrate the data from flat files to database.
- Implementing Data Integration process with Talend BigData Integration Suite
- Also Used Talend BidData Components like tHDFS, tHDFSRowCount, tHIVEConnection, tHIVERow.
- Implemented custom Error handling in Talend jobs and worked on different methods of logging.
- Created Source to Target mappings and various aspects of data access methods based on the capabilities of source and targets using transformations like Expression, Lookup, Update Strategy extensively for Claims reporting for various data files from providers like Baptist, ITS, etc.
- Worked on Scheduling workflows and sessions using Informatica scheduler and have implemented decision task and email task for tracking the successful completion.
- Identify source systems, connectivity, tables, and fields to ensure data suitability for mapping.
- Created flexible mappings/sessions using parameters, variables and heavily using parameter files.
- Developed the Talend jobs and make sure to load the data into HIVE tables & HDFS files and develop the Talend jobs to integrate with Teradata system from HIVE tables
- Worked on different tasks in Workflows like sessions, events raise, event wait, decision, e-mail, command, worklets, Assignment, Timer and scheduling of the workflow.
- Unit testing, code reviewing, moving in UAT and PROD.
- Experienced in creating MFT (Managed File Transfer) Mock jobs for HL7 2.5.1 LAB and ADT transactions in Talend Big Data 6.2.
- Experienced in converting pipe delimited data Ex: HL7V 2.5.1, into JSON structure using Talend Components.
- Used Talend reusable components like routines, context variable and global Map variables.
- Provided support to develop the entire warehouse architecture and plan the ETL process.
- Knowledge on Teradata Utility scripts like FastLoad, MultiLoad to load data from various source systems to Teradata.
- Followed the organization defined Naming conventions for naming the Flat file structure, Talend Jobs and daily batches for executing the Talend Jobs.
- Created Unix Scripts and run them using tSystem for reading the Data from flat files and archiving the Flat files at the specified server.
- Tuned sources, targets and jobs to improve the performance.
- Monitor troubleshoot batches and jobs for weekly and monthly extracts from various data sources across all platforms to the target database.
- Provided the Production Support by running the jobs and fixing the bugs.
- Created mapping documents to outline data flow from sources to targets.
- Extracted the data from the flat files and other RDBMS databases into staging area and populated onto Data warehouse.
- Maintained stored definitions, transformation rules and targets definitions.
Environment: Talend 5.x, XML files, DB2, Oracle 11g, SQL server 2008, SQL, UNIX Shell Scripts, Informatica 8.6.0/9.1, SAP BO, Oracle 11g, Teradata, Big Data, Cloud AWS, Hive, Unix and Tivoli, ETL/Informatica Power Center, SQL*Loader, HP Quality Center, AUTOSYS, Rational Clear case, Rational Clear Quest, Windows XP, TOAD.
Confidential - Phoenix, AZ
ETL/Talend Developer
Responsibilities:
- Interact with Solution Architects and Business Analysts to gather requirements and update Solution Architect Document.
- Analyze and create low level design document (LLD) and mapping document.
- Performed analysis, design, development, Testing and deployment for Ingestion, Integration, provisioning using Agile Methodology.
- Attended Daily Scrum meetings to provide update on the progress of the user stories Rally and to the Scrum Master and also to notify blocker and dependency if any.
- Experienced in creating Generic schemas and creating Context Groups and Variables to run jobs against different environments like Dev, Test and Prod.
- Created Talend Mappings to populate the data into dimensions and fact tables.
- Broad design, development and testing experience with Talend Integration Suite and knowledge in Performance Tuning of mappings.
- Created complex mappings in Talend 6.0.1/5.5 using tMap, tJoin, tReplicate, tParallelize, tJava, tjavarow, tJavaFlex, tAggregateRow, tDie, tWarn, tLogCatcher, etc.
- Created joblets in Talend for the processes which can be used in most of the jobs in a project like to Start job and Commit job.
- Developed jobs to move inbound files to vendor server location based on monthly, weekly and daily frequency.
- Implemented Change Data Capture technology in Talend in order to load deltas to a Data Warehouse.
- Created jobs to perform record count validation and schema validation.
- Created contexts to use the values throughout the process to pass from parent to child jobs and child to parent jobs.
- Developed joblets that are reused in different processes in the flow.
- Developed error logging module to capture both system errors and logical errors that contains Email notification and also moving files to error directories.
- Provided the Production Support by running the jobs and fixing the bugs.
- Experienced in using Talend database components, File components and processing components based up on requirements.
- Responsible for developing, support and maintenance for the ETL (Extract, Transform and Load) processes using Talend Integration Suite.
- Worked in improving performance of the Talend jobs.
- Performed unit testing and also integration testing after the development and got the code reviewed.
- Involved in migrating objects from DEV to QA and then promoting to Production.
Environment: Talend Studio 6.0.1/5.5, Oracle 11i, XML files, Flat files, HL7 files, JSON, TWS, Hadoop 2.4.1, HDFS, Hive 0.13, HBase 0.94.21, Talend Administrator Console, IMS, Agile Methodology, HPSM, Informatica 8.6.0, Oracle11i, BOXIR2 and Windows NT.
Confidential - San Antonio, TX
Lead ETL Consultant/ETL Talend Consultant
Responsibilities:
- Involved in building the ETL architecture and Source to Target mapping to load data into Data warehouse.
- Performed data manipulations using various Talend components like tMap, tJavaRow, tjava, tOracleRow, tOracleInput, tOracleOutput, tMSSQLInput and many more.
- Designed and customized data models for Data warehouse supporting data from multiple sources on real time
- Designed ETL process using Talend Tool to load from Sources to Targets through data Transformations.
- Created Informatica mappings to build business rules to load data.
- Extensive experience on Pentaho designer, Pentaho kettle, Pentaho BI server, BIRT report designer
- Developed advanced Oracle stored procedures and handled SQL performance tuning.
- Involved in creating the mapping documents with the transformation logic for implementing few enhancements to the existing system.
- Teradata was the target database, Source database is a combination of Flat files, Oracle tables, Excel files and Teradata database.
- Loaded data in to Teradata Target tables using Teradata utilities (FastLoad, MultiLoad, and FastExport) Queried the Target database using Teradata SQL and BTEQ for validation.
- Created connection to databases like SQL Server, oracle, Netezza and application connections.
- Prepare the Talend job level LLD documents and working with the modeling team to understand the Big Data Hive table structure and physical design.
- Involved in Dimensional modeling (Star Schema) of the Data warehouse and used Erwin to design the business process, dimensions and measured facts.
- Responsible for developing, support and maintenance for the ETL (Extract, Transform and Load) processes using Talend.
- Maintained stored definitions, transformation rules and targets definitions using Informatica repository Manager.
- Used various transformations like Filter, Expression, Sequence Generator, Update Strategy, Joiner, Stored Procedure, and Union to develop robust mappings.
- Developed mapping parameters and variables to support SQL override.
- Developed Talend ESB services and deployed them on ESB servers on different instances.
Environment: Talend Studio 6.0.1/5.5, Oracle 11i, XML files, Flat files, HL7 files, JSON, TWS, Hadoop 2.4.1, HDFS, Hive 0.13, HBase 0.94.21, Big Data, Talend Administrator Console, IMS, Agile Methodology, HPSM
Confidential - Iowa City, IA
Senior ETL Informatica Developer
Responsibilities:
- Worked on Informatica tools like Power Center Designer, Workflow Manager and Workflow Monitor.
- Performed data manipulations using various Informatica Transformations like Joiner,
- Expression, Lookup, Sorter, Aggregate, Filter, Update Strategy, Normalizer and Sequence Generator.
- Developed InformaticaType-1 and Type-2 Mappings based on the requirements.
- Involved in Mapping, Session and database level optimization techniques to improve the performance.
- Involved in creating complex workflows which utilize the various tasks like Event Wait, Command, Mail decision and Session.
- Extracted data from Flat files and Oracle to load them into Teradata.
- Worked on WLM Scheduler to automate the Workflows.
- Prepared SDLC Work book and conducted walk through before moving to SIT, UAT and Production.
- Involved in the Performance Tuning of Database and Informatica
- Debugged invalid mappings using break points, tested stored procedures, functions, Informatica sessions, batches and the target data
- Wrote stored procedures in PL/SQL and UNIX Shell Scripts for automated execution of jobs
- Identified performance bottlenecks and resolved those issues by Query optimization, dropping Indexes and Constraints, bulk loading.
- Created reusable transformations and Mapplets to use in multiple mappings.
- Created complex/ad-hoc reports using Business Objects reporter and exported to the repository.
- Experienced in Talend Data Integration, Talend Platform Setup on Windows and UNIX systems.
- Designed the Talend ETL flow to load the data into hive tables and create the Talend jobs to load the data into Oracle and Hive tables.
- Used Talend to Extract, Transform and Load data into Netezza Data Warehouse from various sources like Oracle and flat files.
- Monitored and supported the Talend jobs scheduled through Talend Admin Center (TAC)
- Developed the Talend mappings using various transformations, Sessions and Workflows.
- Defects were tracked, reviewed and analyzed.
Environment: Informatica Power Center 9.5, Teradata V12, Oracle 11g, WLM, SQL*Loader, HP Quality Center, AUTOSYS, Rational Clear case, Rational Clear Quest, Talend,Windows XP, TOAD, UNIX.
Confidential - McLean, VA
Sr. ETL Informatica Developer/DWH Engineer
Responsibilities:
- Involved in Systems Study/Analysis and understand the business and implement the same into a functional database design.
- Created source definitions from flat files, Oracle, imported Target definitions and created Reusable Transformations in Informatica Power Center.
- Participated in designing the data model using Erwin.
- Identified and tracked the Slowly Changing Dimensions, heterogeneous Sources and determined the hierarchies in dimensions.
- Created users and user groups with appropriate privileges and permissions, folders and folder permissions in Repository manager.
- Used Unix Shell scripting to get data from source for the transformation to cleanse data as per business rule and also to automate pre-session & post-session processes.
- Developed Transformation Logic in Informatica to cleanse the source data to inconsistencies before loading the data into the staging area which is the source for stage loading.
- Worked on connected & unconnected Lookups, Router, Expressions, Source Qualifier, Aggregator, Filter, Sequence Generator and Joiner Transformations to implement complex business logic.
- Used PMCMD commands for automating Batches and Sessions.
- Extensively used Teradata Utilities including FAST LOAD, FASTEXPORT, TPump and TPT.
- Identifies and fixed bottlenecks and turned the Informatica mappings for better performance.
- Created Test cases for Unit Test. Used TOAD to validate the data with SQL Query.
Environment: Informatica Power Center 9.5, Teradata V12, Oracle 11g, WLM, Windows XP, SQL*Loader, HP Quality Center, AUTOSYS, Rational Clear case, Rational Clear Quest, TOAD, UNIX.
Confidential
ETL Developer/Data Warehouse Engineer
Responsibilities:
- Involved in Requirement Gathering and Business Analysis.
- Involved in Designing of Data Modeling for the Data warehouse. And developed data Mappings between source systems and warehouse components using Mapping Designer.
- Analyzed source database (Sybase), designed mapping (using various transformations like Update Strategy, Look Up, Stored Procedure, Router, Filter, Sequence Generator, Joiner, Aggregate and Expression) to transform and load data on target databases (SQL Server) using Informatica Power Center (ETL Tool).
- Involved in the performance tuning of the Informatica mappings and the sequel queries inside the source qualifier.
- Created, launched & scheduled sessions.
- Improved performance by identifying and rectifying the performance bottle necks.
- Used Server Manager to schedule sessions and batches.
- Involved in creating Business Objects Universe and appropriate reports.
- Analyzed and Resolved issues in Extraction, Transformation and Loading (ETL) Processes, especially in transformation and mapping development.
- Designed ETL process using Informatica Designer to load the data from various source databases and flat files to target data warehouse in Oracle.
- Created parameter based mappings, Router and lookup transformations.
- Debugged invalid mappings using break points tested stored procedures, functions, Informatica sessions, batches and the target data.
- Wrote UNIX Shell Scripts for automated execution of jobs.
- Identified performance bottlenecks and resolved those issues by Query optimization, dropping Indexes and Constraints, bulk loading.
- Created reusable transformations and Mapplets to use in multiple mappings.
- Created complex/ad-hoc reports using Business Objects reporter and exported to the repository.
Environment: Informatica Power Center 8.6, SQL*Loader, HP Quality Center, Oracle 11g, AUTOSYS, Rational Clear case, Rational Clear Quest, Windows XP, TOAD, UNIX.