We provide IT Staff Augmentation Services!

Big Data/ Talend Developer Resume

0/5 (Submit Your Rating)

Dallas, TX

SUMMARY:

  • 3+ years of experience using Talend Integration Suite (6.3.1/5.x) / Talend Open Studio (6.1/5.x) and 2+ years of experience with Talend Administration Center (TAC). Experience working with Data Warehousing Concepts like Kimball/ Inmon methodologies, OLAP, OLTP, Star Schema, Snowflake Schema, Logical/Physical/ Dimensional Data Modeling.
  • Highly Proficient in Agile, Test Driven, Iterative, Scrum and Waterfall software development life cycle.
  • Extensively used ETL methodology for performing Data Profiling, Data Migration, Extraction, Transformation and Loading using Talend and designed data conversions from wide variety of source systems including Netezza,s Oracle, DB2, SQL server, Teradata, Hive, Hana And non - relational sources like flat files, XML and Mainframe Files. Experience in analyzing data using HiveQL and Pig Latin in HDFS. Performed Importing and exporting data into HDFS and Hive using Sqoop.
  • Involved in extracting user's data from various data sources into Hadoop Distributed File Systems (HDFS) using Talend.
  • Automated all the jobs, for pulling data from FTP server to load data into Hive tables, using Talend components.
  • Experience with Big data, Hadoop, HDFS, Map Reduce, Spark and Hadoop Ecosystem (Pig & Hive) technologies.
  • Extensively created mappings in TALEND using tMap, tJoin, tReplicate, tParallelize, tConvertType, tFlowToIterate, tAggregateRow, tSortRow, tFlowMeter, tLogCatcher, tRowGenerator, tNormalize, tDenormalize, tSetGlobalVar, tHashInput, tHashOutput, tJava, tJavarow, tAggregateRow, tWarn, tLogCatcher, tMysqlScd, tFilter, tGlobalmap, tDie etc.
  • Experience in using cloud components and connectors to make API calls for accessing data from cloud storage like Amazon S3 in Talend Open Studio.
  • Experience in creating Joblets in TALEND for the processes which can be used in most of the jobs in a project like to Start job and Commit job.
  • Experience in monitoring and scheduling using Autosys, Control M & Job Conductor (TAC) and using UNIX (Korn & Bourn Shell) Scripting.
  • Expertise in creating sub jobs in parallel to maximize the performance and reduce overall job execution time with the use of parallelize component of Talend in TIS and using the Multithreaded Executions in TOS.
  • Experienced in creating Triggers on TAC server to schedule Talend jobs to run on server. Strong experience in Extraction, Transformation, loading (ETL) data from various sources into Data Warehouses and Data Marts using Informatica Power Center (Designer, Workflow Manager, Workflow Monitor, Metadata Manager).
  • Experience in developing Informatica mappings using transformations like Source Qualifier, Connected and Unconnected Lookup, Normalizer, Router, Filter, Expression, Aggregator, Stored Procedure, Sequence Generator, Sorter, Joiner, Update Strategy, Union Transformations.
  • Worked extensively on Error Handling, Performance Analysis and Performance
  • Tuning of Informatica ETL Components, Teradata Utilities, UNIX Scripts, SQL Scripts etc.
  • Strong decision-making and interpersonal skills with result oriented dedication towards goals.
  • 8+ years of experience in IT Industry involving Software Analysis, Design, Implementation, Coding, Development, Testing and maintenance with focus on Data warehousing applications using ETL tools like Talend and Informatica.

TECHNICAL SKILLS:

ETL TOOLS: Talend Open Studio (TOS) for Big Data6.1/5.x and Talend Data Integration6.3.1/5.x, Informatica Power Center9.x/8.x/7.x.

Databases: Microsoft SQL Server, Oracle12c/11x/10g, DB2, Hive, Sybase, Hbase, Pig.

Programming: T-SQL, PL/SQL, HTML, XML.

Environment: Windows, UNIX

(SunSolaris10, HP,AIX) & Linux

Scripting Languages: Korn shell script & Windows batch scripting, JavaScript

Other Tools: SQL Developer, Aginity WorkBench, Teradata SQL Assistant, Toad, SQL Navigator, Putty, MS- Office, VMWare Workstation.

PROFESSIONAL EXPERIENCE:

Big Data/ Talend Developer

Confidential, Dallas,TX

Responsibilities:

  • Worked in the Data Integration Team to perform data and application integration with a goal of moving more data more effectively, efficiently and with high performance to assist in business critical projects coming up with huge data extraction.
  • Perform technical analysis, ETL design, development, testing, and deployment of IT solutions as needed by business or IT.
  • Experience on writing Hive queries to load the data into HDFS.
  • Performed data manipulations using various Talend components like tMap, tJavarow, tjava, tMysqlRow, tMysqlInput, tMysqlOutput, tMSSQLInput and many more. Analyzing the source data to know the quality of data by using Talend Data Quality.
  • Designed Talend Jobs Using Big Data components like tHDFSInput, tHDFSOutput, tPigLoad, tPigFilterRow, tPigFilterColumn, tPigStoreResult, tHiveLoad, tHiveInput, tHbaseInput, tHbaseOutput.
  • Developed jobs to move inbound files to HDFS file location based on monthly, weekly, daily and hourly partitioning.
  • Created Hadoop cluster connections to access HDFS.
  • Worked on Migration projects to migrate data from data warehouses on SQL Server and migrated those to Mysql.
  • Worked in optimizing the SQL queries for Mysql(5.7) to support Microstrategy reports
  • Worked on develop jobs and scheduled jobs in Talend integration suite.
  • Writing Mysql queries to join or any modifications in the table
  • Used Talend reusable components like routines, context variable and global Map variables.
  • Responsible to tune ETL mappings, Workflows and underlying data model to optimize load and query Performance.
  • Developed Talend ESB services and deployed them on ESB servers on different instances.
  • Implementing fast and efficient data acquisition using Big Data processing techniques and tools.
  • Monitored and supported the Talend jobs scheduled through Talend Admin Center (TAC).
  • Worked on Complex Data Migration from Mysql tables to JSON Structure.
  • Worked on History Data Migration from various data sources like from SQL Server, Oracle to Mysql Tables Using Talend.
  • Acquire and interpret business requirements, create technical artifacts, and determine the most efficient/appropriate solution design, thinking from an enterprise - wide view.

Environment: Talend for Data Integration 6.3.1 & Big Data 6.1,TAC, Mysql(5.7), SQL Server 2012, XML, Hadoop, Hive, Pig, SQL, PL/SQL, HP ALM, Linux, JIRA.

Sr. ETL/ Big Data/Talend Developer

Confidential, Houston, TX

Responsibilities:

Environment:

  • Analyzed the requirements and framed the business logic and implemented it using Talend.
  • Handled importing of data from various data sources, performed transformation like ETL (Extract Transform and Load) and ELT (Extract Load and Transform) into HDFS.
  • Extracted the data from Oracle 12c, transformed and load in HDFS usingTalend Studio ETL TOOL.
  • Analyzed and performed data integration using Talend open integration suite.
  • Worked on the design, development and testing of Talend mappings.
  • Translated the business needs into system requirements by communicating with the business on non - technical level and with the System Analyst on technical level. Coordinating the data execution and loading in ETL with the Big Data Framework Proof of Concept using Talend Open Studio.
  • Worked on Talend components like tReplace, tmap, tsort and tFilterColumn, tFilterRow, tJava, Tjavarow, tConvertType etc.
  • Used Database components like tMSSQLInput, tMsSqlRow, tMsSqlOutput, tOracleOutput, tOracleInput etc.
  • Worked with various File components like tFileCopy, tFileCompare, tFileExist, TFileDelete, tFileRename.
  • Worked on improving the performance of Talend jobs.
  • Created triggers for a Talend job to run automatically on server.
  • Worked on Exporting and Importing of Talend jobs.
  • Created jobs to pass parameters from child job to parent job.
  • Exported jobs to Nexus and SVN repository.
  • Implemented update strategy on tables and used tJava, tJavarow components to read data from tables to pull only newly inserted data from source tables.
  • Observed statistics of Talend jobs in AMC to improve the performance and in what scenarios errors are causing.
  • Created Generic and Repository schemas.
  • Developed project specific Deployment job responsible to deploy Talend jar files on to the windows environment as a zip file, later, this zip file is unzipped and the files are again deployed to the UNIX box.
  • Also, this deployment job is responsible to maintain versioning of the Talend jobs that are deployed in the UNIX environment.
  • Developed shell scripts in UNIX environment to support scheduling of the Talend jobs.
  • Monitored the daily runs, weekly runs and adhoc runs to load data into the target systems.
  • Worked with Data mapping team to understand the source to target mapping rules.

Talend for Data Integration 5.5.2, Talend Open Studio for Big Data, UNIX, Shell script, SQL Server, Oracle,HDFS, Hbase, Hive, ERwin, SVN.

Talend Etl Developer

Confidential, San Diego, CA

Responsibilities:

  • Created the ODS jobs using Talend Open Studio.
  • Used Talend open studio to execute jobs for ODS.
  • Debugged numerous issues in Talend.
  • Worked closely with the administrators with the configuration of Talend Open studio.
  • Developed and tested all the backend programs, Informatica mappings and update processes.
  • Developed Informatica mappings to load data into various dimensions and fact tables from various source systems.
  • Developed, tested stored procedures, Cursors, Functions and Packages in PL/SQL for Data ETL.
  • Used Power Exchange along with Power Center to leverage data by avoiding manual coding on data extraction programs.
  • Created various active and passive transformations like Source Qualifier, Lookup, Router, Normalizer, Aggregator, Filter, Joiner, Expression and standard/reusable mappings using Informatica.
  • Made substantial contributions in simplifying the development and maintenance of ETL by creating re - usable Source, Target, Mapplets and Transformation objects.
  • Responsible for developing and testing the new conformed dimensions that were used by the conformed fact.
  • Used Power Center Workflow Manager to create sessions, and also used various tasks like session, event wait, event raise, email to run with the logic embedded in the mappings.
  • Responsible for validating the Informatica mappings against the pre-defined ETL design standards.
  • Developed incremental and updateable loading through Informatica mappings.
  • Used debugger and breakpoints to view transformations output and debug mappings.
  • Documented all the mappings and the transformations involved in ETL process
  • Used UNIX shell scripting for scheduling tasks.
  • Extracted huge volumes of data from legacy systems and uploaded into Oracle using SQL*Loader and shell scripts.
  • Attended POC for Talend open studio.

Environment: Informatica9.1, Power Exchange, Talend Open Studio 5.1, Oracle 10g, SQL Server, ERwin, PL/SQL, and UNIX shell scripts, PL/SQL, TOAD.

Informatica Developer

Confidential, Atlanta, GA

Responsibilities:

  • Developed technical specifications design (TSD) for VA Claims track
  • Coordinated with SME for technical clarifications
  • Extensively worked on BTEQ scripts to load huge volume of data in to EDW
  • Integrated data in to EDW by sourcing it from different sources like SAP, mainframe copy books (Cobol files), DB2 and Teradata tables
  • Loaded data in to some of the DB2 X - ref tables
  • Developed Infa mappings with DB2 as targets to support ESI application
  • Loaded data in to Landing Zone (LZ) Teradata tables, applied transformations and then loaded the data in to conformed staging area(CSA)
  • Analyzed the source systems before starting the development
  • Developed shell scripts to parameterize the date values for the incremental extracts
  • Loaded data from CSA to EDW by cross referencing the RDM codes
  • Extracted data using BTEQ scripting which includes Fast Export and Multi Load utilities
  • Extensively worked on Informatica 8.6.1 to extract the data and load it in to LZ
  • Coordinated with SIT team for testing different scenarios
  • Developed unit test cases with different scenarios
  • Implemented Audit balancing using BTEQ scriptin
  • Designed Teradata LZ tables with appropriate Primary index
  • Reviewed and translated BRD/BSD in to technical specifications design (TSD)

Environment: Informatica Power Center 9.1/8.6.1, Power Exchange 9.1/8.6.1, SAP R3 Power Connect/ Power Exchange, Windows, Teradata 12.x, IBM DB2 8.x, Oracle 10g, Toad for IBM DB2, BTEQ, ER studio.

ETL/Informatica Developer

Confidential, Monroe, LA

Responsibilities:

  • Created mappings with heterogeneous sources like flat tiles, oracle databases and created targets in oracle using Informatica Mapping Designer. Developed Mappings/Workflows/Scheduling ETL process.
  • Frequently using import and export utility to migrate session from developers folder to subject folder.
  • Developed Re - usable piece of code like Mapplet to use across various modules.
  • Used Power Exchange interface to extract the legacy data.
  • Developed UNIX Shell Scripts to schedule the Informatica jobs.
  • Involved in design changes specific to releases.
  • Designed mapping templates to specify high-level approach.
  • Extensively worked with Informatica components like Source Analyzer, Warehouse Designer, Transformation developer, Mapplet Designer, Mapping Designer, Repository manager, Workflow Manager, Workflow Monitor, Repository server and Informatica server to load data from flat files, SQL Server.
  • Designed the mappings between sources (files and databases) to operational staging targets.
  • Used Aggregator, sequence, look up, expression, filter, Joiner, Rank, Router, Sequence generator, Update Strategy transformations in populating the data process.
  • Designed and developed the Informatica workflows/sessions to extract, transform and load the data into oracle Server.
  • Worked with different Informatica tuning issues and fine-tuned the transformations to make them more efficient in terms of performance.
  • Prepared ETL process flow documents based on the present process flow and business functionalities.

Environment: Informatica Power Center 8.6.x, Informatica Power exchange 8.6.x, Oracle 10g, SQL*Loader, UNIX, Toad

Etl Developer

Confidential

Responsibilities:

  • Extracted data from heterogeneous sources like Oracle, SQL Server
  • Created detailed Technical specifications for Data Warehouse and ETL processes.
  • Used Transformations like Look up, Router, Filter, Joiner, Stored Procedure, Source Qualifier, Aggregator and Update Strategy extensively.
  • Tuned performance of Informatica session for large data files by increasing block size, data cache size, sequence buffer length and target based commit interval. Created Mapplets and used them in different Mappings.
  • Involved in doing error handling, debugging and troubleshooting Sessions using the Session logs, Debugger and Workflow Monitor.
  • Monitoring the Data Quality, Generating weekly/monthly/yearly statistics reports on production processes - success / failure rates for causal analysis as maintenance part and Enhancing exiting production ETL scripts.
  • Worked with SAP and Oracle sources to process the data.
  • Worked on SAP for data migration Human Resources and Finance and converted various objects on Organizational Structure, Addresses, Time, Basic Pay, Bank Details, Recurring Payments, Tax assignment, Insurance Plans, Payroll etc., to generate report from SAP BI system.
  • Worked with Pre and Post Session SQL commands to drop and recreate the indexes on data warehouse using Source Qualifier Transformation of Informatica Power center. Created Unix Shell Scripts to automate sessions and cleansing the source data.
  • Implemented pipeline partitioning concepts like Round-Robin, Key-Range and Pass Through techniques in mapping transformations.
  • Involved in Debugging and Performance tuning of targets, sources, mappings and sessions.
  • Worked with Business Analyst and application users to finalize Data Model, functional and detailed technical requirements.

Environment: Informatica Power Center 8.6/8.5, Oracle9i, SAP, SAP BI 7.0, SQL Server, UNIX Sun Solaris.

We'd love your feedback!