Etl Developer Resume
SUMMARY:
- Excellent skills in Application / Solution analysis, Customizations, Open Interfaces and Conversions, reporting in SAP / Oracle EBS / Mainframe (TSOD)
- Expert in the development of UNIX shell scripts, PL/SQL Stored Procedures, Functions, Triggers and Packages to implement business logic.
- Expertise knowledge in ETL tool Datastage, Informatica, Red Gate and SSIS
- Hands on knowledge of System Administrator / Application Object Library modules
TECHNICAL SKILLS:
Business Suite: SAP, Oracle
Databases: Netezza, Oracle 11g/12c, MS SQL Server, MS Access, DB2
Scripting: SQL, PL/SQL, UNIX Shell Scripting
Programming Languages: JCL, Cobol, VSAM, XML, VB, .NET, UML
Operating Systems: Unix, Linux, AIX, Solaris, Windows NT/XP
Reporting Tools: Cognos / Tableau, Qlikview
ETL Tools: Datastage, Informatica, SSIS, ODI
Query Tools: TOAD, SQL Navigator, SQL Developer, SQL Editor, OraEdit
Other Tools: Clear Case, Erwin, MS Visio, MS Project
PROFESSIONAL EXPERIENCE:
Confidential
ETL Developer
Responsibilities:
- Confidential is loading its Credit Card and personal banking data from Mainframe (DB2 / Oracle) to data mart (Hadoop, Oracle and Netezza), using Datastage / SSIS as ETL tool, also building BI application using Cognos / Tableau / Qlikview
- Used DataStage Designer to design and develop jobs for extracting, cleansing, transforming, integrating, loaded data using various stages like Aggregator, Funnel, Change Capture, Change Apply and copy for project OM4
- Extensively worked on building datastage jobs using various stages, such as Oracle Connector, Funnel, Transformer stage, Sequential file stage, LookUp, Join and Peek Stages, Sort, Merge, Aggregator, Peek, DataSet and Remove Duplicates stages…
- Worked with sequential file, dataset, file set and look up file set stages
- Used the Data Stage Director and its run - time engine to schedule running the solution, testing and debugging its components, and monitoring the resulting executable versions on ad hoc or scheduled basis
- Administrated DataStage to create Repository, User groups, Users and managed users by setting up their profiles and privileges
- Used SSIS to develop jobs for extracting, cleaning, transforming and loading data into data warehouse (Netezza)
- Prepared the completedata mapping/packagefor all the migrated jobs using SSIS(TFS/VSS for version control)
- Defined and developed technical specifications for the automation of the BI application with the clients, translated business model into technical, including database / BI (Cognos) level.
- Upgraded Cognos application from 10 to 11.
- Designed and created BI reporting using Report Studio, including all related drill throughs
- Designed and developed the most suitable user interface such as dashboards, multiple charts types, trends, customer requests for Excel Export and objects for management dashboard reporting
- Developed complex calculated fields for the business logic, field actions, sets and parameters to include various filtering capabilities for the dashboards, to provide drill down features for the detailed reports.
- Generated various report using Python Report lab and sent to Business users to improve their decision making.
- Performed Tableau admin duties that comprised configuration, adding users, groups, managing licenses and data connections, scheduling tasks, embedding views in the Tableau server.
- Worked on Qlikview Server and Publisher, to manage user access, schedule refreshing of documents, mount completed application
- Performance tuning of dashboards was carried out to limit the response time to well under 5 seconCalculation and defining server sizing for optimal performance of QlikView dashboards.
- Designed, developed, implemented, and supported QlikView dashboards using Qlikview components like List Box, Slider, Buttons, charts and Bookmarks, integrated data sources and databases with QlikView, and designed and developed data models and backend queries for presenting data
Confidential
BI /ETL Developer
Responsibilities:
- Imported and exported Repositories across DataStage projects using datastage designer.
- Extensively worked on DataStage Job Sequencer to Schedule Jobs to run jobs in Sequence.
- Created Datastage jobs using different stages like Transformer, Aggregator, Sort, Join, Merge, Lookup, Data Set, Funnel, Remove Duplicates, Copy, Modify, Filter, Change Data Capture, Change Apply, Sample, Surrogate Key, Column Generator, Row Generator, Join, Look up (Normal and Sparse) and Merge stages…
- Administrated DataStage to create Repository, User groups, Users and managed users by setting up their profiles and privileges
- Developed / reviewed the Standardization, Scrubbing, Matching and Survivorship rules in order to get best of the breed record
- Used DataStage Designer to design jobs for extracting, cleansing, transforming, integrating, loaded data using stages like Aggregator, Funnel, Change Capture, Change Apply and copy
- Used Teradata utilities fastload, multiload, tpumpto load data
- Created SSIS Packages using SSIS Designer for export heterogeneous data from OLE DB Source (Oracle), Excel Spreadsheet to SQL Server, created reports and embedded them in Dashboards according to the requirements.
- Worked on SSIS Package, DTS Import/Export for transferring data from Database, load each dimensions, Fact tables, for Configuring packages using XML, Sql Server, Parent package variable methods
- Created Complex ETL Packages using SSIS to extract data from staging tables to partitioned tables with incremental load, using TFS/VSS for version control
- Analyzed Hadoop clusters using big data analytic tools including Pig, Hive, and MapReduce
- Conducted in-depth research on Hive to analyze partitioned and bucketed data
- Defined and developed technical specifications for the automation of the BI application with the clients, translated business model into technical, including database / BI (Cognos) level.
- Designed / defined all database related level objects (tables, views, MVs, functions…) and its loading mechanism, tested / tuned database performance (Oracle 11g Pivot feature), Conducted database (Oracle / Teradata) capacity planning and disaster recovery planning.
- Analyzed / developed BI Model using Framework manager, including relationship defining among objects, published / migrated packages in Dev / Test /Prod, including query item bi-lingual translation and model version control.
- Designed and created BI reporting using Report Studio, including all related drill throughs, designed various charts (revenues, expenses, cost, activity use, management score…), set up default output format (html / pdf / Excel), migrated reports among Dev / Test / Prod, especially automatically associated related geographically pictures.
- Generated various testing cases at database level, tuned report performance.
- Generated Data visualization and presentations with Python's matplotlib, Bokeh, Plotly libraries
- Created UML diagrams artifact and related documentation using Microsoft Visio, mentored and transferred technical skills to client uses.
- Hadoop, SSIS, Python, Datastage 11.5, Cognos 8.4 / 10.2, Oracle / Teradata, Microsoft Visio
Confidential
ETL/BI Consultant
Responsibilities:
- Designed Mappings between sources to operational staging targets, using Star Schema, Implemented logic for Slowly Changing Dimensions (SCD) using Datastage.Extensive designed and developed Parallel and server jobs.
- Extensively worked on building Datastage jobs using various stages like Oracle Connector, Funnel, Transformer stage, Sequential file stage, LookUp, Join and Peek Stages.
- Extensively used Sort, Merge, Aggregator, Peek, DataSet and Remove Duplicates stages.
- Migrated DataStage jobs from development to QA/production environment.
- Worked with different Sources such as Oracle, MS SQL Server and Excel, Flat files
- Used DataStage Administrator to create Repository, User groups, Users and managed users by setting up their profiles and privileges, at database and datastage level
- Developed and reviewed the Standardization, Scrubbing, Matching and Survivorship rules in order to get best of the breed record
- Used DataStage Designer to design and develop jobs for extracting, cleansing, transforming, integrating, and loading data using various stages like Aggregator, Funnel, Change Capture, Change Apply and copy
- Used SSIS to develop jobs for extracting, cleaning, transforming and loading data into data warehouse, such as packages, workflow
- Developed and designed a 10-node Hadoop cluster for sample data analysis
- Regularly tuned performance of Hive and Pig queries to improve data processing and retrieving
- Ran Hadoop streaming jobs to process terabytes of XML data
- Prepared data for complex statistical analysis by working with SQL databases, complex joins/merging using Python's Pandas library
- Designed and developed workflows / sessions control using Control M.
- Used Websphere MQ trigger event, allowing a client application to respond to the delivery of a message.
- Hadoop, SSIS, Python, Datastage 9.5, Oracle / Teradata, SQL Server
Confidential
ETL/BI Developer
Responsibilities:
- Created Datastage jobs using different stages like Transformer, Aggregator, Sort, Join, Merge, Lookup, Data Set, Funnel, Remove Duplicates, Copy, Modify, Filter, Change Data Capture, Change Apply, Sample, Surrogate Key, Column Generator, Row Generator, Etc.
- Extensively worked with Join, Look up (Normal and Sparse) and Merge stages using Datastage.
- Extensively worked with sequential file, dataset, file set and look up file set stages.
- Extensively used Parallel Stages like Row Generator, Column Generator, Head, and Peek for development and de-bugging purposes in Datastage.
- Used the Data Stage Director and its run-time engine to schedule running the solution, testing and debugging its components, and monitoring the resulting executable versions on ad hoc or scheduled basis.
- Maintained the DataStage Production and Development servers. Worked on various admin issues to keep the server up at all times
- Identified and developed Perl scripts for repeating patterns and created ETL scripting templates for code reuse
- Designed Universes for Financial Billing system and Order processing
- Created UNIX complex Korn-shell scripts to automate data loading, extraction to perform regular updates to database tables to keep in sync with incoming data from other sources
- Customized ETL solutions, scheduled ETL jobs
Confidential
ETL/BI Consultant
Responsibilities:
- MDS Pharma has undertaken a strategic initiative “to enable improved decision-making and actions that favorably impact costs, quality of care, outcomes, and satisfaction through an information-driven environment that leverages their data assets for competitive advantage.”
- Defined / configured ETL Process, used ETL tools, built data mart / cube, Created and Monitored sequences using Data stage sequencer, Extensively used Parallel Stages like Join, Merge, Lookup, Filter, Remove Duplicates, Funnel, Row Generator, slowly changing dimensions, Modify, Peek etc. for development / de-bugging purposes. Responsible for metadata management, creating shared containers for reusability, extensively worked with Data Stage Job Sequences to Control and Execute Data Stage Jobs and Job Sequences using various Activities and Trigger.
- Created shared containers to use in multiple jobs, imported and exported Repositories across DataStage projects using Datastage designer
- Migrated DataStage jobs from development to QA/production environment, maintained the Data Stage Production and Development servers.
- Migrated & upgraded Business Objects from 6.5 and Crystal Reports 10 to XI Universe designing and establishing BO environment
- Designed Universes for Financial Billing system and Order processing
- Used Universe Designer, Webi, Deski, published Wizard, imported Wizard in XI, built Universes, stored procedures, free Hand SQL methods and creating complex and Ad-Hoc Reports using Business Objects, creating dashboards using Dashboard Builder, managed performance of Crystal, Excel, QAAWS, and Live Office.
- Prepared reports for deals, facility and outstanding for the information of expiry date and effective date of each category
- Tested the aggregate awareness to ensure the query is pulling correct level of aggregation
- Generated Business Objects reports involving complex queries, Unions and Intersection
- IBM Data stage8.5, Cobol, JCL, VSAM, CICS, Shell, AIX, Oracle, Business Objects, DB2
Confidential
ETL/BI Consultant
Responsibilities:
- Confidential implemented EBS / Data warehouse, hundreds ETL / BI customizations have been developed.
- Identified and analyzed ETL requirements, defined the data model for stage data cube in a data warehouse environment
- Defined / designed / modified application interfaces and Implemented Data Profiling using Oracle Data Integrator.
- Used ETL and OBIEE to effectively integrate heterogeneous data sources and converting raw data into BI application
- Developed / maintained database server (Oracle), coded pl/sql triggers / procedures / functions, developed scripts for ETL functions/tools (extract, transformation and loading) and interface, via manual scripts (SQL*Loader)
- Migrated BI application from Business Objects to OBIEE, including Universe / Query analysis, ETL design, development and Dashboard / Answers development.
- Analyzed the impact for modification, enhancements and fix bug for ETL.
- Co-managed production upgrades / batches.
- OBIEE, BO, ODI, PL/SQL, T-SQL, Shell scripts, Toad, Oracle 10g
Confidential
Oracle Consultant
Responsibilities:
- IRS tax collection and tax law enforcement business requirements definition and analysis
- Coordinated and participated in system enhancements and new system implementations including SIT / UAT testing in a data warehouse environment.
- Coded and unit tested of Informatica mappings for various stages of ETL.
- Performed source-to-target mappings, storage capacity planning, developing very complex ETL processes using Informatica
- Created Sessions, reusable Worklets in Workflow Manager and scheduled the sessions to run at a specified frequency.
- Executed sessions, both sequential and concurrent for efficient execution of mappings and used other tasks like event wait, event raise, email, command and pre/post SQL.
- Developed ETL conversion programs to convert different files (Excel, flat file or Data base) from legacy system to oracle (Java), using Informatica 7 as ETL tool.
- Analyzed ETL Implementation Method approach, packaged production releases and supported implementation of instance upgrade release.
- Created packages/procedures/function and scripts to load data from production database to development database for data cleansing and verification. Wrote and ran SQL statements for retrieving data, clone), provided data migration from lower to upper instance - including tables/materialized views, performance tuning (hints, index, partition…), in charge of all the implementation stage
- Gathered the requirement, completed the proof of concept, designed, developed and tested physical layer, business layer and presentation layer of OBIEE, including Data Mart, ETL, Answers, Reports and Dashboard design.
- Developed / executed test plans / test scripts and conducted Integration test to tune performance and scalability, using ClearCase and ClearQuest for document version control and defect control.
- Used OBIEE KPI to create actionable intelligence analysis dashboard to view performance and drill into the details to understand KPI’s value (May be off target), created action links (send an email, kick off a workflow) to initiate a action based on the status of the KPI, created overall BI Scorecard to assess performance against strategic organizational goals
- Informatica 7/8, OBIEE, Oracle 10g, Toad, ClearCase and ClearQuest
Confidential
ETL/BI Technical Consultant
Responsibilities:
- Involved in implementation of Mobile ETL/BI Applications and external systems communication
- Developed staging tables and Interface PL/SQL scripts to load data into EBS interface tables using SQL*Loader for testing and verification. Wrote and ran SQL statements for retrieving data.
- Gathered business requirements, suggested solutions using Oracle 11i, worked with super users in setting up and configured the instance
- Created ETL Packages, Procedures and Functions that extensively used PL/SQL cursors, user defined object types and exception handling, created packaging production releases and supposed the implementation of the releases, clone), data migration - tables/views, performance tuning (using hints, indexing, partitioning…)Converted PL/SQL procedures scripts to Informatica workflow/mapping to tune performance.
- Optimized PL/SQL to reduce the loading process. Utilized SQL*Loader to perform bulk data loads into database tables from eternal data files
- Created UNIX shell scripts to automate data loading, extraction to perform regular updates to database tables to keep in sync with incoming data from other sources
- Created Universes for various functional areas of high level reports group
- Created Contexts and Aliases to resolve chasm and fan traps in Universes
- Customized LOVs in Universes for quicker performance of reports
- Created Joins, Contexts, and Aliases for resolving Loops and Universe Tuning and checked the Integrity of the Universes
- Linked the Universes to share the common components between the universes
- Used the Aggregate Awareness feature of Business Objects for Optimization / Performance tuning of Universes
- Oracle Application, PL/SQL, Informatica, BO, Toad, Windows, UNIX