Datastage Developer Resume
WA
SUMMARY
- Over 8 years of expertise in software design, development and deployment of large and complex software applications.
- Experience in Data Requirement Analysis, Design, Development of ETL process using IBM DataStage8.X/7.x (WebSphere, Ascential DataStage).
- Expert in all the stages of Software Development Life Cycle (SDLC).
- Experience in Data Warehousing applications, responsible for the Extraction, Transformation and Loading (ETL) of data from multiple sources into Data Warehouse.
- Experience in using various Stages like Join, Merge, Lookup, Remove Duplicates, Sort, Filter, Funnel, Dataset, Change Data Capture, Slowly Changing Dimensions, Switch, Modify, and Aggregator in DataStage designing.
- Experience in design and development of parallel jobs using Parallel Extender for splitting bulk data into subsets and to dynamically distribute to all available nodes to achieve best Job performance, Involved in maintaining logical and physical models.
- Developed efficient mappings for data extraction/transformation/loading (ETL) from different sources to a target data warehouse.
- Excellent Experience in Designing, Developing, Documenting, Testing of ETL jobs and mappings in Server and Parallel jobs using Data Stage to populate tables in Data Warehouse and Data marts.
- Experience in new enhancements in the IBM Websphere DataStage - Multiple Job Compile, Surrogate key generator Stage, Job Report, Message handler options.
- Worked on DataStage production job scheduling process using the Scheduling tools and data stage scheduler.
- Excellent knowledge in Extraction, Cleansing and Modification of data from/to various Data Sources like Flat Files, Sequential files, Comma Delimited files (.csv), XML and Databases like Oracle, ODBC, DB2, Teradata etc.
- Extensive experience in writing UNIX shell scripts for data manipulation and transformation.
- Worked on DataStage client tools like DS Designer, DS Director and DS Administrator, Parallel Extender/Orchestrate.
- Detailed knowledge in Data Warehousing concepts like Dimensional Modeling, Star Schema, Snowflake Schema and ETL.
- Used Both Pipeline Parallelism and Partition Parallelism for improving performance.
- Identified and tracked the slowly changing dimensions, heterogeneous Sources and determined the hierarchies in dimensions.
- Worked with SQL, SQL*PLUS, Oracle PL/SQL Stored Procedures, Triggers, SQL queries and loading data into Data Warehouse/Data Marts.
- Experience in using software configuration management tools like Rational Clear case/Borland star team for version control.
- Skilled in writing technical specification documents, translating user requirements to technical specifications.
- Excellent analytical, interpersonal and communication skills with aptitude to assimilate new technologies.
- Excellent experience in Relational database (RDBMS), Oracle 11g/10g/9i/8i, Microsoft SQL Server, Teradata Load, Teradata MultiLoad, SQL Loader.
TECHNICAL SKILLS
ETL Tools: DataStage 8.X/7.X (Administrator, Parallel Extender/Orchestrate and Server Jobs, Quality Stage/Integrity, Metastage, Manager, Designer, Director), DataStage Plug-In.
Databases: Oracle 11g/10g/9i/8i, MS SQL Server 2000/7.0/6.5 , DB2 UDB.
Reporting Tools: Business Objects, Crystal Reports, Cognos, Autosys, Toad.
Data Cleansing: SQL Scripts, Unix Scripts.
Data Modeling Tools: Designer 2000, Star Schema, Snowflake Schema, ERwin.
Operating Systems: Windows NT 98/2000/2003/ XP, 7, 8, UNIX, LINUX, MS-DOS, Solaris and AIX.
Programming Languages: SQL, SQLPLUS, PL/SQL, C, C++, C#.
PROFESSIONAL EXPERIENCE
Confidential, WA
DataStage Developer
Responsibilities:
- Managed the DataStage repository using DataStage Manager.
- Suggested various changes in the physical model to support the business requirements.
- Developed server jobs to load the data from flat files, text files, tag text files and MS SQL.
- Set various parameters in DataStage Administrator for performance tuning.
- Used DataStage Designer to develop various jobs.
- Utilized shared containers for code reusability and for implementing the predefined business logic.
- Created and scheduled the job sequences by checking job dependencies.
- Wrote complex SQL queries using joins, sub queries and correlated sub queries.
- Performed Unit Testing to check the validity of the data at each stage.
- Used DataStage Director to debug the jobs and to view the error log to check for errors.
- Implemented best practices in the development environment (code standards, code migration).
- Wrote PL/SQL stored procedures, functions and packages and triggers to implement business rules into the application.
Environment: DataStage 8.7, Windows 7, Linux, DB2, UNIX, IBM Mainframe, Metastage, Flat Files, Sequential files.
Confidential, Overland Park, Kansas
DataStage Developer
Responsibilities:
- Involved in status meetings, and interacted with the Business Analyst to get the business rules.
- Design and development Confidential source system data and finally make the data available to satisfy the reporting and analytical requirements of the long distance entry effort.
- Loaded data into Load, Staging and Lookup tables.
- Performance tuned DataStage to reduce batch run time.
- Extensively used Built-in (Sort, Merge, Oracle, Aggregator, DB2 Stages), Plug-in Stages for extraction,transformation and loading of the data.
- Designed the DataStage jobs to Extract the data from Text files, Transform the data according to the business requirements and load them into Oracle tables. As per the data volume, presented the size requirements and partitioning requirements for ODS tables in Oracle.
- Designed Parallel jobs using various stages like Join, Merge, Remove Duplicates, Filter, Dataset, Lookup file set, Modify, Aggregator and Funnel stages.
- Wrote Oracle stored procedures and called the Stored Procedures in Data Stage.
- Create and enhance the existing process of the PL/SQL Procedures and Packages.
Environment: DataStage 7.5, Oracle and MS SQL Server 2005, UNIX, SQL Loader.
Confidential, Princeton, NJ
ETL/DataStage Developer
Responsibilities:
- Interacted with Business Users/Business Analysts and Managers in gathering business requirements.
- Analyzed, designed, developed, implemented and maintained Parallel jobs using IBM info sphere DataStage 8.7.
- Involved in design of dimensional data model - Star schema and Snow Flake Schema
- Generating DB scripts from Data modeling tool and Creation of physical tables in DB.
- Successfully implemented pipeline and partitioning parallelism techniques and ensured load balancing of data.
- Deployed different partitioning methods like Hash by column, Round Robin, Entire, Modulus, and Range for bulk data loading and for performance boost.
- Repartitioned job flow by determining DataStage PX best available resource consumption.
- Experienced in PX file stages that include Complex Flat File stage, DataSet stage, LookUp File Stage, Sequential file stage.
- Implemented Shared container for multiple jobs and Local containers for same job as per requirements.
- Adept knowledge and experience in mapping source to target data using IBM Data Stage 8.x
- Implemented multi-node declaration using configuration files (APT Config file) for performance enhancement.
- Experienced in developing parallel jobs using various Development/debug stages (Peek stage, Row generator stage, Column generator stage, Sample Stage) and processing stages (Aggregator, Change Capture, Transformer, Filter, Sort & Merge, Funnel, Remove Duplicate Stage)
- Worked within a team to populate Type I and Type II slowly changing dimension tables from several operational source files.
- Created some routines (Before-After, Transform function) used across the project.
- Involved in creating UNIX shell scripts for database connectivity and executing queries in parallel job execution.
- Used the ETL DataStage Director to schedule and run jobs, testing and debugging its components & monitoring performance statistics.
- Debug, test and fix the transformation logic applied in the parallel jobs.
- Coordinated with other team members and participated / organize daily meetings to discuss the Dev progress.
- Created, implemented, modified and maintained the business simple to complex reports using Reporting tool (Cognos).
Environment: IBM Info sphere DataStage 8.7, Oracle 11g, DB2, SQL server, Windows, UNIX, MS Visio, Flat files, UNIX, TOAD, Cognos.
Confidential, Harrisburg, PA
DataStage Developer
Responsibilities:
- Participating on Business Analysis Development Phase and Gathering Requirements. Working with Developer Department team for translating business requirements into Data Mart Design.
- Working with Database team for creating logical and physical Data Modeling process using ERwin and guide business group and users session.
- Taking part in planning and management of all Data Warehouse migration process.
- Involved in creating strategy for Star Schemas with Fact and Dimension Tables.
- Translating Business requirements into Data Mart design coordinating with team members Creating Fact, Dimensional and Aggregate Tables and Loading Data Warehouse tables.
- Implemented the Surrogate Key by using Key Management functionality for newly inserted rows in Data Warehouse.
- Developed Shell Scripts for taking backup and recovery of database. Performed physical and logical backup.
- Used Autosys to schedule the DataStage ETL batch jobs on daily, weekly and monthly basis.
- Used DataStage manager to import, create and edit the metadata.
- Used the DataStage Administrator to assign privileges to users or users groups, move, rename or delete projects and manager or publish jobs from development to production status.
- Developed Server side functionality by using PL/SQL and UNIX shell programming.
- Constructed SQL Scripts to validate the data after loading process.
Environment: DataStage EE 7.5/8.1 ERwin 4.1.5, SQL Server 2005, Oracle10g/9i, PL/SQL, Autosys, TOAD, UNIX.
Confidential, Mount Laurel, NJ
DataStage Developer
Responsibilities:
- Interacted with Business Analysts to finalize the requirements and documented the technical design document for Coding.
- Translated business processes into DataStage jobs for building data marts.
- Worked with different sources such as Oracle, SQL Server and Excel, Flat, Complex Flat File, COBOL files.
- Designed DataStage ETL jobs for extracting data from heterogeneous source systems, transforming and finally loading into the data marts.
- Identified source systems, their connectivity, related tables and fields and ensured data consistency for mapping.
- Followed theSoftware Development Life Cycle (SDLC)process in the development of the application. Developed various triggers, functions and stored procedures.
- Created source table definitions in the DataStage repository by studying the data sources.
- Imported and exported repositories across projects.
- Used the Data stage Manager to import and export jobs.
- Written various Triggers, Stored Procedures to perform or automate many of the tasks.
- Tuned DataStage transformations and jobs to enhance their performance.
- Wrote SQL statement and stored procedures in Oracle for extracting as well as loaded data to Oracle tables
- Used Change Data Capture (CDC) technique to capture inserts, deletes and updates and made changes to the target database
- Created reusable components like parameter sets.
- Created various Jobs to serve the recurring requests reducing the manual effort and turnaround time.
- Developed UNIX shell scripts to automate the DataStage Jobs to populate/ refresh to the Data warehouse.
- Used DataStage Director and its run-time engine to schedule and execute developed jobs and job sequences, and used log events to monitor job progress and performance.
- Created FDD (Functional design Documents), TDD (Technical Design Document), Issue resolution Trackers and Unit test case document.
Environment: DataStage 7.5.2/8.1,Oracle 9i, 11g, SQL/PLSQL, UNIX and Windows NT/2000.
Confidential, Princeton, NJ
DataStage Developer
Responsibilities:
- Worked with the Business analysts and the DBA for requirements gathering, analysis, testing, and metrics and project coordination.
- Developed documents like Source to Target mapping for developing the ETL jobs.
- Populated Data Marts at different levels of granularity for the inside Customers using DataStage, SQL scripts and stored procedures.
- Imported the required Metadata from heterogeneous sources at the project level.
- Involved in the deployment of DataStage jobs from Development to Production environment.
- Developed Parallel jobs using Parallel stages like: Merge, Join, Lookup, Transformer (Parallel), and Oracle Enterprise Stage, Dataset, Funnel, Change Data Capture and Pivotal.
- Performed debugging on these jobs using Peek stage by outputting the data to Job Log or a stage.
- Designed and implemented wrappers to execute the DataStage jobs from remote servers.
- Responsible formonitoringall the jobs that are running, scheduled, completed and failed. Troubleshooting the failed jobs is a primer in these situations.
- Used SQL*Loader to load bulk data from various flat files like .csv and so on
- Participate in walk-through and provide approval of Test Plan and Test Cases.
- Conducted test cases as per plan.
- Verify readiness of the product for release through release review.
- Wrote test cases and performed Unit testing.
- Participate in defect reviews.
- Created UNIX job to schedule the jobs to be run the given time and completion of the dependent jobs.
Environment: DataStage 8.1, Oracle 10g, TOAD, SQL Server, UNIX.
Confidential, San Antonio, Texas
DataStage Developer
Responsibilities:
- Involved in meetings with the users to gather and analyze the requirements.
- Worked closely with the Data analyst and business analyst during the design and development of ETL technical specification document.
- Used DataStage as an ETL to extract data from sources like flat files and DB2 and loaded to target DB2 UDB.
- Extensively used parallel stages like Aggregator, Sort, Column Generator, Join, Merge, Head and peek for development and de-bugging purposes.
- Developed jobs to standardize and improving error handling.
- Implemented Surrogate key by using Key Management functionality for newly inserted rows in Data warehouse which made data availability more convenient.
- Implement Slowly Changing Dimensions (Type1 and Type2) using DataStage ETL jobs.
- Implemented Surrogate key by using Key Management functionality for newly inserted rows in Data warehouse which made data availability more convenient.
- Developed Jobs to load Staging and ODS tables.
- Responsible for the design, development, coding, testing and debugging of application to meet the requirements of the users.
- Extensive experience in writing complex queries.
- Extensively used SQL tuning techniques to improve the performance.
- Working on improving performances of the jobs to meet strict time lines.
- Involved in unit, system and end-to-end testing of the design.
- Worked with Squirrel to interact with Oracle and used the tool for testing.
- Worked on call for production support.
Environment: DataStage Enterprise 7.5, IBM AIX 5.2, UDB DB2 LUW 9.6, AQT, PL/SQL, UNIX Scripting, Squirrel.
Confidential
QA Engineer
Responsibilities:
- Back end testing to warehouse using shell scripting
- Created manualTest Casesin Team Foundation Server &HP QC/HP ALMfor variousUser Storiesbased onReleaseandSprintPlan.
- Stored and updated data in the database.
- Worked with DOORS Quality center for requirements documents and writing test cases. Mapping of test cases with the requirement documents.
- Functional, Regression, unit testing.
- Validation in Payment Manager for transactions.
- Validation in mainframe using TSOC and CAMS screens.
- Executed test cases and test scripts for manual and automation testing. Defects are raised in the HP Quality Center/HP ALM for logging, tracking and reporting bugs in the application.
- Test plan walk through Functional, regression, integration, unit and smoke testing.
- Getting data from few tables by using inner join and outer join.
- Used wild cards, order by, and group by functions using SQL for back end testing.
- Estimation of test cases.
- Test results walk through with business team.
- Testing of DEBIT credit, ATM and prepaid card transactions.
- Used Test Director extensively for Requirements, Test planning, and Defect tracking. TSO, File Aid, Access used for application testing.
- Communicated with Developers regarding technical issues.
- Attended Risk review meetings.
- Used Quality center service test for defect tracking and execution of test cases.
- Used SQL for sending queries to database at the back end.
- Performed UAT testing for data ware house with business objects.
- Validated different stages of claim processing.
- Worked on Share Point.
- Conducted day to day meeting in order to improve the process of testing.
- Wrote test plans and test scripts according to the requirement.
- Ran automated scripts for regression testing.
- Wrote test plan, test strategy document and scripts according to requirement.
- Performed UAT testing.
Environment: DOORS, Clear case, ORACLE 10g, JCL, UNIX, XML, PL/SQL, J2EE, JSP, Web services,DB2, SOA, Client/Server, SQL, Quality center, ALM, Agile, Quick Test pro.