Data Architect/etl Architect Resume
Lakeforest, IL
SUMMARY:
- Over 16 years of experience in Design, Modeling, Analysis, Development, Testing and Implementation of Data Warehouse/Data Mart Applications/Master Data Management and Client/Server Applications on Windows and UNIX platforms. And Installation/administration of IBM Ascential Datastage 7.5.x, IBM Infosphere Information Server 8.0/8.5,Java SE7.2.
- Extensive experience in using the Data Warehousing tools, DataStage 11.3/9.1/8.5/8.1/8.0/7.5, Informatica,SSIS,Kalido,SSIS,Java SE.
- Installation and administration of apache Hadoop 2.6.0 in multi node infrastructure, Pig 0.15.0,Hive 1.2.1,Sqoop 1.4.6
- Experience in designing Data Lake and transitioning Datawarehoue data to Hadoop cluster. Hands on processing, storing, archiving and retrieval of large datasets to/from Hadoop HDFS.
- Designing, developing and implementing enterprise data warehouse applications (Inmon/Kimball modeling strategies)
- Strong knowledge in OLTP/OLAP Systems, Normalized (OLTP), Denormalized (DW and ODS) models, Dimensional Strategies, Surrogate Key, Star schema and Snowflake schema.
- Experience in UNIX Shell Programming for Scheduling, File Manipulation etc.
- Experience in integration of various data sources like DB2, Oracle and MS SQL Server into staging area. Designing real - time datawarehouse applications using SOA architecture.
- Proficient in performance optimization of data extraction and loading to/from RDBMS databases and performance tuning of ETL applications
- Experience in evolving strategies and developing Architecture for building a Data warehouse using data modeling tools such as ERWIN, ERStudio,viz-.
- Quick learner, efficient, self-motivated and success oriented. Excellent written and communication skills and ability to work independently in a time sensitive environment.
- Certification in Financial Markets from National Stock Exchange (NSE), India
- Certification in Mutual Funds from Association of Mutual Funds in India (AMFI)
- Recognized as a top performer by Confidential for FY .
- Rewarded as top performer by Tata Consultancy Services (TCS) for FY
TECHNICAL SKILLS:
Data warehousing Tools: IBM WebSphere Datastage V8.1/8.0/8.5, Ascential Data Stage7.5,SSIS, Informatica PowerCenter 6.x, Cognos Powerplay, Toad, Erwin 3.5/4.x, Kalido 7.0/8.0,Kalido MDM,,Pentaho Kettle 3.1,WebFocus8
Hadoop Eco System: Hadoop 2.x, pig 0.15.0, Hive 1.2.1, Sqoop 1.4.6
Languages: Java, SQL, PL/SQL, JDBC, VB 6.0, PERL, Shell Scripting in UNIX, Crystals Reports
Databases: SQL Server 7.0/2012, Oracle 10g/9i/8i/7, MSAccess, MYSQL, DB2
Extended Technologies: XML
Operating Systems: Windows 95/98, Windows NT/2000, AIX, HP-UX,RedHat Enterprise Linux
Version Management Tools: Microsoft VSS, Clear Case, PVCS, TFS
PROFESSIONAL EXPERIENCE:
Data Architect/ETL Architect
Confidential, LakeForest, IL
Environment: IBM WebSphere DataStages EE 8.5, SQL Server 2008, Window Server 2008, Control M,Webfocus8,IBM IS Manager,Microsoft MDM, Hadoop 2.x ecosystem
Roles & Responsibilities:
- Design, Develop and implement Data warehouse/MDM applications
- Involved in designing Data Lake in Hadoop HDFS, identifying sources and defining data extraction methodology from variety of source systems into data lake
- Managing and reviewing Hadoop log files, importing/exporting data from HDFS using Sqoop, creating Hive tables, wrote map reduce job using Pig Latin
- Interacting with business analysts to identify KPI’s and researching HDFS Data Lake/ legacy applications for KPI’s
- Performing data profiling, setting up data governance rules for reference/transactional data
- Mentoring development team, and designing and developing ETL framework, pseudo code to aid development team
- Evaluating cost effective open source technologies, and developing proof of concept using open source technologies
- Performance tuning of Database applications
ETL Architect
Confidential, Fortworth, TX
Environment: IBM Websphere Datastage 8.5, Oracle 10/11g, UNIX Shell Scripting, Java, SSH, XML, Clear Case, ERWIN
Responsibilities:
- Interacted with business analysts to gather business user requirements.
- Generated HLD, LLD, Source to Target mapping, test cases and miscellaneous documents.
- Involved in designing and developing IBM Websphere Datastage ETL jobs.
- Created physical designs for implementation on Oracle database.
- Created generic datastage sequence by integrating korn shell scripts to schedule ETL jobs on Tivoli work scheduler
- Worked extensively on performance tuning of SQL queries on source system database and datawarehouse data base, and optimizing ETL jobs for high throughput.
- Installed IBM Infosphere Information Server on AIX 5.3 using DB2 database as repository database.
- Migrated IBM Ascential Datastage 7.5 to IBM Infosphere Information Server 8.5
- Applied fix pack I/II/III on Information Server Linux server and client windows desktops
- Wrote shell scripts to monitor IBM Information server health and routine cleanup
- Installed Apache Hadoop HDFS cluster,Hive,Sqoop and set up file system on HDFS
- Involved in creating Hive tables, loading with data and writing Hive queries
- Monitoring Hadoop logs, and troubleshooting any warnings/failures regularly
- Tracking status of name node,datanode,resource manager,node manager,history server from command line and web interfaces
- Configuring Hive metastore in Oracle database, and monitoring metastore tables using TOAD
Datastage Team Lead/Data Architect
Confidential, Phoenix, AZ
Environment: IBM Websphere Datastage 8.5, DB2 V9.5, UNIX Shell Scripting, Java, SSH, XML, Information Analyzer, Rational Data Architect, ERWIN
Responsibilities:
- Interacted with business analysts to gather business user requirements.
- Generated HLD, LLD, Source to Target mapping, test cases and miscellaneous documents.
- Used IBM information analyzer for data profiling and data quality analysis report
- Involved in designing and developing IBM Websphere Datastage ETL jobs.
- Created dimensional models using Rational Data Architect.
- Created physical designs for implementation on DB2 database.
- Worked extensively on performance tuning of SQL queries on DB2 database.
- Created Parallel shared containers to collect common performance audit log details from ETL jobs and to gather batch run statistics
- Carried out performance test in the performance environment and tuned the ETL application for performance optimization.
- Wrote shell scripts in UNIX environment.
- Generated test environment and production environment migration documents.
- Installed IBM Infosphere Information Server on Red Hat Linux 6 using Oracle 10g database as repository database.
- Applied OS libraries on red hat Linux 6.0
- Migrated IBM Ascential Datastage 7.5 to IBM Infosphere Information Server 8.5
- Configured Datastage environment file, configuration files, and .odbc.ini files on Information server
- Created Information server roles and users, and set up auto purging of logs on IIS console
ETL Senior Developer
Confidential, CA
Environment: IBM Websphere Datastage 8.1/8.0, Oracle 10g/9i, UNIX Shell Scripting, Java, SSH, XML, WSDL,Pentaho-Kettle 3.1, Microsoft Visio, ERWIN
Responsibilities:
- Interacted with business analysts to gather business user requirements.
- Generated HLD, LLD, Source to Target mapping, test cases and miscellaneous documents.
- Involved in designing and developing IBM Websphere Datastage ETL jobs to function as Service Oriented Architecture(SOA).
- Prepared Data Quality Analysis Review (DQAR) document and identifying bottle necks in source data and preparing remedy for the same
- Created dimensional models using ERWIN.
- Created physical designs for implementation on Oracle database.
- Responsible for extracting, transforming and loading data from Oracle DB, flat files, XML files and loading Oracle target databases(Data Hub).
- Responsible for Extracting, Transforming and Loading data from Oracle, flat files and loading Oracle Database (Data warehouse database).
- Wrote PL/SQL scripts and invoked them using Shell scripts and Datastage ETL jobs.
- Designed, developed and tested jobs using IBM Websphere Datastage.
- Created Parallel shared containers to collect common performance audit log details from ETL jobs and to provide response to the requestor as job output
- Implemented new features of IBM Websphere Datastage V8.0 such as exposing ETL Jobs as webservices, carrying out SFTP from FTP enterprise stages, using WISD Input/Output stages to enable ETL jobs to communicate with each request/response to/from requestor.
- Configured SSH on third party application server and ETL server for carrying out secure copy of files across servers.
- Carried out performance test in the performance environment and tuned the ETL application for performance optimization.
- Wrote shell scripts in UNIX environment.
- Generated test environment and production environment migration documents.
- Involved in writing assembly test cases and preparing Test data document to aid testing team
- Prepared Unit test cases and System test cases and recorded the results accordingly.
- Generated UAT test cases and coordinated the review of the test cases by Business Analysts.
Associate Technical Architect and Senior Developer
Confidential, MN
Environment: Ascential Datastage 7.5.1 EE, Oracle 9i, DB2 UDB, UNIX Shell Scripting, Control-M, SSH, XML, Microsoft Visio, ERWIN
Responsibilities:
- Interacted with business analysts to gather business user requirements.
- Prepared estimation document to estimate the effort, duration and resource requirement during each phase of the project
- Generated HLD, LLD, Source to Target mapping, test cases and miscellaneous documents.
- Prepared Data Quality Analysis Review (DQAR) document and identifying bottle necks in source data and preparing remedy for the same
- Created dimensional models using ERWIN.
- Created physical designs for implementation on DB2.
- Responsible for extracting, transforming and loading data from DB2 UDB, flat files and loading DB2 target databases.
- Responsible for Extracting, Transforming and Loading data from Oracle, flat files and loading Data warehouse databases.
- Wrote PL/SQL scripts and invoked those using Shell scripts and Datastage ETL jobs.
- Designed, developed and tested jobs using Ascential Data stage 7.5.1 EE.
- Configured SSH on Java / ATG server and ETL server for carrying out secure copy of files across servers.
- Wrote shell scripts in UNIX environment.
- Generated test environment and production environment migration documents.
- Prepared Unit test cases and System test cases and recorded the results accordingly.
- Scheduled ETL and Java application jobs on Control-M.
- Generated UAT test cases and coordinated the review of the same by Business Analysts.
- Carried out performance test in the performance environment and tuned the ETL application for performance optimization.
- Monitored Ascential Datastage jobs on Control-M scheduler and handled the troubleshooting of failure jobs.
Senior ETL Developer
Confidential
Environment: Ascential Datastage 7.5.1, Kalido DIW 8.0,Kalido MDM, Oracle, UNIX Shell Scripting, SSH, Cognos Impromptu, SQL Server Analysis Services, ERWIN
Responsibilities:
- Interacted with client to gather business user requirements.
- Generated LLDs, test cases and miscellaneous documents.
- Prepared Data Quality Analysis review (DQAR) document.
- Extracted Transformed and Loaded data from Oracle, flat files and loaded Data warehouse databases.
- Created data model for staging area using ERWIN.
- Involved in Data modeling using Kalido and writing Kalido routines.
- Scheduled Kalido routines and shell scripts using Data Stage sequencer.
- Created reports using Cognos Impromptu reporting tool.
- Wrote PL/SQL scripts and invoked them in Shell scripts and Datastage ETL jobs.
- Designed, developed and tested Datastage jobs using Designer tool.
- Migrated ETL code and Shell scripts across the environments and prepared environment migration document.
- Carried out impact analysis of new business requirements and prepared estimation document for the same.
- Automated loading global reference data into global data warehouse using Kalido MDM tool.
- Developed Datastage ETL jobs to load global reference data from global datawarehouse to regional data warehouse.
- Monitored and handled trouble shooting Ascential datastage and Kalido failure jobs.
- Automated loading of monthly data such as Customer debts and Marketing margin data.
Datawarehouse Developer
Confidential
Environment: Ascential Datastage 7.5.1, Oracle, UNIX Shell Scripting, Cognos Impromptu
Responsibilities:
- Interacted with client to gather business user requirements.
- Generated LLDs, test cases and miscellaneous documents.
- Responsible for Extracting, Transforming and Loading data from Oracle and loading Data warehouse databases
- Wrote PL/SQL scripts and invoked them through shell scripts.
- Designed, developed and tested Datastage jobs using Datastage designer.
- Migrating code to test and production environments.
- Created Datastage sequences to schedule datastage jobs.
- Automated notification of bad data during batch process to Business Analysts.
- Involved with the monitoring and failure analysis of Datastage ETL jobs.