Sr. Pentaho (etl) Developer Resume
Greensboro, NC
SUMMARY:
- Over 8+ years of experience in the field of Information Technology with proficiency in ETL design/development and Data Warehouse Implementation/development.
- Experienced in Design, Development and Implementation of large - scale projects in Financial, Shipping and Retail industries using Data Warehousing ETL tools (Pentaho) and Business Intelligence tool.
- Knowledge about Software Development Lifecycle (SDLC), Agile, Application Maintenance Change Process (AMCP).
- Excellent data analysis skills.
- Experience in Architecting and building Data Warehouse systems and Business Intelligence systems including ETL using Pentaho BI Suite (Pentaho Data Integration Designer / Kettle, Pentaho Report Designer, Pentaho Schema Workbench, Pentaho Design Studio, Pentaho Enterprise Console, Pentaho BI Server, Pentaho Meta Data, Pentaho Analysis View, Pentaho Analyzer & Mondrian OLAP).
- Hands-on experience on Data warehouse Star Schema Modeling, Snow-Flake Modeling, FACT & Dimension Tables, Physical and Logical Data Modeling.
- Installed and configured Pentaho BI Server on different operating systems like Red Hat, Linux and Windows Server.
- Hands on experience on the whole ETL (Extract Transformation & Load) process.
- Experience in creating ETL transformations and jobs using Pentaho Kettle Spoon designer and Pentaho Data Integration Designer and scheduling them on Pentaho BI Server.
- Used bunch of steps in Pentaho transformations including Row Normalizer, Row Demoralizer, Database Lookup, Database Join, Calculator, Add Sequence, Add Constants and various types of inputs and outputs for various data sources including Tables, Access, Text File, Excel and CSV file.
- Integrating Kettle (ETL) with Hadoop and other various NoSQL data stores can be found in the Pentaho Big Data Plugin. This is a Kettle plugin and provides connectors to HDFS, MapReduce, HBase, Cassandra, MongoDB, CouchDB that work across Pentaho Data Integration.
- Loaded unstructured data into Hadoop File System (HDFS)
- Created ETL jobs to load Twitter JSON data and server data into MongoDB and transported MongoDB into the Data Warehouse.
- Worked on the business requirements to change the behavior of the cube by using MDX scripting.
- Experience in using Pentaho report Designer to create pixel-perfect reports of data in PDF, Excel, PL/SQL, SQL, Confidential /SQL, JavaScript, HTML, Text, Rich-Text-File, XML and CSV.
- Good Experience in use of other ETL and reporting tools like DTS, SSIS, SSRS, SSAS and Crystal Reports.
- Experience in performing Data Masking/Protection using Pentaho Data Integration (Kettle).
- Experience in writing shell scripting for various ETL needs.
- Deep knowledge of RDBMS (SQL Server, MySQL, DB2 etc) and NoSQL databases such as MongoDB, DynamoDB and Cassandra.
- Experience in designing and developing SSIS packages for loading data from Oracle, text files, Excel, Flat files to SQL Server database.
- Proficient in writing Confidential - SQL Statements, Complex Stored Procedures, Dynamic SQL queries, Batches, Scripts, Functions, Triggers, Views, Cursors and Query Optimization.
- Quick understanding of Relational Source Database Systems and data models for building accurate transformation logic that can be used in Data Migration and Data Integration.
- Pertinent experience using heterogeneous databases such as Oracle 12c/11g/10g, DB2, MySQL, MS SQL Server 2016/2014/2, XML, and various Flat Files (Structured/Unstructured) systems for integration into Enterprise/Business Analysis DWH.
- Experience in converting stored procedures, functions and triggers written in PL/SQL code into Confidential -SQL.
- Responsible for performance tuningofstored procedures, Database Tables using Table Partitioning, SQL Profiler and Database tuning wizard.
- Well exposed and worked closely with testing teams in performanceunit testing, user acceptance testing and system integration testing.
- Exclusively worked on different flavors of UNIX and Windows operating systems and good Experience in UNIX shell scripting.
- Team player with good interpersonal and problem solving skills, ability to work in team and also work independently.
TECHNICAL SKILLS:
RDBMS: Oracle 12c/11g/10g, SQL Server 2016/2014/2012/2008 , DB2 9.7 FP1, MySQL, MS Access Sybase 15.x.
ETL Tools: Pentaho Data Integration (Kettle), Pentaho BI Server, Pentaho Analysis Tool, Pentaho Report Designer, Pentaho Mondrian, Pentaho Metadata Editor, Pentaho Design Studio, Mondrian OLAP Server
Operating System: Unix, AIX/Linux/Solaris, Windows NT/2000/XP/7, Server 2003.
Languages: Java, C++, C#, SQL, PL/SQL, Confidential -SQL, Shell Scripting, Perl, HTML, CSS, JavaScript,XML, MySQL, DHTML, VB.NET.
Modeling Tools: ER Studio, Erwin 4.1, MS Visio.
Dimensional Data Modeling: Star Schema Modeling, Snow, Flake Schema Modeling, Fact and Dimension Tables, Logical and Physical Data Modeling.
Other Tools: SQLPlus, Toad, AutoSys, SQL developer, SQL Management Studio, Microsoft Visio, red shift s3, Putty, OPENPGP.
PROFESSIONAL EXPERIENCE:
Confidential, Greensboro, NC
Sr. Pentaho (ETL) Developer
Responsibilities:
- Interacted with the Business Analysts to understand the process flow and the business.
- Actively participated in team to gather requirements to develop this BI project and also participated in designing Physical and Logical of Data warehouse.
- Created Data Flow Mappings to extract data from source system and Loading to Target.
- Used Pentaho Data Integration/Kettle to design all ETL processes to extract data from various sources including live system and external files, cleanse and then load the data into target data warehouse.
- Created transformations that involve configuring the following steps: Table input, Table output, Text file output, CSV file input, Insert/Update, Add constants, Filter, Value Mapper, Stream lookup, Join rows, Merge join, Sort rows, Database Lookup, Set Environment Variables.
- Skilled in High Level Design of ETL DTS Package for integrating data from heterogeneous sources (Excel, CSV, Oracle, MySQL, flat file, Text Format Data).
- Created user-friendly and a dynamically rendered custom dashboard to visualize the output data using the Pentaho CDE and CDF.
- Created various types of chart reports in Pentaho Business Analytics having Pie Charts, 3D Pie Charts, and Line Charts, Bar Charts, Stacked Bar Charts and Percentage Bar charts.
- Created single value as well as multi-value drop down and list type of parameters with cascading prompt in the reports.
- Integrating Kettle (ETL) with Hadoop, Pig, Hive, Spark, Storm, HBase, Kafka and other Big Data component for various functionalities and other various NoSQL data stores can be found in the Pentaho Big Data Plugin.
- Created and saved Pentaho jobs in enterprise repository and scheduled them to run in production on weekly basis.
- Used dimension lookup/update step to populate data into SCDs.
- Created Databases, Tables, Stored Procedure, Triggers, Views, Functions, Cursors, Indexes, Derived Tables and Common Table Expressions (CTEs) using Confidential -SQL programming to facilitate efficient data manipulation and data consistency.
- Experience with data integration using hive Scripts, processing the data over HDFS.
- Experienced in performing Data Masking/Protection using Pentaho Data Integration (Kettle).
- Used SQL Loader to import bulk data in Oracle tables from data files of different formats including fixed record, variable record and stream record format.
- Used the Pentaho Enterprise Repository to: create folders, store transformations and jobs, move, lock, revise, delete, and restore artifacts.
- Created and used various re-usable tasks, workflows, worklets, mapplets, and reusable transformations.
- Created cubes using schema workbench on top of DW star schema.
- Dealt with slowly changing dimensions type 1&2 and multi-level hierarchical dimensions.
- Hands on Experience with PL/SQL & Confidential -SQL.
- Involved in Prod Support to research and resolve the daily load issues.
- Scheduled meetings with Senior Data Leads and analyzed the data.
- Prepared the documents for all the modules developed.
Environment: Pentaho BI Server, Pentaho Data Integration (PDI/Kettle), Pentaho Mondrian OLAP Server, Pentaho Metadata Editor, Pentaho Design Studio, Pentaho Report Designer, MS SQL Server, Oracle, Hadoop, Confidential -SQL, PL/SQL, JavaScript, Enterprise Management Studio, MS Query Analyzer, SQL profiler and SSRS.
Confidential, Dallas, TX
Pentaho /ETL Developer
Responsibilities:
- Created user accounts in Pentaho Enterprise Console for end users/Business Analysts who were supposed to view the reports using Pentaho User Console.
- Created mapping documents to define and document one-to-one mapping between source data attributes and entities in target database.
- Used Pentaho Data Integration to create all ETL transformations and jobs.
- Used different types of input and output steps for various data sources including Tables, Access, Text File, Excel and CSV files
- Identify and analyze data discrepancies and data quality issues and works to ensure data consistency and integrity.
- Implemented Slowly Changing Dimension Type 1 and Type2 in ETL jobs for certain Dimensions.
- Wrote Shell scripts in UNIX and PL/SQL scripts to automate daily routine jobs for production databases.
- Modified existing Oracle PL/SQL code of stored procedures, functions and packages.
- Saved Pentaho jobs in enterprise repository and scheduled them to run in production on daily basis.
- Used Pentaho Report designer to create various reports having drill down functionality by creating Groups in the reports and drill through functionality by creating sub-reports within the main reports.
- Implemented security in Pentaho reports by assigning permission to specific users to view the reports.
- Implemented data level security by creating database tables to store usernames, user groups and their allowable permissions, and joined those tables in the report queries to make sure each user should be able to see only the permissible data.
- Deployed reports on Pentaho BI Server to give central web access to the users.
- Created several dashboards in Pentaho using Pentaho Business Analytics Platform.
- Used Pentaho Schema Workbench to create Cubes, Dimensions and fact tables in Pentaho.
Environment: Pentaho BI Server, Pentaho Data Integration (PDI/Kettle), Pentaho Mondrian OLAP Server, Pentaho Metadata Editor, Pentaho Design Studio, Pentaho Report Designer, Pentaho Dashboard Designer, Pentaho Business Analytics, Java, MySQL, Oracle 10g, Oracle SQL Developer, SQL Profiler, Windows Server 2008/XP 64-bit Edition, Linux.
Confidential, San Francisco,CA
Pentaho /ETL Developer
Responsibilities:
- Responsible for developing, support and maintenance for the ETL (Extract, Transform and Load) processes using Pentaho.
- Created several Pentaho Data Integration Transformations &Jobs (aka Kettle with UI as Spoon) to extract data from OLTP systems and load into staging databases.
- Involved in performance tuning of SQL queries and stored procedures using SQL Profiler and Index Tuning Wizard.
- Wrote complex stored procedures in Confidential -SQL using SQL Server Management Studio (SSMS) to summarize the data in staging environment and then developed PDI’s to load the summarized data in Data warehouse Data Mart Dimension and Fact Tables.
- Wrote cursors in the stored procedures to perform row-wise operations.
- Created triggers on master data tables in OLTP to keep track of insertion and updates.
- Created Profiling, Cleansing and auditing plans with pdi jobs and transformation.
- Applied Configuration, Logging, Error reporting to all packages to make package deployment easy and troubleshoot package on run time.
- Implemented Logic with lookup table to maintain Parent- Child relationship and maintain hierarchy.
- Wrote Perl, UNIX scripts for preliminary file check and extracting data from vendors
- Used transformation like Merge Join, Pivot, lookup, Fuzzy lookup and slowly changing Dimension.
- Generated reports using SSRS that could be used to send information to different managers of Different Branch.
- Created different type of reports including Cross-tab, Tabular, Charts, graphs, Sub reports, Drill-down reports with Cascading Parameter prompts
- Created sessions and batches to move data Confidential specific intervals & on demand using Server Manager
- Responsibilities include creating the sessions and scheduling the sessions
- Extracted the data from Oracle, DB2, CSV and Flat files, Confidential -SQL, PL/SQL.
- Understanding the Functional Requirements.
- Designed the dimension model of the OLAP data marts in Erwin.
- Wrote documentation to describe program development, logic, coding, testing, changes and corrections.
- Performed Unit testing and System integration testing. Preparing the documents for test data loading
- Troubleshoot the Productions failure and provide root cause analysis. Worked on emergency code fixes to Production.
Environment: Pentaho BI Server, Pentaho Data Integration (PDI/Kettle), Pentaho Mondrian OLAP Server, Pentaho Metadata Editor, Pentaho Design Studio, Pentaho Report Designer, MS SQL Server, Oracle, MS Access, Confidential -SQL, PL/SQL, SQL server Management Studio, MS Query Analyzer, SQL profiler and SSRS.
Confidential, San Francisco, CA
Pentaho/ETL Developer
Responsibilities:
- Extensively worked with the business and data analysts in requirements gathering and to translate business requirements into technical specifications.
- Prepared detailed design documentation including ETL data mapping documents and report specifications.
- Used Pentaho Data Integration Designer v4.0 to extract data from various sources including flat files of fixed format, Excel, XML, CSVs and databases like IBM DB2, MySQL and SQL Server.
- Used several transformations in Kettle to cleanse, validate and improve quality of data before loading it into target schemas.
- Coached a team of developers to develop Pentaho Reports, Dashboards, XActions and Analyzer Reports for the client.
- Developed complex custom reports using Pentaho Report Designer which includes developing Business Views, Cascading pick-lists, Drill-throughs, Hyperlinks, sub-reports etc, functionality into these reports.
- Problem solving: One such problem resolved was introducing an efficient and cost-savings approach to pass multiple parameters to a report using a dynamic pick-list or list of values, and also developing custom and complex stored procedures from Oracle and/or SQL Server databases which includes supporting both application and custom tables, views, triggers, and functions to aid in the design of these reports. Another was developing an Ad Hoc solution for Abbott laboratories using Pentaho Analyzer. Project was developed and deployed by myself via SDLC, and integrated within the application using Pentaho Java
- Deployed custom dashboard deliverables to clients on time. This was accomplished using the Pentaho Dashboard designer hosted on Pentaho BI Server using Pentaho User Console interface.
- Modified and/or re-created existing reports from excel requirements.
- Integrated custom pentaho reports into the Profiler application with JSP calling the pentaho report after validating parameter values from the application interface in UAT environment before deploying to production
- Created ROLAP cubes using Pentaho Schema Workbench for group web meetings for demonstrating the powerful functionality of Pentaho Mondrian Server with Ad-Hoc reporting. Power users were based out of Germany and Switzerland. Thus, also responsible for constantly communicating updates between business users and local management
- Involved in Weekly meetings to streamline and define requirements and business rules to arrive Confidential a general consensus between the analysts, business users, and software development team.
- Developed UNIX scripts for scheduling the delta loads and master loads using Autosys Scheduler.
Environment: SQL Developer, Oracle 10g, SQL Server 2005/2008, Pentaho BI Suite (Data Integration Designer, Report Designer, Dashboard Designer, Analysis View, Pentaho Analyzer, Design Studio, Mondrian Server), SQL Management Studio 2008, Perl, Confidential -SQL, PL/SQL, SQL Server 2005 & 2008,Windows Server 2008, JavaScript.
Confidential, New York, NY
SSIS/ETL Developer
Responsibilities:
- Designed ETL SSIS packages for loading data files provided by the external vendor into our data warehouse, generating reports for Sales Operations, supporting internal data warehouse applications, and generating Outbound files.
- Automated the entire process of running the reports from Infor ERP LN and validating against existing data for new accounts and inserting them in Salesforce.
- Developed complex SQL queries, Joins, Functions, CTEs, and Stored Procedures to generate reports.
- Implemented SSIS packages to call Sales force API to load data using Task Factory Pragmatic Works.
- Implemented Standardizing Addresses for the legacy data in salesforce by calling Smarty Streets through REST API.
- Responsibilities involve ETL development, unit tests, business validations, integration tests, defect resolution and production support.
- Experience in performing Data Masking using SSIS.
- The development technologies involved working with Microsoft SQL Server 2008, SQL Server Integration Services (SSIS), Task Factory Extension, Visual Studio 2012, Salesforce, MS Excel and TFS.
- Worked closely with Business Analysts, data architects for business requirements, design standards, validations, and UAT tests.
- Designed PowerShell code for SSIS Project deployments in SQL Server SSISDB.
- Validated data from Pivot tables, VLOOKUP validations in Excel.
- Had a chance to optimize legacy system SSIS packages, SQL code using SQL execution plan, Indexing, Update Stats and documented ETL packages going through development process to assist other teams for future development purposes.
Environment: SQL Server 2008, SSIS, Confidential -SQL, SSMS, C#, Perl,Excel, TFS, Visual Studio, Salesforce, info ERR LN, Cognos
Confidential, New York City, NY
SQL Server Developer (SSIS)
Responsibilities:
- Created Databases, Tables, Stored Procedure, Triggers, Views, Functions, Cursors, Indexes, Derived Tables and Common Table Expressions (CTEs) using Confidential -SQL programming to facilitate efficient data manipulation and data consistency.
- Performed Performance Tuning on Various SQL scripts.
- Created ETL packages with different data sources (SQL Server, Flat Files, and Excel etc.) and loaded the data into target data sources by performing different kinds of transformations using SQL Server Integration Services (SSIS).
- Creating various Data Sets, Writing complex queries, optimizing the queries and tuning the performance of Reports (SSRS).
- Worked on different kinds of Transformations (Data Conversion, Lookup, Derived
- Column, For Loop, For Each Loop, Conditional Split, Look up, Slowly Changing Dimension, Union All) in SSIS as per the clients Requirement.
- Design, deployment, and maintenance of various SSRS Reports in SQL Server.
- Increased the performance of the SSIS packages which is currently running in production.
- Provided application DBA support, worked on Replication, Database Backup, Database Shrink, Indexing, Defragmentation.
- Done minor enhancements in VB.net, VBA application and Crystal reports.
Environment: Windows, VB.Net, ASP, MSSQL, VBScript, JavaScript, MS access, Html, XML, BCP, DTS Import/Export, SQL Agents, SQL Mail, SQL Server Management Studio, SQL Server Integration Services(SSIS), SQL, Team Foundation Server (TFS).