Etl/bigdata Engineer Resume
MO
OBJECTIVE
- Over 22 years of IT experience in Software Life cycle.
- Contributing to organizational growth to its optimum efficiency, deliver effective business results, and quality by obtaining a position that allows me to use my strong IT Development and analytical skills and SDLC experience extensively, to design, develop and Cloudera Hadoop development projects in a dynamic team - oriented environment that promotes leadership, learning, and growth.
- Actively involved in the SCRUM and Kanban agile Methodology, working as Cloudera Hadoop Development/Support
- Expertise in Cloudera Hadoop Development/Ingestion/support since 3 yrs.
SUMMARY
- Recently worked for Confidential in ETL, Big Data.
- Worked with the Confidential, in Cloudera Hadoop Data Development and Ingestion team, successfully ingesting the customer files on a daily basis.
- Worked as a Senior Data Analyst, supporting the BI support with different projects like Trickle down, vin solutions, Dealer Zone etc.
- Developed and supported the Cloudera Hadoop, MapReduce, HDFS, HBase, Hive, Pig, Sqoop, Hbase, Scala, Spark, amazon cloud cluster.
- Optimized apache Spark and Hadoop applications in all major distributions include Hortonworks, Cloudera and EMR.
- Supported the BI Support team, vigorously working with all the customer tickets with different applications using Informatica work flows.
- Get data from all databases such as Oracle, mysql, Mssql, and Teradata using Spark and Sqoop
- Experienced in all important bigdata ecosystems such as Hadoop, hive, sqoop, Kafka, Cassandra, Hbase, Phoenix, Redshift, Alluxio, and Flink
- Hands on experience to install all Cloudera, Hortonworks, and Apache Hadoop/spark in AWS google cloud environment.
- Experienced Talend, Qubole to implement data warehouse and cloud applications.
- Extensive experience in all important AWS Services like IAM, Redshift, Rds, EC2, EMR, CloudWatch, and AWS Pipes.
- Extensively working personality with Mobile Technology, CRM, and Billing with Quikview of Confidential Internal application along with the Amdocs Billing.
- Close intimate work nature with Amdocs Billing system on daily basis.
- I have strong functional knowledge in data warehousing, billing, collections, vouchers, and Internet transaction solutions, GUI (Graphical User Interface), and mobile technology systems. Multiple times, I was recognized as a Strong Analysis and Developemnt Resource with strength.
- Proven ability to work on multiple projects within given timeline by delivering projects successfully and efficiently,
- Expertise in creating Data Analysis reports, Data/test Coverage, Requirements, planning Final test reports.
- Cloudera Hadoop support/BI ETL support.
- Excellent understanding of SDLC, Agile, Waterfall and Test methodologies from project scope to post deployment.
- Expertise in DATA Analysis and support Environments like UNIX, Oracle, SQL Server, Netezza databases.
- Expertise in supporting the Migration of the data from Legacy system to new systems
- Expertise in writing and executing SQL queries
- Expertise in analyzing data coming from external sources before loading to big data environment.
- Expertise in Unix Shell scripting.
- Understanding and acting accordingly to all the server side coding in order to deal with some of the critical Intermediate server issues.
- Descriptive Programming. Proven Problem - Solving and technical Debugging Skills.
TECHNICAL SKILLS
Operating System: UNIX, Linux, Windows
Software: Cloudera/hortonworks Cloudera Hadoop, PIG, HIVE, Scala/Spark, TOAD, PL/ SQL, SQL, MS-Office, SQL Developer
Database: DB2/400, Netezza,ORACLE, MS Access, ES9000, MS SQL
Application Server: Oracle Web Server, JSP Server, SQL Server, IIS, Solaris, XMPP
Automated Test Tools: QTP, Win Runner, Load Runner, Clear Quest, Silk Test6.0, AMDOCS
Bug Tracking Tool: Quality Center(ALM), Rational Clear Quest, PVC Tracker, JIRA, Rally, BUGZilla
Testing methodologies: Black Box, White Box, System Integration, Regression, End-to-End, Performance, CIT, SIT, UAT, Data Validation & Verification, Data Driven,Functional, Smoke, Positive and Negative testing,boundary testing, bottom up, RBT, SBT,ATDD.
Reporting Tools: Test Director (Test Manager), Quality Center, Team Track, Crystal Reports, Rally,ALM,PVCTracker, Crystal Reports, Oracle Business Intelligence, Tableau
Web tools: Flash, Dream weaver, Adobe Photoshop, Web sphere, Web logic, eCom, COLDFUSION
Language: C, C#, SQL, PL/SQL, SQL*PLUS, COBOL,.NET
Scripting Languages: VB SCRIPT, JAVA Script, HTML,DHTML,TSL,XML, UNIX Shell Scripting
SDLC Methodology: Agile, SCRUM and KANBAN
Training: QTP, QC, Load runner, Configuration Management, HIPPA, Agile.
ETL/BI: Informatica 9.5.1/9.1.0/9.0.1/8.6 , OBIEE, Micro strategy
PROFESSIONAL EXPERIENCE
Confidential, MO
ETL/BigData Engineer
Responsibilities:
- Working on the COMCAST XMBI Mobile Data Stack.
- Working with the MSP (devices) and MIS ( Assurant that is Arterra files, CIF(financial), CTP, PRODUCTREF,PRODUCT DETAILS
- Inventory, Device accessory,,xmpp assurant, medallia, pulling the data from the Oracle databases, and using the Pentaho repository to ingest and also for transformations and Hadoop Cloudera to store the data in impala. Call the data from Hadoop into Pentaho server for transformation and putting back into Hadoop and also to MIS databases.
- Checking for the Integration, Unit testing, and also the connections and making sure the jobs and transformation run in QA, EA-LAB, Dev and Prod.
- Working with the Charter SFTP server for Arterra, Assurant, Verizon and SOLODB and XDW data And MIS and MSP through oracle.
- Using the JIRA for unit test and also for issue maintenance.
- And Zendesk, pulling the data from sftp servers for all the customer tickets through python scripting.
- Working on Scoop and Scala to put the data into Hadoop system from Data warehouse.
- Confidential Big Data Engineer(BI/ETL)(Full Time)Jan 2015 to Jan 2018
Confidential
BI/ Hadoop SUPPORT
Responsibilities:
- Supporting the ETL/BI QA for all the ETL projects,
- New Car - Front end modifications on the web pages and the customer browsing information and the new car specials etc.
- Information is captured as the SSM feed and daily files are loaded, the data warehousing testing should be validated based on the ETL process and the mapping documents.
- The movement of the data into the backend databases like Netezza, INTM, Oracle and MSTR (Micro strategy) Report.
- Worked as the Lead for the Trickle Down project for the Regression testing of the New Features added to the Old Packages.
- The updated packages are being moved properly to the backend databases will be tested and the same need to be shared with the whole group with the status meeting and emails and also the graphical reports.
- Dealer zone is the project when 52 tags and 7 dar tags are been added to the front end web site the same need to the tested from the backend side when the front end ESO group is done the testing and when deployed.
- Conducting Regular Meetings with the QA team and also with the developers and Managers for any issues and status.
- Conduction war rooms to expedite the work as well as the issue fixing and also talking with the SSM data feed group.
- Technology ETL/BI, Informatica, Oracle and Netezza databases using the Tool Toad and Aginity.
- MSTR reports through Micro strategy.
- Working for Confidential on datawarehouse, BI and ETL Projects.
- Supporting and working with the ETL projects like NEW CAR, HAYSTACK, SPECTRUM, DEALER ZONE, SLA DAR
- SLA DAR most visibility and important data project for the company
- Complete the DAR and car/dealer process two hours earlier by developing Month to Date DAR Process using the DAILY CUSTOMER METRICS Table.
- Making sure the ETL CODE reflects the design
- Validating the wrk table and the design code (DAR Daily SQL)
Environment: Cloudera Hadoop, HIVE, PIG, ALM/QC, XMPP, Agile, RUP, UAT, Netezza, Toad, Aginity, SQL Developer, Putty, SQL NAVIGATOR, Sun Solaris, Red Hat Linux, HP-UX, Oracle 11G, SQL Server, Micro Strategy, Informatica 9.0.1.
Confidential
DATA ANALYST/LEAD
Responsibilities:
- Working with the Quick View Confidential application made with Flex and Java, which is a Customer Service Management (CSM) application.
- Dealing with all the Socs, Services, Bans and IMEI and SIMS with different environments, depending on the Monthly Builds.
- Working with the Customer Service defects from the Production.
- Working with the Quick View Confidential application made with Flex and Java, which is a Customer Service Management (CSM) application.
- Dealing with all the Socs, Services, Bans and IMEI and SIMS with different environments, depending on the Monthly Builds.
- Working to retrieve different types of data for different environment using SQL Server database.
- Working with the Scrum Master and other offshore team.
Environment: Web 2.0, QC, XMPP, Agile, RUP, UAT, SQL Developer, Putty, SQL NAVIGATOR, Sun Solaris, Red Hat Linux, HP-UX, Oracle 10G/11G, QTP, Flex Monkey, SQL Server, Confidential Tools for IMEI.
Confidential, Lindberg, GA
DATA Analyst/LEAD
Responsibilities:
- Working with the Unified Inbox and the Address Book in the 4G technology
- Hands on experience in analysing IVR technology with extensive knowledge of IVR applications, data analysis procedures, and tools
- Encore application is the web based application with the Java application servers.
- This application is made to check and analyse the functionality of the widely used SMS, MMS and also Voice recognition and vice versa which is useful to prompt the callers to gather and transfer the calls.
- Also analyzing and reporting the results of the IVR on different devices like web, IPAD and also on the Android phones.
- Created the analysis reports and provided team co-coordination to complete for further reporting and feedback for future versions of the tested app.
- Supported the ETL process of Warehouse Management System of the IVR application.
- Supported and analyzed with cross functional team leads and managers and shared biweekly cycles of scrum team defect and status.
- Successfully analyzed and reported telephone line transmission, SMS/MMS, Audio, Video, and webcam pictures by extensive analysis and support the Pre Prod and Production environments.
- Analyzed the test results of different levels of the stage environment with the femoto cell to get the signals onto mobile.
- Analyzed and reported the HUI technology with Femoto cell.
- Analyzed the recording data of Voice Message/Video call/telephone line transmission, SMS/MMS, Audio, Video and webcam. And also assisted the UAT/Stress for the acceptance for Pre and Production deployment.
- Played a major role in data analysis of the end-to-end functionality for the Mobile Applications.
- Multiple sets of data along with SMS, MMS, Video, Voice and Emoticons are analaysed in all the enivironments like Pre-Prod and Production.
- Analyzed the data of uploaded instant and also old pictures.
- Recorded Video calls data was analyzed and the reports are given as per the requirement.
- Performing multi-tasking like data analysis /support by Working with the Master Number Management and working with the voice plus photo teams in giving in time best analysis and reports.
- Analyzing the data on the Emoticons from the direct selection and also with the short code typing.
- Working with the Jmeter with the server side script running.
- Supporting the Loads of the Jmeter scripts for SMS/MMS, MMS with vcard.jmx files, MMS with vCal, EP1 Inbound services, BSCA Services.JMX, Metaswitch, Jmx, Login.jmx.
- Attending meetings with Business/BA’s/Developer in BR/SR/FRD/ADD stages which would be useful get the real expected data and would create data analysis requirements and reports in beforehand so would be useful in the real analysis.
- Create User Define Function, Maintain User define function library for Automation Scripts.
- Testing with the QTP automation framework mostly with the descriptive programming of vb script.
- Writing and enhancing the scripts as per the Requirement changes. Working with the object repositories both local and shared.
- Automated the MMS and SMS concepts with the Data driven framework using data in the excel sheet.
- Multiple sets of data along with SMS, MMS, Video, Voice and Emoticons are tested in all the environments like QA, SIT and Production.
- Different options for time recording with min of 1 sec to max of 4 min voice and video were tested with QTP.
- Design, Document and Gain approval for test Deliverables for Test Automation.
- Review Test Results to Validate Completion of Testing. Review test deliverables to ensure compliance to the Test Strategy.
- Analyzing results of performance and stress test execution and deliverables reviews to evaluate product performance.
- Analyzing test Metrics to determine quality and forecast release readiness.
- Participated in Peer Reviews and Walkthroughs for SRS, Design, Test Planning, Test case Specification artifacts, and Source code
- Performed back-end testing using SQL queries to make sure that data entered correctly into the Source Database.
- Create Regression and Functional Automation Scripts using Descriptive Programming.
- Created and Implemented an Automation strategy that converted manual test cases to QTP scripts.
- Created and maintained Virtual test lab Resource via VMware Virtualization.
- Create User Define Function, Maintain User define function library for Automation Scripts.
Environment: Web 2.0, QC, XMPP, Agile, RUP, UAT, PL/SQL, SIT, TOAD, SQL Developer, Putty, SQL NAVIGATOR, Sun Solaris, HP-UX, Oracle 10G/11G, QTP, Load Runner, UNIX / Red hat Linux Shell Scripts
Confidential, Alpharetta, GA
DATA ANALYST/QA
Responsibilities:
- Involved in successfully executing multiple Modules, which includes QPASS, Bad Debt, Manual JE, Roamer, Rollover, VPDP, Warehouse Management System, commissions and compensations.
- Performed testing on UI (User Interface), PC Client, Middleware, and web user interface solution.
- Successfully created and executed test cases by utilizing Agile, RUP, and interactive test procedures.
- Performed system integration testing (SIT) of Web 2.0 applications. Responsible for messaging protocols including XMPP.
- Prepared the Setup scripts and uploaded the test cases into Quality Center. Worked on Manual JE testing both SIT and with UAT.
- Uploaded the test scripts for testing in MPQA1 (SIT environment), MPQA2 (UAT environment), MSQA6 (Pre-Prod and Prod).
- Prepared the SIT test summary.
- Worked extensively with the performance test plan for Roamer and manual testing projects, VUGEN scripts, and its enhancements.
- Extensively used Web based application (ASPEN) for MANUAL JE testing and created new featured Manual JE in Phase 3.
- Tested the breadcrumbs, navigation, and content on the web page and also compatibility check and orphan pages.
- Worked with commission and compensation projects in VPDP.
- Participated in BRD and FRD meeting with users. Working with the NFRD test cases. Defect tracking and discussion about the defects with developers. Leading up all Test Efforts across the Board for business.
- Analyzing test Metrics to determine quality and forecast release readiness.
- Responsible for creating risk analysis reports, test plan, test data specifications, test matrices, test results, test status reports and final test reports. Retesting the fixed defects. Worked on Flat files.
- Worked very effectively and efficiently with ATDD development methodology (Agile Methodology).
- Checking the mappings in Informatica 8.6 and analyze the sources and target mappings how the data is transmitting and write test cases on these mappings. Check the source vs. Target based on the mappings done in Informatica and make sure all the required data populated in target tables based on the Filter transformations used in Informatica.
- Wrote test cases on Informatica data flow (how the data is populating in Target tables from Source using different transformations).
- Wrote the whole project test cases single handed. Worked with the developers and Business Analysts.
- Opened and maintained the defect tracking and retested the defects.
- Aggressively used the SQL Developer for the Backend testing. Used ASPEN front end JAVA APPLICATION for testing.
- Set up the entire environment for testing. Wrote the full QPASS project test cases single handed.
Environment: Web 2.0, QC, XMPP, Agile, RUP, UAT, PL/SQL, SIT,TOAD, SQL Developer, Putty, SQL NAVIGATOR, Informatica 8.6, Sun Solaris, HP-UX, Oracle 10G, Informatica, UNIX / Red hat Linux Shell Scripts