Hadoop Administrator/developer Resume
Houston, TX
SUMMARY:
- More than 9 years of experience in requirement analysis, design, development, implementation, support of Java/J2EE, Hadoop, B2B, and BPM applications.
- Familiar with all aspects of technology projects including Business Requirements, Design Specification, Design Patterns and Deployment.
- Experience in using Flume to transfer log data files to Hadoop Distributed File System (HDFS)
- Extensive experience with Hive/HQL to query data from Hive tables in HDFS
- Experience in Using NoSQL databases like Hbase, Cassandra Worked on multiple clusters and nodes (including Namenodes & DataNodes).
- Extensive experience in Data/Big Analytics using Hive, Pig.
- Expert in developing and testing MapReduce programs to give data.
- Extensive experience in developing Mappers, Reducers, JobTrackers, Workflows in Hadoop.
- Hands on Experience ingesting data to HDFS using SQOOP, FLUME, KAFKA, FTP, SPARK, STORM.
- Processed real time streaming data in Hadoop using storm
- Brought stream data processing to data platform using apache storm.
- Good working knowledge of Amazon EC2
- Familiar with Hadoop administration tasks.
- Experience in source code Management using HARVEST,CVS, VSS and Rational Clearcase.
- Expertise in installation, configuration of Oracle SOA Suite 11g,Oracle Service Bus (OSB) 11g and Oracle Weblogic Server.
- Adept in Oracle Fusion Middleware toolsets like Oracle SOA Suite 11g, Oracle Weblogic, Oracle Mediator, Oracle Service Bus (OSB), Oracle Business Activity Monitoring (BAM), Oracle Business Process Management (OBPM), Business Rules Service and Business Events.
- Experience in BPEL process Orchestration for integrating ERP applications, Databases, legacy applications, Flat Files and Portals.
- Excellent experience in XML technologies - XML, XSD, XSLT, Xpath and Web Services - SOAP, WSDL
- Experience in Oracle OSB to build service proxies like load balancing, dynamic routing, fault handling to back - end SOA services.
- Designed and developed applications using object oriented Design approach using Unified Modeling Language (UML) andimplementation of various J2EE design patterns, which includes Singleton, MVC, Value Object, Data Access Object (DAO), Front Controller, Business Delegate, Service Locator and Session Facade.
- Expertise in building large - scale SOA solutions using Oracle SOA Suite components like BPEL Process, Business Rules, Human Task, Mediator and various activities like Assign, Invoke, Switch, Flow, Java embedding, Wait and While.
- Expertise in developing and deploying BPEL Processes using file adapters, FTP adapter, DB adapter and JMS adapter.
- Handled various faults in BPEL using Fault Handling Framework.
- Integrated BPEL events with BAM and created dashboards with various types of graphs.
- Worked on SO AImplementations involving Web Services using SOAP,WSDL and XMLtechnologies.
- Experience in packaging and deployment of SOA suite components in different environments and defined security and management policies.
- Experience in using SOA Suite adapters like Database, JMS, File and Oracle Applications adapter.
- Experience on administrating and web services monitoring using Oracle Web Service Manager (OWSM), Configuring components, policies, steps and alerts etc.
- Expertise in using Oracle11g/10g/9i, SQL Server 2008, DB2 databases
- Proficient working experience in SQL, PL/SQL,StoredProcedures, Functions, Packages, DB Triggers, Indexes, SQL *Loader, SQL *Plus, and TOAD.
- Extensively worked in Tuning SQL Statements and Procedures using explain plans, hints for enhancing the performance in various schemas across databases.
- Knowledge on DBA activities such as Performance Tuning, Backup and Recovery.
- Sound understanding of other Relational and Normalized databases such as Oracle 10g, 11g, MS SQL Server.
- Strong oral and written communications skills, analytical and interpersonal skills.
- Good team spirit and organized working habits with a willingness to learn new concepts.
TECHNICAL SKILLS:
Application Development Tools: TOAD 9.7/9.0, SQL Developer, SQL* PLUS, Eclipse- Kepler IDE
Big Data: Hadoop, HDFS, MapReduce, Apache Pig, Sqoop, Hive, Flume, ZooKeeper, Clusters, Oozie, Ambari, Big Analytics, MPP, Cloudera's CDH4, Hortonworks, HBase, Cassandra.
Oracle Fusion Middleware: SOA Suite 10g/11g - WebLogic 10.1.x, Oracle Application Server 10g, BPEL, Mediator, Business Rules, HumanWorkflow, Adapters - Ebusiness Suite Adapter, DB Adapter, JMS Adapter, File Adapter, FTP Adapter, Oracle Service Bus, Oracle B2B- EDI-X12, EDI-HIPAA, RosettaNet
Business Activity Monitoring: Databases and Related API Oracle, Microsoft SQL server, PL/SQL, SQL
Programming Languages: Java, C/C++, Shell scripting, PERL
XML: XML, XSD, XSL, DTD, DOM and SAX parser API
SOAP and Web services: WSDL, SOAP, UDDI Registry
GUI Technologies: HTML, Java Script, DHTML, AJAX
Other Tools: ERWIN, ETL, Visio, Microsoft Office Suite, JDeveloper, Eclipse, SOAP UI
Operating Systems: Solaris, HP-UX, Linux (Red Hat), Windows
PROFESSIONAL EXPERIENCE:
Confidential, Houston, TX
Hadoop Administrator/Developer
Responsibilities:
- Gathered business requirements by conducting meetings with Business, documenting them.
- Worked closely with Hadoop Architect in designing the Hadoop Eco System.
- Involved in implementation of Hadoop Cluster Environments.
- Worked with ETL Team to ingest data into HDFS
- Used spark with Yarn and got performance results compared with map reduce
- Migrated Hadoop Cluster from 1.0 to 2.0 version.
- Developed some machine learning algorithms using Mahout for data mining for the data stored in HDFS
- Developed Flume extensively in ingesting log data files from Application Servers, Web Servers to a HDFS
- Used Kafka to ingest real time data into HDFS
- Configured Zookeeper Services
- Developed Oozie Workflow manager to schedule Hadoop jobs and monitored them
- Used Sqoop to load data from Oracle database to Hadoop Distributed File System (HDFS)
- Extensively used Hive/HQL queries to give to results to users.
- Improved performance of Map Reduce Programs.
- Configured Qlikview to connect to Hadoop Cluster.
- Implemented Solr Search with hadoop
- Developed map reduce programs using Java and Scala.
- Developed and modified Unix Scripts to move data from FTP server to Hadoop
- Uploaded documentation on SharePoint server
- Optimized queries using partitioning and bucketing
- Worked with cluster computing frameworks like apache storm
- Have knowledge in running event processing using storm
- Integrated Kafka and Storm by using Avro for serializing and deserializing the data and Kafka procedure and consumer
- Wrote data with Storm by providing Storm-Kafka connectors
- Processed unbounded streams of data, continuous computation, ETL and real time processing using Storm
- Ingested data with Hadoop MapReduce and Storm cluster framework
Environment: Hadoop 1.2.1, Sqoop 1.4.4, Hive 0.10.0, Flume 1.4.0, Oozie 3.3.0, Pig 0.11.1, Hbase 0.94.11, Oracle 11g/10g, Apache Cassandra,SQL Server 2008, MySQL 5.6.2, Java, SQL, PLSQL, Toad 9.7, Eclipse Kepler IDE, Microsoft Office 2007, MS Outlook 2007, SharePoint Teamsite.
Confidential, Plano, TX
Hadoop Developer/Analyst
Responsibilities:
- Gathered requirements from different team like ICD9, Medicare, Medicaid, Claims team
- Used Visio to design work flows
- Developed Map Reduce programs
- Data Ingestion into hadoop using SQOOP
- Involved in loading data into HIVE
- Developed HQL queries to pull the data
- Used Storm for Real time feeds ingestion into HDFS
- Did POC using SPLUNK
- Worked with different teams to co-ordinate Hadoop migration
- Resolved Hadoop Production Support requests logged in Remedy
- Worked with claims team to discuss about claims ingestion issue into Hadoop
- Used Ambari for monitoring cluster
- Analyzed the slow peformance of Hadoop map reduce programs and recommended fixes
- Developed test cases to hand over to business to test map reduce, pig, hive programs
- Provided recommendations to move data from Oracle DB to Cassandra No SQL Database
- Used GIT code Repository for code reviewing
- Implemented map reduce programs to perform complex joins.
- Designed and implemented custom writable, custom input formats, custom partitioners and custom comparators.
- Developed PIG UDFs for loading and storing of data
- Developed Hive Internal and external tables
- Maintained PIG Scripts for Analytics purposes.
- Worked with Administrator teams to allocate HDFS Quotas.
Environment: Hadoop, Java/J2EE, Cassandra,Zookeeper, NoSQL, Cassandra, JSON, XML, JUnit, Hector, Hudson, git, SVN
Confidential, Chicago, IL
Java/J2EE/Fusion Developer
Responsibilities:
- Creating server-side JAVA architecture using Java Servlets.
- Design and deployment of EJBs.
- Developed architecture to generate dynamic Server side pages.
- Coded XML pages thatuse XSLT and display data to the user.
- Developed and deployed EJBs,Servlets and JSPs on WebLogicServer.
- Implemented Struts with WebLogic Application server, JSPas view,Servletsas Controller, and Java Beansas model.
- Installation, configuration and clustering of BEA WebLogic Server on Windows NT platform
- Developed dynamic content of presentation layer using JSP.
- Developed the user interface for administration console using Java.
- Developed of server side utilities usingServlets, Java Beans, XML, and EJB.
- Involved in developing the core business logic using Session & Entity Beans(EJBs)and deploying them in Web Logic Application server.
- Worked on the full life cycle of the project starting from user requirements.
- Building Software modules using Apache Ant.
- Object Oriented design and OOP methodology was used extensively.UML and USE-CAS Ewere the tools for theOOD purposes.
- Developed Sequence diagramsand Class diagrams using UML.
- Developed JMS Session bean, which is a wrapper over the MQ Series Queue interfaceto handle the Matching and Routing queues.
- Worked with JUnit unit testing framework for testing various java modules.
- Assisted the QA Tester in developing Test Scenarios and Test Casesand their documentation
- Extensively used BPEL to determine dynamic partner link calls, updating databases via BPEL, generating proxy stub classes to call BPEL process from a web application, etc
- Bug fixing the BPEL and ADF code and unit testing
- Modified the transformations to accommodate the changes from frontend ADF code.
- Edited DB Adapters SQL(custom) Statements to fit for the DB change requests
- Changed the FTP Adapter for the changes in Tumbleweed, Legacy System that process the PDF files
- Helped the Release Management Team to migrate from Oracle SOA Suite 10g to 11g R2
- Created Use cases and test case for brand new BPEL process.
- Prepared XSD for BPEL which is used as an input for the process.
- Created Scheduler process like NightlyOps CleanPromise ToPay Scheduler, Common Move Account To Franchise BPEL process.
- Configured File, FTP Adapters to create XML files from inbound generic files downloaded from vendor sites which are available in CSV, flat and XML formats.
- Configured DB Adapters to insert data into custom tables which is picked from the flat files, message queues and also for polling and select.
- Worked with many Jasper report to generate PDF outputs.
Environment: Java2.0, J2EE 1.4, EJB 1.1/2.0, JSP, OOP, Servlets, JDBC, UML, Rational Rose, Use Case, SQL, PL/SQL, JMS, Struts framework, ANT, BEA WebLogic,Oracle SOA 10.3.5,SOA 11g,BAM and OWSM, Apache Web Server 1.3 / 2.0, Siebel, EBS, Linux AS 4 servers, BEA Weblogic, Aqualogic,Jdeveloper 10g, 11g.
Confidential, Minneapolis, MN
J2EE/Oracle Developer
Responsibilities:
- Developed and deployed Session Beans that invoke factory services to do database operations and call PL/SQLprocedures to retrieve and manage customer account summary, details, account transfer status, billed revenue information, sales agents hierarchy information etc.
- Designed and developed custom tags to decouple presentation logic from business logic in JSP pages to handle incoming requests and improve maintainability and flexibility of JSP pages.
- Participated in discussions with other groups to understand the business like Bonds and Fixed Income Trading.
- UsedJava Message Service(JMS) for reliable and asynchronous exchange of information such as change in the prices and the values of the shares etc.
- Designed and developed Message driven beans that consumed the messages from the Java message queue.
- Involved in Design (OOD) and Identification of Components and preparing Technical Use Cases.
- Wrote SQL, statements to retrieve the data from the database and used JDBC for database connectivity.
- Involved in the developing and deploying the Entity Beans in the WSA Dusing deployment tool.
- Involved in creating all Java Server Pages,JavaBeans and invoking EJB Components through JNDI.
- Responsible for the design and development coordination of JSF/MVC based web application.
- Developed a utility web site for Portal Development.
- Analyzed and investigated on the new technology JSF and used that framework for the application
- Extensively used high level backend pl/sql programming involving different databases
- Involved in the development of new procedures, functions, triggers and also updating the old ones based on the change requests
- Developed new forms which are used to extract data from different systems and criteria. The results are written to a flat file
- Wrote PLSQL blocks, Procedures, Functions, Packages for Migration of flat files and to transfer in exact format in Database Tables.
- Prepared technical specifications document for the procedure which is used to update the purchase order information from one database to another database.
- Utilized tools like TOAD during development of the application.
- Involved in direct discussion with the End-Users to study the requirements and helped in the development of functional specifications to meet their requirements.
- Used SQL Loader for Data Conversions and to load the flat files from other different sources.
- Tuned SQ LOADER and SQL Scripts to load data faster..
- Fine tuned SQL Queries using explain plan, hints for maximum efficiency and performance
- Involved in the production support and solved the tickets that are raised by the end users.
Environment: Java, J2EE, Oracle 10g/11g, SQL, PL/SQL, UNIX, SQL*Plus, SQL*Loader, Toad, Forms & Reports 9i, Linux.
Confidential, New York, NY
Java/J2EE Developer
Responsibilities:
- Created Form Beans, Value Beans, and Action classes as a part of the Struts framework
- Used Action Servlets, Action mappings and Action form beans to implement the project
- Client side development was done using the HTML and validations were implemented using JavaScript.
- Developed Multi threaded Java Servlets.
- Designed HTML and DHTML using Dream Weaver.
- Developed Java Script for Client side validation.
- Designed JSP pages for collecting input from the users and for user interaction.
- Used Dream Weaver for designing Internet based Web Pages
- Used Web sphere Application server as IDE tool
- Deployed all the JSP, Java Servlets and the action classes in a .jar and.war format in the IBM Web Sphere 5.1 server
- Designed the database objects such as tables, procedures and views
- Developed database schema, packages and the JDB Clayer, the application.
- Unit testing and rigorous integration testing of the whole application.
- Actively involved in unit testing using test cases.
- UsedJunitto write the test cases
- Understanding the business logic to modify existing SQL Code and Performance Tuning if it requires.
- Used EXPLAIN PLAN, ANALYZE, HINTS to tune queries for better performance and also Extensive Usage of Indexes
- Involved into updating procedures, functions, triggers, and packages based on the change request.
- Coordinate with the front end design team to provide them with the necessary stored procedures, packages and the necessary data.
- Experience with Business Objects
- Involved in writing procedures, functions, triggers, and packages for new requirements.
- Developed Informatica Mappings using various Transformations and PL/SQL packages to extract, transformation and loading of data.
- Involved in large and complex data conversion from the existing system to the new system which did help End-Users to use both old and new systems and helped in the development of new system that replicates the old system as is.
- Read data from flat files and uploading that data into oracle tables using SQL*loader.
- Involved in the production support and solved the tickets that are raised by the end users.
- Involved to prepare user level and system level maintenance documentation.
Environment: Java, JSP, Servlets, IBM Web Sphere 5.1, Jakarta Struts, WSAD, Dream Weaver,Oracle 10g, SQL, PL/SQL, UNIX, SQL*Plus, SQL Loader, Toad.
Confidential, Raritan, NJ
Java/J2EE Developer
Responsibilities:
- Preparing the requirements document
- Involved in application development using Java, JSP, J2EE, SOA Web Service, Rational Application Developer.
- Implemented Web Services using Java
- Responsible for testing using JUnit, SoapUI and providing the solution for performance issue.
- Used CVS as a version control system
- Coordination and support for web service interface development.
- Implementing the required changes in the System
- Preparing the Test Plan to describe the scenario for Testing Requirements and preparing the Test Results.
- Writing PL/SQL Procedures, Functions and Packagesto meet the module requirements.
- Used EXPLAIN PLAN, AUTOTRACE,SQL TRACE, and TKPROF for tuning the SQL queries.
- Monitoring of the reports on daily, weekly and monthly basis.
- Creating Ad-hoc reports according to the request send, for the monthly analysis of the customer's data.
- Used SQL*Loader to load data from external system and developed PL/SQL programs to dump the data from staging tables into base tables.
- Developed Dynamic SQL scripts and DBMS SQL Packages for Large data applications.
- Developed packages for extracting and loading data into SQL SERVER database to create reports using SSRS.
Environment: Java, J2EE, WebSphere, Red Hat LinixOracle 10g/11g, PL/SQL, SQL Loader & SQL* Plus, SQL *Navigator, Oracle Forms 10g, Reports 10g, DB2 9, MS SQL Server 2005, SQL Developer, TOAD, UNIX.
ConfidentialJava/Oracle Developer
Responsibilities:
- Coordinated with the business users to collect requirements.
- Modifying & Creating PL/SQL Procedures, Functions, Packages, Triggers, SQLQueries for Reports
- Performance tuning: Partitioning the fact tables and materialized views to enhance the performance.
- SQL queries, SQL Tuning, PL/SQL Tuning & Performance Tuning.
- Migrating and Converting Old Database of unify toOracle, and Process Improvements.
- To communicate with colleague & Support Team for Enhancement.
- To study the System and Preparation the Specifications.
- Developed standard reports as per user requirements.
- Implementation of assigned work.
- Got trained on Java
- Developed HTML, JSP
- Modified existing Business Logic in Java.
Environment: Oracle 9i, SQL, PL/SQL, Forms 6i, Reports 6i, Java, UNIX.