Sr. Hadoop Developer Resume
Memphis, TN
SUMMARY:
- Over 7+ years of progressive experience in the IT industry with proven expertise in architecting and implementing Software Solutions using Big Data Technologies and Java based technologies.
- Over 3 years of experience on Hadoop working environment includesMap Reduce, HDFS, HBase, Zookeeper, Oozie, Hive, Sqoop, Pig, Cassandra and Flume.
- Domain experience in Insurance, Retail, Health Care industries
- Well versed in developing complex Map Reduce programs using Apache Hadoop for analyzing Big Data.
- Expertise in Database Design, Creation and Management of Schemas, writing Stored Procedures, Functions, DDL, DML SQL queries
- Good knowledge of No - SQL databases Cassandra and HBase
- Implemented batch processing solution to certain unstructured and large volume of data by using Hadoop Map Reduce framework.
- Hands on experience in writing Ad-hoc Queries for moving data from HDFS to HIVE and analyzing the data using HIVE QL.
- Working experience in importing and exporting data using Sqoop from Relational Database Systems (RDBMS) to HDFS
- Extending HIVE and PIG core functionality by using custom User Defined Function’s (UDF), User Defined Table-Generating Functions (UDTF) and User Defined Aggregating Functions (UDAF) for Hive and Pig.
- Developed Pig Latin scripts for data cleansing and Transformation.
- Working Knowledge in configuring and monitoring tools like Ganglia and Nagios.
- Worked on HBase to load and retrieve data for real time processing using Rest API.
- Imported data from RDBMS to column families in Cassandra through Storage Handler
- Experienced with different file formats like CSV, Text files, Sequence files, XML, JSON and Avro files.
- Good knowledge on Data Modelling and Data Mining to model the data as per business requirements.
- Involved in unit testing of Map Reduce programs using Apache MRunit.
- Good knowledge on data analysis with R.
- Experience developing applications using Java, J2EE, JSP, MVC, Hibernate, JMS, JSF, EJB, XML, AJAX and web based development tools.
- Experience working with popular frame works like Spring MVC, Hibernate.
- Implemented SOAP based web services.
- Used Curl scripts to test RESTful Web Services.
- Experience in database design using PL/SQL to write Stored Procedures, Functions, Triggers and strong experience in writing complex queries for Oracle.
- Experience working with Build tools like Maven and Ant.
- Experienced in both Waterfall and Agile Development (SCRUM) methodologies
- Strong Problem Solving and Analytical skills and abilities to make Balanced & Independent Decisions.
- Good Team Player, Strong Interpersonal, Organizational and Communication skills combined with Self-Motivation, Initiative and Project Management Attributes.
TECHNICAL SKILLS:
Hadoop Core Services: HDFS, Map Reduce, Spark, YARN
Hadoop Distribution: Hortonworks, Cloudera
NO SQL Databases: HBase, Cassandra, Mongo DB
Hadoop Data Services: Hive, Pig, Sqoop, Flume
Hadoop Operational Services: Zookeeper, Oozie
Monitoring Tools: Gangila, Cloudera Manager
Programming Languages: C, Java, Python, SQL, PL/SQL, Unix Shell Scripting
IDE Tools: Eclipse, RAD, Net beans
Cloud Computing Services: AWS (Amazon Web Services)
Java & J2EE Technologies: Core Java, Servlets, JSP, JMS, EJB, Junit
Frameworks: Hibernate, Spring, Struts
Version Control Tools: CVS, SVN, Rational Clear Case
Application Servers: Web Logic, Web Sphere, Jboss, Tomcat.
Databases: Oracle, MySQL, Postgress, Teradata
Operating Systems: UNIX, Windows, LINUX, Macintosh OSX
Build Tools: Jenkins, Maven, ANT
Visualization Tools: Tableau
PROFESSIONAL EXPERIENCE:
Confidential, Memphis, TN
Sr. Hadoop Developer
Responsibilities:
- Responsible for building scalable distributed data solutions using Hadoop.
- Implemented nine nodes CDH3 Hadoop cluster on Red hat LINUX.
- Involved in loading data from Oracle database into HDFS using Sqoop queries.
- Implemented Map reduces programs to get Top K Results using Map Reduce programs by fallowing Map Reduce Design Patterns.
- Involved in loading the created HFiles into HBase for faster access of large customer base without taking Performance hit.
- Implemented working with different sources using Multi Input formats using Generic and Object Writable.
- Implemented best income logic using Pig scripts and Joins to transform data to Confidential custom formats.
- Implemented custom comparators and partioners to implement Secondary Sorting.
- Worked on tuning the performance of Hive queries.
- Implemented Hive Generic UDF's to implement business logic.
- Responsible to manage data coming from different sources.
- Configured Time Based Schedulers that get data from multiple sources parallel using Oozie work flows.
- Installed Oozie workflow engine to run multiple Hive and pig jobs.
- Used Zookeeper for providing coordinating services to the cluster.
- Coordinated with end users for designing and implementation of analytics solutions for User Based Recommendations using R as per project proposals.
- Assisted monitoring Hadoop cluster using Gangila
- Implemented test scripts to support test driven development and continuous integration.
- Configured build scripts for multi module projects with Maven and Jenkins CI.
- Involved in story-driven agile development methodology and actively participated in daily scrum meetings.
Environment: Hadoop, Map Reduce, HDFS, Pig, Hive, Oozie, Java, Linux, Maven, Oracle 11g/10g, Zookeeper, SVN, Gangila
Confidential, Dallas, TX
Hadoop Developer
Responsibilities:
- Imported Data from Different Relational Data Sources like RDBMS, Teradata to HDFS using Sqoop.
- Imported Bulk Data into HBase Using Map Reduce programs.
- Perform analytics on Time Series Data exists in HBase using HBase API.
- Designed and implemented Incremental Imports into Hive tables.
- Used Rest ApI to Access HBase data to perform analytics.
- Worked in Loading and transforming large sets of structured, semi structured and unstructured data
- Involvedin collecting, aggregating and moving data from servers to HDFS using Apache Flume
- Written Hive jobs to parse the logs and structure them in tabular format to facilitate effective querying on the log data.
- Involved in creating Hive tables, loading with data and writing hive queries that will run internally in map reduce way.
- Experienced in managing and reviewing theHadooplog files.
- Migrated ETL jobs to Pig scripts do Transformations, even joins and some pre-aggregations before storing the data onto HDFS.
- Worked with Avro Data Serialization system to work with JSON data formats.
- Worked on different file formats like Sequence files, XML files and Map files using Map Reduce Programs.
- Involved in Unit testing and delivered Unit test plans and results documents using Junit and MRUnit.
- Exported data from HDFS environment into RDBMS using Sqoop for report generation and visualization purpose.
- Worked on Oozie workflow engine for job scheduling.
- Created and maintained Technical documentation for launching HADOOP Clusters and for executing Pig Scripts.
Environment: Hadoop, HDFS, Map Reduce, Hive, Oozie, Sqoop, Pig, Java, Rest API, Maven, MRUnit, Junit.
Confidential, Newark, NJ
Hadoop Developer
Responsibilities:
- Developed PIG UDF'S for manipulating the data according to Business Requirements and also worked on developing custom PIG Loaders.
- Developed Java Map Reduce programs on log data to transform into structured way to find user location, age group, spending time.
- Implemented Row Level Updates and Real time analytics using CQL on Cassandra Data.
- Collected and aggregated large amounts of web log data from different sources such as web servers, mobile and network devices using Apache Flume and stored the data into HDFS for analysis.
- Developed PIG scripts for the analysis of semi structured data.
- Worked on the Ingestion of Files into HDFS from remote systems using MFT (Managed File Transfer).
- Analyzed the web log data using the HiveQL to extract number of unique visitors per day, page views, visit duration, most purchased product on website.
- Integrated Oozie with the rest of the Hadoop stack supporting several types of Hadoop jobs out of the box (such as Map-Reduce, Pig, Hive, and Sqoop) as well as system specific jobs (such as Java programs and shell scripts).
Environment: Hadoop, HDFS, Map Reduce, Hive, Pig, Sqoop, Oozie, Cassandra, Java, Shell Scripting, MySQL, SQL
Confidential, San Francisco, CA
Senior Java / J2EE Developer
Responsibilities:
- Used HTML, JavaScript, Servlets, and JSP in the presentation tier. Developed business delegates to communicate with the Enterprise Java Beans (EJB) and web services residing on the business tier.
- Responsible and active in the analysis, definition, design, implementation, management, and deployment of full software development life cycle of project.
- Development of the application that was used to create dynamic JSPs, given input from a database table containing information about the contents of the JSP being developed.
- Implemented Business Logic Components using Spring Core and navigation using Spring MVC.
- Implemented Persistent entities and Persistent Layers using Hibernate API and Interfaces.
- Implemented SOAP based Web Services for Enterprise Logging Application Module.
- Responsible for coding SQL Statements and Stored procedures for back end communication using JDBC
- Developed an API to write XML documents from a database. Utilized XML and XSL Transformation for dynamic web-content and database connectivity.
- Involved in the development of presentation layer and GUI framework in JSP. Client Side validations were done using JavaScript.
Environment: J2EE, Java, XML, JAXB, JNDI, SOAP, Web sphere Application Server, Linux, SQL Server, Rational Rose, and Oracle
Confidential
Java Developer
Responsibilities:
- Used Hibernate ORM tool as persistence Layer - using the database and configuration data to provide persistence services (and persistent objects) to the application.
- Implemented Oracle Advanced Queuing using JMS and Message driven beans.
- Responsible for developing DAO layer using Spring MVC and configuration XML’s for Hibernate and to also manage CRUD operations (insert, update, and delete).
- Implemented Dependency injection of spring frame work.
- Developed and implemented the DAO and service classes.
- Developed reusable services using BPEL to transfer data.
- Participated in Analysis, interface design and development of JSP.
- Configured log4j to enable/disable logging in application.
- Developed Rich user interface using HTML, JSP, AJAX, JSTL, Java Script, JQuery and CSS.
- Implemented PL/SQL queries, Procedures to perform data base operations.
- Wrote UNIX Shell scripts and used UNIX environment to deploy the EAR and read the logs.
- Implemented Log4j for logging purpose in the application.
- Involved in code deployment activities for different environments.
- Implemented agile development methodology.
Environment: Java, Spring, Hibernate, JMS, EJB, Web logic Server, JDeveloper, Sql Developer, Maven, XML, CSS, JavaScript, JSON.