SUMMARY

• 7+ years of working knowledge in Hadoop and its stack like HDFS, Map Reduce, Apache Pig, Hive, Hbase and Sqoop.
• 4+ years of experience in Web development using Java.
• Extensive knowledge in Hadoop eco - system technologies like Apache Pig, Apache Hive, Apache Scoop, Apache Flume and Apache Hbase.
• Experience in UNIX shell scripting.
• Experiences in analyzing data using HiveQL and Pig latin and custom MapReduce programs in Java.
• Experienced in writing Pig UDFs and Hive UDFs and UDAFs in the analysis of data.
• Worked with Hbase which is a NoSQL database.
• Experienced in importing and exporting data from relational database into HDFS using Scoop.
• Experienced in configuring flume to stream data into HDFS.
• Good knowledge of Hadoop architecture and its components like HDFS, MapReduce, Job Tracker, Task Tracker, Name Node and Data Node.
• Hands on experience in Application Development using Java, Hadoop, RDBMS and Linux shell scripting.
• Excellent knowledge in Java and SQL in application development and deployment.
• Good knowledge on relational databases like MySQL, Oracle and NoSQL databases like Hbase.
• Experience in Extraction, Transformation and Loading (ETL) of data from multiple sources.
• Basic knowledge in Apache spark for fast large scale in memory MapReduce.
• Experience in using open source projects like Apache commons, ANT, Log4J and Junit to increase the productivity and software quality.
• Diverse experience in utilizing Java tools in business, web and client server environments including Java platform, JSP, Servlet, Java beans, JSTL, JSP custom tags, EL, JSF and JDBC.
• Expertise in developing the presentation layer components like HTML, CSS, JavaScript, JQuery, XML, XSLT, XSD, JSON and AJAX.
• Experience in source control repositories like SVN and Git.

TECHNICAL SKILLS

Big Data Ecosystem: HDFS, HBase, Hadoop MapReduce, Zookeeper, HivePig, Sqoop, Flume, Oozie, Cassandra

Languages: C, C++, Java, PHP, SQL/PLSQL

Methodologies: Agile, waterfall.

Database: Oracle 10g, DB2, MySQL, MongoDB, CouchDB, MS

SQL server, Amazon EC2:

Web Tools: HTML, Java Script, XML, ODBC, JDBC, Java BeansEJB, MVC, Ajax, JSP, Servlets, Java Mail, Struts, Junit

IDE / Testing Tool: Eclipse.

Operating Systems: Windows, UNIX, Linux

Scripts: JavaScript, Shell Scripting

PROFESSIONAL EXPERIENCE

Confidential, Sunnyvale, CA

Senior Hadoop Developer

Responsibilities:

• Used Scoop to dump data from relational database into HDFS for processing.
• Configured flume to capture the news from various sources for testing the classifier.
• Wrote extensive MapReduce Jobs in Java to train the classifier.
• Wrote MR jobs using various Input and Output formats. Also used custom formats whenever necessary.
• Developed workflow in Oozie to automate the tasks of loading the data into HDFS and pre-processing, analyzing and training the classifier using MapReduce jobs, Pig jobs and Hive jobs.
• Used OpenNLP in the removal of stop words and Stemming of the words.
• Used Pig and Hive in the analysis of data.
• Created/modified UDF and UDAFs for Hive whenever necessary.
• Loaded and transformed large sets of structured, semi structured and unstructured data.
• Supported Map Reduce Programs those are running on the cluster.
• Managed and reviewed Hadoop log files to identify issues when job fails.
• Developed Pig UDFs for preprocessing the data for analysis.
• Involved in writing shell scripts in scheduling and automation of tasks.
• Worked on Hive for further analysis and for generating transforming files from different analytical formats to text files.
• Recently started using Mahout for machine learning in identifying a more subtle classifier.

Environment: Hadoop (Gen 1), HDFS, MapReduce (MR1), Pig, Hive, Scoop, Oozie, Mahout, Java, Linux Shell Scripting and Big Data.

Confidential, Melville, NY

Hadoop/Big data Developer

Responsibilities:

• Installed and configured Hadoop and Hadoop stack on a 7 node cluster.
• Developed MapReduce programs to parse the raw data, populate staging tables and store the refined data in partitioned tables.
• Involved in data ingestion into HDFS using Scoop and Flume from variety of sources.
• Responsible for managing data from various sources.
• Got good experience with NoSQL database Hbase.
• Designed and implemented MapReduce-based large-scale parallel relation-learning system.
• Worked with NoSQL databases like Hbase in creating Hbase tables to load large sets of semi structured data coming from various sources.
• Evaluated the use of Zookeeper in cluster co-ordination services.
• Installed and configured Hive and also wrote Hive UDAFs that helped spot market trends.
• Used Hadoop streaming to process terabytes data in XML format.
• Involved in loading data from UNIX file system to HDFS.
• Implemented Fair schedulers on the Job tracker to share the resources of the Cluster for the Map Reduce jobs given by the users.
• Involved in creating Hive tables, loading the data using it and in writing Hive queries to analyze the data.
• Gained very good business knowledge on health insurance, claim processing, fraud suspect identification, appeals process etc.

Environment: CDH4 with Hadoop 1.x, HDFS, Pig, Hive, Hbase, MapReduce, Java, Sqoop, Flume, Oozie, Linux, UNIX Shell Scripting and Big Data.

Confidential, Monroe, MI

Hadoop/Java Developer

Responsibilities:

• Worked extensively in creating MapReduce jobs for search and analytics in the identification of various trends.
• Worked on data analytics using Pig and Hive on Hadoop.
• Evaluated Oozie for workflow orchestration in the automation of MapReduce jobs, Pig and Hive jobs.
• Installed and configured Hadoop Map reduce, HDFS, Developed multiple Map Reduce jobs in java for data cleaning and preprocessing.
• Captured data from existing relational database into HDFS using Scoop.
• Captured the data logs from web server into HDFS using flume for analysis.
• Experienced in writing Pig scripts and Pig UDFs to pre-process the data for analysis.
• Experienced in managing and reviewing Hadoop log files.
• Built front end using JSP, Servlets, HTML and JavaScript to create user friendly and appealing interface.
• Used JSTL and built custom tags whenever necessary.
• Used Expression Language to tie beans to UI components.
• Gained very good business knowledge on health insurance, claim processing, fraud suspect identification, appeals process etc.

Environment: CDH4 with Hadoop 1.x, HDFS, Pig, Hive, Oozie, Scoop, flume, Servlets, JSPs, JSTL, HTML, JavaScript, JQuery, CSS, Linux Shell Scripting.

Confidential, Woburn, MA

Application Developer J2EE

Responsibilities:

• Wrote Servlets and JSPs to generate UI for an internal application.
• Developed user interface using Java Server Faces UI component framework.
• Developed POJOs and Java beans to implement business logic.
• Managed data to and from the database using JDBC connections.
• Used Spring JDBC to write some DAO classes to interact with the database to access account information.
• Involved in creation of tables and indexes and wrote complex SQL queries.
• Designed, customized and implemented the search feature using Lucene internally.
• Used expression language to wire UI components to the business objects in the model layer.
• Used Git as version control system to manage the progress of the project.
• Used Junit framework for unit testing of the application.

Environment: Servlets, JSPs, Java Beans, POJO, JSF, HTML, CSS, JavaScript, JQuery, SQL, JDBC, JSTL, EL, Stored Procedures, Lucene, Git, Junit.

Confidential, Billerica, MA

Software Engineer

Responsibilities:

• Involved in designing, coding, debugging, documenting and maintaining a number of applications.
• Participated in Hadoop training and development as a part of cross training program.
• Prepared use cases and designed class diagrams and object models.
• Involved in the creation of SQL tables and indexes and also wrote queries to read/manipulate data.
• Used JDBC to establish connection between the database and the application.
• Implemented controllers layer using servlets and JSPs.
• Implemented view layer using JSPs, JSTL and EL and also made custom JSP tags.
• Created the user interface using HTML, CSS and JavaScript.
• Created/modified shell scripts for scheduling and automating tasks.
• Wrote unit test cases using Junit framework.

Environment: Java (Jdk 1.6), Servlets, JSPs, Java Beans, POJO, JSF, HTML, CSS, JavaScript, JQuery, SQL, JDBC, JSTL, EL, Custom JSP tags, Junit, Oracle 9i/10g.