Hadoop Developer Resume
Franklin Lakes, NJ
SUMMARY
- 7 years of IT experience in Analysis, Design, Development, Implementation, Integration and testing of Application Software in web - based environments, distributed n-tier products and Client/Server architectures.
- Experience in SDLC and Object Oriented Application Design and Programming.
- Experience in OO Design using IBM Rational Rose and UML.
- Strong programming skills in Core Java technology
- Strong programming skills in advanced frameworks like Spring, Struts and Hibernate.
- Strong experience with SOA and WebServices.
- Strong hands on experience of 3 years with Big Data Technologies including Hadoop (HDFS & MapReduce), PIG,HIVE, HBASE, ZOOKEEPER, SQOOP
- Extensive knowledge in Kafka, Scala Streaming, StormMapreduce.
- Experience in creating complex SQL Queries and SQL tuning, writing PL/SQL blockslike Stored Procedures, Functions, Cursors, Index, triggers and packages.
- Experience with databases like DB2, Oracle 9i, Oracle 10g, MySQL, SQL Server and MS Access.
- Experience in administering NoSQL databases stores and design, integrating and implementation.
- Experience with advance technologies like MongoDB and advanced framework like Spring Roo.
- Extensive experience in creating Class Diagrams, Activity Diagrams, Sequence Diagrams using Unified Modeling Language(UML)
- Experienced in SDLC, Agile (SCRUM) Methodology, Iterative Waterfall Model
- Extensive experience in Software design methodologies. Proficient in web application development using Web Logic and Web Sphere (JSP, JDBC) Apache Struts, applying design patterns, and Web Services
- Expertise in developing web based GUIs using JSP, HTML, XHTML JavaScript and CSS.
- Extensive experience in Java, JSP, JDBC.
- Expertise in creating XML, XML Schemas, DOM/SAX Parser and web designing using HTML, CSS and Java Script.
- Expertise in deploying applications in Apache and Tomcat 4.0/5.0/5.5, WebSphere 6.0, Weblogic 8.1
- Strong hands on experience with Production Support
- Experience in developing ANT scripts to build and deploy Java web applications.
- Leadership experienced with a focus on QA Processes and Methodologies.
- Experience in developing test cases, performing Unit Testing, Integration Testing, experience in QA with test methodologies and skills for manual/automated testing using tools like WinRunner, JUnit.
- Experience with various version control systems Clear Case, CVS, PVCS and VSS, SVN.
- Experience with Business Rules engines like JRules and Drools.
- Excellent communication skills, team player, quick learner, organized, resilient and self-motivated.
TECHNICAL SKILLS
Big Data Ecosystem: Hadoop 2.0, MapReduce, HDFS, HBase, Zoo Keeper, Hive, Pig, Sqoop, Impala
Java/J2EE: Java 6, Ajax, Log4j, JSP 2.1 Servlets 2.3, JDBC 2.0, XML, Java Beans
Methodologies: Agile, UML, Design Patterns, Scrum
Frameworks: Struts, Hibernate, Spring, Hadoop Framework - Cloudera
DataBase: Oracle 10g, PL/SQL, MySQL
Application Server: Apache Tomcat 5.x 6.0
Web Tools: HTML, Java Script, XML, XSL,DOM
IDE/ Testing Tools: NetBeans, Eclipse
Scripts: ANT, SQL, HiveQL, NoSQL Shell Scripting, Unix
Testing API: JUnit
PROFESSIONAL EXPERIENCE
Confidential, Franklin Lakes, NJ
Hadoop Developer
Responsibilities:
- Installed/Configured/Maintained Apache Hadoop clusters for application development and Hadoop tools like Hive, Pig, HBase, Zookeeper and Sqoop.
- Wrote the shell scripts to monitor the Hadoop daemon services and respond accordingly to any warning or failure conditions.
- Managing and scheduling Jobs on a Hadoop cluster.
- Deployed Hadoop Cluster in the local, pseudo-distributed and distributed mode.
- Developed Pig Latin scripts to extract the data from the web server output files to load into HDFS
- Developed the Pig UDF’S to pre-process the data for analysis
- Develop Hive queries for the analysts
- Developed workflow in Oozie to automate the tasks of loading the data into HDFS and pre-processing with Pig.
- Cluster co-ordination services through Zookeeper.
- Collected the logs data from web servers and integrated in to HDFS using Flume.
- Implemented Fair schedulers on the Job tracker to share the resources of the Cluster for the Map Reduce jobs given by the users.
- Managed and reviewed Hadoop log files
- Assisted in designing, development and architecture of Hadoop and HBase systems.
- Coordinated with technical teams for installation of Hadoop and third related applications on systems.
- Formulated procedures for planning and execution of system upgrades for all existing Hadoop clusters.
- Supported technical team members for automation, installation and configuration tasks.
- Suggested improvement processes for all process automation scripts and tasks.
- Created system for capturing, reporting, and correcting error data.
- Work with developers, DBAs, and systems support personnel in elevating and automating successful code to production.
- Provide on-call support to production system to resolve any issues.
- Gather requirements and design of data warehouse and data mart entities.
- Conducted peer design and code reviews and extensive documentation of standards, best practices, and ETL procedures.
Environment: Hadoop, Hbase, HDFS, Hive, Java (jdk1.6), Pig, Zookeeper, Oozie, Flume.
Confidential, Jersey City, NJ
Hadoop Developer
Responsibilities:
- Pro-actively monitored systems and services, architecture design and implementation of Hadoop deployment, configuration management, backup, and disaster recovery systems and procedures.
- Involved in analyzing system failures, identifying root causes, and recommended course of actions.
- Worked with systems engineering team to plan and deploy new Hadoop environments and expand existing Hadoop clusters.
- Monitored workload, job performance and capacity planning using Cloudera Manager.
- Installed and configured Flume, Hive, Pig, Sqoop and Oozie on the Hadoop cluster.
- Used Flume to collect, aggregate, and store the web log data from different sources like web servers, mobile and network devices and pushed to HDFS.
- Analyzed the web log data using the Hive query language.
- Exported the analyzed data to the relational databases using Sqoop for visualization and to generate reports for the BI team.
- Integrated Oozie with the rest of the Hadoop stack supporting several types of Hadoop jobs out of the box (like Java Map Reduce, Pig, Hive, Sqoop) as well as system specific jobs (such as Java programs and shell scripts).
- Involved in installing and configuring Kerberos for the authentication of users and Hadoop daemons.
Environment: CDH4, Flume, Hive, Sqoop, Pig, Oozie, Cloudera Manager, Java, Linux, CentOS
Confidential, Morrisville, NC
Hadoop Developer
Responsibilities:
- Installed and configured Hadoop Map Reduce, HDFS, developed multiple Map Reduce jobs in Java for data cleaning and preprocessing.
- Good understanding and related experience with Hadoop stack - internals, Hive, Pig and Map/Reduce
- Wrote Map Reduce jobs to discover trends in data usage by users.
- Involved in defining job flows
- Involved in managing and reviewing Hadoop log files
- Involved in running Hadoop streaming jobs to process terabytes of text data
- Load and transform large sets of structured, semi structured and unstructured data
- Responsible to manage data coming from different sources
- Supported Map Reduce Programs those are running on the cluster
- Involved in loading data from UNIX file system to HDFS.
- Installed and configured Hive and also written Hive QL scripts.
- Responsible to manage data coming from different sources
- Involved in creating Hive tables, loading with data and writing hive queries which will run internally in map reduce way
- Implemented Partitioning, Dynamic Partitions, Buckets in HIVE.
- Monitor System health and logs and respond accordingly to any warning or failure conditions.
- Used Clearcase for version control.
- Used JUnit for unit testing and Continuum for integration testing.
Environment: Hadoop, Map Reduce, HDFS, Hive, Java (jdk1.6), Hadoop distribution of Horton Works, Cloudera, MapR, Flat files, Oracle 11g/10g, UNIX Shell Scripting, Clearcase, JUnit.
Confidential, Minneapolis, MN
Java Developer
Responsibilities:
- Involved in the complete development, testing and maintenance process of the application
- Responsible for gathering the requirements doing the analysis and formulating the requirements specifications with the consistent inputs/requirements
- Developed JSP as an application controller.
- Designed and developed HTML front end screens and validated forms using JavaScript.
- Used Frames and Cascading Style Sheets (CSS) to give a better view to the WebPages.
- Deployed the web application on Web Logic server
- Used JDBC for database connectivity.
- Developed necessary SQL queries for database transactions.
- Involved in testing, implementation and documentation.
- Written Java script code for Input Validation.
- Front End was built using JSPs, JavaScript and HTML.
- Built Custom Tags for JSPs.
- Built the report module on reports based from Crystalreports.
- Integrating data from multiple data sources.
- Generating schema difference reports for database using toad.
- Built Prototypes for internationalization.
- Wrote Stored Procedures in DB2.
Environment: Java, JSP, Web Logic 5.1, HTML, JavaScript, JDBC and SQL, PL/SQL, Oracle 8, UNIX.
Confidential
Jr. Java Developer
Responsibilities:
- Involved in Design, Development and Support phases of Software Development Life Cycle (SDLC)
- Reviewed the functional, design, source code and test specifications.
- Involved in developing the complete front end development using Java Script and CSS.
- Author for Functional, Design and Test Specifications.
- Implemented Backend, Configuration DAO, XML generation modules of DIS.
- Analyzed, designed and developed the component.
- Used JDBC for database access.
- Used Data Transfer Object (DTO) design patterns.
- Followed UML standards, created class and sequence diagrams.
- Unit testing and rigorous integration testing of the whole application.
- Preparing and executing test cases.
- Actively involved in the system testing.
- Developed XML parsing tool for regression testing.
- Prepared the Installation, Customer guide and Configuration document which were delivered to the customer along with the product.
Environment: Java, JavaScript, HTML, CSS, JDK 1.5.1, JDBC, Oracle10g, XML, and UML