We provide IT Staff Augmentation Services!

Hadoop Consultant Resume

4.00/5 (Submit Your Rating)

CA

SUMMARY:

  • Having 9+ years of IT experience in developing, delivering of software using wide variety of technologies in all phases of the development life cycle. Expertise in Java technologies as consultant, proven ability in project based leadership, teamwork and good communication skills.
  • Very Strong Object - oriented concepts with complete software development life cycle experience - Requirements gathering, Conceptual Design, Analysis, Detail design, Development, Mentoring, System and User Acceptance Testing.
  • Strong knowledge of Java with experience ranging from introduction of version 1.3, adoption of version 1.6 and commitment to use SWING for component architecture of product interface.
  • Hands-on development and implementation experience in Big Data Management Platform (BMP) using Hadoop, Map Reduce, Hive and other Hadoop related eco-systems as a Data Storage and Retrieval systems.
  • Highly knowledgeable in Writer Comparable, Writer interfaces, Mapper and Reducer abstract classes, Hadoop Data Objects such as IntWritable, ByteWritable, Text objects.
  • Experience in building infrastructure by utilizing DHCP, PXE, DNS, KICKSTART, and NFS.
  • Experience in designing and building complete Hadoop ecosystem comprising of PIG, HIVE, Sqoop, Oozie, Flume and Zookeeper.
  • Experience in upgrading the existing Hadoop infrastructure to latest releases.
  • Experience in understanding Big Data business requirements and providing them Hadoop based solutions.
  • Experience in analyzing existing Hadoop cluster, understanding the performance bottlenecks and providing the performance tuning solutions accordingly.
  • Experience in Data transfer from structured data stores to HDFS using Sqoop.
  • Experience in writing Map Reduce programs to perform Data processing and analysis.
  • Experience in analyzing data with Hive and Pig.
  • Experience in using Oozie for managing Hadoop jobs.
  • Experience in cluster coordination using Zookeeper.
  • Experience in loading logs from multiple sources directly into HDFS using Flume.
  • Experience in setting up cluster monitoring tools like Nagios and Ganglia for hadoop.
  • Working experience on Amazon Web Services (AWS) (EMR, S3, EC2)
  • Good Understanding of distributed systems and parallel processing architectures.
  • Excellent verbal and written communication skills.
  • Hands on experience in Agile and Scrum methodologies.
  • Extensively development experience in different IDE’s like Eclipse, NetBeans.
  • Expertise in relational databases like Oracle, My SQL.
  • Extensive experience in working with the Customers to gather required information to analyze, debug and provide data fix or code fix for technical problems, build service patch for each version release and unit testing, integration testing, User Acceptance testing and system testing and providing Technical Solution documents for the Users

TECHNICAL SKILLS:

Languages: Java, C, C++, SQL, PL/SQL

Big Data Framework: HDFS, Hive, Pig, Flume, Oozie, Zookeeper, HBASE and Sqoop.

Databases: Oracle 8i/9i/10g, MySQL

Web/Application Servers: BEA WebLogic 7.0/8.1, Tomcat 4x/5x, JBOSS 4.2.3

IDE Tools: Eclipse 3.3, NetBeans 6

Version Control Tools: CVS, SVN

Operating Systems: Windows XP/2000/NT, Linux, UNIX

Tools: Ant, Maven, WinSCP, Putty

PROFESSIONAL EXPERIENCE:

Confidential, CA

Hadoop Consultant

Responsibilities:

  • Dumped the data from different sources using Sqoop into HDFS for analyzing.
  • Developed transformation logic using Hive for data sources.
  • Performed hive transformations on data to transform into structured way to find IOS S/W numbers and Features used .
  • Implemented Partitioning, Dynamic Partitions, buckets in Hive to analyze and process the data.
  • Designed PDI jobs to integrate Sqoop, Hive and Hadoop file system operations.
  • Streamlined Hadoop jobs and workflow operations using PDI job.
  • Exported the analyzed data to the relational databases using Sqoop for visualization and to generate reports by our BI team.
  • Migrated code to different environment using bash scripts.
  • Installed and configured various components of Hadoop ecosystem and maintained their integrity.
  • Implemented Fair scheduler on the job tracker to allocate the fair amount of resources to small jobs.
  • Supported Map Reduce jobs those are running on the cluster.
  • Managed and reviewed Hadoop Log files.
  • Worked on Hadoop cluster maintenance activities such as metadata backups, file system checks.
  • Involved in product life cycle developed using Scrum methodology.
  • Involved in mentoring team in technical discussions and Technical reviews.
  • Involved in code reviews and verifying bug analysis reports.

Environment: JDK1.6, RedHat Linux, HDFS, Map-Reduce, Hive, Sqoop, Oozie, Netezza, Teradata, DB2, Pentaho.

Confidential, CA

Hadoop Administrator

Responsibilities:

  • Installed, configured and deployed Cloudera Hadoop cluster for development, production
  • Worked on setting up high availability NameNode for major production cluster and designed automatic failover
  • Commissioning and Decommissioning nodes to Hadoop cluster.
  • Worked on configuring security for hadoop cluster.
  • Responsible to manage data coming from different sources
  • Installed and configured Zookeeper for Hadoop cluster.
  • Involved in HDFS maintenance, Upgrading the cluster to latest versions of CDH
  • Configured Hive metastore, which stores the metadata for Hive tables and partitions in a relational database.
  • Configured Flume for efficiently collecting, aggregating and moving large amounts of log data
  • Imported/exported data from RDBMS to HDFS using Sqoop.
  • Developed the Pig UDF’S to pre-process the data for analysis.
  • Developed workflow in Oozie to automate the tasks of loading the data into HDFS and pre-processing

Environment: JDK1.6, RedHat Linux, HDFS, Map-Reduce, Hive, Pig, Sqoop, Oozie, Teradata, Oracle.

Confidential, Choochee, CA

Hadoop Developer

Responsibilities:

  • Developed Map Reduce program for parsing and loading the streaming data into HDFS information regarding messaging objects
  • Developed Hive queries to pre-process the data for analysis by imposing read only structure on the stream data
  • Data pipelines setup from scratch including design and implementation.
  • Developed workflow using Oozie for running Map Reduce jobs and Hive Queries
  • Used Sqoop for exporting data into MYSQL.
  • Configured job flows, jobs management using Fair scheduler.
  • Developed Cluster coordination services through Zookeeper.
  • Developed workflow to push data directly into HDFS using Flume.
  • Worked with Agile Methodologies and actively participated in Scrum.
  • Performance tuning for infrastructure and Hadoop settings for optimal performance of jobs and their throughput
  • Design and developed ant scripts to build the application.
  • Managing the version control for the deliverables by streamlining and re-basing the development streams of the SVN.

Environment: Java 1.6, Hadoop, Map Reduce, Hive, MySQL, Eclipse, JUnit, Log4j, Windows and Linux, Scrum.

Confidential, PRINCETON, NJ

Java /J2EE Developer

Responsibilities:

  • Responsible for developing various modules, front-end and back-end components using several design patterns based on client’s business requirements.
  • Designed and Developed application modules using Spring and Hibernate frameworks.
  • Developed user interface using JSP and JSTL as View.
  • Used JUnit for Testing Java Classes.
  • Used SCRUM Agile methodology.
  • Worked with Maven for build scripts and Setup the Log4J Logging framework.
  • Involved in the Integration of the Application with other services.
  • Involved in Units integration, bug fixing, and testing with test cases.
  • Fixed the bugs reported in User Testing and deployed the changes to the server.

Environment: Java/JDK, J2EE, Spring 2.5, Spring MVC, Hibernate, Eclipse, IBM Web sphere, XML, JSTL, Java script, Maven2, Web Services, Jquery, subversion, RESTful, JUnit, Log4J, Windows, Linux, Oracle.

Confidential

Java /J2EE Developer

Responsibilities:

  • Developed the user interface screens using Swing for accepting various system inputs such as contractual terms, monthly data pertaining to production, inventory and transportation.
  • Involved in designing Database Connections using JDBC.
  • Involved in design and Development of UI using HTML, JavaScript and CSS.
  • Involved in creating tables, stored procedures in SQL for data manipulation and retrieval using SQL SERVER 2000, Database Modification using SQL, PL/SQL, Stored procedures, triggers, Views in Oracle.
  • Developed the business components (in core Java) used for the calculation module (calculating various entitlement attributes).
  • Involved in the logical and physical database design and implemented it by creating suitable tables, views and triggers.
  • Created the related procedures and functions used by JDBC calls in the above components.
  • Involved in fixing bugs and minor enhancements for the front-end modules.

Environment: Eclipse, IBM Websphere, Java/JDK, JSP, Servlets, JDBC, PL/SQL, XML, XSLT, Struts Framework, Rational Suite, Oracle, HTML/DHTML

We'd love your feedback!