Senior Hadoop Developer Resume
Charlotte, NC
SUMMARY:
- Overall 7+ years of experience in Analysis, Design, Development, Testing, Implementation, Maintenance and Enhancements on various IT Projects.
- 3+ years of experience on BIG DATA using HADOOP framework and related technologies such as HDFS, HBASE, Map Reduce, HIVE, PIG, FLUME, OOZIE, SQOOP, and ZOOKEEPER.
- Experience creating real time data streaming solutions using Apache Spark/Spark Streaming/Apache Storm, Kafka and Flume.
- Designing data models in Cassandra and working with Cassandra Query Language.
- Hands on experience in running SQL style applications on HBase using Apache Phoenix.
- Extensive involvement in relational database / data warehouse environment including experience with slowly changing dimensions (SCD), operational data stores, and data marts.
- Experience in handling messaging services using Apache Kafka.
- Extensive experience in Unix Shell Scripting
- Expertise in writing Hadoop Jobs for analyzing data using Hive and Pig.
- Experience with Amazon S3, Amazon EC2, Amazon Elastic MapReduce(EMR).
- Experience in importing and exporting data using Sqoop from HDFS to Relational Database Systems(RDBMS) and vice - versa.
- Experience in working with MapReduce programs using Apache Hadoop for working with Big Data.
- Extensively used ETL methodology for supporting Data Extraction, transformations and loading processing, using Informatica.
- In depth understanding/knowledge of Hadoop Architecture and various components such as HDFS, Job Tracker, Task Tracker, Name Node, Data Node and MapReduce concepts.
- Worked on Cascading API for Hadoop application development and work flows.
- Experience in analyzing data using Hive QL, Pig Latin, and custom MapReduce programs in Java.
- Knowledge of job workflow scheduling and monitoring tools like Oozie and Zookeeper.
- Experience in optimization of Map reduce algorithm using combiners and partitioners to deliver the best results.
- Good understanding of NoSQL databases like HBase, Cassandra, MongoDB.
- Good understanding of Data Mining and Machine Learning techniques.
- Expertise in core Java, J2EE, Multithreading, JDBC,Web Services Shell Scripting and proficient in using Java API’s for application development.
- Solid background in Core Java concepts like Threads, Collections Framework, Java Reflection and also have handsome experience in building Class diagrams, activity diagrams, sequence diagrams, activity diagrams and flow charts using Rational Rose and Visio.
- Strong application knowledge of various design patterns including MVC, VO, Session Facade, Abstract Factory, Business Delegate, Singleton, Command, Front Controller and DAO for the development of Multi-Tier distributed Enterprise Applications.
- Strong database connectivity skills which includes Oracle, MYSQL, and DB2 and in programming with SQL, PL/SQL, and Stored Procedures, Triggers, Functions and Packages besides writing DDL, DML and Transaction queries with development tools like SQL developer.
- Proficient in Working with Various IDE tools including Eclipse Galileo, IBMRational Application Developer (RAD).
- Worked on different operating systems like UNIX/Linux, Windows XP and Windows 2K
- Excellent working experience in Scrum / Agile framework and Waterfall project execution methodologies.
- Very good experience in customer specification study, requirements gathering, system architectural design and turning the requirements into final product.
- Strong technical background, excellent analytical ability, good debugging skills, good communication skills, team player, goal oriented and ability to quickly learn new technologies as required.
SKILLS SETS:
Technologies: Hadoop (Cloudera-CDH, Hortonworks-HDP), Hive, Pig, Cassandra, Sqoop, Storm, Kafka Mahout, Oozie, HBASE, Spark, Tableau,DataStage 8.1/7.x/6.x, Java Beans, Servlets, JSP, JDBC, EJB, JNDI, JMS, RMI.
Architecture & Framework: Client-Server, MVC, J2EE, Struts, Spring, Hibernate.
Database: Cassandra, HBase, Oracle 11g, SQL server 2008,MySQL
IDE: Eclipse, WSAD, Net Beans, IBM RAD, JBuilder.
Design Methodology : UML, Water Fall, Perl, Agile
Operating Systems : Windows 9x/NT/XP/2003/VISTA/7/8, Linux, Unix
Conceptual knowledge : Web Services, Socket Programming.
GUI : HTML, XML, XSLT, JSF, AJAX, JavaScript, CSS, JQuery, YUI.
Query Languages : SQL, PL/SQL.
Programming Language : Core Java APIs (I/O, Thread, Collection), C, C++, AJAX, Python, JavaScript.
Design patterns : Business Delegate, Business Object, Value Object, Front Controller, Database Access Object, Factory, Singleton, Session Facade.
Tools : BEA WebLogic 8.1, JBOSS, IBM Websphere Application Server 6.1, Tomcat 6.0, JUnit 4.0, ANT, Log4j, Mercury Quality Centre, Rational Clear Quest. ANT, Maven, SVN, Toad
Design & Control : UML, Rational Rose, CVS, Clear Case
PROFESSIONAL EXPERIENCE:
ConfidentialRole: Senior Hadoop Developer
Charlotte, NC
Responsibilities:
- Experience in working with Hadoop clusters using Cloudera (CDH3) distributions.
- Used the Spark - Cassandra Connector to load data to and from Cassandra.
- Created data-models for data using the Cassandra Query Language.
- Involved in writing Unix/Linux Shell Scripting for scheduling jobs and for writing PIG scripts and Hive QL.
- Handled importing of data from various data sources, performed transformations using PIG, Hive, Hbase and loaded data into HDFS.
- Performed data standardization, cleansing the data before loading into the Data Warehouse.
- Develop ETL solutions utilizing Informatica to fetch data from external sources to populate the Data warehouse.
- Created User defined types to store specialized data structures in Cassandra.
- Worked with a team in order to develop and maintain an internal Log Delivery System, utilizing Apache Kafa and Storm.
- Continuous monitoring and managing the Hadoop cluster using Cloudera Manager.
- Experienced in designing, developing, documenting, and testing of ETL jobs and mappings in Server and Parallel jobs to populate tables in Data Warehouses with high degree involvement of Informatica.
- Migrated projects from Oracle to Hbase using Apache Phoenix , hands on experience in tuning JDBC queries to Phoenix as the later does not support Union, Intersect, Minus.
- Updated Hbase using Phoenix command line tools
- Utilized Amazon Map Reduce framework to run Hadoop MapReduce programs with Amazon S3 as source (input) and destination (output).
- Used Sqoop to extract data from Oracle SQL server and MySQL databases to HDFS
- Loaded the dataset into Hive for ETL (Extract, Transfer and Load) operation.
- Experience in Requirement Gathering and Analysis, System Design, and Developments phases of Software Development Life Cycle (SDLC).
- Developed workflows in Oozie for business requirements to extract the data using Sqoop
- Developed MapReduce(YARN) jobs for cleaning, accessing and validating the data
- Wrote MapReduce jobs using Pig Latin
- Used Hive and Impala to query the data in HBase
- Hive scripts were written in Hive QL to de-normalize and aggregate the data
- Automated the work flows using shell scripts (Bash) to export data from databases into Hadoop
- Used JUnit framework to test the Unit testing of the application
- Hive queries for data were written to meet the business requirements
- Developed product profiles using Pig and commodity UDFs.
- Designed workflows by scheduling Hive processes for Log file data, which is streamed into HDFS using Flume
- Developed schemas to handle reporting requirements using Tableau
- Imported data from oracle database to HDFS using Sqoop.
- Actively participated in weekly meetings with the technical teams to review the code
- Involved in loading data from UNIX file system to HDFS
- Created UDF's for Hive and PIG in Java
- Implemented test scripts to support test driven development and continuous integration
- Responsible to manage data coming from different sources
- Have deep and thorough understanding of ETL tools and how they can be applied in a Big Data environment
- Participate in requirement gathering and analysis phase of the project in documenting the business requirements by conducting workshops/meetings with various business users
- Involved in moving all log files generated from various sources to HDFS for further processing through Flume
Environment: Hadoop, Map Reduce, Hive QL, Hive, HBase, Sqoop, Cassandra, Flume, Tableau, Impala, Oozie, MYSQL, Oracle SQL, Java, Unix Shell, YARN, Pig Latin.
ConfidentialRole: Hadoop Developer
Milwaukee, WI.
Responsibilities:
- Worked on analyzing Hadoop cluster using different big data analytic tools including Pig, Hive, and MapReduce
- Monitored workload, job performance and capacity planning using Cloudera Manager.
- Responsible for writing Hive Queries for analyzing tera-bytes of customer data from Hbase and put the results in CSV file.
- Supported MapReduce Programs those are running on the cluster
- Worked on debugging, performance tuning of Hive & Pig Jobs
- Implemented test scripts to support test driven development and continuous integration
- Worked on tuning the performance Pig queries
- Involved in loading data from LINUX file system to HDFS using Kettle
- Importing and exporting data into HDFS and Hive using Sqoop
- Experience working on processing unstructured data using Pig and Hive
- Gained experience in managing and reviewing Hadoop log files.
- Wrote Hive UDFS to format the data.
- Involved in scheduling Oozie workflow engine to run multiple Hive and Pig jobs
- Created and maintained Technical documentation for launching HADOOP Clusters and for executing Hive queries and Pig Scripts
Environment: Hadoop, HDFS, Pig, Hive, MapReduce, Sqoop, Oozie, LINUX, and Big Data.
ConfidentialRole: Senior Java/J2EE Developer
Detroit, MI
Responsibilities:
- Actively Participated in JAD (Joint application development) sessions for requirements gathering and documenting business process.
- Used JSP, Struts, JSTLtags, Java Script for building dynamic web pages.
- Added tag libraries like Display tag, Tiles and Validator for extended flexible page design.(For more flexible page design introduced tag libraries like Display tag and Validator tags.)
- Incorporated J2EE design Patterns (Business Delegate, Singleton, Data Access Object, Data Transfer Object, MVC) for the Middle Tier development.
- Data access framework by Spring is used for automatically acquiring and releasing database resources and exception handling by spring data access hierarchy for better handling of database connections with JDBC.
- Established communication among external systems using Web Services (SOAP).
- Implemented several JUnit test cases.
- Implemented a logging application, Web Logging for better trace the data flow on application server using Log4J.
- Used Clear Case as a version control of the application with developments streams.
- Worked with team of Developers and Testers to resolve the issues with the server timeouts and database connection pooling issues.
- Initiated Profiling using RAD for finding the Objects memory leaks.
Environment:Java1.4,J2EE1.3,Struts1.1,HTML,JavaScript,JSP1.2,Servlets2.3,Spring1.2,ANT,Log4j1.2.9, PL/SQL, Oracle8i/9i, SQL Navigator5.5, WebSphere Application Server 5.1/6.0, RAD6.0, IBM Clear case.
ConfidentialRole: Java/J2EE Developer
Dover,NH
Responsibilities:
- Contributed to the Design, Created Class diagrams, Sequence diagrams and Activity Diagrams.
- Worked with cross-browser issues related to the front end that is created using JavaScript, CSS, and HTML5.
- Worked on how to produce the graphs using JavaScript.
- Created the scripting code to validate the data.
- Worked on different J2EE Design Patterns such as Front controller, Session Facade, ServiceLocator, Singleton and DAO.
- Developed/modified the bean components to incorporate new business level validations.
- Implemented SAX parsing of XML production data.
- Developed the Core Services which consume the partners Web Services and exposed them as Web services using Axis 1.2.
- Deployed Web Services in Web Sphere.
- Wrote queries, stored procedures and functions using SQL, PL/SQL
- Developed the Business logic Java components.
- Developed the email component using Java Mail to send confirmation emails to users.
- Developed the Search Widget using JSP, Struts, Tiles, JavaScript and AJAX.
- Was responsible for deploying and testing entire Web Services layer developed as a part of Core, using SoapUI.
- Involved in coding Data Layer, which is wrapper class over database.
- Generated build file using Ant 1.6.
- Responsible for performing end-to-end system testing of application writing JUnit test cases
- Developed test environment for testing all the Web Service exposed as part of the core module and their integration with partner services in Integration test.
- As part of the development team Contributed for Application Support in Soft launch and UAT phase.
Environment: Java, Servlets, JSP, JDBC, Struts, Tiles, AJAX, JAXP, XML, XSL, XSLT, SOAP,JSON, Web Services,Java Mail, Eclipse, Web Sphere, Axis, JUnit, Oracle 8, PL/SQL, Ant, Rational Rose 2000,TOAD, Edit plus, HTML, Windows NT, Unix, Java / J2EE Patterns.
Confidential
Role: Java Developer
Responsibilities:
- Responsible for coordinating on-site and off-shore development teams in various phases of the project.
- Involved in developing dynamic Jsp and doing page validations using Java Script.
- Involved in database schema design and review meetings.
- Designed a nightly build process for updating the catalogue and intimating the user of the pending authorization.
- Used automated test scripts and tools to test the application in various phases.
- Coordinated with Quality Control teams to fix issues that were identified.
- Involved in writing Stored Procedures using Oracle.
- Responsible for building projects in deployable files (WAR files and JAR files)
- Designed and developed base classes, framework classes and common re-usable components.
- Involved in performance tuning, debugging production problems during testing and deployment phases of the project
- Involved in re-factoring the existing components to meet the current application requirements
- Used various Java and J2EE APIs including JDBC, XML, Servlet, JSP, and JavaBean.
- Support Production Team members in the development and testing of production
- Implementation Plans and the Midrange group during Migrations.
- Involved in testing, maintenance and production support of the application.
- Responded to requests from Technical Team members to prepare a TAR and configured files for Production migration.
Environment: J2EE, Hibernate, JSP, Servlets, Java beans, Java Script, Oracle Application Server OC4J, JDeveloper, Apache ANT 1.6.1, Windows 2000.