SUMMARY:

• Around 8 years of experience on gathering System Requirements, Analyzing the requirements, Designing and developing systems including 3+ years in Big Data.
• Excellent Knowledge in understanding Big Data infrastructure, distributed file systems - HDFS, parallel processing - MapReduce framework and complete Hadoop ecosystem - Hive, Pig, Sqoop, Hbase, NoSQL, Oozie, Flume and Kafka.
• In depth understanding of Hadoop components such as Resource Manager, Node Manager, Application Master, Application Manager, Data Node and ZooKeeper.
• Experienced Hadoop/Java developer having end to end experience in developing applications in Hadoop ecosystem.
• Experience in loading streaming data into HDFS and analyzing the data with tools like Hive.
• Experience in data management and implementation of Big Data applications using Hadoop frameworks.
• Hands on experience in application development using Java, RDBMS, and Linux/Unix shell scripting.
• Knowledge in job/workflow scheduling and monitoring tools like Oozie.
• Experience in Hadoop 2.0 (MRv2) YARN architecture.
• Experience in using Flume to load the log data from multiple sources directly into HDFS.
• Experience in using different file formats like Sequence, AVRO, ORC, and Parquet.
• Experienced with improving data cleansing process using Pig Latin operations, transformations and join operations files and CSV and using different compression Techniques.
• Expertise with optimizing query performance and data load times in PIG, Hive and Map Reduce applications.
• Experience in processing data using the in-memory computing capabilities like Apache Spark written in Python.
• Experience in using search engine frameworks like Solr and Elastic Search.
• Experience in building dataflows using NiFi.
• Experience in implementing security on Hadoop cluster using Ranger.
• Experience in working on clusters secured by Kerberos.
• Experience in working with Apache Spark streaming API in the active cluster environment.
• Experienced in extending Hive and Pig core functionality by writing custom UDFs using Java.
• Experience with agile/scrum methodologies to iterate quickly on product changes, developing user stories and working through backlog.
• Experience using tools like JIRA and Confluence.
• Experience in understanding the client’s Big Data business requirements and transform it into Hadoop centric technologies.
• Good knowledge of all phases of the Iterative Software Development Life Cycle (SDLC).

TECHNICAL SKILLS:

Hadoop: HDFS, Map-Reduce, Hive, Pig, Sqoop, Flume, Kafka, Hbase, Phoenix, Oozie

Languages: Core Java, J2EE

Web Technologies: Servlets, JSP, JDBC, RMI

Scripting: HTML, JavaScript, AJAX, DHTML, XML, Shell and Python scripting

Frameworks: Spring2.x, Hibernate

Web Services: JAX-WS, JAX-B, WSDL, UDDI, SOAP

Application/Web Servers: IBM WebSphere 6.1, Apache Tomcat

RDBMS/Databases: Oracle10g and MS-SQL Server

Operating Systems: UNIX, Linux

Build Tools: ANT, Maven.

Version Control Tools: Subversion, CVS, and GIT

Continuous Integration: Jenkins

PROFESSIONAL EXPERIENCE:

Confidential, NY

Hadoop Developer

Roles and Responsibilities:

• Collaborating with business users to contribute to the analysis of functional requirements.
• Developed and validated the parallel ingestion mechanism to handle incremental loads, incremental updates to existing records.
• Converted complex oracle queries into optimized hive queries and configured them to run the queries with TEZ execution engine.
• Converted an existing standalone python EP calculator (Catastrophe modelling) algorithm into Map Reduce streaming job by passing on the mapper and reducer codes, which are written in python.
• Developed an end-to-end workflow which polls a SFTP server for data, creates respective hive tables from the layouts provided, processes the fixed width input file into delimited output files, which are finally ingested into the respective hive tables.
• Developed a multiple output Map Reduce job to process COBOL fixed width files into delimited text files.
• Integrated a masking logic into the Map Reduce code to mask PII data in both structured and unstructured data.
• Worked on various POC’s to implement the functionalities of the Waterline, Dataguise, Data Robot, Cassandra and Zaloni (Bedrock and Mica) tools.
• Worked on implementing an Enterprise Data Lake which includes File validation, Data profiling, Data Quality check and provisioning Data to user workspaces.
• Developed an end-to-end workflow to build a real time dashboard using Kibana, Elastic Search, Hive and Flume.
• To work closely with the release team to address issues if any, during the release work activities.
• Using Oozie for designing workflows and scheduling various jobs in the Hadoop ecosystem.
• Developing Sqoop scripts for the extractions of data from various RDBMS databases into HDFS.
• Developing scripts to automate the workflow of various processes using python and shell scripting.
• Developing Map Reduce programs in java for applying business rules on the data and optimizing them using various compression formats and combiners.

Technologies Used: JDK1. 7, HDP 2.3, Red Hat Linux, HDFS, Map-Reduce, Hive, Pig, Sqoop, Flume, Zookeeper, Oozie, Netezza, HBase, Phoenix, Kafka, Solr, Elastic Search.

Confidential, NY

Hadoop Developer

Roles and Responsibilities:

• Optimized Map Reduce Jobs to use HDFS efficiently by using various compression mechanisms.
• Expertise in using Data organizational design patterns in Map Reduce to convert business data into custom format.
• Implemented Custom JOINS to create tables containing the records of Items using Spark SQL.
• Expertise in optimization of MapReduce algorithms using Combiners, Partitioners and Distributed Cache to deliver best results.
• Experienced with handling different compression codec's like LZO, GZIP, and Snappy.
• Experienced with handling data from different sources at a time to reducer using Object Writable in Mapreduce programs.
• Analyzed the data by performing Hive queries and running Pig scripts.
• Created and worked Sqoop jobs with incremental load to populate Hive External tables.
• Developed Hive scripts for end user / analyst requirements to perform ad hoc analysis.
• Expert in optimizing performance in hive using partitions and bucketing concepts.
• Experienced with handling different optimization techniques in Hive joins.
• Solved performance issues in Hive and Pig scripts with understanding of Joins, Group and aggregation and how does it translate to MapReduce jobs.
• Developed UDFs in Java as and when necessary to use in HIVE queries
• Developed Oozie workflow for scheduling and orchestrating the ETL process.
• Knowledge on visualization tools like Tableau.
• Experienced with writing build jobs using Maven anBi d integrate that with Jenkins.

Technologies Used: JDK1.6, Red Hat Linux, HDFS, Map-Reduce, Hive, Pig, Sqoop, Flume, Zookeeper, Oozie, Netezza, Teradata, DB2, HBase, NoSQL,Hbase.

Confidential, Jersey City, NJ

Java Developer

Roles and Responsibilities:

• Developed Admission & Census module, which monitors a wide range of detailed information for each resident upon pre-admission or admission to your facility.
• Involved in comprehensive library of problems, goals and approaches. You have the option of tailoring (adding, deleting, or editing problems, goals and approaches) these libraries and the disciplines you will use for your care plans.
• Involved in development of General Ledger module, which streamlines analysis, reporting and recording of accounting information. General Ledger automatically integrates with a powerful spreadsheet solution for budgeting, comparative analysis and tracking facility information for flexible reporting.
• Developed UI using HTML, JavaScript, and JSP, and developed Business Logic and Interfacing components using Business Objects, XML, and JDBC.
• Designed user-interface and checking validations using JavaScript.
• Managed connectivity using JDBC for querying/inserting & data management including triggers and stored procedures.
• Developed various EJBs for handling business logic and data manipulations from database.
• Involved in design of JSP’s and Servlets for navigation among the modules.
• Designed cascading style sheets and XML part of Order entry Module & Product Search Module and did client side validations with java script.

Technologies Used: J2EE, Java/JDK, JDBC, JSP, Servlets, JavaScript, EJB, JNDI, JavaBeans, XML, XSLT, Oracle 9i, Eclipse, HTML/ DHTML, SVN.

Confidential, Jersey City, NJ

Java Developer

Roles and Responsibilities:

• Designed the application UML Class Diagrams, Sequence diagrams using RSA.
• Involved in creating technical design for the project along with core team members.
• Interaction with business requirements team and developed business process
• Developed task utility services necessary for generating documents
• Develop utility classes using TDD
• Developed code to validate the state of Business Objects (POJO’S) using singleton pattern
• Used spring DI for object creation throughout the project
• Designed XML files to implement most of the wiring need for Hibernate annotations and spring configurations.

Technologies Used: JDK1.5, J2EE, HTML, JSP, Springs (IOC and AOP), JSF, Web Sphere, Hibernate, JavaScript, Maven, CSS, DB2, XML, UML, XSLT, FTP, HTTP, RSA7.0, JUnit, Log4j, Apache Velocity, JMS, JDBC, EJB and Web Services

Confidential

Java Developer

Roles and responsibilities:

• Data layer integration with relational databases like Oracle.
• Handling queries and data wrappers for relational databases using JDBC driver.
• Design, Develop and Handle database table schemas as per business requirements.
• Wrote Servlets to fetch and manipulate data from the database.
• Data service development using Java and Oracle.
• Extensively used Java Collection Framework.
• Utilized Servlets to handle various requests from the client browser and send responses
• Based on these definitions, the tool generates the XML file, which in turn is used to create Java code and HTML templates
• Development of services using Java, Oracle developing Java applications and deployed in Tomcat 5.0 web server
• Involved in the development of user interfaces

Technologies Used: Core Java, Servlets, JSP, HTML, Ajax, Sql.