SUMMARY:

• 8+ Years of working experience in IT Development including around 3 years of experience in Hadoop Technologies like HDFS, MapReduce, Yarn, Pig, Hive, HBase, Sqoop, Oozie, Flume, Spark and Zookeeper.
• Well experienced Mapper, Shuffling, Practitioner, and Reducer and Combiner process along with custom Partitioning for efficient Bucketing.
• Analyzing and recommending the Big Data solutions.
• Importing and exporting data into HDFS and Hive using Sqoop/FTP.
• Extensive experience in using Flume to transfer log data files to Hadoop Distributed File System (HDFS).
• Used tools like SQOOP, Kafka to ingest data into Hadoop.
• Experience in writing test cases and unit testing.
• Experience in developing Shell scripts and Python Scripts for system management.
• Extensive experience in Extraction, Transformation and Loading (ETL) of data from Multiple sources into Data Warehouse and Data Mart.
• Developed Pig Latin scripts using operators such as LOAD, STORE, DUMP, FILTER, DISTINCT, FOREACH, GENERATE, GROUP, COGROUP, ORDER, LIMIT, UNION, SPLIT to Extract data from data Files to load into HDFS.
• Experience in working with ApacheSqoop in efficiently transferring bulk data between Apache Hadoop and structured data stores such as relational databases.
• Hands on experience in developing the applications with Java, J2EE, J2EE - Servlets, JSP, EJB, SOAP, Web Services, JNDI, JMS, JDBC2, Hibernate, Struts, spring, XML, HTML, XSD, XSLT, PL/SQL, Oracle10g and MS-SQL Server RDBMS.
• In depth knowledge of Object Oriented programming methodologies (OOPS) and object Oriented features like Inheritance, Polymorphism, Exception handling and Templates and Development experience with Java technologies.
• Implemented SOA architecture with Web Services using SOAP and deploying them on Apache Tomcat server.
• Expertise in shell scripting on UNIX platform.
• Hands on experience in designing and developing distributed systems.
• Experience in installing, administering, and supporting Linux operating systems and Hardware in an enterprise environment.
• Expertise in typical system administration and programming skills such as storage Capacity Management, performance tuning.
• Experience in designing and developing applications spanning full life cycle of software Development (SDLC) from writing functional specification, designing, implementing Documentation, unit testing and support.
• Handled several techno-functional responsibilities including estimates, identifying Functional and Technical gaps, requirements gathering, designing solutions, development, Developing Documentation and production support.
• An individual with excellent interpersonal and communication skills, strong business Acumen, Creative problem solving skills, technical competency, team-player spirit, and Leadership skills.

TECHNICAL SKILLS:

Programming Language: Scala, Java.

Hadoop/Big Data Stack: Hadoop Map Reduce, Cloudera Manager 5.4.x, Oozie, Pig, Flume NG, Kafka, Sqoop, Spark, Hive, Yarn, Hadoop Eco System HDFS, Hive, Pig, Sqoop, Oozie, Flume, Tools, Technologies and Utilities Java, Servlets, JDBC, JSP, Swings, MVC, Hibernate, Struts 1.x, Web Services, Junit, Maven.

Databases: Oracle, SQL Server

NO SQL: HBase, Cassendra

PROFESSIONAL EXPERIENCE:

Sr. Hadoop Developer

Confidential - Minneapolis, MN

Responsibilities:

• Was hired for the Hadoop Developer role and the primary responsibilities include Design, implement and maintain an application that receives a transaction-based and data Generated from Kroger retail stores across all US locations.
• Job duties involved the design, development of various modules in Hadoop Big Data Platform and processing data using MapReduce, Hive, Pig, Scoop and Oozie.
• Write and test the MapReduce code to do aggregations on identified and validated data. Processed Mobile Offers data for Kroger stores in USA across various locations and Ingested the data into Hadoop HIVE tables.
• Scheduled Multiple Map Reduce jobs in Oozie. Involved in extracting the promotions data for stores in USA by writing the map reduce jobs and automating it with UNIX shell script.
• Prepared Use Cases, UML diagrams and Vision diagrams.
• Responsible for working with different teams in building Hadoop Infrastructure
• Experience using Cloudera and Horton Works platform and their eco systems. Hands on Experience in installing, configuring and using ecosystem components like Hadoop MapReduce, HDFS, Pig, Hive, Sqoop and Flume.
• Gathered business requirements in meetings for successful implementation and POC and moving it to Production.
• Implemented POC to migrate map reduce jobs into Spark RDD transformations using Scala.
• Implemented different machine learning techniques in Scala using Scala machine learning Library.
• Developed Spark applications using Scala for easy Hadoop transitions.
• Successfully loaded files to Hive and HDFS from Oracle, Netezza and SQL Server using SQOOP
• Uses Talend Open Studio to load files into Hadoop HIVE tables and performed ETL Aggregations in Hadoop HIVE.
• Designing & Creating ETL Jobs through Talend to load huge volumes of data into Cassandra, Hadoop Ecosystem and relational databases.
• Developed some machine learning algorithms using Mahout for data mining for the data Stored in HDFS
• Used Flume extensively in gathering and moving log data files from Application Servers to a central location in Hadoop Distributed File System (HDFS)
• Responsible for cluster maintenance, adding and removing cluster nodes, cluster
• Monitoring and troubleshooting, manage and review data backups, manage and review Hadoop log files.
• Extensively used Hive/HQL or Hive queries to query data in Hive Tables
• Introduced Tableau Visualization to Hadoop to produce reports for Business and BI team.
• Creating UDF functions in Pig & Hive and applying partitioning and bucketing techniques in Hive for performance improvement
• Creating indexes and tuning the SQL queries in Hive and Involved in database connection By using Sqoop
• Defined a standard layout and standard set of attributes that are a part of all application Logs
• Used File System Check (FSCK) to check the health of files in HDFS
• Used Sqoop to import data from SQL server to Cassandra
• Integration of Cassandra with Talend and automation of jobs.
• Maintenance and Troubleshooting in Cassandra Cluster
• Monitored Nightly jobs to export data out of HDFS to be stored offsite as part of HDFS Backup.
• Used Pig for analysis of large data sets and brought data back to HBase by Pig.
• Scheduled, monitored and debugged various MapReduce, Pig, Hive jobs using Oozie Workflow.
• Closely worked with Hadoop security team and infrastructure team to implement security.
• Implemented authentication and authorization service using Kerberos authentication Protocol.
• Created schema and data base objects in HIVE.
• Developed Unix Scripts to data loading and automation
• Involved in of big data ecosystem to end-users.

Environment: Hadoop 1.2.1, MapReduce, Sqoop 1.4.4, Hive 0.10.0, Flume 1.4.0, Oozie 3.3.0, Pig 0.11.1,Hbase 0.94.11, Scala, Zookeeper 3.4.3, Talend Open Studio 5.6.0, Talend 5.5, Oracle 11g/10g, Apache Cassandra, SQL Server 2008, MySQL 5.6.2, Java, SQL, PL/SQL, UNIX shell script, Eclipse Kepler IDE, Microsoft Office 2007, MS Outlook 2007.

Hadoop Developer

Confidential - Boston, MA

Responsibilities:

• Loading files to HDFS and writing Hive queries to process required data.
• Loading data to hive tables and writing queries to process.
• Involved in loading data from LINUX file system to HDFS.
• Load and transform large sets of structured, semi structured and unstructured data.
• Experience in managing and reviewing Hadoop log files.
• Worked on Hive for exposing data for further analysis and for generating transforming Files from different analytical formats to text files.
• Importing and exporting data into HDFS and Hive using Sqoop.
• Involved in creating Hive tables, loading with data and writing hive queries that will run Internally in MapReduce way.
• Worked on configuring multiple MapReduce Pipelines, for the new Hadoop Cluster.
• Performance tuned and optimized Hadoop clusters to achieve high performance.
• Written Hive queries for data analysis to meet the business requirements.
• Monitored System health and logs and respond accordingly to any warning or failure Conditions.
• Responsible to manage the test data coming from different sources.
• Involved in scheduling Oozie workflow engine to run multiple Hive and pig jobs
• Weekly meetings with technical collaborators and active participation in code review Sessions with senior and junior developers.
• Created and maintained Technical documentation for launching Hadoop Clusters and for Executing Hive queries and Pig Scripts
• Implemented schedulers on the Job tracker to share the resources of the cluster for the MapReduce jobs given by the users.
• Extensive hands on experience in Hadoop file system commands for file handling Operations.

Environment: Hadoop, Map Reduce, HDFS, Hive 0.10.1, Java, Hadoop distribution of Cloudera, Pig 0.11.1, HBase 0.94.1, Linux, Sqoop 1.4.4, Kafka, Zookeeper 3.4.3, Oozie 3.3.0, Tableau.

Hadoop Developer

Confidential - Mountain View, CA

Responsibilities:

• Installed and configured Hive on the Hadoop cluster.
• Work closely (face-to-face) with the Accountants, Financial Analysts, Data Analysts, and Data Scientists, Statisticians, Compliance, Sales, Marketing, Pricing Strategists, Product
• Development and Business Analysts to create solutions for their issues.
• Developed Simple to complex Map/reduce streaming jobs using Java language that are Implemented Using Hive and Pig.
• Optimized Map/Reduce Jobs to use HDFS efficiently by using various compression
• Mechanisms.
• Effectuate server relocation projects and participate in 3rd party audits when working With TurboTax
• Handled importing of data from various data sources, performed transformations using Hive, MapReduce, loaded data into HDFS and Extracted the data from MySQL into HDFS Using Sqoop.
• Analyzed the data by performing Hive queries (HiveQL) and running Pig scripts (Pig Latin) to study customer behavior.
• Tested Apache(TM) Tez, an extensible framework for building high performance batch and interactive data processing applications, on Pig and Hive jobs.
• Used Impala to query the Hadoop data stored in HDFS.
• Working as a Hadoop consultant for converting the Oracle Stored Procedures based Data
• Warehouse Solution to Hadoop based Solution.
• Filtered, transformed and combined data from multiple providers based on payer filter criteria using custom Pig UDFs.
• Used the RegEx, JSON and Avro SerDe's for serialization and de-serialization packaged
• With Hive to parse the contents of streamed log data and implemented Hive custom UDF's.
• Extensively used InformaticaPower Center in end-to-end of Data warehousing ETL
• Routines, which includes writing custom scripts, data mining and data quality process.
• Implemented business logic by writing UDFs in Java and used various UDFs from Piggybanks and other sources.
• Continuous monitoring and managing the Hadoop cluster using Cloudera Manager.
• Experience in using Sqoop to migrate data to and fro from HDFS and My SQL or Oracle
• Deployed Hive and HBase integration to perform OLAP operations on HBase data.
• Worked with application teams to install operating system, Hadoop updates, patches, version upgrades as required.
• Exported the analyzed data to the relational databases using HIVE for visualization and to generate reports for the BI team.
• Perform data analysis on large datasets and present results to risk, finance, accounting, and pricing, sales, marketing, and compliance teams.
• Experienced on loading and transforming of large sets of structured, semi structured and unstructured data.
• Analyzed large amounts of data sets to determine optimal way to aggregate and report on it.

Environment: Hadoop 0.20.2 - PIG, Hive, Cloudera manager, 30 Node cluster with Linux-UbuntuKafka, Zookeeper, Sql, Oracle 11g/10g, Eclipse IDE, Tableau, Oozie, Sqoop

Sr. Java Developer

Confidential - Charlotte, NC

Responsibilities:

• Analyzed and reviewed client requirements and design.
• Followed agile methodology for development process.
• Developed presentation layer using HTML5, and CSS3, Ajax.
• Developed the application using Struts Framework that uses Model View Controller (MVC)
• Architecture with JSP as the view.
• Extensively used Spring IOC for Dependency Injection and worked on Custom MVC
• Frameworks loosely based on Struts.
• Used Restful Web services for transferring data between applications.
• Configured spring with ORM framework Hibernate for handling DAO classes and to bind
• Objects to the relational model.
• Adopted J2EE design patterns like Singleton, Service Locator and Business Facade.
• Developed POJO classes and used annotations to map with database tables.
• Used Java Message Service (JMS) for reliable and asynchronous exchange of important
• Information such as Credit card transactions report.
• Used Multi-Threading to handle more users.
• Developed Hibernate JDBC code for establishing communication with database.
• Worked with DB2 database for persistence with the help of PL/SQL querying.
• Used SQL queries to retrieve information from database.
• Developed various triggers, functions, procedures, views for payments.
• XSL/XSLT is used for transforming and displaying reports.
• Used GIT to keep track of all work and all changes in source code.
• Used JProfiler for performance tuning.
• Wrote test cases which adhere to a Test Driven Development (TDD) pattern.
• Used Junit, a test framework which uses annotations to identify methods that specify aTest.
• Used Log 4J to log messages depending on the messages type and level.
• Built the application using MAVEN and deployed using Web Sphere Application server.

Environment: Java 8, Spring framework, Spring Model View Controller (MVC), Struts 2.0, XML, Hibernate 3.0,UML, Java Server Pages (JSP) 2.0, Servlets 3.0, JDBC4.0, Junit, Log4j, MAVEN, Win 7, HTML, REST Client Eclipse, Agile Methodology, Design Patterns, Web Sphere 6.1.Java/J2EE Developer.

Java Developer

Confidential

Responsibilities:

• Implemented Microsoft Visio and Rational Rose for designing the Use Case Diagrams, Class
• Model, Sequence diagrams, and Activity diagrams for SDLC process of the application.
• Deployed GUI pages by using JSP, JSTL, HTML, DHTML, XHTML, CSS, JavaScript, AJAX
• Configured the project on Web Sphere 6.1 application servers
• Implemented the online application by using Core Java, Jdbc, JSP, Servlets and EJB 1.1,Web Services, SOAP, WSDL
• Communicated with other Health Care info by using Web Services with the help of SOAP,WSDL JAX-RPC
• Used Singleton, factory design pattern, DAO Design Patterns based on the applicationrequirements
• Used SAX and DOM parsers to parse the raw XML documents
• Used RAD as Development IDE for web applications.
• Preparing and executing Unit test cases
• Used Log4J logging framework to write Log messages with various levels.
• Involved in fixing bugs and minor enhancements for the front-end modules.
• Doing functional and technical reviews
• Maintenance in the testing team for System testing/Integration/UAT
• Guaranteeing quality in the deliverables.
• Conducted Design reviews and Technical reviews with other project stakeholders.
• Was a part of the complete life cycle of the project from the requirements to theproduction support.
• Created test plan documents for all back end database modules.
• Implemented the project in Linux environment.
• Environment: JDK 1.5, JSP, Web Sphere, JDBC, EJB2.0, XML, DOM, SAX, XSLT, CSS, HTML,
• JNDI, Web

Environment: WSDL, SOAP, RAD, SQL, PL/SQL, JavaScript, DHTML, XHTML,, Java Mail, PL/SQL Developer, Toad, POI Reports, Windows XP, Red Hat Linux.

Java Developer

Confidential

Responsibilities:

• Responsible for writing functional and technical documents for the modules developed.
• Extensively used J2EE design Patterns.
• Used Agile/Scrum methodology to develop and maintain the project.
• Developed and maintained web services using XMPP and SIP protocols.
• Developed business logic using Spring MVC.
• Developed DAO layer using Hibernate, JPA, and Spring JDBC.
• Used Oracle 10g as the database and used Oracle SQL developer to access the database.
• Used Eclipse Helios for developing the code.
• Used Oracle SQL developer for the writing queries or procedures in SQL.
• Implemented Struts tab libraries for HTML, beans, and tiles for developing User Interfaces.
• Extensively used Soap UI for Unit Testing.
• Involved in Performance Tuning of the application.
• Used Log4J for extensible logging, debugging and error tracing.
• Used Oracle Service Bus for creating the proxy WSDL and then provide that to consumers
• Used JMS with Web Logic Application server.
• Used UNIX scripts for creating a batch processing scheduler for JMS Queue.
• Need to discuss with the client and the project manager regarding the new developmentsand the errors.
• Documented all the modules and deployed on server in time.
• Involved in Production Support and Maintenance for Application developed in the Red Hat Linux Environment.

Environment: Java 1.5, Spring, Hibernate, XML, XSD, XSLT, WSDL, Web services, XMPP, SIP, JMS, SOAP UI, Eclipse, IBM-UDB, Web logic, Oracle 10g, Oracle SQL developer.