PROFESSIONAL SUMMARY:

• An accomplished, results - driven IT professional with around 7.8 years of experience in the IT industry and 2.4+ years of experience in the Hadoop ecosystem and Apache Spark.
• Experienced with Hadoop distributions like Cloudera, Hortonworks and depth knowledge on Hadoop architecture including MapReduce, Yarn, Hive, Sqoop, HDFS, Spark, Oozie, Zookeeper in a range of industries such as Financing sector and Health care.
• Experience in importing and exporting data using Sqoop from HDFS to Relational Database Systems and vice-versa.
• Expertise in working with RDBMS databases like MySQL and SQL Server.
• Rich experience in automating Sqoop and Hive queries using Oozie workflow.
• Experience in strong and analysing data using HiveQL, SparkQL and MapReduce programs.
• Experience with AWS components like Amazon EC2 instances, S3 buckets and Cloud Formation templates
• Experienced in Developing Spark application using Spark Core, Spark SQL and Spark Streaming API’s.
• Analysed the data by performing Hive queries and used HIVE UDF’s.
• Tuning, and Monitoring Hadoop jobs and clusters in a production environment.
• Experience in designing tables and views for reporting using Impala.
• Experienced in working with different file formats - Avro, Parquet.
• Experience in different compression techniques like Gzip, LZO, Snappy and Bzip2.
• Experience in web development using Java and front-end using JavaScript, HTML, XML and CSS.
• Proficient in Automation (Selenium IDE), Manual (Blackbox), Database, functional and non-functional, negative, regression, system integration (SIT), acceptance (UAT), security, browser compatibility and system testing.
• Strong Communication skills of written, oral, interpersonal and presentation
• Proven ability to interface with stakeholders to ensure solutions to meet customer requirements.
• Adept at managing multiple projects simultaneously. Strategic planning through operational and technology design for optimizing Software development, architectural design, creating various proof of concepts.
• Exceptional ability to learn new technologies and to deliver outputs in short deadlines.

TECHNICAL SKILLS:

Hadoop: Hadoop 2.2, HDFS, MapReduce, Hive, Sqoop, Zookeeper, Yarn, Spark, Kafka, Hbase, Flume, Impala and Hue. Hadoop management & Security Cloudera Manager, Hortonworks

Web Technologies: HTML, XHTML, XML, XSL (XSLT, XPATH), XSD, CSS, JavaScript

Database: Microsoft SQL Server, MySQL, SQL, RDBMS, Familiar with NoSQL- Hbase

Programming Languages: Java, J2EE

Web Servers: Apache Tomcat 5.x

OS/Platforms: Mac OS X 10.9.5, Windows2008/Vista/2003/XP/2000/NT, Linux, Unix

Client side: JavaScript, CSS, HTML

Testing & Case Tools: Junit, Rational Clear Case, JBuilder, Selenium

Cloud Components: AWS (S3 Buckets, Ec2, Cloud Formation)

PROFESSIONAL EXPERIENCE:

Confidential, IL

Hadoop Developer

Responsibilities:

• Used SQOOP to import data from RDMS sources into HDFS using Avro File format.
• Performed transformation, cleaning and filtering on imported data using Hive, Spark and loaded data into HDFS.
• Used Hive to Partition and Bucketing, for optimizing data storage and compute-various metrics for reporting
• Developed Spark core and Spark SQL scripts using Scala for faster data processing.
• Converted Hive queries into Spark transformations using Spark RDDs.
• Responsible for creating shell script to run Hive jobs.
• Worked with different file formats like Avro and Parquet.
• Worked with HBase to conduct quick look ups (Update, Inserts and Deletes) in Hadoop.
• Used Oozie workflow to automate all the jobs.
• Exported the analysed data into Tableau Server using SCOOP for visualization and to generate reports for BI team.
• Used Git hub as code repository and version control.
• Used Agile methodology in developing the application, participated in stand-up meetings and Sprints.
• Working on a sample POC to collect large amount of log data using Kafka in HDFS for further analysis.
• Continuous monitoring and managing the Hadoop cluster through Cloudera Manager.

Environment: HDFS, Yarn, MapReduce, Hive, Sqoop, Oozie, HBase, Kafka, Impala, Spark SQL, Eclipse, Oracle, Teradata, Cloudera.

Confidential, NC

Hadoop Developer

Responsibilities:

• Involved in the Complete Software development life cycle (SDLC) to develop the application.
• Worked with the Data Science team to gather requirements for various data mining projects.
• Validating the source file for Data Integrity and Data Quality by reading header and trailer information and column validations.
• Used Hive to do transformations, joins, filter and some pre-aggregations before storing the data onto HDFS.
• Experience in importing data from various data sources like MySQL and Netezza using Sqoop, performed transformations using Hive, and loaded data back into HDFS.
• Import and export data between the environments like MySQL, HDFS and deploying into productions.
• Worked on partitioning and used bucketing in HIVE tables and setting tuning parameters to improve the performance.
• Experience in Oozie workflow scheduler template to managevarious jobs like Sqoop, MR, Hive, Shell scripts.
• Involved in importing and exporting data from Sqoop using Spark.
• Actively participating in the code reviews, meetings and solving any technical issues.
• Implemented SparkSQL to access hive table into Spark for faster processing of data.

Environment: Java API,Hadoop2.x, Spark, Cloudera CDH 5.x, Kafka, HDFS, Hive 0.12, MapReduce, Sqoop 1.4.3, Oozie, Agile, JIRA

Confidential

System Engineer

Responsibilities:

• Writing Test cases using element locators, web driver methods, java programming features.
• Responsible in Development, Testing, Maintenance, and Production Support of ALM application.
• Involved in regression testing of RESTful web services at the different phases of this application.
• Involved in architecting, developing and testing SOAP based web services.
• Received appreciations for scaling up quickly and taking up extra modules after having completed the assigned modules before time.
• Involved in Analysis of requirements, low level designing, implementation of the services, Bug Fixing, System & Integration Testing, Onsite Coordination and undertaking the complete responsibility of a few of requirements.
• Involved in designing, coding, Peer Code Reviews and was also responsible for performing functional, technical walk through, code reviews and debugging of various modules.
• Done Functional testing, Regression testing, Integration testing, Database testing, user interface and System testing.
• Preparing test reports for all the releases.
• Was one of the points of contact for any production issue for this application.
• Worked on Agile methodologies.

Environment: Selenium, Java, MySQL, SVN, Javascript, SQL Server 2008, XML, Excel, HTML, SOAP web services, SOAP UI Tool, Quality Assurance

Confidential

Process Associate

Responsibilities:

• Prepared test plans based on the requirement walk through given by the client.
• Discussed issues with the major stakeholders on calls and raised defects whenever required.
• Designed and executed test cases in ALM.
• Prepared test execution summary report for the completed
• Prepared in-house defect tracking tool using Java, HTML, CSS,

Environment: Selenium, Java, MySQL, SVN, Quality Assurance