SUMMARY:

• Over 10 years of IT experience in analyzing, designing, development, and maintenance of critical web based business applications and applications involving with Big Data Ecosystem, Java/J2EE.
• Over 3+ years of extensive experience with Big Data, Data Science and Data Analytical Platforms.
• Over 2+ Years of experience in Apache Spark with Scala, Python using Azure Cloud and AWS.
• Implemented Horton Works in AWS Cloud stack to reduce Overhead of Server Maintenance.
• Implemented Azure Security Deployment Life cycle across multiple Big Data Applications.
• Architected and Implemented solution to migrate existing .Net, Sql Server applications into Spark Real time systems using Scala programming.
• Implemented Data Pipelines/Automated Frameworks to ingest data from a vast range of sources - from applications (ex: salesforce), databases (ex: Oracle, Sql Server) and Splunk Repository.
• Designed and developed complete Application suite for ingesting data into HDFS and maintaining CDC (Change Data Capture) in Hive tables.
• Good exposure in writing complex Map reduce jobs, Pig Scripts and Hive data modeling.
• Proficient knowledge and hands on experience in writing shell scripts in Linux and Unix.
• Good Experience in developing Applications using Java, J2EE (Servlets, JSP, struts, spring) and Python.
• Developed web-based applications using Java API, J2EE, Web Services, both SOAP WSDL and REST, MVC framework, spring, Hibernate Struts, Oracle and Sql.
• Hands on experience in installing, configuring, and using Hadoop ecosystem components like Hadoop Map Reduce, HDFS, HBase, Oozie, Hive, Sqoop, Pig, and Flume.
• Developed Hive/MapReduce/Spark Python modules for ML & predictive analytics in Hadoop/Hive/Hue on AWS.
• Have good experience creating real time data streaming solutions and batch style large scale distributed computing applications using Apache Spark, Kafka and Flume.
• Experience in working with as well as Hosting Hadoop in cloud environments in AWS Cloud using EC2, EMR and AWS Cloud Watch.
• In-depth understanding of NoSQL databases such as HBase and Cassandra.
• Worked on the development of Dashboard reports for the Key Performance Indicators using Power BI.
• Good Exposure in providing solutions using SOA, Distributed Computing & Enterprise Service Bus.

TECHNICAL SKILLS

Hadoop Eco System: HDFS YARN, MapReduce, Pig, Hive, Sqoop, Flume and Zookeeper

Hadoop Distributions: HDP (Hortonworks), Cloudera, IBM Big Insights

Real Time/Stream processing: Apache Storm, Apache Spark

Distributed message broker: Apache Kafka

Programming Languages: Scala, Java, J2EE, XML, SQL, PL/SQL, HiveQL, Pig Latin, Python

Scripting Languages: Shell, Perl

Markup Langauages: JSON, XML, YAML

Relational Databases: DB2 V 9.0, MySQL, Microsoft SQL Server and Oracle

NoSQL Databases: HBase, Cassandra

Application/Web Servers: Apache Tomcat, JBoss, Websphere, MQ Series, Data power, Web services

Framework: JUnit and JTest, LDAP, Spark Test Base

IDE: Eclipse, IntelliJ Idea, Microsoft Visual Studio Enterprise 2015

Versioning Tools: GIT, SVN, Librarian

Workflow Scheduler System: UC4, Oozie, Azure Data Factory

PROFESSIONAL EXPERIENCE:

Confidential

Java/J2EE Developer

Responsibilities:

• Involved in Design, Development and Support phases of Software Development Life Cycle (SDLC). Used agile methodology and participated in Scrum meetings.
• Developed the application using Spring Framework that leverages Model View Layer (MVC) architecture UML diagrams like use cases, class diagrams, interaction diagrams (sequence and collaboration) and activity diagrams were used.
• Design architecture following J2EE MVC framework.
• Data from UI layer sent through JMS to Middle layer and from there using MDB message retrieves Messages and will be sent to MQSeries.

Confidential,

Principal Engineer, Big Data

Responsibilities: 

• Architecture and implemented solutions for AWS Data Lake (AWS Cloud) using Open source stack.
• Implemented Security modules for Data Platform which ensures the Authority and Authorization.
• Developed CI (Continuous Integration) solutions using Gitlab, Git and created Yaml configuration files.
• Developed Spark streaming modules for pulling data from RabbitMQ and Kafka.
• Developed Python, Spark modules for ingesting data from Splunk, External DSaaS (Data Science as a service) like Mintigo, Eloqua.
• Involved in the Cluster deployments using Automated Scala Packages.
• Experimented and hands-on various Apache projects like NiFi, Superset, Neo4J, Titan.
• Implemented solution to handle Salesforce Hard deletes there by syncing the data across environments.
• Implemented recommendation system for Big Data Infrastructure.

Environment: AWS Cloud Stack (S3, RedShift, RDS, EMR, EC2), HDP 2.5.0, Rabbit MQ, Splunk 6.6.1, Spark 2.0, Hive, Map Reduce, Spark.

Sr. Big Data Engineer

Responsibilities: 

• Developed Data Pipeline using Spark Streaming, Spark Batch and Spark Sql in pulling the data from Event Hub (Scalable Publish-Subscribe service ).
• Developed scripts to deploy the HDI(HDInsight) Clusters into Azure Cloud.
• Developed and Deployed Azure Data Factories, Azure Data sets and Data Pipelines.
• Ingested huge amount of Data from Cosmos, Azure Sql to Azure Cloud Storage.
• Implemented VSTS (Visual Studio Team Services) Continuous Delivery Pipeline for Big Data Applications.
• Developed various modules to Perform Cleansing, Aggregation and Time series generation using Scala Programming.
• Framework involving with Scope and PowerShell scripting in pulling the data from various Cosmos Streams and loading into Azure Storage System.
• Developed Spark streaming/batch applications using Java APIs and Scala programming for generating/processing data for Data Science team.
• Integrated Scala Applications with Data Processing framework and created Power BI Dashboards.
• Implemented Azure Data Catalogue(C#) for users to understand and consume data sources.
• Implemented Spark Test base and Code Coverage tools for Automated Testing.

Environment: HDI 3.4.1, Spark 1.6.1, Hive, Map Reduce, HDFS, Spark, HBase, Livy, Cosmos, Power BI, Linux

Sr. Big Data Developer,Enterprise Business Intelligence

Responsibilities: 

• Launching and Setup of Hadoop Cluster on AWS, which includes configuring different components of Hadoop.
• Experience with EC2 Instances, EMR (Elastic Map Reduce), S3, AWS Cloud watch and AWS Kinesis.
• Developed framework involving with Sqoop in pulling the data from various Relation databases (ex: Sql Server, Oracle, Netezza, Teradata) and applications into Hive tables.
• Developed Framework (Shell scripting and Sqoop commands) to import and export data into HDFS and vice versa.
• Developed Spark streaming applications using Java APIs and Scala programming for Data Science team.
• Integrated Core Java Applications with Data Processing framework.
• Creating Hive tables and working on them using Hive QL. 
• Supported the existing MapReduce Programs those are running on the cluster. 
• Experience migrating MapReduce programs into Spark transformations using Spark and Scala.
• Wrote jobs and analyzed data using Spark and Spark R.
• Wrote programs in java and at times Scala to implement intermediate functionalities like events or records count from the HBase.

Environment: HDP2.2, Hive, Sqoop 1.4.5, Cassandra, Map Reduce, HDFS, Spark, HBase, Kafka.

Big Data Engineer.

Responsibilities:

• Understanding Client’s DW Application, Interfaces and Business involved.
• Involved in creating Data Lake by extracting customer’s Big Data from various data sources into Hadoop HDFS. This included data from Excel, Web sac, databases and also log data from servers.
• Worked on data load from various sources i.e., Oracle, MySQL, DB2, MS SQL Server, Cassandra, MongoDB, Hadoop using Sqoop and Python script.
• Developed MapReduce programs to cleanse the data in HDFS obtained from heterogeneous data sources to make it suitable for ingestion into Hive schema for analysis.
• Work closely with architect and clients to define and prioritize their use cases and iteratively develop APIs and architecture.
• Used Hive data warehouse tool to analyze the unified historic data in HDFS to identify issues and behavioral patterns.
• Worked with Business Developer team in generating customized reports and ETL workflows in Data Stage.

Environment: HDP2.0, Hive, Sqoop 1.4, Oozie 3.2.0, Cassandra, Map Reduce, HDFS, Hbase, Splunk, Kafka.

Big Data Consultant

Environment: Apache Hadoop 2.0, Hive 0.9, Pig 0.10, Oozie 3.2.0, Map Reduce, HDFS, Hbase, Python, Eclipse.

Responsibilities:

• Understanding Danske DW Application, Interfaces and Business involved.
• Extensively used ETL processes to prepare consolidated Marts in relation to business requirements.
• Involved in extracting customer’s Big Data from various data sources into Hadoop HDFS. This included the data from Excel, Web sac, databases and log data from servers.
• Configured the Hadoop Cluster in Local (Standalone), Pseudo-Distributed, Fully-Distributed Mode.
• Involved in Cassandra database design, integration and implementation.
• Back-end Java developer for Data Management Platform (DMP) and building RESTful APIs in front of different types of NoSQL storage engines allowing other groups to quickly meet their Big Data needs.
• Used the machine learning libraries of Mahout to perform advanced statistical procedures like clustering and classification to determine the probability of payment default.
• Used Hive data warehouse tool to analyze the unified historic data in HDFS to identify issues and behavioral patterns.
• Partitioned and queried the data in Hive for further analysis by the BI team.

Hadoop Developer / Data Power Developer

Responsibilities:

• Worked on Hadoop cluster set up, administration, maintenance, monitoring and support.
• Analyzing requirements for Optimization and tuned the Hadoop environment to meet the business requirements.
• Designed and documented REST/HTTP APIs, including JSON data formats and API versioning strategy.
• Loaded the customer profiles data, customer claims information, billing information etc. onto HDFS using Sqoop and Flume.
• Gathered requirements from Engineering (Statistical analysis) and Finance (Financial Reporting) teams to design solutions on the Hadoop ecosystem.

Environment: IBM Big insights, Data Power, Hadoop, HDFS, Pig, Hive, MapReduce, Sqoop, LINUX.