SUMMARY:

• O ver 8 years of experience with core strengths in Banking and Insurance industry.
• Designed and implemented data ingestion techniques for real time data coming from various data sources
• Built predictive analytics models to generate actionable insights
• Performed business analysis by defining the requirements. Designed Functional specifications and technical specifications
• Spark: Spark Core, Spark SQL, Spark Streaming
• Data Collection and exploration (Python) + Data Visualization
• Hive performance tuning
• Highly proficient with SQL query - based applications
• Productionizing Big Data Applications
• PySpark API’s working knowledge
• Schedulers and streaming applications knowledge
• Experience in Importing and exporting data from RDBMS into HDFS and vice-versa using Sqoop
• Managed multiple tasks and worked under tight deadlines and in fast pace environment
• Possess good communication, interpersonal, analytical skills and a go-getter personality

TECHNICAL SKILLS:

Analytical Tools: Programming Data Mining Python Big Data Spark AWS NoSQL

Databases: Jupyter Notebook, Tableau, Zeppelin Python, SCALA, RApriori, KNN, Na ve Bayes, C4.5m Data Manipulation, Numpy, Pandas, Matplotlib, scikit-learn Hadoop, Hive, Sqoop, pig, Impala, HDFS, Kafka Spark Core, Spark SQL, Spark Streaming, PySpark, Spark Structured Streaming, GraphX, MLlib EMR, S3, Kinesis MongoDB, Cassandra, HBase SQL Server, Oracle, Teradata

PROFESSIONAL EXPERIENCE:

Confidential, CHICAGO, IL

BIG DATA ENGINEER

Responsibilities:

• Creating the tables in Hive and integrating data between Hive & Spark
• Performed hive queries by extracting data from Hadoop into Hive
• Imported data’s from MYSQL to HBase and performed Hive Queries based on requirements
• Developed python scripts to collect data from source systems and store it on HDFS to run analytics
• Created Hive Partitioned and Bucketed tables to improve performance
• Created Hive tables with User defined functions
• Involved in code review and bug fixing for improving the performance
• Worked on the core and Spark SQL modules of Spark extensively using programming languages Python and Scala
• Developed Spark core and Spark SQL scripts using Scala for faster data processing
• Perform extensive studies of different technologies and capture metrics by running different algorithms
• Design and implement data ingestion techniques for real time data coming from various source systems
• Defining the data layouts and rules and after consultation with ETL teams
• Worked on UNIX environment to automate day to day activities
• Worked on Spark Streaming to consume the data from Kafka
• Worked on NoSQL databases like MongoDB
• Worked on new tools integration with Big Data ecosystem like AtScale, NiFi
• Worked in aggressive AGILE environment and participated in daily Stand-ups/Scrum Meetings

Environment: Python, HDFS, Hadoop, PL/SQL, Hive, Spark, Scala, AGILE, Spark SQL, PySpark, Kafka, Sqoop, MongoDB, MYSQL, Unix

Confidential CHICAGO, IL

BIG DATA ENGINEER

Responsibilities:

• Data analysis using open source tools
• Defining the requirements for data lakes/pipe lines
• Transforming the data using Spark applications for analytics consumption
• Design and implement data ingestion techniques for real time data coming from various source systems
• Creation of regulatory reports and analysis. Defining the data streams
• Used Hive to analyze the partitioned and bucketed data and compute various metrics for reporting
• Importing and exporting data into HDFS and Hive using Sqoop
• Written Hive queries for data analysis to meet the Business requirements
• Experience in managing and reviewing Hadoop log files
• Worked in aggressive AGILE environment and participated in daily Stand-ups/Scrum Meetings

Environment: HDFS, HBase, SQL, Hive, Spark, Python, AGILE, Spark SQL, PySpark, Scala, Sqoop, Oracle, Quality Center, Windows

Confidential

BIG DATA ENGINEER

Responsibilities:

• Defining the requirements for data lakes/pipe lines
• Defining the data layouts
• Design and implement data ingestion techniques for real time data coming from various source systems
• Developed Hive QLs for ETL transformations
• Defining the data layouts and rules and after consultation with ETL teams
• Importing and exporting data into HDFS and Hive using Sqoop
• Written Hive queries for data analysis to meet the Business requirements
• Experience in managing and reviewing Hadoop log files
• Reporting to the project manager on the tasks and also identifying risks & issues
• Worked in aggressive AGILE environment and participated in daily Stand-ups/Scrum Meetings

Environment: HDFS, Oracle RDBMS, Mongo DB, Spark, SQL, Sqoop, Python

Confidential

SQL DEVELOPER - FREELANCING

Responsibilities:

• Creating database objects such as tables, views, stored procedures, Triggers etc.
• Identifying columns for Primary Keys in all the tables at the design time and creating them
• Creating functions to provide custom functionality as per the requirements
• Identifying of potential blocking, deadlocking and write code to avoid those situations
• Ensuring that the code is written keeping in mind any security issues such as SQL Injection
• Developing reports in SQL Server Reporting Services
• Creating Entity Relationship (ER) Diagrams to the proposed database

Confidential

DATABASE SUPPORT ADMIN

Responsibilities:

• Maintaining the database system in house
• Setting user privileges within the database environment
• Supporting the updating of records from different locations of the Air Defense branch
• Providing the department with records generation as per the requirements
• Help with troubleshooting with any database issues and maintaining the backup and access of the database