Azure-Databricks-Spark developer Resume

SUMMARY

Overall 10 years of experience In Industry including 4+Years of experience As Developer using Big Data Technologies like Databricks/Spark and Hadoop Ecosystems.
Hands on experience on Unified Data Analytics with Databricks, Databricks Workspace User Interface, Managing Databricks Notebooks, Delta Lake with Python, Delta Lake with Spark SQL.
Good understanding of Spark Architecture with Databricks, Structured Streaming. Setting Up AWS and Microsoft Azure with Databricks, Databricks Workspace for Business Analytics, Manage Clusters In Databricks, Managing the Machine Learning Lifecycle
Hands on experience Data extraction(extract, Schemas, corrupt record handling and parallelized code), transformations and loads (user - defined functions, join optimizations) and Production (optimize and automate Extract, Transform and Load)

TECHNICAL SKILLS

PROFESSIONAL EXPERIENCE

Confidential

Azure-Databricks-Spark developer

Responsibilities:

Experience in developing Spark applications using Spark-SQL in Databricks for data extraction, transformation, and aggregation from multiple file formats for Analyzing& transforming the data to uncover insights into the customer usage patterns.
Extract Transform and Load data from sources Systems to Azure Data Storage services using a combination of Azure Data factory, T-SQL, Spark SQL, and U-SQL Azure Data Lake Analytics. Data ingestion to one or more Azure services (Azure Data Lake, Azure Storage, Azure SQL, Azure DW) and processing the data in Azure Databricks
Develop Spark applications using pyspark and spark SQL for data extraction, transformation, and aggregation from multiple file formats for analyzing and transforming the data uncover insight into the customer usage patterns
Hands on experience on developing SQL Scripts for automation
Responsible for estimating the cluster size, monitoring, and troubleshooting of the Spark databricks cluster
Ability to apply the spark DataFrame API to complete Data manipulation within spark session
Good understanding of Spark Architecture including spark core, spark SQL, DataFrame, Spark streaming, Driver Node, Worker Node, Stages, Executors and Tasks, Deployment modes, the Execution hierarchy, fault tolerance, and collection

Confidential

Spark Developer

Responsibilities:

Processed Data into HDFS by developing solutions, analyzed the Data using MapReduce
Import Data from various systems/sources like MYSQL into HDFS
Involving on creating Table and then applied HiveQL on those tables for Data validation
Involving on loading and transforming large sets of structured, semi structured and unstructured data
Extract, Parsing, Cleaning and ingest data
Monitor System health and logs and respond accordingly to any warning or failure conditions
Managed and reviewed Hadoop log files
Involving in loading data from UNIX file system to HDFS
Provisioning Hadoop and Spark clusters to build the On-Demand Data warehouse and provide the Data to Data scientist

Confidential

JSW Store Attendant

Responsibilities:

Assist Warehouse Manager with all paperwork related to warehouse shipping and receiving
Used SAP (Material Management), enter order and other shipping information
Sorted and Placed materials or items on racks, shelves or in bins according to predetermined sequence such as size, type style, color, or product code
Sorted and placed materials or items on racks, shelves or in bins according to predetermined sequence such as size, type, style, color or color or product code
Label and organize small parts on automated storage machines