SUMMARY

• Over 13+ years of experience in analysis, design, development, implementation and testing of web - based distributed applications
• Azure Certified Professional - DP-200.1 Implementing Azure Data Solution and AZ-200.1 Azure Technology Development Solution
• Having 5 + years’ experience in Big Data technologies as Hadoop Developer/Lead with strong expertise in HDFS, Hive, IMPALA, Sqoop, Cassandra, ParAccel, Pig, Map Reduce, Hbase/Phoenix 4.7, Kafka, SPARK/Scala, Oozie, Bedrock Workflow, Talend 6.2 Big Data Platform and hands on experience in designing optimized solutions using various Hadoop components like Map reduce, Hive, Sqoop, HDFS, Hbase, Oozie have domain expertise in Media & Entertainment and Manufacturing, Healthcare, Telecom Domain
• Has 1 year experience using Azure Cloud Environment
• Created ADF V2, SHIR, VM, Databricks, Blob Storage, Azure SQL Resources using Ansible Scripts
• Created pipelines, datasets, linked services in Azure Data factory and Integration run time
• Build Transformations using Data bricks, Spark SQL, SCALA/Python stored in to ADLS
• Proficiency in Hadoop and its Ecosystem and Java /J2EE related technologies
• Involved in development and enhancement projects and worked on Horton works HDP platform 1.3 and 2.1.4 distribution system/Cloudera/MapR, Hadoop ecosystems like HDFS, Map Reduce, Hive, Impala, Sqoop, Flume, Oozie,No SQL Databases - Cassandra, HBase, Talend 6.2 Big Data Intergration. and Analytical Database - Paraccel, Datalake
• Excellent understanding of Hadoop architecture and its components such as HDFS, JobTracker, TaskTracker, NameNode, DataNode and MapReduce programming paradigm.
• Have good experience in extracting and generating statistical analysis using Business Intelligence tool Tableau for better analysis of data.
• Experience in creating complex SQL Queries and SQL tuning,
• Very Good knowledge and Hands-on experience in Data warehousing
• Exposure to Cloudera development environment and management using Cloudera Manager.
• Worked on Ambari for cluster management(Cluster Health Check)
• Expertise in all major phases of a SDLC including Design, Development and Deployment, i mplementation and support.
• Working experience in AGILE and WATERFALL models.
• Expertise in preparing the test cases, documenting and performing unit testing and Integration testing.
• Expertise in cross-platform (PC/Mac, desktop, laptop, tablet) and cross-browser (IE, Chrome, Firefox, Safari) development.
• Skilled in problem solving and troubleshooting, strong organizational and interpersonal skills.
• Possesses professional and cooperative attitude, Adaptable approach to problem analysis and solution definition.
• Good team player with strong analytical and communication skills.

TECHNICAL SKILLS

Languages: Scala, core java, Python

Programming Architecture: Map Reduce, PIG

Databases: Cassandra, Paraccel, HBase/Phoenix, Hive, Impala, Azure SQL

File Systems: HDFS, ADLS

Tools & Utilities: Apache Spark SQL/Streaming, Sqoop, Ambari, Jira, Putty, Winscp, IntelliJ, Git, SVN Squirrel, Talend 6.2 (Big Data integration), Tableau 8.2, Oozie, Zookeeper

Domain Knowledge: Media & Entertainment, Manufacturing, Healthcare, Telecom, Oil & Gas

PROFESSIONAL EXPERIENCE

Azure Data Engineer

Confidential

Environment: Azure ADF V2, ADLS, Data Bricks, Spark SQL, Python/Scala, Ansible Scripts, Azure SQL DW(Synopsis), Azure SQL DB, Cloudera, SQOOP, Spark SQL, Shell Scripting, Oozie, Spark

Responsibilities:

• Provide Azure BI and Analytics Solution to the business requirements.
• Created ADF V2, SHIR, VM, Databricks, Blob Storage, Azure SQL Resources using Ansible Scripts
• Created pipelines, datasets, linked services in Azure Data factory and Integration run time
• Created Logic Apps
• Review existing computer systems to determine compatibility with projected or identified needs; research and select appropriate system
• Perform design analysis, development and performance tuning for S&T application.
• Developed ADF V2 Pipeline to ingest data from Source RDBMS to Azure Data Lake.
• Build Transformations using Data bricks, Spark SQL, Scala/Python stored in to ADLS Refine area and stored in azure blobs
• Troubleshooting errors and issues in the Ansible Scripts and ADF Pipelines and resolving them.
• Created Trigger jobs for using ADF pipelines
• Worked with customer teams and other stakeholders to capture requirements and provide technical solutions to requirements.
• Performed required activities to move software upgrades in applications through Development, Quality and Certification to Production systems.
• Use continuous integration for creating builds, unit tests and SAST SCANS for code coverage analysis. Git Version control tool is used and Other similar professional responsibilities as needed.

Sr Data Engineer

Confidential, Fort Worth, TX

Environment: Azure ADF V2, ADLS, Data Bricks, Spark SQL, Python/Scala, Ansible Scripts, Azure SQL DW(Synopsis), Azure SQL DB

Responsibilities:

• Developed Data ingestion Pipeline using Nifi (POC)
• Provide Azure BI and Analytics Solution to the business requirements.
• Created ADF V2, SHIR, VM, Databricks, Blob Storage, Azure SQL Resources using Ansible Scripts
• Created pipelines, datasets, linked services in Azure Data factory and Integration run time
• Developed ADF V2 Pipeline to ingest data from Source RDBMS to Azure Data Lake.
• Build Transformations using Data bricks, Spark SQL, Scala/Python stored in to ADLS Refine area and stored in azure blobs
• Parsing json events into hive using spark sql/scala
• Involved Low level design for Hive, Hbase, Shell scripts to process data.
• Worked on ETL scripts to pull the data from DB2/Oracle/Teradata/SAP HANA/Sql Server Data Base into HDFS (Sqoop)
• Developed hive hql to upload data from different sources.
• Involved for Hive/Hbase Database Schema design.
• Involved Sprint Planning and Sprint Retrospective meetings
• Daily Scrum Status meeting.
• Proposed an automated system using Shell script to sqoop the job.
• Worked in Agile development approach.
• Monitoring production jobs and fixes
• Created the estimates and defined the sprint stages.
• Developed a strategy for Full load and incremental load using Sqoop.