SUMMARY

• Experienced in complete Software Development Life Cycle including Requirement Analysis,
• Design, Development, Database Design, Deployment, Testing, Debugging, and Application Maintenance.
• Expertise with teh tools in Hadoop Ecosystem including Pig, Hive, HDFS, MapReduce, Sqoop, Storm, Spark, Kafka, Yarn, Oozie, and Zookeeper.
• Developed, updated and maintained a suite of RShiny dashboards for modeling and visualizing data related to refinery outages, pipeline flow, ship tracking, etc., used by BP's traders and analysts
• Authored data extraction, manipulation and modeling functions for BP's internal Python Library.
• Wrote web - scraping Python scripts to gather data from terminal operator and government websites, as well as other publicly available data.
• Coordinated with RAD Team to fulfill analyst requirements working as a team, not in a silo and developed aMATLABalgorithm which determines an object's dimensions from digital images.
• Gained domain knowledge of teh Oil & Gas trading industry-crude and refined balances, demandforecasting, data sources and draw/build indicators and working knowledge at software such as MULTISIM, PROTEUS ISIS, ARES PCB DESIGN, PSPICE, andMATLAB.
• Mastering in implementing MVC architecture using PYTHON's Django, Flask for web based applications in OOP concepts like Multi-Threading, Exception Handling, and Collections.
• Real Time experience in Robotic Process Automation (RPA) used KAPOW and BLUE PRISM tools for automation process.
• Identified and fixed cause (s) of teh reported issues by checking batch loading and python scripts scheduled incronjobs, or manually started fixing scripts, modified and wrote new python scripts andUNIXshell scripts, etc.
• Involved in various development methodologies like Waterfall, Scrum and AgileExperienced with WAMP (Windows, Apache, MySQL, and Python/PHP) and LAMP (Linux, Apache, MySQL, and Python/PHP) Architecture andMATLABfor data analysis
• Experience of using Python modules like NumPy, Matplotlib, Pickle, Pandas, PySide, SciPy, wxPython, PyTables, PyQtetc, for generating complex graphical data, creation of histograms, etc.
• Exploring with teh Spark for improving teh performance and optimization of teh existing algorithms in Hadoop using Spark Context, Spark-SQL, Data Frame, Pair RDD's, Spark YARN.
• Skilled in Object-Oriented Design and Programming using Python and MySQL dB for database connectivity andCassandrato improvise software development process.
• Experience with Data migration from Sqlite3 to ApacheCassandradatabase and Cassandradata model designing, implementation
• Used Python, Django,CSS,HTML, and jQuery, XML, AJAX and JSON for developing web applications and having good experience in using NoSQL databases like ApacheCassandra.
• Worked on AJAX framework to transform Datasets and Data tables into HTTP Serializable JSON strings.
• Experienced with JSON based RESTful web services, and XML based SOAP web services and AWS based HybridCloudMigration.
• Hands on experience installing, configuring, testing Hadoop ecosystem components and experience with AWS services like EC2, VPC,CloudWatch,CloudFormation, IAM, S3, RDS, Elasticache, SNS, SQS, SWF.
• Python scripting with focus on Devops tools, CI/CD and AWSCloudArchitecture and hands-on Engineering.
• Worked on various applications using python integrated IDEs like Eclipse, Sublime Text, PyCharm, Jupyter, IPython, PyScripter, and NetBeans.
• Experience of working with relational databases like Oracle, SQL SERVER, PostgreSQL, MySQL, DB2 and non-relational databases like MongoDB and Cassandra.
• Proficient in developing complex SQL Queries, Stored Procedures, Triggers, Cursors, Functions, and Packages along with performing DDL and DML operations on teh database.
• Experienced with Amazon SQS, and Amazon Web Services like EC2, Redshift, Lambda and S3.
• Written and developed scripts for automating tasks using Jenkins and UNIX shell scripting.
• Experienced in project deployment using Heroku, Jenkins and Amazon Web Services (AWS).
• Worked on various Operating Systems like UNIX, Linux, Windows, and Mac OS.
• Experienced with version control systems like Git, GitHub, CVS, and SVN to keep teh versions and configurations of teh code organized.
• Experienced with Unit Testing, System Integration Testing (SIT) and User Acceptance Testing (UAT).
• Programmed Oracle Triggers, PL/SQL stored procedures andCrontasks.
• Good amount of experience in handling errors/exceptions and debugging issues in large-scale applications.

PROFESSIONAL EXPERIENCE

Sr. Python ETL Developer

Confidential, Bellevue, WA

Responsibilities:

• Automation of infrastructure using Terraform and Ansible. Implemented cloud infrastructure automation using internal web-services and distributed worker queues (celery).
• Worked on development of internal testing tool framework written in Python and active use of Teradata, Netezza, Unix, Python, Excel/VBA, QUARTZ, R, SAS,MATLAB, R
• Developed GUI using Python and Django for dynamically displaying teh test block documentation and other features of python code using a web browser.
• Experience in working with tools like Attunity and having knowledge on ETL tools like Talend and wrote Python modules to view and connect teh ApacheCassandrainstance.
• Extensively worked on Spark SQL, Dataframes, RDD's to improve teh performance of teh application.
• Wrote scripts in Python for extracting data fromHTMLfile and develop optimized python models from existing R, SAS andMATLABmodels
• Worked on using different file formats like JSON, Sequence files, AVRO file, Parquet file formats.
• Scheduled SAS and BTEQ Scripts inUNIXCRONJob's and tan used tables in for reports.
• Used Spark SQL and Hive SQL to process structured and un structured data and used AmazonCloudEC2 along with Amazon SQS to upload and retrieve project history.
• UsedApacheStorm for extracting teh data by designing a topology as per client requiremenDeveloped views and templates with Python and Django's view controller and templating language to create a user-friendly website interface.
• Used JavaScript and JSON to update a portion of a webpage and designedCassandraschema for teh APIs.
• Utilized theUnixcrontab for scheduling teh automation of modifying teh data in teh database
• Responsible for debugging teh project monitored on JIRA (Agile) and experience with R language andMATLABfor data analysis.
• Wrote Python scripts to parse JSON documents and load teh data in database designing and developing various machine learning frameworks using Python, R, andMATLAB.
• Used Python and Django to interface with teh jQuery UI and manage teh storage and deletion of content.
• Developed automated processes that run daily to check disk usage and perform clean-up of file systems onUNIXenvironments using shell scripting andCRON.
• Handle and operate ETL using Hadoop, Hive, MySQL, SSIS, Shell,AWSEMR,AWSS3, Pentaho, and PostgreSQL platforms.
• Created database using MySQL, wrote several queries to extract data from database. Have Knowledge onESPscheduler and in creating job doc sets to schedule ETL jobs.
• Worked on Extracting, Transforming and Loading (ETL) data from Excel, Flat file, to MS SQL Server by using DTS and SSIS services.
• Worked on Extraction, Transformation and Loading of data usingInformatica.
• Experienced inInformaticadata quality (IDQ), power center, data cleansing, data profiling, data quality measurement and data validation processing.
• Developing several complex mappings inInformaticaa variety of Power center transformations, Mapping Parameters, Mapping Variables, Mapplets& Parameter files in Mapping Designer using both teh InformaticaPowercenter and IDQ.
• CreatedInformaticamappings with PL/SQL procedures/functions to build business rules to load data.Extensively worked on Facts and Slowly Changing Dimension (SCD) tables.
• Performed Loading operation of historical data using full load and incremental load into Enterprise Data Warehouse.
• Developed several complexmappingswith transformations,MappingParameters,MappingVariables, Mapplets& Parameter files inMappingDesigner using Informatica Power Center.
• Generated variouscronscripts to notify through trigger emails about important alerts and events
• Written complex SQL procedures to perform aggregations insemanticlayer. Also written complex procedures to perform dynamic query generation for UI dashboard reporting
• Involved in building Data Marts and multi-dimensional models like Star Schema and Snowflake schema.
• Filtered data from Transient Stage to EDW by using complex T-SQL statements in Execute SQL Query Task and in Transformations and implemented various Constraints and Triggers for data consistency and to preserve data integrity.
• Writing T-SQL scripts, dynamic SQL, complex stored procedures, functions, triggers and SQLCMD.
• PL/SQLand Data Model development for Java web applications.Used data conversion tasks in SSIS to load teh data from flat file to SQL Server database.
• MS SQL Server Configuration, Performance Tuning, Client-Server Connectivity, Query Optimization, Database Maintenance Plans and Database Consistency Checking DBCC commands.
• Built MDX queries and Data Marts Expression (DMX) queries for Analysis Services cubes& Reporting Services.Performed database transfers and queries tune-up, integrity verification, data cleansing, analysis and interpretation.
• Setup automated cron jobs to upload data into database, generate graphs, bar charts, upload these charts to wiki and backup teh database.
• Utilize SSIS for teh access and operation of Shell script, JSON, and Hive onAWSEMR and S3 files, for teh purpose of syncing teh loading databases.
• Utilize shell script to communicate and sync up multiple database platforms and file systems, applying teh ability to pass parameters and arguments betweenAWSEMR, Hive, and JSON.
• Developed Merge jobs in Python to extract and load data into MySQL database and programmatically controlled teh COMSOL Multiphysics® model in theMATLAB® graphical user interface to perform case studies and customize plots and data processing.
• Work with MySQL andPostgreSQLto develop an optimized databases.
• Successfully migrated teh Django database from SQLite to MySQL to PostgreSQL with complete data integrity.
• Designed front end using UI,HTML, Bootstrap, Node JS, underscore JS, Angular JS,CSS, and JavaScript.
• Followed AGILE development methodology to develop teh application.
• Used and customized Apache server to for checking our developed project.
• For queuing teh message services as per teh category of payment will be assigned and tracked by usingApacheKafkaUsed WebSphere, HTML/CSS,JavaScriptto add functionality to Delta Sky Club Liquor Inventory ApplicationUsedGit to resolve and coding teh work on python and portlet.

Environment: Spark, Python 2.7, Django 1.4, HTML5,CSS, XML, MySQL, JavaScript, Angular JS, Backbone JS, JQuery,MATLAB, CSSBootstrap, Mongo DB, T-SQL, JavaScript, Eclipse, Git, GitHub, AWS, Linux, Shell Scripting. Cassandra

Sr. Python Developer

Confidential, New York, NY

Responsibilities:

• Worked with a team of developers on Python applications for cards management of teh bank.
• Designed teh database schema for teh content management system and added support for Amazon AWS S3 and RDS to host static/media files and teh database into AmazonCloud.
• Designed and developed teh website UI usingHTML,CSS, XHTML, AJAX and JavaScript.
• Involved in teh development of Web Services using SOAP for sending and getting data from teh external interface in teh XML format.
• Used Selenium Library to write fully functioning test automation process that allowed teh simulation of submitting different requests from multiple browsers to web application.
• Created Data tables utilizing PyQt to display customer and policy information and add, delete, update customer records and developed aMATLABalgorithm which determines an object's dimensions from digital images.
• UsedPythonand Django creating graphics, XML processing, data exchange and business logic implementation.
• Involved in creating multiple SSIS packages to extract data from Flat Files, Excel, JSON, SQL Server, andPostgreSQLdatabase.
• Utilize in-depth knowledge of Technical experience in LAMP and other leading-edge products and technology in conjunction with industry and business skills to deliver solutions to customer.
• Developed multiple spark batch jobs in Scala using Spark SQL and performed transformations using many APIs and update master data in Cassandra database as per teh business requirement.
• Wrote manycronjobs to automate teh free up teh disk space by deleting all teh temporary files
• Written Spark-Scala scripts, by creating multiple udf's, spark context, Cassandra sql context, multiple API's, methods which support dataframes, RDD's, dataframe Joins, Cassandra table joins and finally write/save teh dataframes/RDD's to Cassandra database.
• As part of teh POC migrated teh data from source systems to another environment using Spark, SparkSQL.
• Developed and implemented core API services usingPythonwith spark and associated with working withcloudcomputing platforms like Openstack.
• Representation of teh system in hierarchy form by defining teh components, subcomponents usingPythonand developed set of library functions over teh system based on teh user needs.
• Fault tolerance in teh presence of machine failure usingApacheKafka.

Environment: Spark,Python, Django, HTML5/CSS, MS SQL, MySQL, JavaScript, MATLAB, Eclipse, Linux, Shell

Python Developer

Confidential, Austin, TX

Responsibilities:

• Generated Python Django Forms to record data of online users.
• Created PHP/MySQL back-end for data entry from Flash. I had to assist teh Flash developer send teh correct data via query strings.
• Developed monitoring and notification tools using Python and worked on HTML5, CSS3, JavaScript, AngularJS, Node.JS, Git, REST API,MongoDB, intelliJ IDEA.
• Design and Setting up of environment ofMongodBwith shards and replica sets. (Dev/Test and(Production).
• Private VPN using Ubuntu, Python, Django, CherryPy, Postgres, Redis, Bootstrap, Jquery,Mongo, Fabric, Git, Tenjin, Selenium, Sphinx, Nose
• Created Data tables utilizing PyQt to display patient and policy information and add, delete, update patient records.
• Wrote Python routines to log into teh websites and fetch data for selected options.
• Designed Cassandra schema for teh APIs.
• Handled all teh client side validation using Javascript.
• Good experience with Oops concepts.
• Designed and created backend data access modules using PL/SQL stored procedures and Oracle.
• Good experience in writing SQL Queries and implementing stored procedures, functions, packages, tables, views, Cursors, triggers.
• Experience in using collections in Oracle for manipulating and looping through different user defined objects.
• Developed Python/Django application for Google Analytics aggregation and reporting.
• Developed and executed User Acceptance Testing portion of test plan.
• Generated Python Django forms to record data of online users and used PyTest for writing test cases.
• Involved in capturing teh requirements for serial functional interface and other software requirements specification document.

Environment: Python, MySQL, Django, Cassandra, MongoDB, Flask, PHP, XML, Linux,HTML, Java script, Windows

Software Developer

Confidential, San Francisco, CA

Responsibilities:

• Designed and developed Use-Case Diagrams, Class Diagrams, Object Diagrams using UML Rational Rose for OOA/OOD techniques.
• Programmed teh components using C/C++.
• Software release to internal clients by compiling teh code into C/C++ libraries and implementing teh APIs.
• Created UI usingHTMLCSSand Javascript.
• Created Servlets and Beans to implement Business Logic.
• Used SAX/DOM Parser for parsing teh data to Oracle Database.
• Implemented teh presentation layer withHTML, DHTML, AJAX,CSSand JavaScript.
• Designed object model, data model, tables, constraints, necessary stored procedures, functions, triggers, and packages for Oracle Database.
• Designed and created backend data access modules using PL/SQL stored procedures and Oracle.
• Developed and executed User Acceptance Testing portion of test plan.
• ImplementedApacheSpark data processing project to handle data from RDBMS and streaming sources using Scala.Developed rich user interface usingCSS,HTML, JavaScript and JQuery.
• Setup automatedcronjobs to upload data into database, generate graphs, bar charts, upload these charts to wiki and backup teh database.
• Used Django framework for application development.
• Created entire application using Python, MySQL and Linux.
• Enhanced existing automated solutions, such as teh Editorial Tool for automated request reporting and teh Who's Who site, by adding new features and fixing bugs.
• Created database using MySQL, wrote several queries to extract/store data.
• Used Perforce for teh version control.

Environment: C, C++,HTML,CSS, Shell Scripting, PL/SQL, JDBC, PVCS, Oracle, Linux and Windows, Python, Django, MySQL, Windows, Linux,HTML,CSS, JQuery, JavaScript, Apache, Linux, Perforce.