Python Developer With Pyspark Resume
Houston, TX
SUMMARY
- 8+ years of IT Experience in designing, developing, testing and implementing of various stand - alone and client-server architecture-based enterprise application software in Python on different domains.
- Skilled experience in Python with proven expertise in using new tools and technical developments (libraries used: libraries- Beautiful Soup, Jasy, numpy, Scipy, matplotlib, Pickle, PySide, python-twitter, Pandas dataframe, networks, urllib2, MySQL dB for database connectivity) to drive .com
- Good experience in developing web applications implementing Model View Control architecture using Django, Flask, Pyramid and Zope Python web application frameworks.
- Good knowledge on front end frame works like CSS Bootstrap.
- Experience with Design, code, and debug operations, reporting, data analysis and web applications utilizing Python.
- Worked on JavaScript library (Python), SRSS, PySpark to augment browser based applications with functional capability
- Experience in providing highly available and fault tolerant applications utilizing orchestration technologies like Kubernetes and Apache Mesos on Google Cloud Platform.
- Experienced in all facets of full CM process with tools such as SVN, GIT, PVCS, Clear Case, Clear Quest, Perforce, Cruise Control, Jenkins, Bamboo, Chef and Puppet.
- Experienced in creating AWS IAM and Security Group in Public and Private Subnets in VPC. Created AWS Route53 to route traffic between different regions.
- Used for DevOps in Windows-based systems.
- DevOps role converting existing AWS infrastructure to Server-less architecture (AWS Lambda, Kinesis) deployed viaCloudFormation.
- Used all major ETL transformations to load the tables through Informatica mappings.
- Good Experience with Django, a high-level Python Web framework.
- Used virtualenv and pip to work in segregated environments for specific project requirements and to update old technologies, SRSS, PySpark, and NodeJS.
- Sound experience in Object Oriented Programming using concepts like Multi-Threading, Exception Handling and Collections.
- Knowledge on Cloud innovations including Infrastructure as a Service, Platform as a Service, and Software as a Service supplier (Iaas, PaaS, and SaaS).
- Everything running on Linux and making use of Google cloud SDKs.
- Knowledge about setting up Python REST API Frame work using Django.
- Experience in working with Python ORM Libraries including Django ORM.
- Experience in implementing Model View Control (MVC) architecture using server-side applications like Django, Flask and Pyramid for developing web applications.
- Extended Testing framework by adding some helper classes and methods
- Good Knowledge in implementation of Python best Practices (PEP-8).
- Good at writing SQL Queries, Stored procedures, functions, packages, tables, views, triggers using relational databases like Oracle, MySQL, DB2.
- Proficient in using editors Eclipse, PyCharm, PyScripter, Notepad++ and Sublime Text while developing different applications.
- Experienced in developing test automation framework with Python scripting& Selenium.
- Having good knowledge in using NoSQL databases like Apache Cassandra (1.2, 2.0 and 2.1), Kubernetes and Mongo DB (2.6, 2.4), Orient DBF net.
- Having experienced in Agile Methodologies, PCI, Scrum stories and sprints experience in a Python based environment, Data analytics, data wrangling and Excel data extracts.
- Worked on various operating systems like Windows, UNIX and LINUX.
- Experienced in Shell Scripting and UNIX commands.
- Shares expertise throughout the organization and may provide user training for products
- Excellent in Business Analytics. Interpersonal and communication skills, efficient time management and organization skills, ability to handle multiple tasks and work well in team environment.
TECHNICAL SKILLS
Programming Languages: C, C++, Python-3.7 & 2.7, SQL and Shell Scripting
Python Libraries: Python, Django, Flask, Beautiful Soup, httplib2, Jinja2, HTML/CSS, Bootstrap, jQuery, Numpy, Matplotlib, Pickle, Pyside, SciPy, wxPython, PyTables, pdb
Frameworks: Django, PySpark, web2py, pyramid, Flask, Mongodb, Hadoop/ Big Data and CSS Bootstrap,RASA
Technologies: HTML, CSS, DOM, SAX, Java Script, JQuery, AJAX, XML, AngularJS,Version Control GIT (GitHub), SVN
Protocols: TCP/IP, HTTP/HTTPS, SNMP, SMTP
IDE's/ Development Tools: NetBeans, Android Studio, Py Charm, Eclipse and Sublime Text.
Version Control: GIT (GitHub), SVN
Application Tools: Net Beans, Eclipse (SE, EE, Android), Visual Studio, MySQL
Deployment Tools: Heroku, Jenkins
Tracking Tools: Bugzilla and JIRA.
Methodologies: Agile, Scrum and Waterfall
Databases: Access, SQL Server, MySQL, Teradata, Oracle
Software packages: MS Word, Advanced MS Excel, MS PowerPoint, SQL Plus
Reporting Tools: SSRS, Tableau, MS-Word, and MS-PowerPoint.
Operating systems: Linux/Unix, Windows Variants
Cloud Environment: AWS Services, EC2, ELB, VPC, RDS, AMI, IAM, Cloud formation, S3, Cloud Operating Systems
PROFESSIONAL EXPERIENCE
Confidential, Houston, TX
Python Developer with PySpark
Responsibilities:
- Involved in analysis, specification, design, and implementation and testing phases of Software Development Life Cycle (SDLC) and used agile methodology for developing application.
- Participate in requirement gathering and analysis phase of the project in documenting the business requirements by conducting workshops/meetings with various business users
- Containerized and Deployed the ETL and REST services on AWS ECS through the CI/CD Jenkins pipe.
- Worked on optimizing and memory management of the ETL services
- Implemented public segmentation using unsupervised machine learning algorithms by implementing K-means algorithm by using PySpark using data munging.
- Generated Python Django forms to record data of online users and used PyTest for writing test cases.
- Recorded the online users' data using Python Django forms and implemented test case using Pytest.
- Worked on Python Open stack API's, used Python scripts to update content in the database and manipulate files
- Experienced in writing Spark Applications in Scala and Python (Pyspark)
- Analyzed the SQL scripts and designed it by using PySpark SQL for faster performance
- Designed and deployed machine learning solutions in Python to classify millions of previously unclassified Twitter users into core data product
- Designed and developed the UI of the website using HTML, XHTML, AJAX, CSS and JavaScript, also used Bootstrap as a mechanism to manage and organize the html page layout
- Wrote and executed various MYSQL database queries from Python using Python-MySQL connector and MySQL db package.
- Imported data from SQL Server DB, Golang, Azure SQL DB to power BI to generate reports.
- Implemented Okta Single Signon Authentication for several applications making using of Python SDKs, SAML and OpenID Connect.
- Created and maintained various DevOps related tools for the team such as provisioning scripts, deployment tools and staged virtual environments using Docker.
- Implemented new metrics on the MedeAnalytics' Software as a Service (SaaS) Patient Access Analytics platform.
- Hand-on experience in Microsoft Azure cloud platform and merge with python
- Azure cloud Merge with python to store the data into cloud with High security.
- Working on the migration of mobile application from skava to cloud (Google Cloud) by making the chunk of code to microservices.
- Hands on experience inRest APIs, AWS, Open Shift,perl,BDDPL/SQLAmazon Web Services AWS provisioning and good knowledge of AWS services like EC2, S3, Red shift, Glacier, Bamboo, ELB (Load Balancers), RDS, SNS, SWF and EBS etc and Azure.
- Designed Data Quality Framework to perform schema validation and data profiling on Spark (Pyspark).
- Provision RHEL and SUSE server for SAAS based application leveraging chef recipes.
- Development of new IPTV/IP Video SaaS lines of business for major satellite operators (Intelsat, SES); Enterprise
- Used PySpark-SQL to load JSON data and create schema RDD, Data Frames and loaded it into Hive Tables and handled Structured data using Spark-SQL.
- Developed views and templates with Python and Django's view controller and templating language to create a user-friendly interface using MVC architecture.
- Utilized Python Libraries like Boto3, numPY for AWS. Used Pandas library for statistical Analysis
- Used Python and Django to interface with the jQuery UI and manage the storage and deletion of content.
- Worked with team of developers on Python applications for RISK management
- Using Git version control tool to coordinate team-development.
- Implemented machine learning schemes using Python libraries scikit-learn and scipy.
- Implemented PySpark using Python and utilizing data frames and temporary table SQL for faster processing of data.
- Worked on several python packages like Matpoltlib, Pillow, Numpy, SQL Alchemy, and sockets.
- Implemented AWS high-availability using AWS Elastic Load Balancing (ELB), which performed balance across instances.
- Involved in development of Web Services using SOAP for sending and getting data from the external interface in the XML format
- Responsible for debugging the project monitored on JIRA (Agile).
Environment: AWS, Windows, Python3.7, Angular JS, libraries - (NumPy, SciPy, Pandas, SNMP, PySpark, PyCharm, PyQuery, Matpoltlib), SQL Alchemy, MVC, HTML5, Scala,Ansible, OAuth 2.0, OIDC,Kubernetes, Kafka, Shell Scripting, JSON, Apache Web Server, SQL, UNIX.
Confidential, Woonsocket, RI
Python Developer with PySpark
Responsibilities:
- Developed back-end components using Python 3.7 on Django 1.7 framework and front-end using HTML5, CSS/CSS3, JavaScript to improve responsiveness and performance of Web application.
- Understanding of SQL queries and database management system such as PostgreSQL.
- Worked on application using python integrated IDEs like Eclipse, IntelliJ, Sublime Text and PyCharm.
- Developed customized API with token-based authentication for Django using Rest Framework, SRSS, PySpark.
- Worked on Amazon Web Services(AWS Kinesis, EC2) infrastructure with automation and configuration management tools such as Chef and Puppet.
- Installed data sources like SQL-Server, Cassandra and remote servers using the Docker containers as to provide the integrated testing environment for the ETL applications.
- Written Ansible Playbooks with Python SSH as the Wrapper to Manage Configurations of Azure Nodes and Test Playbooks on Azure instances using Python SDK and automated various infrastructure activities like continuous deployment, application server setup, stack monitoring using Ansible playbooks.
- Azure cloud Merge with python to store the data into cloud with High security.
- Worked on google cloud platform (GCP) services like compute engine, cloud load balancing, cloud storage, cloud SQL, stack driver monitoring and cloud deployment manager.
- Developed Node-based front-end applications for easy accessing, tracking and monitoring inventory system for internal usage with multi-level permission, SRSS, PySpark, Node JS.
- DevOps role converting existing AWS infrastructure to Server-less architecture (AWS Lambda, Kinesis) deployed viaCloudFormation.
- Developed numerous new features and enhancements for the Web application using Python/Django, HTML/CSS/JavaScript, as well as automating our development environment using Ansible.
- Developed and maintained an infrastructure for a new project using Ansible, Redis, Postgres and Celery.
- Worked on optimizing and memory management of the ETL applications developed in Go-Lang and python and also reusing the existing code blocks for better performance.
- Implemented AWS Kinesis, EC2 S3 and EFS for improved efficiency of static storage and file sharing system, Jira.
- Provided ongoing maintenance, support and enhancements in existing systems and platforms, bug fixing and tracking (JIRA) and managing new application modules.
- Developed and executed White box test cases using Python, Unitest/Pytest/Robot framework & PyCharm/Ride.
- Perform data transformations with Apache Spark (PySpark) on Databricks to write to AWS S3.
- Used AWS Glue and Redshift Spectrum for ETL and querying semi-structured data from S3 respectively.
- Adjusted and followed scheduled plans set for Sprint planning events, prepared Sprint backlogs along with development and final reports for Sprint reviews, SRSS, PySpark.
- Provision RHEL and SUSE server for SAAS based application leveraging chef recipes.
- Development of new IPTV/IP Video SaaS lines of business for major satellite operators (Intelsat, SES); Enterprise
- Developed dynamic web pages using python DjangoFrameworks and developed various templates for different customers and saved their User preferences.
- Developed views and templates with Django's MVT (model, view, and template) to create a user-friendly website.
- Monitored database performance with the help of AWS CloudWatch and communicated with users to consume resources optimally.
- Microservice architecture development using Python and Docker on an Ubuntu Linux platform using HTTP/REST interfaces with deployment into a multi-node Kubernetes environment.
- Developed web applications and REST full web services and APIs using Python, Django.
- Developed tools using Python, XML to automate some of the manual tasks.
- Used Python Flask framework to build modular and maintainable applications.
- Used Celery with Django to complete asynchronous tasks, like huge report generation, sending mails to respective people.
- Wrote and executed various Postgre SQL database queries from python using Psycopg2 connector.
- Involved in Agile methodology and scrum process.
- Participated in day-to-day meeting, status meeting, strong reporting and effective communication with project manager and developers.
Environment: Python 3.7,AWS,Devops, OAuth 2.0, OIDC,Ansible, PySpark, Kubernetes, Django, Django Rest, Flask, React, Redux, Circle CI, Vagrant, Splunk, New Relic Server, GitHub, Ansible, HTML5/CSS,SNMP, PostgreSQL, Amazon Web Service (AWS), S3, EC2, EBS, PyCharm, Microsoft Visual Code, Linux, Shell Scripting, JIRA.
Confidential, Kansas City, MO
Python Developer with PySpark
Responsibilities:
- Worked as a Python Developer for Confidential to help them complete the process of development and testing for their two most heavily used websites. in a small Confidential team of developers, QA, and engineering resources supporting a total of 30 websites.
- Implemented new metrics on the MedeAnalytics' Software as a Service (SaaS) Patient Access Analytics platform.
- Involved in analysis, specification, design, and implementation and testing phases of Software
- Developed and deployed Business feature set incorporated with APIs using Django REST Framework and Django Serializes module along with Java Technologies for the back-end and UI development with Angular.
- Set up base Python structure with the Create-Python-App package, SRSS, PySpark.
- Developed modules for contact management including Database design, Back-end development and implementing front-end using Bootstrap and JavaScript.
- Used Django Migrations for designing and expanding of the database.
- Development of new IPTV/IP Video SaaS lines of business for major satellite operators (Intelsat, SES); Enterprise
- Hands on experience in migrating Business from a physical data centre environment to AWS.
- Successfully migrated the Django database from SQLite to MySQL to PostgreSQL with complete data integrity.
- Stored and maintained Test Pass Results in SQL Server database. Created base tables, indexes, and stored procedures in SQL Server, SRSS, and PySpark
- Working as an application developer experienced with Perl, BDD, PL/SQL, controllers, views and models in Django and Linux Shell Scripting - Bash and Python scripting.
- Restful web services using Python REST API Framework.
- Implemented the application using Python Spring IOC (Inversion of Control), Django Framework and handled the security using Python Spring Security.
- Tested entire frontend and backend modules using Python on Django Web Framework.
- Developed Server-side automation using Node JS scripting and connecting different types of SQL and NoSQL stores from Node JS.
- Used object-relational mapping (ORM) solution, technique of mapping data representation from MVC model to Oracle Relational data model with an SQL-based schema
- Implemented Performance tuning and improved the Performance of Stored Procedures and Queries.
- Installed and configured py Builder for application builds and deploying it.
- Used Selenium Library to write fully functioning test automation process that allowed the simulation of submitting different we request from multiple browser to web application
- Developed and Deployed SOAP based Web Services on Tomcat Server
- Used Jenkins for continuous integration for code quality inspection and worked on building local repository mirror and source code management using Git hub.
- Used IDE tool to develop the application and JIRA for bug and issue tracking.
- Wrote unit testing codes using unit test, resolving bugs and other defects using Firebug,
- Used JIRA to assign, track, report and audit the issues.
- Used GIT to coordinate team development
Environment: Python 2.7,AWS,Devops, Django, Django Rest, PySpark, Flask, React, Redux, Circle CI, Vagrant, Splunk, New Relic Server, GitHub, Ansible, HTML5/CSS, PostgreSQL, Ansible, Kubernetes, Amazon Web Service (AWS), S3, EC2, EBS, PyCharm, Microsoft Visual Code, Linux, Shell Scripting, JIRA.
Confidential
Python Developer
Responsibilities:
- Work with team of developers on python applications for RISK management.
- Design, develop, test, deploy and maintain the website.
- Designed and developed the UI of the website using HTML, AJAX, CSS and JavaScript
- Designed and developed data management system using MySQL.
- Rewrite existing Python/Django modules to deliver certain format of data.
- Used Django Database API's to access database objects.
- Wrote python scripts to parse XML documents and load the data in database.
- Handled all the client-side validation using JavaScript.
- Expertise in writing Constraints, Indexes, Views, Stored Procedures, Cursors, Triggers, and User Defined function.
- I have also used Selenium Library to write fully functioning test automation process that allowed the simulation of submitting different we request from multiple browser to web application.
- Used JQuery for all client-side Java script manipulation.
- Created unit test/regression test framework for working/new code
- Using Subversion control tool to coordinate team-development.
- Developed entire frontend and backend modules using Python on Django Web Framework.
- Responsible for debugging and troubleshooting the web application.
- Developed views and templates with Python and Django's view controller and templating language to create a user-friendly website interface.
Environment: Python, Django 1.3, MySQL, Linux, HTML, XHTML, SVN, CSS, AJAX, Bugzilla, JavaScript, Apache Web Server, Django