We provide IT Staff Augmentation Services!

Sr. Python/data Engineer Resume

4.00/5 (Submit Your Rating)

SUMMARY

  • Around 5 years of IT experience in design, development, testing and implementation of various stand - alone and client-server architecture-based enterprise application software using various technologies, analyzing complex business requirements & mapping them to system specifications.

TECHNICAL SKILLS

Programming Languages: Python, C#, C

Scripting Languages: Scala, Shell Script (Bash, ZSH, KSH), Unix

Query Languages: SQL, PL/SQL, HQL

Operating Systems: Windows Vista/XP/7/8/10, Linux, Unix, OS X

Python Libraries/Packages: HTTPLib2, Urllib2, BeautifulSoup, Py Query, Numpy, Scipy, Pickle, PyQT, PySide, wxPython, PyTables, DataFrames, SQLAlchemy, Twisted, Pygame, pyGtk, Pandas, Matplotlib

AWS Services: EC2, S3, VPC, ELB, IAM, Glacier, Aurora, RDS, DynamoDB, Cloud Front, Cloud Watch, Route 53, Elastic Beanstalk (EBS), AutoScaling, Security Groups, EC2 Container Service (ECS), Code Commit, Code Pipeline, CodeBuild, Code Deploy, Dynamo DB, Auto Scaling, Security Groups, Red shift, CloudWatch, CloudFormation, CloudTrail, OpsWorks, Kinesis, IAM, SQS, SNS, SES

WebDevelopment: CSS, HTML, XML, DHTML, JavaScript, JSON, AJAX, XSLT, XAML and Angular JS, Angular2, Angular4, TypeScript, Node JS, NPM, React JS, Redux, BackBone Js, Knockout JS, D3.JS, Vue.JS, JQuery, Ember JS, Ext JS

WebServers: WebSphere, WebLogic, Apache, Gunicorn

Python Framework: DJango, Zope, Bottle,Swagger, Pyramid/Turbogears, Pylons, Flask, Web2py, Gunicorn, Totnado, CherryPy, Spring

Servers: Apache Web Server 2.0, NginX, Tomcat, JBoss, WebLogic

Cloud Computing: Amazon Web Services (AWS), Heroku, Google Cloud, Microsoft Azure

Methodologies: Agile, Scrum, Waterfall, TDD, ATDD, BDD

IDEs: PyDev, PyScripter, Pycharm, Spider, Python IDLE, Brackets, Eclipse, JBuilder, Jdeveloper, Eric, Komodo Edit, Komodo IDE, Cloud9 IDE, DrPython and NetBeans

Version Controls: Git, SVN/Subversion, Perforce, Team Foundation Server, Rational ClearCase, SCM, CVS, GitHub

Build Tools: Apache Ant, Apache Maven, Gradle, Grunt, Gulp, WebPack, MSBuild, Nant, Rake, Make

Virtualization: VMware Workstation, virtual Box

Design Patterns: MVC, Front Controller, Session Facade, IOC, Singleton, Data Access Object, Business Delegate

Architectures: LAMP, WAMP

PROFESSIONAL EXPERIENCE

Sr. Python/Data Engineer

Confidential

Responsibilities:

  • Developed web-based applications using Python, Django, Angular, XML, CSS, HTML, JavaScript and jQuery and Deployed NoSQL technologies like MongoDB, Cassandra, and relational databases like Oracle, SQLite, PostgreSQL and MySQL databases.
  • Automated RabbitMQ cluster installations and configuration using Python/Bash and Deployed and monitored scalable infrastructure on Amazon Web Services (AWS) & configuration management using Chef.
  • Worked with Python ORM Libraries including SQLAlchemy, CherryPy, Pyramid and Bottle for quick service development.
  • Spearheaded project to rewrite provider API suite and migrate from DataProc to DataFlow saving Confidential $500k annually in server costs.
  • Utilized Apache Spark with Python to develop and execute Big Data Analytics and Machine learning applications, executed machine Learning use cases under Spark ML and Mllib.
  • Worked on importing data from MySQL DB to HDFS and vice-versa using Sqoop to configure Hive metastore with MySQL, which stores the metadata for Hive tables.
  • Used Oozie workflow engine Used Spark Streaming APIs to perform transformations and actions on the fly for building common learner data model which gets the data from Kafka in Near real time and persist it to Cassandra. to run multiple Hive and Pig Scripts with the help of Kafka for the real-time processing of data to navigate through data sets in the HDFS storage by loading Log File data directly into HDFS.
  • Used Pandas API to put the data as time series and tabular format for east timestamp data manipulation and retrieval.
  • Automated on-boarding process from 40+ steps to 3 steps for granting licensor’s access to the Enhanced Provider API, which reduced wait time from several weeks to less than three days, excluding time in legal negotiations.
  • Developed the notification service by posting the JSON request in AWS API Gateway, AWS Glue Validating the response in Lambda by getting the data from DynamoDB and sending the notification through AWS SNS.
  • Leveraged Data pipelines to serve various business use cases such as ingesting into AWS S3 Data Lake, exporting data to consumers like ServiceNow, Sales Force, Oracle Cloud etc and Deployed automation, used Configuration Management Tool, Chef and Integrate the dynamic pages with Angular JS and j-query to make the pages dynamic.
  • Created Terraform scripts for EC2 instances, Elastic Load balancers and S3 buckets. Implemented Terraform to manage the AWS infrastructure and managed servers using configuration management tools like Chef and Ansible. Developed by creating REST APIs and used them to access data from different suppliers and to gather network traffic data from servers.
  • Orchestrated solutions for CI pipelines by connecting together TeamCity, Github, Elasticsearch and Artifactory through creative use of Puppet roles, Python scripts, Java microservices and even Scala pipelines.
  • Used SQLAlchemy as Object Relational Mapper (ORM) for writing ORM queries. Developed a module to build Django ORM queries that can pre-load data to greatly reduce the number of databases queries needed to retrieve the same amount of data.
  • Installed Jenkins/Plugins for GIT Repository, Setup SCM Polling for Immediate Build with Maven and Maven Repository and Deployment of apps using custom modules through Puppet as a CI/CD Process.
  • Performed data cleaning and feature selection using MLlib package in PySpark and working with deep learning frameworks such as Caffe, Neon.
  • Worked with Boto3 as interface for AWS API and TensorFlow to make numerical computations using data flow graphs.
  • Scraped and retrieved web data as JSON using Scrapy, presented with Pandas library. Worked with Scrapy for webscraping to extract structured data from website to analyze the specific data of a website and work on it.
  • Leveraged OAuth for token based authentication using Internal Authentication Framework (IAF) and corporate LDAP for single sign on using django-auth-ldap backend. Used Google Analytics API to analyze web traffic and OAuth for authentication.
  • Used Kubernetes to deploy scale, load balance, and worked on Docker Engine, Docker HUB, Docker Images, Docker Compose for handling images for installations and domain configurations.
  • Designed and developed internal compliance tooling and frameworks for SOX and SOC-1 compliance, integration with internal build and deployment tooling as well as runtime environments including GCE, Docker and Kubernetes.
  • Developed tooling for managing user access, as well as for periodic user access review. Currently working on tooling for distributed policy management, deployment and verification.
  • Designed and developed the backup strategy and tooling for Cassandra and Bigtable. Prior to Google releasing tooling for Bigtable we had to develop internal tooling to manage our backups.
  • Employed TDD and OOP best practices to rapidly deliver quality code in an Agile environment with short production timelines.
  • Used Test Driven Approach for developing the application and implemented the unit tests using Python Unit Test framework.
  • Developed a fully automated continuous integration system using CVS, Git, Gerrit, Jenkins, MySQL and custom tools developed in Python and Bash. Wrote Python code to automate Jupyter Notebook to run self-contained scripts on SAS server.
  • Used PyQuery for selecting particular DOM elements when parsing HTML. Used Wireshark, live http headers, and Fiddler debugging proxy to debug the Flash object and help the developer create a functional component.

Python developer

Confidential

Responsibilities:

  • Implemented responsive user interface and standards throughout the development and maintenance of the website using the HTML, CSS, JavaScript, jQuery and Managed Virtual servers (Ubuntu, Linux and Windows) on AWS EC2 using Open-Source Chef server.
  • Leveraged Django caching and query optimization to achieve a performance improvement of a factor of 7, for a legacy e-commerce application in advance of a major marketing event.
  • Developed Wrapper in Python for instantiating 3 tier client-server application and Deploy and monitor scalable infrastructure on Amazon web services (AWS).
  • Consumed external APIs and wrote RESTful API using Django REST Framework and AngularJs. Equally comfortable working within the Django ORM or writing native SQL in SQL Server.
  • Developed the notification service by posting the JSON request in AWS API Gateway, AWS Glue Validating the response in Lambda by getting the data from DynamoDB and sending the notification through AWS SNS.
  • Deployed Python scripts with Cloud Formation templates to automate installation of Auto scaling, EC2, VPC, DynamoDB, cloud formation, Beautiful soup and other services.
  • Used Spark API over Hortonworks Hadoop YARN to perform analytics on data in Hive and Performed Unit testing, Integration Testing, GUI testing using Py-test and web application testing using Selenium Python bindings and Proficient in specialization areas related to Chef for Cloud Automation.
  • Developed and maintained data acquisition pipelines, integrated with 3rd party data providers and created scrapers for client sites when APIs were not available.
  • Involved in Python OOD code for quality, logging, monitoring, and debugging code optimization. Skilled in using Collections in Python for manipulating and looping through different user defined objects.
  • Automated the cloud deployments using chef, Python (boot & fabric) and AWS Cloud Formation Templates. Worked on automation using the Python scripting language, Git on Cygwin32 and XML and used Jenkins and pipelines to drive all micro services builds out to the Docker registry and then deployed to Kubernetes, Created Pods and managed using Kubernetes.
  • Configured internode communication between Cassandra nodes and client using SSL encryption and Created new PL/SQL stored procedures for new Oracle Forms and Reports development.
  • Developed views and templates with Python and Django's to create a user-friendly website interface and Proficient in developing front-end with React-Redux, HTML, JSX, CSS, SCSS, Bootstrap, TypeScript, ES6 and Webpack.
  • Designed and implemented various web applications in both LAMP (Linux, Apache, MY SQL,) Environments and Involved in Data pipelines using python for medical image pre-processing, Training and Testing.

Python developer

Confidential

Responsibilities:

  • Developed entire frontend and backend modules using Python on Django Web Framework on MySQL. Used Django APIs for database access. Implemented database access using Django ORM.
  • Worked on several python packages like Matplotlib, Pillow, NumPy, SQL Alchemy, and sockets.
  • Designed and deployed machine learning solutions in Python to classify millions of previously unclassified users into core data product.
  • Developed Python Scripts to automate various system tasks, OS patches for dev environments, and deployment of applications to testing/prod environments using Python OS modules.
  • Implemented and managed continuous delivery systems and methodologies on AWS.
  • Managed Amazon Web Services (AWS) infrastructure with automation and configuration management tools such as uDeploy, Puppet or custom-built. designing cloud-hosted solutions, specific AWS product suite experience.
  • Developed application using Amazon Web Service (AWS) like EC2, cloud search, Elastic load balancer ELB, Cloud Deploy and monitor scalable infrastructure on Amazon web services (AWS) & configuration management using puppet.
  • Used Git for source code version control and integrated with Jenkins for CI/CD pipeline, code quality tracking and user management with build tools Maven.
  • Worked on front end frame works like CSS, Angular JS, Bootstrap for development of Web application.
  • Developed and tested many features for dashboard using Python, ROBOT/ Bottle framework, Bootstrap, CSS, and HTML5, CSS3, JavaScript, AJAX, J2EE, JSON, jQuery UI, XML and created complex dynamic HTML UI using jQuery and automated Regression analysis for determining fund returns based on index returns (Python/Excel).
  • Developed views and templates with Python and Django's view controller and templating language to create a user-friendly interface using MVC architecture.
  • Used Python and Django interface with the jQuery UI and manage the storage and deletion of content.
  • Designed and developed the UI of the website using HTML, XHTML, AJAX, CSS and JavaScript, also used Bootstrap as a mechanism to manage and organize the html page layout.
  • Designed and developed data management system using MySQL. Built application logic using Python 2.7. Designed Cassandra schema for the APIs.
  • Developed embedded firmware in C to create custom images on Oscilloscope, DSP: Digital Communication, Adaptive. Targeted to pique interest on DSP and embedded systems for beginners.
  • Worked on development of SQL and stored procedures, trigger and function on MYSQL.
  • Developed shopping cart for Library and integrated web services to access the payment (E-commerce). Designed and developed a horizontally scalable APIs using Python Flask.
  • Enabling of the embedded Linux systems to support the development of the enterprise the network Unified Thread Management (UTM) products.
  • Used PhP language on lamp server to develop page. Developed dynamic interaction page on .net MS visual basic 2014, using SQL developer tools.

Software Developer

Confidential

Responsibilities:

  • Involved in Design, Development and Support phases of Software Development Life Cycle (SDLC).
  • Designed and developed components using Python with Django framework. Implemented code in python to retrieve and manipulate data. Developed Python Framework using Django to perform scan software unit monitoring.
  • Involved in development of the enterprise social network application using Python, Twisted, and Cassandra and responsible for setting up Python REST API framework and spring frame work using Django.
  • Developed Rest API's using python with flask and Django framework and done the integration of various data sources including RDBMS, Shell Scripting, Spreadsheets, and Text files.
  • Experienced in Object Oriented Analysis in developing Python Back-End framework using various design patterns. Used Python programming and Django for the backend development, Bootstrap and Angular for frontend connectivity and MongoDB for database.
  • Developed web sites using Django, Python, XHTML, CSS, and JavaScript. Developed and designed e-mail marketing campaigns using HTML and CSS. Tested and implemented applications built using Python.
  • Developed views and templates with Python and Django’s view controller and templating language to create a user-friendly website interface.
  • Developed of Python APIs to dump the array structures in the Processor at the failure point for debugging. Handling Web applications - UI security, logging, backend services.
  • Applied breadth of knowledge in programming (Python), Descriptive, Inferential, and Experimental Design statistics, advanced mathematics, and database functionality (SQL, Hadoop).
  • Developed Spark/Scala, Python for regular expression (regex) project in the Hadoop/Hive environment with Linux/Windows for big data resources. Used clustering technique K-Means to identify outliers and to classify unlabeled data.

We'd love your feedback!