We provide IT Staff Augmentation Services!

Senior Data Engineer Resume Profile

3.00/5 (Submit Your Rating)

Summary of Qualifications

Confidential is a seasoned software developer and team manager with experience in both technology and financial consulting. He has delivered complete Business Intelligence and Data Warehouse solutions to corporate clients. He has also developed multiple financial engineering solutions to analyse numerical financial data, as well as sentiment analysis of news articles and other textual data. Current interests are mining social media data, natural language processing, data science, and big data applications using Hadoop.

Career History

Confidential

Senior Data Engineer and Data Analyst

  • HipCricket, Inc. is a leading mobile ad and sms messaging service provider.
  • Ad clients include Fortune 100 companies such as Dodge, Ford, Miller Coors, and MGM.
  • Technologies Used: AWS Redshift/Postgresql, AWS EC2 Linux, Hadoop/Pig, SQL Server, R and RStudio, Python, Perl, Git version control, Tableau, Aginity Workbench, and Navicat.
  • Worked with sales teams to create performance reports for Mobile Ad and SMS Messaging clients using Tableau and Excel.
  • Analyzed Real-Time Bidding data RTB from ad-serving exchanges to create optimization models on KPIs such as CTR, CPM, and impressions data.
  • Dealing with multiple terabytes of mobile ad data stored in AWS using Elastic MapReduce and Redshift Postgresql.
  • Augmented HipCricket RTB and ad-serving data with third-party vendor data from Neustar, Comscore, and Axciom to created targeted customer segmentation ad delivery.
  • Created customer segmentation using machine learning clustering algorithms.
  • Created summary KPI reports for SMS messaging clients using Tableau. Reports focus on performance over time, geography, by brand, and campaign.

Confidential

Analytical Software Developer

  • Data Engineer, Business Intelligence Developer, and Technical Architect.
  • Technologies Used: Open Source stack: RedHat Linux, Python, Perl, PHP, Mysql 5.6, Java, and SVN. Windows Stack: Windows Server 2012 R2, .Net, and InetSoft Business Intelligence. Oracle Stack: Oracle 11.2. Cloud Stack: AT T Silver Lining, AT T ITO Cloud, AWS EC2.
  • Project Lead for SCRS 2.0 Short Code Reporting Server . Assumed late SCRS Cloud P2V Project and delivered on schedule. Previous team failed to deliver project within one year. My team delivered in less than eight months.
  • Developed architecture for SCRS system doing a P2V from Windows server to Linux Cloud platform.
  • Re-engineered system with new data modelling design, new ETL and SQL code. System went from ad hoc reporting system to an enterprise data warehouse with dimensional modelling design.
  • Ad Hoc reporting and analysis provided to Short Code Provisioning Team, Short Code Campaign Marketing team, and Legal. Reporting on 552 million worth of annual revenue data.
  • Developed data interfaces to Neustar provided Short Code Campaign Management System.
  • Received AT T Director Level Certificate of Appreciation for delivery of SCRS 2.0.
  • SCRS Cloud Implementation project handles roughly 500 gigs worth of data, including transactional, campaign management, and short code/network related data.
  • SCRS reporting layer developed using InetSoft Business Intelligence software. Created reports using Style Studio after gathering requirements from end users. Custom reports developed using Javascript and SQL.
  • Created all scheduler and data back-up jobs using Python/Perl/Bash/Cron/SCP/SSH/FTP.
  • Developer for Black Flag API Reporting team.
  • Developed executive dashboards on overall Black Flag API environment and usage.
  • Developed Perl, Python, and Bash scripts to do data verification between Mysql and Oracle databases as part of database migration.
  • Developed scripts in PLSQL and Python to ingest API data from internal web servers.
  • Developed scripts in Python to ingest mobile ad data from Pubmatic API.
  • Developed tool for Customer Text Response triage team. Web UI developed with PHP/Mysql interface. System responds to approximately 20,000 customer text messages per month.
  • Passed all required AT T Agile Training.

Confidential

BI Applications Developer

  • Business Intelligence BI Applications developer.
  • Technologies Used: Open Source stack: Linux, Python, Hadoop, Pig, Mysql, JAVA, GIT, Gerrit, Jenkins, Puppet, Pyunit, Netezza, Open Source Scheduler, VM Ware Player. Windows Stack: Windows 2008, .Net, SQL 2008, QL2, Active Batch , Team Foundation Server Reporting, SSAS, SSRS, Omniture Insight, Tableau.
  • Developed apps for Data Acquisition DAQ , Operational Data Store ODS , and Enterprise Data Warehouse EDW .
  • Apps manage 100 terabytes of data on IBM Neteeza database.
  • Apps manage 150 data sources including web analytics and financial data.
  • Apps track over 1 billion page views per month.
  • Use of Omniture Insights to harvest web traffic data.
  • Developed apps for Operation Metadata OMD MS SQL Server database. Database table schema design, C management code, and stored procedures.
  • Utilized SQL Server Management Studio SSMS .
  • C code done with OOP design.
  • Developed DAQ code to acquire data in C .Net framework
  • Key DAQ scripting components written in Python and QL2.
  • Developed DAQ code applying Google Adsense and DART APIs for web analytics in Python.
  • Developed Apache Hadoop components using Java and Pig CDH3 .
  • Cloudera Certified Developer for Apache Hadoop Sept. 2012 .
  • IDEs and tools used include Visual Studio 2010, Eclipse, PyCharm, Neteeza Aginity, TFS version control, Git version control, and Sublime Text 2.
  • ActiveBatch Workflow/Scheduler.
  • Utilize Amazon Web Services AWS cloud storage tools to process web analytics data using Hadoop tool set.
  • Utilize Linux flavors Centos 5.0 and Ubuntu 11.0 using VMWare Fusion.
  • Worked in Agile development using SDLC paradigm.
  • Experience in extraction of data using Google DART Adsense, Yahoo, Twitter, and Facebook APIs. Familiar with REST-ful APIs in Python.
  • Experience with parsing data in JSON, XML, and AVRO formats.
  • Experience with schema design in MS SQL Server and Mysql.
  • Experience with Git and TFS version control.

Confidential

Ruby on Rails Developer Intern

  • Web applications developer using Ruby on Rails.
  • Development on mainly on backend data processing applications.
  • Completed entire Ruby on Rails Tutorial by Michael Hartl.
  • Development done using Rails 3.0, Ruby Mine, Heroku, Sublime Text 2, Spork, rspec, Rubygems, RVM, Git.
  • Developed apps to do Oauth2.0 authentication to Twitter.
  • Developed Twitter newsfeed app using JQuery.

Confidential

Consultant/Principal

Confidential

Investment/Technology Consultant

  • Developed proprietary Sentiment Index by annotating financial news corpora spanning 11 years, 23 million lines of text, and 225,000 individual annotated sentiment observations.
  • Sentiment scoring algorithm implemented using proprietary software tools.
  • Daily collections of financial news articles from over 30 websites using web-scraping.
  • Latent Semantic Algorithms LSA to categorize domain and topic.
  • Tools: Perl, Mysql, and R statistical package.
  • Sentiment Index is the basis for a trading strategy which is currently managing 10mm.
  • Developed trading apps using Interactive Brokers Trader Work Station api in Java using Eclipse Juno IDE.

Confidential

Investment/Technology Consultant

  • Technology and Portfolio manager for the KCAP ETF Opportunity Fund, LP.
  • Developed real-time Algorithmic Trading Software using .Net tools and Interactive Brokers Trader Work Station api, and fully integrated accounting back-end system.
  • Algorithms developed to analyse real-time market data and trade using proprietary decision algorithm.
  • Designed database schemas in MS SQL to handle order flow.
  • Applied delegates/event processing for real time database updates and order execution.
  • Object-Oriented design using features of .Net and C to process real-time events.
  • Web-scraping tools to get pricing and financial data from Yahoo Finance.
  • Developed winform GUIs to query transaction DB in SQL Server using T-SQL
  • Tools: Visual Studio, .Net 3.0, C , Visual Basic, Mysql, MS T-SQL, Perl.

Confidential

Technology Consultant

  • 2 Ranking for Hedge Fund Administrator by Institutional Investor's Alpha Awards.
  • Developed architecture for front-to-back end solution for Hedge Fund accounting data system.
  • Worked with prime brokers include Goldman Sachs, Morgan Stanley, UBS, and Interactive Brokers to develop fully automated data acquisition system.
  • Worked directly with hedge fund administration accountants in reconciliation of:
  • Daily account balances/NAV.
  • Daily trades/positions.
  • Monthly fee accrual calculations.
  • Cash balances/payments.
  • Used Perl as scripting language and Mysql for database architecture.

Confidential

Portfolio Manager

  • Equity portfolio manager 40mm . Small cap value focus with options and futures overlays.
  • Created NYSE/NASDAQ tick database to produce proprietary technical analysis.
  • Consulted with Investor's Business Daily IBD national newspaper based in Los Angeles to identify bugs in their premium charting package.
  • Implemented bug fixes using Perl, Mysql, R Statistics Package, and Linux.

Confidential

Director

  • Co-founder of B2B e-commerce start-up. 1 million in seed capital. 7 employees.
  • Architect of CGI-Perl web server infrastructure.

Confidential

Trader and Risk Manager

  • Trader in European fixed income electronic futures options with focus on arbitrage market-making.

Proprietary Trading Analyst

  • Together with two professors of Finance MIT PhDs programmed a complete pricing and risk management system from scratch for the Japanese equity warrants and convertible bonds arbitrage book.
  • Tools: C, UNIX, Perl, and Sybase SQL.
  • Qualifications/Skills
  • Languages: Japanese fluent speaker reader. Working knowledge of Chinese Cantonese Mandarin .
  • Certified Advanced C and .Net Programming Skills through Bellevue College.
  • Cloudera Certified Developer for Apache Hadoop Sept. 2012 .
  • Microsoft Office: Excel, Powerpoint, Word.

We'd love your feedback!