SKILL:

• .NET/C#, MVC, JAVA, JavaScript, jQuery, Bootstrap, XSLT, CSS, HTML, Entity Framework
• PowerShell, Hybernate, NiFi, Hadoop, HDFS, Hortonworks, Scala, Spark, Zookeeper

PROFESSIONAL EXPERIENCE

Big Data Architect / Scala Developer

Confidential

Responsibilities:

• Developed Spark 1.6.3/Scala framework to process new MUREX front end of day credit/risk trade data and create output files for downstream risk systems.
• Used maps for broadcasting all and MUREX end of day tables to increase spark efficiency.
• Setup jenkins/UrbanCodeDeploy for CI built pipeline.
• Created shell scripts to trigger spark jobs and automated peripheral activities as pushing hdfs files into edge node locations for downstream systems to pick up files.
• Created Solace connection module to lookup data in solace queues. Used JMS messages to send Solace payload. Retrieved response data from solace queue and broadcasted to use as map for spark application.
• Created java spring boot application to for spark application scheduling.
• Assisted QA in troubleshooting QA issues and application problems.
• Created technical and deployment documentation for application.
• Supported production support when issues arose in production environment.
• Worked closed wif business analyst to understand required logic and that Jira tickets were correct described.
• Importing/exporting data from LINUX file system to HDFS.

Big Data Architect / Scala Developer

Confidential

Responsibilities:

• Developed/implemented project where ETL processes related to ATM/terminal transactions where to moved from legacy applications/hardware to a big data stack to save ~$4 million.
• Developed Spark Streaming 2.2 application using Scala to create/migrate real time stream of existing batch transformations on existing legacy Teradata platform.
• Used Spark structured streaming API to write/read from kafka topics and leverage checkpointing for failure recovery.
• Created "commands" topic in kafka that Spark ingested to be able use to trigger a graceful shut down.
• Created stream listeners that logged progress and throughput info.
• Created UDFs to use for transformation logic in dataframes.
• Transformed RDDs into DataFrames to leverage DF functionality.
• Used Spark broadcasting to cache look up tables on nodes.
• Used Scalatest and log4j for unit testing and logging.
• Developed NIFI flows for ingress and egress flow from spark application.
• Develop successful real - time pipeline to read/process/output 300 million events per month.
• Developed against Horton Data Works 2.4.2 platform.
• Analyze Teradata/DataStage ETL code, develop new simplified SQL code to be developed in Scala for Spark.
• Export Teradata output tables and compare wif new streaming system output for accuracy and demos for stake holders.
• Created Datawarehouse and data lakes based on requirements in Hive for data validation and ad hoc user needs.
• Create proof of concept pipeline for initial demo on recommended architecture.
• Work successfully in an unestablished big data development environment to get code results ready quickly and efficiently for clients to see progress and continue to believe in success of project.
• Created Nifi 1.3 cluster for and Zookeeper cluster for redundancy and failover.
• Created Kafka .10 topics and partitions for performance tuning against Spark app to maximize performance.
• Creating HIVE External tables wif partitions on daily ingesting files for analysis by business analysts. Created datamarts in HIVE for specific business use cases to simply teh datasets teh business analysis would of to deal wif.
• Importing/exporting data from LINUX file system to HDFS.
• Managed DEV/QA/UAT edge node environments for production life cycle.
• Used Maven/POM for dependency management.
• Assist other teams in development of python scripts for machine learning algorithms. Leveraged Pandas, numpy, skelearn for preprocessing of data for /testing against regression algorithms.

Big Data Developer

Confidential

Responsibilities:

• Started Big data practice to discover and leverage existing big data sets related to oil and gas wells.
• Engaged wif business analysts to understand their business requirements and build appropriate datamarts for their specific needs.
• Using Hive, Pig, NiFi and SQOOP to ingest, manipulate, analysis, export and store data leveraging HortonWorks Hadoop platform.
• Create Java UDFs for Hive tables. Created data warehouses and data lakes in Hive and SQL.
• Used Pig as ETL tool to do transformations, event joins, pre-aggregations before storing teh data onto HDFS.
• Responsible for developing data pipeline using NiFi to extract data from weblogs and store in HDFS.
• Imported and exported Data from/to SQL to/from HDFS/Hive using Sqoop.
• Created Hive tables and involved in data loading and writing Hive UDFs.
• Used Hive to analyze teh partitioned and bucketed data and compute various metrics for reporting.
• Automated workflows using shell scripts to pull data from various databases into Hadoop.
• Traversed Hadoop system to identify job failure issues.
• Created various data lakes for various business units and continued in assisting them to be able extract/analyze correct data.

Java Spring Developer

Confidential

Responsibilities:

• Created web application using Spring/Boot to surface various legacy databases for business users.
• Used SPRING boot to quick start projects and work wif clients through iterative processes to deliver to their needs.
• Leveraged hibernate for repository code layer. Spring MVC for code clarity and manageability.
• Used SOLID design principals to keep code manageable and efficient.
• Leveraged SQL to analyze and create datalakes for specific department needs.
• Used H2 database for testing and SQL for delivery databases using SPRING profiles and properties files.
• Used Thymeleaf templating for view layers.

Architect / Java Developer

Confidential

Responsibilities:

• Created custom solutions wifin teh SharePoint environment including integration wif external applications such as Active Directory, web services and databases.
• Developed Java Spring application to create oil well inventory database application. Used JPA/hibernate for database/table creation. Maintained and continued to develop application as user needs changed.
• Supported all 500+ SharePoint sites team. Setup/maintained integration wif Active Directory, Office Web apps, Exchange inbound/outbound mail configurations in relation to SharePoint and minimized features and shared services enabled as needed only.
• Created and managed Workflows (Nintex 2013). Implemented complex workflows for HR on boarding/termination wif integration into external systems using web APIs and InfoPath forms.
• Established/maintain Azure Site-to-site VPN wif on premise infrastructure to extend development VM’s into teh cloud to reduce costs by 66%. Setup SharePoint Online Hybrid environment (O365) to allow extend external vendors access files wifout compromising on premise system. Setup included SSO and bidirectional search functionality.
• Designed and developed custom C# SharePoint 2010/2013 Web Parts, Timer Jobs, and Branding.

SharePoint Architect/Administrator

Confidential

Responsibilities:

• Administered multiple SharePoint 2007 farms and 2 SharePoint 2010 farms wif 2000+ users. Designed and managed SharePoint hierarchy, from web apps to subsites for manageability and ease of troubleshooting.
• Adhered to company change protocols and maintained/created SharePoint site structures, user groups, Site Collections Content DBs, naming conventions etc. Supported all users and their collaboration sites.
• Created and managed SharePoint workflows using SharePoint designer as well as Nintex Workflows and InfoPath forms. Customized branding initiatives for intranet. Created custom PowerShell and .net scripts for health checks and VIP client requests.
• Designed/implemented a suitable backup/disaster recovery plan for teh farm using AvePoint Backup 5.4 for granular backup, SQL 2008 backups for Content DB recovery and HP Datacentres NetVeritas solution to back up OS. 3 Tier backup plan resulted in multiple levels of recoverability wif variable speeds of recovery depending on Recovery need.
• Increased fail over capacity and Load balancing ability to eliminate single point of failure and increase access speeds for users. Migrated existing 3 server (x86) farm running windows 2003 and SQL 2000 SP4 farm into a 5 server (x64) farm wif Windows 2008 Network Load Balancer and SQL 2008 R2 in a failover cluster. Integrated MS Forefront for SharePoint 2007 as anti-virus solution for farm.
• Enabled automation and ease of use for departments by utilizing custom workflows and modifications to SharePoint sites. This included using SharePoint Designer for workflows to creating console application using teh .net framework API's for SharePoint to modify alert templates for specific lists, custom access request pages, custom views for libraries, etc.

SharePoint Administrator

Confidential

Responsibilities:

• Designed SharePoint site collections using SharePoint designer 2007, customized master, layout pages including CSS. Integrated Outlook 2007 and MS Office 2007 into SharePoint for interoperability.
• Plan, design, develop, deploy, test and support existing functionality and create custom solutions wifin teh SharePoint environment including integration wif external applications such as Active Directory, CRM, ERP, web services and databases.
• Analyzing existing company data, organizing data into units wif functional requirements and integrating into SharePoint architecture. Setting up mandatory field data (xml metadata) for all documentation imported and existing on system. Developed specific workflow control and automation for regular business needs such as Expense sheets, vacation sign off, HR activities Developed SharePoint Master and Core CSS pages.
• Managed change control and release protocols for SharePoint 2007 projects. Upgrades from MOSS 2003 and Service pack installations.
• Created custom workflows, InfoPath forms, a central report depository to reduce redundant workload for managers across all departments. Increased productivity and accuracy of forms and day to day activities.
• Review usage logs/patterns to establish best possible change windows to minimize user impact. Resulted in maximum uptime for employee resulting in more productivity. Running Loadrunner to measure workload capacities.