Sr. Spark /scala Developer Resume
Boston, MA
SUMMARY:
- Building competitive intelligence and analyses around Big Data made by the successful installation, configuration and administration of Hadoop ecosystem components, machine learning and advanced data processing using Scala and R.
- 15+ years of experience in all stages of Software engineering and Software Development Life Cycle - System Study, Analysis, Design, Development and Implementation of core Business Software and Enterprise Applications using Windows and web-related tools.
- Lead our customer teams in architecture guidance wide range of topics including tool selection.
- 3+ year’s development experience in Hadoop ecosystem components.
- Worked on both Cloudera, Hortonworks platforms and plain apache versions too
- Helped with deployments Hadoop Eco system onto cloud(AWS/AZURE)
- Capable of processing large sets of structured, semi-structured and unstructured data and supporting systems application architecture.
- Able to assess business rules, collaborate with stakeholders and perform source-to-target data mapping, design and review.
- Familiar with data architecture including data ingestion pipeline design, Hadoop information architecture, data modeling and data mining, machine learning and advanced data processing. Experience optimizing ETL workflows.
- 3 years of experience in Java, R, Python and Scala
- 11 years of experience in .Net Framework Architecture, .Net Application Architecture, Web Services Architecture, Microsoft VS .Net Development Environment, C#, VB .Net, ASP.Net, Web Services, Web API’s, ADO .Net, LINQ.
- Have excellent skills in developing code for database applications, extensively used LINQ, ADO.NET access data from RDBMS databases like SQL Server, Oracle 9i and MS Access.
- Experience/Knowledge about the version controlling tools such as Visual Source Safe, TFS and PVCS.
- Strong organizational and management skills
TECHNICAL SKILLS:
Big Data Ecosystems: Hadoop, MapReduce, HDFS, HBase, Zookeeper, Hive, Pig, Sqoop, Cassandra, Oozie, Flume, Spark, Machine level algorithms and R, Kinesis,Lambda, S3, EC2, VPC, Redshift, IAM, EBS, Cloudfront, Snowball, Cloudwatch, DynamoDb, Glacier, SES, SNS
Programming Languages: Java,Scala, C#, VB.net, ASP.NET, WCF
Scripting Languages: ASP.net, JavaScript, XML, HTML, Python and Bash
Databases: NoSQL, Oracle, SQL Server
UNIX Tools: Apache, Yum, RPM, putty, mobaXterm
Tools: Eclipse, CVS, Ant, MS Visual Studio, Pandas, Zeppelin
Platforms: Windows(2000/XP), Linux, Solaris, Centos
Application Servers: IIS
Methodologies: Agile, UML, Design Patterns
PROFESSIONAL EXPERIENCE:
Confidential, Boston MA
Sr. Spark /Scala Developer
Responsibilities:
- Suggested the platform selection along with tools with in the HDP stack
- Supported the architecture, design, and deployment of HDP data lack along with security features and work with IT teams for instilling best practices
- Provide expertise and guidance around HDP development Provide mentoring and to co-workers
- Build scripts to deployment systems configuration activities
- Leading meetings, discussions, white boarding critical issues.
- AWS Cloud Implementation Security management
- Provide expertise and guidance around HDP development
- Provide troubleshooting support on Hadoop related issues encountered on Hortonworks clusters
- Developed agents to for analyzing the data
- Created HBase tables to load large sets of structured, semi-structured and unstructured data coming from different vendor systems.
- Supported code/design analysis, strategy development and project planning.
- Developed MapReduce programs to parse the raw data, populate staging tables and store the refined data in partitioned tables in the EDW.
- Created Hive queries that helped OR analysts spot emerging bottlenecks by comparing fresh data with EDW tables and historical metrics.
- Enabled speedy reviews and first mover advantages by using Oozie to automate data loading into the Hadoop Distributed File System and PIG to pre-process the data.
- Provided design recommendations and thought leadership to sponsors/stakeholders that improved review processes and resolved technical problems.
- Managed and reviewed Hadoop log files.
- Tested raw data and executed performance scripts.
- Shared responsibility for administration of Hadoop, Hive and Pig.
- Installed and configured MapReduce, HIVE and the HDFS; implemented CDH3 Hadoop cluster on CentOS. Assisted with performance tuning and monitoring.
- Created HBase tables to load large sets of structured, semi-structured and unstructured data coming from different vendor systems.
- Supported code/design analysis, strategy development and project planning.
- Created reports for the BI team using Sqoop to export data into HDFS and Hive.
- Developed multiple MapReduce jobs in Java for data cleaning and preprocessing.
- Assisted with data capacity planning and node forecasting.
- Collaborated with the infrastructure, network, database, application and BI teams to ensure data quality and availability.
- Moving MapReduce programs to spark job using Scala
- Administrator for Pig, Hive and Hbase installing updates, patches and upgrades
- Gather, Analyze requirements and Design, development of Architectural Demos
- Used third party tools like Kendo controls
- Developed UI with Kendo controls,JQuery, HTML5, CSS3.0, AJAX, JSON
- Developed application with ASP.NET MVC 4.0 /C#, WCF
- Development of WCF Services to provide SOA model
- Design/Development of the Data Service Layer with entity framework
- Analyzing concurrency Rules and requirements and process of Integration with other departments systems.
- Connecting ORSOS Webservices to get daily case list
- Analyzing third party tools like Kendo controls
- Talking to other dept Vendors in the Integration process.
- XSLT/XML/XSD to communicate among systems.
- Requirement Analysis with Doctors for Airway/BP/CO2 management.
- Designing new systems to meet the quality measures
- Planning and Division of work among team members and guiding team members.
- Developed workflows using WWF and Services with WCF to integrate different systems
- Used Team Foundation Server for version control.
Environment: Microsoft Windows Server 2008, Internet Information Server (IIS), Microsoft .Net Framework4.0, WCF/LINQ/WPF/WWF, Microsoft Visual Studio .Net 2010, C#.Net, Web Services, ASP.Net 4.0, ADO .Net, SQL Server 2008
Confidential, Billerica MA
. NET Consultant
Responsibilities:
- Requirement Analysis of Real ID-Driver License for the state of Wisconsin /Delaware.
- Preparing GAP-Analysis Report between product and Customer requirements.
- Providing Architectural Solution to meet the need of the Wisconsin State Requirements.
- Developed and Exposed WCF Services to WI-Sate to get Processed Customer Photo, Signature and Documents
- Provided WCF Services to DMV-Systems to Complete the Workflow of Driver-License Issuance using N-Tire Architecture.
- Interfaced with external Q-Matic Queuing system to get Customers, which is part of workflow.
- Design/Development of the Service Layer and Business components
- Workstation Development using Rich UI Windows Desktop applications with WPF using composite Framework.
- Implemented Active Directory security mechanism to Authenticate and Authorize users to use different applications/functionalities with Role based Authorization
Environment: IIS, .Net Framework3.5, WCF/WPF/LINQ, Visual Studio .Net 2008, C#.Net, Web Services, Oracle Data Provider for .Net, ASP.Net 3.5, ADO .Net, Oracle 11g, XML, XSLT, Report Viewer, Ajax, CAB/SCSF
Confidential, NEWYORK NY
Sr. NET Consultant
Responsibilities:
- Suggesting the technologies to use from MS stack.
- Developed conceptual model/logical model /physical model of database.
- Determining the technologies to use in the project base on the current and future requirements of the system by developing the prototypes using WWF and ASP.NET
- Prepared LLD, Planning of the work and division of the work
- Developed WCF Services. Developed Web application using Asp.net
- Support to testing team during testing phase
- Scheduled Deployments of the application to SIT/UAT
Environment: Windows Server 2003, IIS, .Net 2.0/3.5, WCF/WWF, SharePoint server2007 Visual Studio .Net 2005, C#.Net 2005/2008, ASP .Net 2.0, ADO .Net, Oracle10g
Confidential, St .Petersburg, FL
Sr.NET Consultant
Responsibilities:
- Building shopping Cart functionality, by using Adaptor, Container, and Enumerator pattern
- Build tests using unit-tests of .net
- Developed shopping cart, wish-list modules in C#
- Developing web-services, which communicate with ERP system (Seibel) for checking the availability of the product, reserve the product, ordering and shipping
Environment: Microsoft Windows Server System 2003, Internet Information Server (IIS), Microsoft .Net Framework 2.0/3.0, WCF, SharePoint server2007 Microsoft Visual Studio .Net 2005, Microsoft Visual C#.Net 2005, Web Services, SQL Data Provider for .Net, ASP .Net 2.0, ADO .Net, SQL Server 2005
Confidential
Software Engineer, Team Lead
Responsibilities:
- Extracting Business Rules from Existing Systems during the Analysis phase.
- Used Application blocks for Data access and Configuration and security, cashing
- Developed SSIS packages in SQL Server2005
- Migrated PowerBuilder/ Sybase application to C#/ SQL server2005
- Developed number of stored procedures, views, functions, indexes views during migration from Sybase to SQL 2000
- Developed DTS Packages to perform daily tasks of transaction processing with automation and also distribute the data between the systems
- Involved in converting the High Level Design Document and Detail Design Document of the system to class diagrams and sequence diagrams in the rational unified process methodology.
- Developed web services to communicate with Core banking servers
- Used XML extensively for data definitions, data storage for ease of interoperability between core banking and internet banking.
- Database modeling using Rational Rose
- Developing/ Enhancing and maintaining different Use Case diagrams of the application captured from system requirements using Rational Rose, Rational Requisite Pro tool.
- Writing Trigger, Stored Procedure, Views and User defined procedure Developed User Interfaces, designed and coded the functionality for WISeBank
- Developing Report using BO
- Developed application in VB6.0. Developing Database Layer with ActiveX Dll and supporting testing team
- Writing stored procedures with T-SQL, Bug fixing, Releasing the product after bug fixing
- Written User & Functional Documentation.
- Developed reports using Crystal Reports.
Environment: VB.Net, VB 6.0, VSS 6.0, SQL Server 2005/2000/7.0, Oracle, IIS, C#.Net, ADO .Net, Business Objects, Linux, PL-SQL, Rational Rose, Rational Requisite Pro, Toad, Visual Studio 2005, C#.Net 2005, ASP .Net 2.0, ADO .Net, SVN, Nuint, PowerBuilder, Sybase