Hadoop Admin Resume
Broomfield, CO
SUMMARY
- 7 +years of experience in IT industry which includes 2+ years of experience in Big Data and Hadoop Ecosystem tools.
- 4 years of experience in Oracle Database Administration.
- 1 year of experience in SAP Business objects.
- Excellent working knowledge of HBase and data pre - processing using FLUME-ng.
- Experience in writing MapReduce jobs using apache Crunch.
- Hands on experience in installing, configuring Hadoopecosystems like Flume-ng, Hbase, Zoo Keeper, Oozie, Hive, Sqoop,Hue, Pig andHue with CDH3.x &4.x.
- Experience in Big Data analysis using PIG and HIVE and understanding of SQOOP and Puppet.
- Experience in developing customized UDF’s in java to extend Hive and Pig Latin functionality.
- Good understanding of HDFS Designs, Daemons, federation and HDFS high availability (HA).
- Strong experience working with Test Driven Development (TDD)techniques such as JUunit and Mockitoalong with code coverage tools like Emma.
- Well versed indesigning and implementing MapReduce jobs using JAVA on Eclipse to solve real world scaling problems.
- Solid understanding of the high volume, high performance systems.
- Fair amount of experience working using with MapReduce
- Good knowledge in integration of various data sources like RDBMS, Spreadsheets, Text files, JSON and XML files.
- Basic Knowledge of UNIX and shell scripting.
- Have flair to adapt to new software applications and products, self-starter, have excellentcommunication skills and good understanding of business work flow.
TECHNICAL SKILLS
Big Data Technologies: Hadoop (HDFS, Hive, Pig, Flume, Oozie, Zookeeper, HBASE, Cassandra, Cloudera Manager and Sqoop), Datameer
Operating System: RHEL 5, 6, Ubuntu, Windows 2000/2003/XP/Vista.
Script Languages: Shell scripts, PERL, Python
Networks: LVM, NFS, DHCP, DNS
RDBMS: MySQL, Postgre SQL
Protocols: SSH, SFTP, HTTP, nslookup, TCP/IP, and tcpdump.
Programming Languages: C, SQL, PL/SQL, HTML, Java, XML and Pig Latin
Version control: SVN
PROFESSIONAL EXPERIENCE
Confidential - Broomfield, CO
Hadoop Admin
Responsibilities:
- Responsible for architecting Hadoop clusters Translation of functional and technical requirements into detailed architecture and design.
- Working on a live 100 nodes Hadoop Cluster Running of CDH4.
- Working with highly unstructured and semi structure data of 90 TB in size (270 TB in replication factor of 3)
- Installed and configured multi-nodes fully distributed Hadoop cluster of large number of nodes. " Provided Hadoop, OS, Hardware optimizations,
- Setting up the machines with Network Control, Static IP, Disabled Firewalls, Swap memory. '
- Installed and configured Cloudera Manager for easy management of existing Hadoop cluster.
- Working on setting up high availability for major production cluster and designed automatic failover control using zookeeper and quorum journal nodes
- Implementing Fair scheduler on the job tracker to allocate fair amount of resources to small jobs.
- Performing operating system installation, Hadoop version updates using automation tools.
- Configuring Oozie for workflow automation and coordination.
- Implementing rack aware topology on the Hadoop cluster.
- Configuring Zookeeper to implement node coordination, in clustering support.
- Configuring Flume for efficiently collecting, aggregating and moving large amounts of log data from many different sources to HDFS.
- Involving in collecting and aggregating large amounts of streaming data into HDFS using Flume and defined channel selectors to multiplex data into different sinks.
- Implementing Kerberos Security Authentication protocol for existing cluster.
- Good experience in troubleshoot production level issues in the cluster and its functionality. . Backed up data on regular basis to a remote cluster using distcp.
- Regular Commissioning and Decommissioning of nodes depending upon the amount of data. installing and maintain puppet-based configuration management system
- Deployed Puppet, Puppet Dashboard, and Puppet DB for configuration management to existing infrastructure.
- Using Puppet configuration management to manage cluster.
- Experience working on API. Generated reports using the Tableau report designer.
- Generating reports using the Tableau report designer
Environment: Cloudera Hadoop, Map reduce, HDFS, HBASE, OOZIE, JIRA, GITHUB
Confidential - Chantilly, VA
Hadoop Admin
Responsibilities:
- Installed and configured Hadoop MapReduce, HDFS, Supported multiple MapReduce-YARN jobs in Java for data cleaning and preprocessing.
- Importing and exporting data into HDFS and Hive using Sqoop.
- Experienced in detaining job flows.
- Experienced in managing and reviewing Hadoop log files.
- Experienced in running Hadoop streaming jobs to process terabytes of XML format data.
- Load and transform large sets of structured, semi structured and unstructured data.
- Expertise in Hadoop cluster task like commissioning nodes, decommissioning nodes without any effect to running jobs and data.
- Responsible to manage data coming from different sources.
- Supported Map Reduce Programs those are running on the cluster.
- Implemented Fair schedulers on the Job tracker to share the resources of the cluster for the map reduce jobs given by the users.
- Cluster coordination services through Zoo Keeper.
- Involved in loading data from UNIX file system to HDFS. .
- Involved in creating Hive tabbies, loading with data and writing Hive queries that will run internally in MapReduce way.
- Automated all the jobs, for pulling data from FTP server to load data into Hive tables, using Oozie workflows.
- Production support responsibilities include cluster maintenance, Backup and recovery.
- Gained very good business knowledge on health insurance, claim processing, fraud suspect identification, appeals process etc.
- Production support responsibilities include cluster maintenance and on call support on weekly rotation 24/7 Environment.
- REL, CentOS, Ubuntu. CDH3, Cloudera Manager, HDFS, Map, Reduce, Hbase, MySQL, Shell Scripts, Ganglia, and Nagios.
Environment: Eclipse, Linux, Hadoop, HBase, Sqoop, Pig, Hive, Flume, Zoo keeper.
Confidential, Sacramento, CA
Oracle Database Admin
Responsibilities:
- Configuration and deployment of Real Application Clusters (RAC) on different nodes and experience in adding and removing nodes.
- Scheduled a regular hot backup process and involved the in the backup activities involved in installation,
- Managing two nodes Real Application Cluster databases on Red hat Linux. Database refresh from Production to Test as needed.
- Responsible for moving data from one database to other using Traditional exp/imp and Data Pump Export/import.
- Up-gradation and Migration of Database from 10g to 11g and applying 10patches whenever required.
- Performed SQL and PLISQL tuning, oracle contention and monitored production database using automatic workload repository {AWR) and automatic database diagnostics monitor (ADDM).
- Backup/Recovery of Oracle databases using Oracle RMAN.
Environment: Linux, UNIX, Microsoft Operating Systems, Tools, SQL, PL SQL, RMAN, AWR, ADDM
Confidential
Oracle Database Administrator
Responsibilities:
- Installation and configuration Standby and RAC Database.
- Up-gradation and Migration of Database from 9i to 10g and applying patches whenever required.
- Experience with Oracle Supplied Packages such as DBMS*SQL and DBMS JOB.
- Perform Backup and Recovery on Oracle databases.
- Responsible for moving data from one database to other using Traditional exp/imp and Data Pump Export/import.
- Backup/Recovery of Oracle databases using Oracle RMAN.
- Monitoring the database performance using OEM Grid Control ' Database tuning, Application Tuning & performance monitoring. Fine tuning initialization parameters, I/O, Memory and Operating System kernel parameters.
- Performed SQL and PLISQL tuning, oracle contention and monitored production database using automatic workload repository {AWR) and automatic database diagnostics monitor (ADDM).
- 'Experience in Database Cloning and refreshing using Traditional exp/imp, Data Pump Export & Data Pump import.
- Scheduled a regular hot backup process and involved the in the backup activities involved in installation,
- Configuration and deployment of Real Application Clusters (RAC) on different nodes and experience in adding and removing nodes.
- Managing two nodes Real Application Cluster databases on Red hat Linux. Database refresh from Production to Test as needed.
Environment: Oracle 9i/10g RAC with Red hat, MS SQL 2005, Expdp/lmpdp, RMAN, Disaster Recovery Oracle Enterprise Manager (OEM), Toad, SQL plus, Shell Scripting.
Confidential
SAP Business objects consultant
Responsibilities:
- Creating, managing users and user groups.
- Used Web to create, analyze and publish reports to various user groups.
- Developed and created classes with Dimensions, Detail and Measure objects.
- Building Universe, retrieving data using personal data files, Free hand SQL methods and creating complex Adhoc reports.
- Created Standard reports and Ad hoc reports using Business Objects Reporter.
- Created Web intelligence Ad-hoc and canned reports.
- Wrote formulas, parameters and conditions depending upon selection criteria.
- Created hierarchy and conditions at the design level.
Environment: SAP ECC 6.0, SAP BI 7.0, APO, SCM, FI, CO-PA SD, MM, Business Objects 3.1, Xcelsius 2008, Crystal Reports.