SUMMARY

• Over 8 years of IT professional experience with 3+ years of experience as Hadoop Administrator.
• Experience with complete Software Design Life Cycle including design, development, testing and implementation of moderate to advanced complex systems.
• Good experience in installation, configuration and management of clusters in Cloudera (CDH4) & Hortonworks (HDP 2.2) distributions using CM and Ambari 2.2.
• Experience inHadooparchitecture and its various components such as HDFS, Job Tracker, Task Tracker, Name Node, Data Node and MapReduce concepts.
• Good understanding and hands on experience of Hadoop Cluster capacity planning, performance tuning, cluster monitoring, troubleshooting.
• Hands on experience in installation of various Hadoop Ecosystem components like Spark, MapReduce, HDFS, Hive, HBase, and Zookeeper and Hadoop Daemons.
• Experience in adding and removing nodes to an existing Hadoop cluster.
• Experience in configuring Zookeeper to provide high availability and Cluster service co - ordination.
• Extensive experience on commissioning, decommissioning, balancing and managing nodes.
• Have configured various property files like core-site.xml, hdfs-site.xml, mapred-site.xml andhadoop-env.sh to set up required environment according to the job requirement.
• Good knowledge on analyzinglog files for Hadoop ecosystem servicesand troubleshooting.
• Experience in setting up automated monitoring and escalation infrastructure for HadoopCluster usingGanglia and Nagios.
• Strong knowledge in Name node high availability and recovery of Name node metadata and data residing in the cluster.
• Experience in configuration and management of security for Hadoop cluster using Kerberos.
• Hands on experience on configuring aHadoopcluster in a professional environment and on Amazon Web Services (AWS) using an EC2 instance.
• Good knowledge in NOSQL databases like HBase, Cassandra.
• Experience in Ranger, Knox configuration to provide the security for Hadoop services.
• Designed and Developed the required data warehouse model using Star schema for the generalized model
• Used forward engineering approach for designing and creating data marts for OLAP model
• Have deep and thorough understanding of ETL tools and how they can be applied in a Big Data environment.
• Experience in designing data models for databases and Data Warehouse/Data Mart/ODS for OLAP and OLTP environments.
• Having Strong Experience in LINUX/UNIX Administration, expertise in Red Hat Enterprise Linux 4, 5 and 6.
• Possesses excellent oral and written communication skills and works well in team environment.

TECHNICAL SKILLS

Big Data Ecosystem: HDFS, HBase, MapReduce, Zookeeper, Hive, Pig, Sqoop, Kafka, Flume, Oozie, Spark.

Database: SQL Server 2000/2005/2008 , MySQL, NoSQL Databases HBase, Redis,MongoDB, Cassandra

Scripting Languages: UNIX shell scripting (bash), Python, SQL, PIG LATIN, HIVEQL

ETL tools: Talend Open Studio, ER studio

Network: Http/Https, TCP/IP, ssh, Ftp, Telnet.

Security: Kerberos Security.

Operating Systems: Unix, Linux (RHEL, Ubuntu, CentOS), AIX, Windows XP, Server 2000, 03, Server 2008.

PROFESSIONAL EXPERIENCE

Confidential, Boca Raton, Florida

Hadoop Admin

Responsibilities:

• Installed and configured a Horton Works HDP 2.2 using Ambari and manually through command line.
• Worked on analyzing Hadoop cluster and different big data analytic tools including Pig, HBase and Sqoop.
• Responsible for building scalable distributed data solutions using Hadoop.
• Worked on installing cluster, commissioning & decommissioning of datanode, namenode recovery.
• Created HBase tables to store variable data.
• Managing and reviewing Hadoop log files and debugging failed jobs.
• Implemented Kerberos Security Authentication protocol for production cluster.
• Worked on tuning the performance Pig queries.
• Worked with Infrastructure teams to install operating system, Hadoop updates, patches, version upgrades as required.
• Backed up data on regular basis to a remote cluster using distcp.
• Responsible to manage data coming from different sources.
• Involved in data analysis projects using Elastic Map Reduce on the Amazon Web Services (AWS) cloud.
• Cluster coordination services through Zookeeper.
• Loaded the dataset into Hive for ETL Operation.
• Installation of spark, storm, kafka and configuring it as per the requirements.
• Exported the analyzed data to the relational databases using Sqoop for visualization and to generate reports.
• Implemented Fair scheduler to allocate fair amount of resources to small jobs.
• Worked with the BI team by partitioning and querying the data in Hive.
• Involved in analyzing large amounts of data sets to determine optimal way to aggregate and report on it.

Environment: HADOOP HDFS, MapReduce, HORTON WORKS, Ambari, EC2 (AWS), Yarn, Hive, Pig, Kafka, Oozie, Sqoop, ER studio, HBase.

Confidential, Hartford, CT

Hadoop Admin

Responsibilities:

• Experienced on setting up Horton works cluster and installing all the ecosystem components through Ambari and manually from command line.
• Cluster maintenance, Monitoring, commissioning and decommissioning of data nodes, troubleshooting, manage and review log files.
• Actively involved in installation performance tuning, patching, regular backups, user account administration, upgrades and documentation.
• Installation of new components and removal of them through Ambari.
• Configured Zookeeper to coordinate the servers in clusters to maintain the data consistency.
• Periodically reviewedHadooprelated logs and fixed errors.
• Commissioned new cluster nodes for increased capacity and decommissioned servers with hardware problems.
• Responsible for adding new eco system components, like spark, storm, flume, knox with required custom configurations based on the requirements and Hadoop daemons.
• Developed Python, Shell/Perl Scripts and Power shell for automation purpose.
• Implemented Kerberos Security Authentication protocol for existing cluster.
• Worked with Ranger, Knox configuration to provide centralized security to Hadoop services.
• Hands on experience with NoSQL databases like Hbase, Cassandra and MongoDB.
• Working experience on maintaining MySQL databases creation and setting up the usersf and maintain the backup of databases.
• Managed and reviewed Hadoop Log files as a part of administration for troubleshooting purposes.
• Performing Linux systems administration on production and development servers (Red Hat Linux, CentOS and other UNIX utilities).

Environment: HDP 2.2, HDFS, Python, HBase, Kafka, Yarn, MongoDB, Hive, Impala, Oozie, Pig, Sqoop, HBase, Shell Scripting, MySQL, RHEL, Cent OS, Ambari.

Confidential, Tarrytown, NY

Hadoop Admin

Responsibilities:

• Installed Cloudera CDH 4 cluster (name node and data nodes).
• MonitoredHadoopcluster and file system usage on name and data nodes.
• Created user accounts and given users the access to theHadoopcluster.
• Performed HDFS cluster support and maintenance tasks like adding and removing nodes.
• Lead various data conversion initiatives that included testingHadoopSQL performance using bucketing and partitioning.
• Installed and configured Puppet that included the installation and configuration of the Puppet master, agent nodes and an admin control workstation.
• Use of Sqoop to import and export data from HDFS to RDBMS and vice-versa.
• Implemented HIVE security using Kerberos.
• Exported all log files generated from various sources to HDFS for further processing.
• Monitoring Hadoop Cluster through Cloudera Manager and Implementing alerts based on Error messages. Providing reports to management on Cluster Usage Metrics.
• Working with data delivery teams to setup new Hadoop users. This job includes setting up Linux users, setting up Kerberos principals and testing HDFS, Hive.
• Worked on YUM configuration and package installation through YUM.
• Worked with application teams to install operating system, Hadoop updates, patches, version upgrades as required.
• Performed installation of Apache web server for Ganglia to publish charts and graphs in the web console.
• Set up automated monitoring forHadoopcluster using Ganglia, which helped figure out the load distribution, memory usage and provided an indication for more space.

Environment: Hadoop 2.5, HDFS, Map Reduce, Hive, Flume, Sqoop, Cloudera Manager, CDH4, HBase, Puppet, Pig.

Confidential

Linux Administrator

Responsibilities:

• Provisioning, building and support of Linux servers both physical and virtual using VMware for production, QA and developers environment.
• Installed, configured and Administrated of all UNIX/LINUX servers, includes the design and selection of relevant hardware to support the installation/upgrades of RHEL, Cent OS, Ubuntu operating systems.
• Utilized bash and ksh shell scripting to automate daily system administration tasks
• Managed data backup of UNIX, Windows, Virtual servers, disk storage tier1 and tier2 backups.
• Ensured that various backup life cycle policies, and daily backup jobs are running, and failed jobs are fixed.
• Maintained configuration and security of the UNIX/LINUX operations systems with the enterprise's computing environment.
• Monitored system activities and fine-tuned system parameters and configurations to optimize performance and ensure security of systems.
• Responsible for maintenance of development tools and utilities and to maintain shell, Perl automation Scripts.
• Created and maintained detailed procedural documentation regarding operating system installation and configuration of software packages.
• Used Agile/scrum Environment and used Jenkins, GitHub for continuous integration and deployment.
• Responsible for configuring real time backup of web servers. Log file was managed for troubleshooting and probable errors.
• Responsible for reviewing all open tickets, resolve and close any existing tickets.
• Performance Tuning, Client/Server Connectivity and Database Consistency Checks using different Utilities.
• Installation, Configuration, upgrade and administration of Ubuntu, Red Hat Linux.
• Worked with File System includes UNIX file System and Network file system.

Environment: RHEL 4/5, Ubuntu, CentOS 4/5, SUSE Linux 10.1/10.3, VMware, Jenkins, Agile, Yum, Rpm.

Confidential

Linux Administrator

Responsibilities:

• Installation and configuration of Red Hat Linux, Solaris, Fedora and Cent OS on new server builds as well as during the upgrade situations.
• Log management like monitoring and cleaning the old log files.
• Administration of RHEL4.x, 5.x which includes installation, testing, tuning, upgrading and loading patches, troubleshooting both physical and virtual server issues.
• System audit report like no. of logins, success & failures, running cron jobs.
• System performance for hourly basis or daily basis.
• Remotely copying files using sftp, ftp, scp, winscp, and filezilla.
• Created user roles and groups for securing the resources using local operating System authentication.
• Experienced in tasks like managing User Accounts and Groups, managing Disks and File systems.
• Installing Red Hat Linux using kick start and applying security polices for hardening the server based on company’s policies.
• Install and configure Instruction Detection System (IDS) like Tripwire, Snort, and Lids.
• Configuring & monitoring DHCP server.
• Taking backup using tar and recovering during the data loss
• Wrote bash scripts for job automation.
• Manage system installation, troubleshooting, maintenance, performance tuning, managing storage resources, network configuration to fit application and database requirements.
• Responsible for modifying and optimizing backup schedules and developing shell scripts for it.
• Performed regular installation of patches using RPM and YUM.
• Maintained LVM, VxVM and SVM file systems along with NFS.

Environment: Linux (Red Hat Enterprise, CentOS), Windows 2000/NT, HP, IBM, Solaris, Oracle 8i, Cisco routers/switches, RPM, YUM, DHCP.