Sr Cassandra/kafka Engineer Resume
SUMMARY
- 11+ years IT experience in banking and retail domains and my area of expertise have been NoSQL database administration & Customer information maintenance. My forte is Cassandra, Kafka, Solr, Spark with exposure to MDM & DB2
- Managed Development, Technology re - engineering projects, NoSQL database administration, messaging brokers administration involving Cassandra, Kafka, Spark, Solr, Zookeeper, Linux, MDM and DB2
- Have involved in Cassandra, Kafka, DB2 database administration and data modelling activities such as capacity planning, design, installation, maintenance, optimization monitoring and support.
- Closely worked with Datastax & Hortonworks Support in resolving Cassandra / Kafka bugs with new JIRA opened and get fix for DSE bugs.
- Have provide solution to application team with Cassandra/ Kafka issues related to CQL consistence level, timeouts, OOM scenario, Spark jobs, CQL queries.
- Experience with Data migration from DB2 to Cassandra database involves extensive Cassandra data model designing and implementation.
- Experience with Agile and waterfall development and project methodologies
- Dynamic, hard-working, ability to work in-groups as well as independently with initiative to learn new technologies/tool quickly and emphasis on delivering quality services
- Have strong ability to build productive relationships with peers, management, and clients using strong communication, interpersonal, organizational, and planning skills
- Excellent analytical, debugging and communication skills.
- Worked cross-functionally with other product teams, QA and DevOps teams extensively
- Represented project in organization level seminars and conferences on Implementation of Big data technologies in the projects.
TECHNICAL SKILLS
Databases: Cassandra, Kafka, Spark, Solr, Zookeeper, DB2
Reporting Tools: Datastax Opscenter, Splunk
Operating Systems: Unix, Windows, Linux, OS X
Environments: RedHat Linux, Virtual Machine environment
Languages: SQL, CQL, Python, shell scripting & Java
Product: MDM, DSE, Hortonworks
Software: Datastax DevCenter, IBM Data Studio, ERwin Data Modeller, SVN
PROFESSIONAL EXPERIENCE
Sr Cassandra/Kafka engineer
Confidential
Responsibilities:
- Architecture, design and capacity planning, implementation for various applications for Cassandra infrastructure
- Subject matter expert in different Big Data technologies like Cassandra, Kafka Solr and Spark.
- Seamless Installation of 70+ new Apache/DSE Cassandra, Apache/Hortonworks/confluent Kafka and Solr
- Upgradation & expansion ofthe current systems and decommission ofthe failure items includes add new node/new data centre into Cassandra cluster, also removing failure node and DC.
- Defining Linux / OS requirements for Cassandra / Kafka considering Datastax recommendations
- Closely worked with Datastax & Hortonworks Support in resolving Cassandra / Kafka issues or bugs with fixes/patching developed for an existing/new JIRA opened within DSE / Hortonworks. Few of Solution provided for issues listed below
- Issues during DSE upgrade. Conversion from Apache Cassandra to Datastax Cassandra
- DSE Spark executor detaching from master issue
- OOM issue during nodetool rebuild after bootstrapping new node
- Timeouts occurred due to long running nodetool repair / opscenter repairs
- Race condition issue during new DC addition
- Security issue fix for nodetool/dsetool SSL enabling/ JMX SSL enabling
- Commitlog overflow issue
- Tuneable repairs using open source tools
- SASL SSL setup on Kafka brokers with Kerberos
- Consumer rebalancing for high volume, high number of partitions
- Console offset resetting during production issues
- Implementing Downgrade Consistency Level policy for queries with business requirement Consistency level ALL
- Performing backup, bulk load, run repairs, monitoring using custom build scripts as well as Opcenters.
- Create Cassandra DB objects like keyspace, column family, indexes, Solr indexes, users and explored CQL to support various application activities.
- Installing Kafka message broker clusters zookeeper clusters and involved in creating Kafka topics, partitioning planning, SSL / Kerberos security setup, Monitoring and support.
- Provide cross-functional support to various application team which are using Cassandra/Kafka on all DB issues and queries
- Analysis is carried out to resolve various performance issues by suggesting better techniques and practices as part of issue resolution.
- Handle issues, bugs and provide solutions with root cause analysis and involve Datastax for product support if needed
- Active co-ordination & follow-up is being done all hardware, network issues by working with AppleNetwork/System engineering teams
- Technical leadership for resolving complex programming tasks. Several operational tasks are performed along with other tasksto managethe health of allthe clusters are also taken upregularly
- Responsible for ensuring quality deliverables within the stipulated timelines
Environment: Datastax Enterprise Cassandra, Apache Cassandra, Apache/Hortonworks/Confluent Kafka, Zookeeper, Solr, Spark, Python, Java, Datastax Opscenter, devcenter and Linux
Cassandra database Administrator
Confidential
Responsibilities:
- Deployment of new Datastax Cassandra, upgradation & expansion ofthe current systems and decommission ofthe failure servers
- Architecture and design for various applications with different business requirements
- Daily monitoring of alerts, Splunk event logs and Opscenter dashboards in frequent intervals
- Subject matter expert in different Big Data technologies like Apache Cassandra, DSE, Apache Kafka and Apache Spark
- Upgradation & expansion ofthe current systems and decommission ofthe failure items includes add new node/new data centre into Cassandra cluster, also removing failure node and DC
- Responsible to provide cross-functional support to various application team which are using Cassandra, MDM on all DB issues and queries
- Involved in documentation of Cassandra/Kafkaoperations/solutions, daily communication with team
- Active co-ordination & follow-up is being done all hardware, network issues by working with AppleNetwork/System engineering teams
Environment: Apache Cassandra, Datastax Opscenter, devcenter, Linux, Unix, oracle, IBM MDM and web services
DB2 admin/Cassandra DBA
Confidential
Responsibilities:
- Responsible for due diligence of Enterprise Contact History application currently runs on MDM product, DB2 Backend Database, Cassandra Database, batch scheduled in mainframes.
- Completed a POC on Apache Cassandra database version 2.0 by creating local DB and services
- Provide road map for the Cassandra conversion and implementation support
- Develop Cassandra Data model considering current functionality and business need of application
- Manage execution of Cassandra implementation on new platform
- Closely associated with Cassandra DBA in implementing Cassandra data model in application environment to ensure solution is not affecting existing business as usual
- Worked on data conversion by extracting data from DB2, reform data, and load data into Cassandra nodes. Provided support in application code implementation and post production support
- Suggested flight mode implementation to open live traffic on both data models
Environment: Apache Cassandra, Datastax DevCenter, FTP, IBM MDM, DB2, MQ, ERwin Data Modeller
Technical Developer
Confidential
Responsibilities:
- Involved in various phases of Banking Customer information project life cycle including impact analysis, development, testing, implementation and support activities.
- Worked extensively in data extraction, transformation and loading from source to target systems using IBM utility Unload-load, NDM
- MDM Upgrade, ECH Conversion, Channels on boarding, Linux Upgrade are the major projects worked/handled flawlessly.
- Participate in work group meeting to understand and provide valid input which helps to come up with accurate requirement
- Creation of High level design, Low level design documents, Unit Test Plan and Unit test scripts
Environment: DB2, COBOL, IMS, IBM Data Studio, DB2, MQ, ERwin Data Modeller, OS390 & Unix