Job Seekers, Please send resumes to resumes@hireitpeople.com
Must have skills:
- Spark streaming
- Apache Kafka
- Python scripting
- No SQL
Detailed Job Description:
- Seeking an experienced Data Engineer Big Data individual for Atlanta.
- Candidate must have Big Data engineering experience and must demonstrate an affinity for working with others to create successful solutions.
- Will work on NS Big Data Platforms Cloudera.
- Must be a very good communicator, and have some experience working with business areas to translate their business data needs and data questions into project requirements.
- Will participate in all phases of the Data Engineering life cycle
Skills and Experience:
- 3+ years of experience with high-velocity high-volume stream processing: Apache Kafka and Spark Streaming
- Experience with real-time data processing and streaming techniques using Spark structured streaming and Kafka
- Deep knowledge of troubleshooting and tuning Spark applications
- 3+ years of experience with data ingestion from Message Queues (Tibco, IBM, etc.) and different file formats across different platforms like JSON, XML, CSV
- 3+ years of experience with Big Data tools/technologies like Hadoop, Spark, Spark SQL, Kafka, Sqoop, Hive, S3, HDFS, or Cloud platforms e.g. AWS, GCP, etc.
- 3+ years of experience building, testing, and optimizing Big Data data ingestion pipelines, architectures and data sets
- 2+ years of experience with Python (and/or Scala) and PySpark
- 2+ years of experience with NoSQL databases, including HBASE and/or Cassandra
- Knowledge of Unix/Linux platform and shell scripting is a must
- Strong analytical and problem solving skills
Minimum years of experience required: 6+
