Job Seekers, Please send resumes to resumes@hireitpeople.com

Detailed Job Description:

• Data platform to be moved from hdfs to aws.
• EC2, S3, databases, batch processing using spark.
• Start of data fabric, platform of data.
• Want to take it cafeteria style model, walk and pick what data you want and use it.
• Current is specific data shapes in S3, then to database for apps (it doesn’t scale, it’s one off).
• They want the future to be able to build to scale and democratic the data.
• Spark is in clusters now and they want to move to a platform service and allow database as a service (auto load data shape).
• AWS is required as they want to build a platform team around this resource going forward.
• Spark is also required (Scala is base code, some in Python, but they want to move to Java and Scala.
• They also want to make this self-service.  Need to drive system by metadata (semantic layer, model, etc.).
• Abstract design for meta data systems.  They are using AIRFLOW.
• Moving toward Severless app framework.

Experience required: 5 Years