Description:-

We are looking for a Systems Architect to install Hadoop and drive a couple use cases with Spark and Hive. The results of this POC will need to be presented to a larger management audience, with a dependent subscription deal, so presentation skills/soft skills are required. 

Description of Services.

Scope of Work:

1. Provide requirements for HDP2.2, Hive, Stinger. next and Spark
2. Install/Upgrade HDP 2.2 on an existing 6-Node Cluster
3. Configure HDP 2.2, Hive, Stinger. Next and Spark
4. Process one day worth of app log thru LMF scripts using Spark
5. Upload User Dim (approx. 200 M records) to New Hive table
6. Process incremental User Dim using update feature of Hive
7. Create external Hive table on result set of LMF and run hive query by linking user dim using Tez and spark
8. Configure Hive 14 for ACID and demo INSERT/UPDATE/DELETE capabilities
9. Configure Hive View on Ambari, and demonstrate access to Hive/Tez via the view
10. Findings & Results Documentation - Hortonworks will capture and provide documentation depicting the physical cluster design and configurations along with the findings and results of the above use cases via agreed format for Customer’s HDP 2.2 POC project
11. Migration of two Pig Scripts from existing cluster to new POC cluster. Migration includes related testing & QA.