Years of experience on each of the Technical must have skills: 5+
Any Certifications required: GCP Professional data engineer, but optional
Detailed Job Description:
What will this resource be doing? Development of Spark scala program in GCP environment and loading and validating data into GCP storage and Big query tables.
What outcome or deliverables are expected at the end of this assignment? Spark programs checked in Git repository along with CI/CD pipelines built
Additional Sills: Working knowledge of Google Cloud Platform on GCS, Bigtable, BigQuery Good understanding of cloud design considerations and limitations and its impact on Pricing. Proficient in a Python Scala programming for building and automating data pipelines.
Should have experience in building streaming pipelines using Kafka Knowledge of Application Monitoring using Google Stackdriver Experience with Big Data on GCP BigQuery, PubSub, Spark using Scala
