GCP Series: Google Data Analytics Pipeline and Cloud Functions

4 years ago   •   2 min read

By CloudNerve.com

How to Build a Scalable Data Analytics Pipeline with Google Cloud Platform

Overview:  Google Cloud Functions

Google Cloud Functions

Scalable pay-as-you-go functions as a service (FaaS) to run your code with zero server management.

  • No servers to provision, manage, or upgrade

  • Automatically scale based on the load

  • Integrated monitoring, logging, and debugging capability

  • Built-in security at role and per function level based on the principle of least privilege

  • Key networking capabilities for hybrid and multi-cloud scenarios

Key features

Simplified developer experience and increased developer velocity

Cloud Functions has a simple and intuitive developer experience. Just write your code and let Google Cloud handle the operational infrastructure. Develop faster by writing and running small code snippets that respond to events. Connect to Google Cloud or third-party cloud services via triggers to streamline challenging orchestration problems.

Pay only for what you use

You are only billed for your function’s execution time, metered to the nearest 100 milliseconds. You pay nothing when your function is idle. Cloud Functions automatically spins up and backs down in response to events.

Avoid lock-in with open technology

Use open source FaaS (function as a service) framework to run functions across multiple environments and prevent lock-in. Supported environments include Cloud Functions, local development environment, on-premises, Cloud Run, and other Knative-based serverless environments.

USE CASE #1:

Video and image analysis

Use Cloud Functions with Video Intelligence API and Cloud Vision API to retrieve relevant information from videos and images, enabling you to search, discover, and derive insight from your media content.

Workflow moves from left to right: A blue box with right-facing white triangle (“Play”) flows to Cloud Storage / (Full-length video) icon. Flow branches up to Front end built on App Engine and down to Cloud Functions then Video Intelligence. Flow moves along lower path with arrow labeled “Video Metadata” to icon labeled Cloud Storage / (Video annotation JSON). Flow then moves up and left back to Front end build on App Engine.

USE CASE #2:

Sentiment analysis

Use Cloud Functions in combination with Cloud Natural Language API to reveal the structure and meaning of text and add powerful sentiment analysis and intent extraction capabilities to your applications.

Sentiment analysis workflow diagram: Starting on left, icon labeled Text Message has speech bubble on top: “Devoxx is such an amazing event!” Solid arrow leads right to box labeled Twilio, from which dashed arrow leads left to the Text message. From Twilio, solid arrow leads right to Cloud Functions icon, from which dashed arrow leads left to Twilio. From Cloud Functions, bi-directional arrows lead to Natural Language API / Sentiment extraction icon and to BigQuery/Trend analysis icon

Google Cloud Functions Documentation

https://cloud.google.com/functions/docs

Code Samples:

https://cloud.google.com/functions/docs#code-samples

Spread the word