GCP Series: Google Cloud Platform Dataflow and Big Query

Google DataFlow on Google Cloud Platform:

https://cloud.google.com/dataflow/docs

https://cloud.google.com/dataflow/docs/quickstarts/quickstart-python

Unified stream and batch data processing that’s serverless, fast, and cost-effective.

  • Fully managed data processing service

  • Automated provisioning and management of processing resources

  • Horizontal autoscaling of worker resources to maximize resource utilization

  • OSS community-driven innovation with Apache Beam SDK

Reliable and consistent exactly-once processing

Google Big Query on Google Cloud Platform:

https://cloud.google.com/bigquery/docs

https://cloud.google.com/bigquery/docs/quickstarts/quickstart-command-line

Serverless, highly scalable, and cost-effective multicloud data warehouse designed for business agility.

  • Democratize insights with a secure and scalable platform with built-in machine learning

  • Power business decisions from data across clouds with a flexible, multicloud analytics solution

  • Run analytics at scale with 26%–34% lower three-year TCO than cloud data warehouse alternatives

  • Analyze petabytes of data using ANSI SQL at blazing-fast speeds, with zero operational overhead