Google DataFlow on Google Cloud Platform:
https://cloud.google.com/dataflow/docs
https://cloud.google.com/dataflow/docs/quickstarts/quickstart-python
Unified stream and batch data processing that’s serverless, fast, and cost-effective.
-
Fully managed data processing service
-
Automated provisioning and management of processing resources
-
Horizontal autoscaling of worker resources to maximize resource utilization
-
OSS community-driven innovation with Apache Beam SDK
Reliable and consistent exactly-once processing
Google Big Query on Google Cloud Platform:
https://cloud.google.com/bigquery/docs
https://cloud.google.com/bigquery/docs/quickstarts/quickstart-command-line
Serverless, highly scalable, and cost-effective multicloud data warehouse designed for business agility.
-
Democratize insights with a secure and scalable platform with built-in machine learning
-
Power business decisions from data across clouds with a flexible, multicloud analytics solution
-
Run analytics at scale with 26%–34% lower three-year TCO than cloud data warehouse alternatives
-
Analyze petabytes of data using ANSI SQL at blazing-fast speeds, with zero operational overhead