Google Cloud Observability – Monitoring & Logging Suite

Google Cloud Observability (formerly Operations Suite)

Google Cloud Observability (formerly known as Google Cloud Operations Suite) provides integrated monitoring, logging, and trace managed services for applications and systems running on Google Cloud and beyond. It is built on a “One Observability” foundation that unifies Cloud Logging, Cloud Monitoring, and Cloud Trace, providing a single pane of glass for the generation, collection, routing, storage, and consumption of telemetry at scale.

Google Cloud Observability Suite
Credit Priyanka Vergadia

Cloud Monitoring

  • Cloud Monitoring collects measurements of key aspects of the service and of the Google Cloud resources used.
  • Cloud Monitoring provides tools to visualize and monitor this data.
  • Cloud Monitoring helps gain visibility into the performance, availability, and health of the applications and infrastructure.
  • Cloud Monitoring collects metrics, events, and metadata from Google Cloud, AWS, hosted uptime probes, and application instrumentation.
  • Cloud Monitoring supports PromQL (Prometheus Query Language) for querying all metrics, including Google Cloud system metrics, Kubernetes metrics, custom metrics, and log-based metrics.
  • Cloud Monitoring supports SLO Monitoring to define Service Level Objectives, track error budgets, and create alerting policies based on budget burn rate.
  • Cloud Monitoring provides customizable observability dashboards available in GKE, Compute Engine, Cloud Run, Cloud Functions, Cloud Storage, and many other Google Cloud services.

Google Cloud Managed Service for Prometheus

  • Google Cloud Managed Service for Prometheus lets you globally monitor and alert on workloads using Prometheus and OpenTelemetry, without having to manually manage and operate Prometheus at scale.
  • It supports managed collection on GKE, Compute Engine, and Cloud Run.
  • Prometheus metrics can be used with dashboarding, alerting, and SLO monitoring features inside Cloud Monitoring.
  • It supports PromQL-based alerting policies alongside Cloud Monitoring metrics and dashboards.
  • Grafana dashboards work natively with Managed Prometheus for visualization.

Cloud Logging

  • Cloud Logging is a service for storing, viewing and interacting with logs.
  • Answers the questions “Who did what, where and when” within the GCP projects.
  • Maintains non-tamperable audit logs for each project and organization.
  • Logs buckets are a regional resource, which means the infrastructure that stores, indexes, and searches the logs are located in a specific geographical location.
  • Cloud Logging supports OTLP-formatted log ingestion using OpenTelemetry Collector and the Telemetry API.
  • Cloud Logging supports log-based alerting policies to notify when specific messages appear in logs.

Log Analytics

  • Log Analytics (GA since January 2023) brings capabilities to search, aggregate, or transform logs at query time directly within Cloud Logging.
  • Log Analytics is powered by BigQuery, allowing SQL queries on log data without separate exports.
  • Log Analytics supports pipe syntax (available since February 2025) for more intuitive querying.
  • It enables security log analytics for auditing cloud usage and detecting potential threats using Community Security Analytics (CSA) queries.
  • Log Analytics reduces costs and accelerates time to value compared to traditional BigQuery log exports.

Error Reporting

  • Error Reporting aggregates and displays errors produced in the running cloud services.
  • Error Reporting provides a centralized error management interface, to help find the application’s top or new errors so that they can be fixed faster.
  • Error Reporting can analyze log entries routed to log buckets in non-global regions, provided the log sink is in the same project as the log bucket.
  • Error Reporting supports Slack and webhook notifications for error alerts.

Cloud Profiler

  • Cloud Profiler helps with continuous CPU, heap, and other parameters profiling to improve performance and reduce costs.
  • Cloud Profiler is a continuous profiling tool that is designed for applications running on Google Cloud:
    • It’s a statistical, or sampling, profiler that has low overhead and is suitable for production environments.
    • It supports common languages and collects multiple profile types.
  • Cloud Profiler consists of the profiling agent, which collects the data, and a console interface on Google Cloud, which lets you view and analyze the data collected by the agent.
  • Cloud Profiler is supported for Compute Engine, App Engine, GKE, and applications running on on-premises as well.
  • Continuous profiling alleviates the need to develop accurate predictive load tests and benchmarks for the production environment.

Cloud Trace

  • Cloud Trace is a distributed tracing system that collects latency data from the applications and displays it in the Google Cloud Console.
  • Cloud Trace helps understand how long it takes the application to handle incoming requests from users or applications, and how long it takes to complete operations like RPC calls performed when handling the requests.
  • Cloud Trace can track how requests propagate through the application and receive detailed near real-time performance insights.
  • Cloud Trace automatically analyzes all of the application’s traces to generate in-depth latency reports to surface performance degradations and can capture traces from all the VMs, containers, or App Engines.
  • Cloud Trace supports OTLP span ingestion from applications instrumented with OpenTelemetry.
  • Trace data is stored in observability buckets with expanded regional support (40+ regions as of 2026).

Trace Explorer

  • The Trace Explorer provides an interactive span duration heatmap, span rate line chart, and span duration percentile chart.
  • It allows filtering by service, application, error state, and other properties.
  • The Grouped tab shows how many spans different services are generating with latency statistics per service and span name.
  • Trace Explorer supports App Hub annotations to identify registered services and workloads.
  • It provides a flyout view to explore span attributes, events, and associated log data.

OpenTelemetry Integration

  • Google Cloud Observability natively supports OpenTelemetry Protocol (OTLP) for metrics, traces, and logs ingestion.
  • The Google-Built OpenTelemetry Collector lets you send correlated OTLP traces, metrics, and logs to Google Cloud Observability from applications instrumented using OpenTelemetry SDKs.
  • The Telemetry API (telemetry.googleapis.com) implements the OpenTelemetry Line Protocol and supports http/protobuf, http/json, and gRPC protocols.
  • Applications can be instrumented once with OpenTelemetry and deployed on Compute Engine, GKE, or Cloud Run, querying and alerting on data together.
  • GKE offers a curated OpenTelemetry ingestion pipeline for simplified setup.

Application Monitoring

  • Application Monitoring provides an application-centric approach to monitoring integrated with App Hub.
  • It provides pre-curated dashboards with relevant metrics and logs mapped to user-defined applications in App Hub.
  • It incorporates best practices from Google SRE for optimized troubleshooting.
  • Application Monitoring creates dashboards for App Hub applications, services, and workloads.
  • It displays a topology view of the application showing resource interactions.
  • App Hub-specific labels let Google Cloud Observability construct the hierarchy of dashboards and filter telemetry data.

Cloud Debugger (Shut Down)

⚠️ SERVICE SHUT DOWN

Cloud Debugger was deprecated on May 16, 2022 and shut down on May 31, 2023.

This content is maintained for historical reference and may still appear on certification exams as a legacy question.

Replacement:

  • Snapshot Debugger (Open Source) – An open-source replacement that uses Firebase Realtime Database as the backend. It provides the same snapshot and logpoint functionality. Note: The repository was archived in July 2024.
  • Cloud Logging + Error Reporting – Use structured logging and error reporting for production debugging.
  • OpenTelemetry tracing – Use distributed tracing for understanding application behavior in production.
  • Cloud Debugger allowed inspecting the state of an application at any code location without stopping or slowing down the running app.
  • Cloud Debugger made it easier to view the application state without adding logging statements.
  • Cloud Debugger added less than 10ms to the request latency only when the application state was captured.
  • Cloud Debugger could be used with or without access to the app’s source code.
  • Cloud Debugger supported Cloud Source Repositories, GitHub, Bitbucket, or GitLab as the source code repository.

Debug Snapshots

  • Debug Snapshots captured local variables and the call stack at a specific line location in the app’s source code without stopping or slowing it down.
  • Certain conditions and locations could be specified to return a snapshot of the app’s data.
  • Debug Snapshots supported canarying wherein the debugger agent tested the snapshot on a subset of the instances.

Debug Logpoints

  • Debug Logpoints allowed injecting logging into running services without restarting or interfering with the normal function of the service.
  • Debug Logpoints were useful for debugging production issues without having to add log statements and redeploy.
  • Debug Logpoints remained active for 24 hours after creation, or until they were deleted or the service was redeployed.
  • If a logpoint was placed on a line that received lots of traffic, the Debugger throttled the logpoint to reduce its impact on the application.

GCP Certification Exam Practice Questions

  • Which service provides distributed tracing to help identify latency bottlenecks?
    • Cloud Trace
  • Which Google Cloud Observability feature allows you to query logs using SQL without exporting to BigQuery?
    • Log Analytics
  • What happened to Cloud Debugger?
    • It was deprecated in May 2022 and shut down in May 2023. The open-source Snapshot Debugger is the replacement.
  • Which service allows you to run Prometheus at scale without managing the backend infrastructure?
    • Google Cloud Managed Service for Prometheus
  • Which protocol does Google Cloud Observability natively support for vendor-agnostic telemetry ingestion?
    • OpenTelemetry Protocol (OTLP)

Related Posts

References

Google Cloud Monitoring – Metrics, Alerts & SLOs

Google Cloud Monitoring

  • Cloud Monitoring collects measurements of key aspects of the service and of the Google Cloud resources used.
  • Cloud Monitoring provides tools to visualize and monitor this data.
  • Cloud Monitoring helps gain visibility into the performance, availability, and health of the applications and infrastructure.
  • Cloud Monitoring collects metrics, events, and metadata from Google Cloud, AWS, hosted uptime probes, and application instrumentation.
  • Using the BindPlane service, data can be collected from over 150 common application components, on-premise systems, and hybrid cloud systems.
  • Cloud Monitoring was formerly known as Stackdriver Monitoring (rebranded in 2020 as part of Google Cloud Operations Suite).
  • Cloud Monitoring supports PromQL (Prometheus Query Language) as the recommended query language for creating charts, dashboards, and alerting policies.
  • Cloud Monitoring integrates with OpenTelemetry via the OTLP (OpenTelemetry Protocol) for vendor-agnostic metrics, traces, and logs ingestion.

Cloud Monitoring – Metrics Scope (formerly Workspaces)

ℹ️ Terminology Update: The concept of “Workspaces” has been replaced by Metrics Scopes and Scoping Projects. The functionality remains similar but with key improvements — a Google Cloud project can now be associated with multiple metrics scopes (previously limited to one Workspace).

  • Cloud Monitoring uses Metrics Scopes to organize monitoring information across multiple projects.
  • A Metrics Scope defines the set of resource containers whose time-series data a project can chart and monitor.
  • Every metrics scope is hosted by a designated Google Cloud project called the scoping project.
  • The scoping project stores all configuration content for dashboards, alerting policies, uptime checks, notification channels, and group definitions.
  • By default, the metrics scope of a Google Cloud project only includes that project.
  • A Google Cloud project can be associated with multiple metrics scopes (improved from the previous limit of one Workspace).
  • Up to 375 projects can be monitored per metrics scope.
  • You can use the Google Cloud console, the Google Cloud CLI, and the Cloud Monitoring API to modify the list of resource containers included in a metrics scope.
  • Metrics data remains in the monitored projects — the scoping project only stores configuration.
  • If you delete the scoping project, you also delete the metrics scope configuration.

Cloud Monitoring Query Languages

  • PromQL (Recommended)
    • PromQL is the open-source Prometheus Query Language and is the recommended query language for Cloud Monitoring.
    • Supports creating charts, dashboards, and alerting policies.
    • Offers complex conditions with features such as ratios, dynamic thresholds, and combined metrics.
    • Wide user base and extensive community resources available.
    • Fully integrated with Google Cloud Managed Service for Prometheus.
  • MQL (Deprecated)
    • Monitoring Query Language (MQL) was deprecated starting October 22, 2024.
    • As of July 22, 2025, MQL is no longer available for new charts, dashboards, and alerting policies in the Google Cloud console.
    • Existing MQL charts, dashboards, and alerting policies continue to work.
    • MQL can still be created via the Cloud Monitoring API (not the console).
    • Google recommends migrating to PromQL for all new workflows.

Cloud Monitoring Metrics

  • Metrics are a collection of measurements that help you understand how the applications and system services are performing.
  • Measurements might include the latency of requests to a service, the amount of disk space available on a machine, the number of tables in the SQL database, the number of widgets sold, and so forth.
  • Cloud Monitoring provides over 1,500+ free built-in metrics for Google Cloud services.
  • Metric Value type includes
    • For measurements consisting of a single value at a time
      • BOOL, a boolean
      • INT64, a 64-bit integer
      • DOUBLE, a double-precision float
      • STRING, a string
    • For distribution measurements, the value isn’t a single value but a group of values.
      • The value type for distribution measurements is DISTRIBUTION.
      • Values in distribution include the mean, count, max, and other statistics, computed for a group of values.
      • Latency metrics typically capture data as distributions
  • Metric Kind includes
    • Gauge metric – Value is measured at a specific instant in time for e.g, CPU utilization, current temperature.
    • Delta metric – Value is measured as the change since it was last recorded for e.g., metrics measuring request counts are delta metrics; each value records how many requests were received since the last data point was recorded.
    • Cumulative metric – Value constantly increases over time for e.g., a metric for “sent bytes” might be cumulative; each value records the total number of bytes sent by a service at that time.

Cloud Monitoring Agent

  • Google Cloud’s operations suite provides agents for collecting metrics and logs on Linux and Windows VM instances.
  • Ops Agent (Recommended)
    • The primary and recommended agent for collecting telemetry from Compute Engine instances.
    • Combines logging and metrics into a single agent with YAML-based configurations.
    • Uses Fluent Bit for logs (high-throughput logging) and the OpenTelemetry Collector for metrics.
    • Supports OTLP receiver for collecting OpenTelemetry metrics and traces from instrumented applications.
    • Can send data via the Telemetry API (OTLP-based) instead of proprietary Cloud Monitoring/Logging APIs.
    • Supports third-party application integrations (Apache, MySQL, PostgreSQL, etc.).
    • Available for both Linux and Windows VM instances.
  • Legacy Monitoring Agent (Deprecated)

    ⚠️ Deprecated: The legacy Monitoring agent (5.x version) has been deprecated and decommissioned. The legacy installation scripts (stack-install.sh and install-monitoring-agent.sh) are also deprecated and decommissioned. Google strongly recommends migrating to the Ops Agent.

    • The legacy monitoring agent is a collectd-based daemon that gathers system and application metrics from VM instances.
    • By default, the legacy monitoring agent collects disk, CPU, network, and process metrics.
    • Google strongly recommends using the Ops Agent instead.
    • The Ops Agent and a legacy agent cannot run on the same machine.

Cloud Monitoring – Alerting

  • Cloud Monitoring alerting policies help monitor resource metrics and notify when specific conditions are met.
  • Alerting Policy Types:
    • Metric-based alerting policies — monitor metric values and trigger when thresholds are crossed.
    • Log-based alerting policies — notify when a particular message or pattern appears in logs (created from Logs Explorer).
    • PromQL-based alerting policies — use PromQL queries for complex conditions including ratios, dynamic thresholds, and combined metrics.
    • Forecasted metric-value alerting policies — use historical trends to predict when a metric will cross a threshold in the future (forecast window: 1 hour to 2.5 days).
  • Notification Channels include email, SMS, Slack, PagerDuty, webhooks, Pub/Sub, and mobile app notifications.
  • Alerting policies can be configured to auto-close incidents after a specified duration.
  • Supports snooze functionality to temporarily suppress alerts during maintenance windows.

Cloud Monitoring – Uptime Checks & Synthetic Monitors

  • An uptime check is a request sent to an application endpoint to determine whether it responds.
  • Uptime checks support both public and private endpoints.
  • Public uptime checks can monitor:
    • URLs (HTTP, HTTPS, TCP)
    • Kubernetes LoadBalancer Services
    • VM instances
    • App Engine services
    • AWS load balancers
  • Private uptime checks can monitor resources without external IP addresses using Service Directory for service registration on private networks.
  • The availability of a resource can be monitored by creating an alerting policy that creates an incident when the uptime check fails.
  • The results of uptime checks can be observed in the Monitoring uptime-check dashboards.
  • For non-publicly available resources (public checks), the resource’s firewall must be configured to permit incoming traffic from the uptime-check servers.

Synthetic Monitors

  • Synthetic monitors (GA since November 2023) execute automated test scripts against application endpoints from Google Cloud data centers worldwide.
  • Synthetic monitors run single-purpose 2nd gen Cloud Run functions deployed on Cloud Run.
  • Can simulate critical user journeys (login, search, checkout) and verify multi-step workflows.
  • Support custom scripting using Mocha-based test frameworks for complex validation scenarios.
  • Provide more flexibility than standard uptime checks for complex application testing.

Cloud Monitoring – Service Monitoring & SLOs

  • Cloud Monitoring provides built-in Service Monitoring for defining and tracking SLOs (Service Level Objectives).
  • Key concepts:
    • Service-Level Indicator (SLI) — a measurement of performance (e.g., latency, availability).
    • Service-Level Objective (SLO) — a statement of desired performance (e.g., 99.9% availability over 30 days).
    • Error Budget — starts at (1 – SLO) and declines as actual performance misses the SLO.
  • Supports alerting on error budget burn rate to proactively notify before SLO violations.
  • Auto-detects services from GKE, Cloud Run, App Engine, and Istio/Anthos Service Mesh.
  • Can define custom services for any workload with metrics.

Google Cloud Managed Service for Prometheus

  • Managed Service for Prometheus provides fully managed, globally scalable Prometheus monitoring integrated with Cloud Monitoring.
  • Allows monitoring and alerting on workloads using Prometheus and OpenTelemetry without manually managing Prometheus at scale.
  • Supports managed collection (Google-managed collectors on GKE) and self-deployed collection (custom Prometheus setups).
  • Prometheus metrics can be used with Cloud Monitoring dashboarding, alerting, and SLO monitoring features.
  • Fully compatible with PromQL — chart Prometheus metrics alongside GKE, load balancer, and other Cloud Monitoring metrics.
  • Available on GKE clusters running Kubernetes version 1.25 or later.
  • Also supported on attached clusters (EKS, AKS) and Anthos clusters.

Application Monitoring & Gemini Cloud Assist

  • Application Monitoring (2025) automatically labels and brings together key telemetry for applications into a centralized experience.
  • Makes it easy to discover, filter, and correlate trends across metrics, logs, and traces for registered App Hub applications.
  • Displays trace spans associated with registered applications with annotations for identifying services and workloads.
  • Gemini Cloud Assist Investigations integrates with Application Monitoring for AI-assisted troubleshooting.
  • Gemini Cloud Assist understands cloud infrastructure, application code, organizational policies, and real-time operational state to provide intelligent operations support.

Cloud Monitoring – OpenTelemetry & OTLP Support

  • Cloud Monitoring supports the OpenTelemetry Protocol (OTLP) for direct metrics ingestion.
  • Supported OTLP protocols include http/protobuf, http/json, and grpc.
  • Enables vendor-agnostic telemetry pipelines using OpenTelemetry SDKs and collectors.
  • The Google-Built OpenTelemetry Collector lets you send correlated OTLP traces, metrics, and logs to Google Cloud Observability.
  • The Telemetry API (telemetry.googleapis.com) implements the OpenTelemetry Line Protocol natively.
  • Supports delta temporality for cumulative metrics export, allowing clients to flush memory between exports and reduce resource consumption.

GCP Certification Exam Practice Questions

  • Questions are collected from Internet and the answers are marked as per my knowledge and understanding (which might differ with yours).
  • GCP services are updated everyday and both the answers and questions might be outdated soon, so research accordingly.
  • GCP exam questions are not updated to keep up the pace with GCP updates, so even if the underlying feature has changed the question might not be updated
  • Open to further feedback, discussion and correction.
  1. You need to monitor resources that are distributed over different projects in Google Cloud Platform. You want to consolidate reporting under the same Cloud Monitoring dashboard. What should you do?
    1. Use Shared VPC to connect all projects, and link Cloud Monitoring to one of the projects.
    2. For each project, create a Cloud Monitoring account. In each project, create a service account for that project and grant it the role of Monitoring Editor in all other projects.
    3. Configure a single metrics scope, and add all projects to the same scoping project.
    4. Configure a single Cloud Monitoring account for one of the projects. In Cloud Monitoring, create a Group and add the other project names as criteria for that Group.
  2. You are asked to set up application performance monitoring on Google Cloud projects A, B, and C as a single pane of glass. You want to monitor CPU, memory, and disk. What should you do?
    1. Enable API and then share charts from projects A, B, and C.
    2. Enable API and then give the metrics.reader role to projects A, B, and C.
    3. Enable API and then use default dashboards to view all projects in sequence.
    4. Enable API, create a metrics scope under project A, and then add projects B and C to the metrics scope.
  3. Your company uses the Ops Agent on Compute Engine VMs. A developer asks about monitoring application-specific metrics using OpenTelemetry. What should you recommend?
    1. Install the legacy monitoring agent alongside the Ops Agent.
    2. Use Cloud Monitoring API to write custom metrics directly.
    3. Instrument the application with OpenTelemetry SDK and configure the Ops Agent’s OTLP receiver to collect the metrics.
    4. Deploy a separate Prometheus server to collect the metrics.
  4. You want to create an alerting policy that predicts when your disk usage will exceed 90% within the next 24 hours based on current trends. What type of alerting condition should you use?
    1. Metric threshold condition
    2. Log-based alerting policy
    3. PromQL-based alerting policy with rate function
    4. Forecasted metric-value condition
  5. You need to define availability and latency targets for your Cloud Run service and get alerted when the error budget is being consumed too quickly. What Cloud Monitoring feature should you use?
    1. Create custom metrics and threshold-based alerting policies.
    2. Use Service Monitoring to define SLOs with SLIs and configure burn rate alerting.
    3. Create an uptime check and monitor the uptime percentage.
    4. Use synthetic monitors to simulate user traffic and measure latency.
  6. Your team is migrating from MQL-based alerting policies to PromQL. Which statement about MQL deprecation is correct?
    1. All existing MQL dashboards and alerting policies stopped working on July 22, 2025.
    2. MQL was completely removed from Cloud Monitoring on October 22, 2024.
    3. Existing MQL charts, dashboards, and alerting policies continue to work, but new MQL assets can no longer be created via the Google Cloud console (only via API).
    4. MQL can only be used for dashboards but not for alerting policies.
  7. You want to monitor a web application running on a private VPC network without a public IP address. Which Cloud Monitoring feature should you use?
    1. Standard public uptime checks with VPC firewall rules.
    2. Synthetic monitors with Cloud Run functions.
    3. Private uptime checks using Service Directory for service registration.
    4. Custom metrics pushed from the application using the Cloud Monitoring API.
  8. You are running a Kubernetes workload on GKE and want to use Prometheus-style monitoring without managing your own Prometheus infrastructure. What should you use?
    1. Install a self-managed Prometheus server on the cluster.
    2. Use the legacy monitoring agent on each node.
    3. Use Google Cloud Managed Service for Prometheus with managed collection.
    4. Export all metrics to BigQuery for analysis.

References