Fix Wazuh GCP Pub/Sub Ingestion

As organizations continue moving workloads to Google Cloud Platform (GCP), collecting cloud telemetry has become essential for maintaining visibility across cloud resources.

Wazuh integrates with Google Cloud Pub/Sub to ingest security events, audit logs, network activity, and other cloud-generated logs into a centralized security monitoring platform.

When this ingestion pipeline functions correctly, security teams receive timely alerts and maintain complete visibility into their cloud environment. When it breaks, however, important security events may never reach Wazuh.

Reliable cloud log ingestion is the foundation of effective threat detection.

Missing authentication logs, delayed audit events, or incomplete network telemetry can prevent analysts from detecting privilege escalation, unauthorized API activity, or lateral movement.

Even short interruptions in log collection create blind spots that attackers can exploit.

Google’s own security guidance emphasizes maintaining centralized logging and monitoring as a core cloud security practice.

Unfortunately, GCP Pub/Sub ingestion issues are not uncommon.

Administrators frequently encounter problems such as authentication failures, permission errors, growing Pub/Sub subscription backlogs, delayed event delivery, duplicate logs, or entire categories of Google Cloud logs failing to appear inside the Wazuh dashboard.

These issues may stem from misconfigured IAM roles, expired service account credentials, incorrect Pub/Sub subscriptions, network connectivity problems, or Wazuh module configuration errors.

In this guide, you’ll learn how Wazuh GCP Pub/Sub ingestion works, identify the most common causes of ingestion failures, troubleshoot each issue step by step, and implement best practices that keep your cloud logging pipeline reliable and scalable.

You’ll also learn how to verify successful ingestion and prevent future outages before they affect your security monitoring.


How Wazuh GCP Pub/Sub Ingestion Works

Understanding the complete ingestion pipeline makes troubleshooting significantly easier.

Rather than treating Pub/Sub as a standalone component, it’s helpful to view it as one stage in a larger event processing workflow that begins inside Google Cloud and ends with searchable security events in Wazuh.

Overview of the Architecture

At a high level, Google Cloud services generate operational and security logs that are collected by Cloud Logging. Log sinks export selected log entries into Google Cloud Pub/Sub topics.

Wazuh then subscribes to those topics, retrieves new messages, parses the events, and forwards them into the Wazuh manager for analysis, rule matching, indexing, and dashboard visualization.

The architecture generally follows this flow:

  1. Google Cloud service generates logs.
  2. Cloud Logging receives the events.
  3. A Log Sink exports matching logs to Pub/Sub.
  4. Pub/Sub stores and queues the messages.
  5. Wazuh retrieves messages from the subscription.
  6. Wazuh decodes, analyzes, and indexes the events.
  7. Events become searchable in the Wazuh Dashboard.

This decoupled architecture allows Google Cloud services and Wazuh to operate independently while providing reliable message delivery.

Google Cloud Services

Nearly every major Google Cloud service can generate logs suitable for security monitoring. Depending on your environment, these may include:

  • Identity and Access Management (IAM)
  • Compute Engine
  • Cloud Storage
  • Cloud SQL
  • Kubernetes Engine (GKE)
  • Cloud Functions
  • Cloud Run
  • Virtual Private Cloud (VPC)
  • Cloud DNS
  • Security Command Center

These services publish operational events into Cloud Logging, where administrators decide which logs should be exported to Pub/Sub.

Pub/Sub Topics and Subscriptions

Google Cloud Pub/Sub acts as the messaging layer between Google Cloud Logging and Wazuh.

The process involves two primary components:

  • Topic: Receives published log messages.
  • Subscription: Allows Wazuh to consume those messages.

A single topic can support multiple subscriptions, allowing multiple security tools to receive the same cloud events independently.

One advantage of Pub/Sub is its durability. Messages remain available until acknowledged or until the subscription retention period expires, reducing the likelihood of data loss during temporary outages.

Google recommends designing Pub/Sub subscriptions with proper acknowledgment deadlines, retry policies, and monitoring to maintain reliable message processing.

Wazuh Modules Involved

Several Wazuh components participate in cloud log ingestion:

  • Cloud integration modules
  • Log decoders
  • Detection rules
  • Wazuh Manager
  • Filebeat
  • Wazuh Indexer
  • Wazuh Dashboard

Once Pub/Sub messages arrive, Wazuh parses the JSON payload, applies decoders and rules, generates alerts when applicable, and indexes the events for searching and visualization.

If downstream indexing problems occur after successful ingestion, see How to Fix a Yellow Cluster Status in Wazuh Indexer and Fixing Rejected Index Templates and Invalid Mapping Types in Wazuh.

Event Flow from GCP to Wazuh

A typical event follows this sequence:

  1. An administrator modifies an IAM policy.
  2. Cloud Audit Logs records the action.
  3. Cloud Logging captures the event.
  4. A Log Sink exports it to Pub/Sub.
  5. Wazuh pulls the message from the subscription.
  6. The event is decoded.
  7. Detection rules evaluate the event.
  8. Alerts are indexed.
  9. Analysts view the alert in the Wazuh Dashboard.

Every stage represents a potential failure point, making it important to understand where events stop flowing during troubleshooting.

Supported Google Cloud Log Sources

Wazuh can ingest many different Google Cloud log sources, enabling broad cloud visibility.

Cloud Audit Logs

Cloud Audit Logs are among the most valuable data sources because they record administrative actions, API calls, permission changes, and authentication events.

Security teams commonly monitor:

  • IAM policy modifications
  • Service account activity
  • Project configuration changes
  • Resource creation and deletion
  • Administrative API calls

Google identifies Audit Logs as a foundational component of cloud governance and security monitoring.

VPC Flow Logs

VPC Flow Logs provide metadata about network traffic entering and leaving Google Cloud resources.

These logs help identify:

  • Unexpected outbound connections
  • Port scanning
  • Network reconnaissance
  • Communication with suspicious IP addresses
  • Firewall rule effectiveness

Cloud DNS Logs

Cloud DNS logs capture DNS queries generated within Google Cloud environments.

These logs are useful for detecting:

  • Command-and-control communication
  • Malware domain lookups
  • Data exfiltration attempts
  • DNS tunneling
  • Suspicious domain resolutions

Cloud Storage Logs

Cloud Storage logs provide visibility into object access and administrative operations.

Examples include:

  • Bucket permission changes
  • Object downloads
  • File uploads
  • Bucket creation
  • Object deletion

These events are particularly useful for monitoring sensitive data access.

Security Command Center Findings

Organizations using Google Cloud Security Command Center can forward findings into Wazuh for centralized analysis.

Common findings include:

  • Vulnerability detections
  • Publicly exposed resources
  • Misconfigured IAM permissions
  • Security recommendations
  • Threat detections

Combining these findings with endpoint telemetry provides analysts with broader context during investigations.

Other Supported Services

Depending on your deployment, additional Google Cloud services may also export logs through Pub/Sub, including:

  • Google Kubernetes Engine (GKE)
  • Cloud Functions
  • Cloud Run
  • Cloud SQL
  • Load Balancers
  • Cloud Armor
  • Cloud NAT
  • Identity-Aware Proxy (IAP)

As Google Cloud continues expanding its logging capabilities, additional services can typically be integrated using Cloud Logging export sinks.


Common Symptoms of Wazuh GCP Pub/Sub Ingestion Problems

Pub/Sub ingestion failures rarely present as a complete outage.

More often, administrators notice subtle indicators that logs are no longer flowing correctly.

Recognizing these symptoms early can significantly reduce troubleshooting time.

No Events Appearing in Wazuh

The most obvious symptom is that Google Cloud events stop appearing entirely.

You may observe:

  • Empty cloud dashboards
  • No recent cloud alerts
  • Searches returning zero GCP events
  • No new indexed cloud logs

This usually indicates problems with:

  • Service account authentication
  • Pub/Sub subscription configuration
  • Wazuh cloud integration settings
  • IAM permissions
  • Network connectivity

Delayed Log Delivery

Sometimes logs eventually arrive but are delayed by several minutes or even hours.

Delayed ingestion often results from:

  • Growing subscription backlogs
  • Slow processing
  • Resource bottlenecks
  • Rate limiting
  • Network latency

Delayed security telemetry reduces incident response effectiveness because detections occur well after malicious activity has already happened.

Authentication Failures

Authentication problems commonly prevent Wazuh from connecting to Google Cloud services.

Typical causes include:

  • Expired service account keys
  • Invalid JSON credential files
  • Deleted service accounts
  • Incorrect credential paths
  • Disabled service accounts

Authentication failures often appear in Wazuh logs alongside Google Cloud API error messages indicating invalid credentials or token generation failures.

Permission Denied Errors

Even when authentication succeeds, insufficient IAM permissions can prevent Wazuh from reading Pub/Sub messages.

Common missing permissions include:

  • Pub/Sub Subscriber
  • Pub/Sub Viewer
  • Logging Viewer
  • Service Account Token Creator (depending on the authentication method)

Permission-related errors typically reference HTTP 403 responses or “Permission denied” messages.

Pub/Sub Subscription Backlog Growing

A rapidly increasing subscription backlog usually indicates that Wazuh is not consuming messages as quickly as Google Cloud is publishing them.

Possible causes include:

  • Wazuh service failures
  • Network interruptions
  • CPU exhaustion
  • Memory pressure
  • Parsing bottlenecks
  • Subscription misconfiguration

Google Cloud Monitoring provides metrics for tracking unacknowledged messages, making backlog growth one of the earliest indicators of ingestion issues.

Duplicate Events

Duplicate events can create unnecessary alert noise and inflate storage usage.

This may occur when:

  • Messages are repeatedly redelivered
  • Acknowledgments fail
  • Multiple subscriptions ingest identical logs
  • Retry policies repeatedly process the same message

If duplicate alerts are becoming difficult to manage after ingestion, see How to Reduce False Positives in Wazuh.

Missing Specific Log Types

Sometimes only certain Google Cloud services stop sending logs while others continue working normally.

Examples include:

  • Cloud Audit Logs arriving correctly while VPC Flow Logs are absent
  • Cloud Storage logs missing
  • Security Command Center findings never appearing
  • DNS logs failing to export

This typically indicates issues with:

  • Cloud Logging sink filters
  • Export configuration
  • Service-specific logging settings
  • Incorrect Pub/Sub routing

High Resource Usage During Ingestion

Large cloud environments may generate thousands of events per second.

If Wazuh lacks sufficient CPU, memory, or storage throughput, ingestion performance can degrade.

Warning signs include:

  • High CPU utilization
  • Memory exhaustion
  • Slow indexing
  • Increased ingestion latency
  • Growing processing queues

If ingestion-related resource consumption affects the Wazuh manager or indexer, the issue may be related to broader performance bottlenecks.

See Why Is Wazuh Using High CPU? Troubleshooting Guide and How to Tune OpenSearch Heap Size to Stop Wazuh High Memory Crashes for additional optimization guidance.


Common Causes of Wazuh GCP Pub/Sub Ingestion Failures

Although Pub/Sub ingestion failures may appear similar on the surface, they often originate from very different configuration problems.

Identifying the root cause before making changes can significantly reduce troubleshooting time and prevent unnecessary modifications to a working environment.

Incorrect Service Account Permissions

One of the most common causes of ingestion failures is assigning insufficient IAM permissions to the service account used by Wazuh.

The service account must have permission to:

  • Read messages from the Pub/Sub subscription
  • Access exported Cloud Logging data when required
  • Authenticate with Google Cloud APIs

Without the appropriate IAM roles, Wazuh may authenticate successfully but fail when attempting to pull messages from the subscription.

Common symptoms include:

  • HTTP 403 errors
  • Permission denied messages
  • Empty Pub/Sub reads
  • Subscription backlog growth

Always follow the principle of least privilege by granting only the permissions required for log ingestion rather than broad project-level administrative roles.

Invalid Credentials

Wazuh authenticates to Google Cloud using a service account credential file, typically in JSON format.

Ingestion will fail if:

  • The JSON file is malformed.
  • The file belongs to the wrong project.
  • Credentials were copied incorrectly.
  • The private key has been corrupted.
  • The wrong credential file is referenced in the configuration.

Authentication failures often generate errors indicating that credentials cannot be loaded or verified.

Whenever credentials are regenerated, update the Wazuh configuration to point to the new file.

Pub/Sub Subscription Misconfiguration

Even when logs are successfully published to a Pub/Sub topic, Wazuh cannot consume them if the subscription is configured incorrectly.

Common problems include:

  • Wazuh pointing to the wrong subscription
  • Subscription deleted accidentally
  • Subscription attached to the wrong topic
  • Message retention period too short
  • Incorrect acknowledgment settings
  • Subscription filters excluding desired events

Google recommends monitoring subscription health and backlog metrics to quickly identify configuration or processing problems.

Incorrect Wazuh Module Configuration

Configuration mistakes inside Wazuh are another frequent source of ingestion failures.

Examples include:

  • Incorrect project ID
  • Typographical errors
  • Invalid XML syntax
  • Missing required parameters
  • Disabled cloud integration modules
  • Incorrect subscription names

Even a single incorrect parameter can prevent Wazuh from connecting successfully.

If Wazuh reports XML parsing errors after editing the configuration, see How to Fix ossec.conf Syntax Errors in Wazuh Agents.

API Access Disabled

Several Google Cloud APIs must be enabled before Pub/Sub ingestion can function properly.

Depending on your deployment, these commonly include:

  • Pub/Sub API
  • Cloud Logging API
  • IAM API
  • Resource Manager API

If one of these APIs has been disabled, Wazuh may receive authorization or resource errors despite having valid credentials.

Google Cloud projects created through automation or infrastructure-as-code occasionally omit required APIs, making this an easy issue to overlook.

Network Connectivity Problems

Reliable communication between Wazuh and Google Cloud is essential.

Connectivity issues may include:

  • DNS resolution failures
  • Internet routing problems
  • VPN interruptions
  • Packet filtering
  • TLS handshake failures
  • Proxy misconfiguration

Symptoms often include:

  • Connection timeouts
  • Retry loops
  • Intermittent ingestion
  • Long delivery delays

Testing connectivity from the Wazuh server to Google Cloud endpoints can quickly determine whether the issue is network-related.

Expired Authentication Keys

Service account keys may eventually be rotated or revoked as part of an organization’s security policies.

When this happens:

  • Existing authentication fails.
  • New access tokens cannot be generated.
  • Pub/Sub connections terminate.
  • Wazuh stops receiving cloud events.

Organizations that enforce regular credential rotation should document a corresponding procedure for updating Wazuh immediately after new keys are issued.

Google Cloud IAM Policy Changes

Security administrators frequently modify IAM policies to improve security or comply with organizational standards.

Unfortunately, permission changes may unintentionally remove access required by Wazuh.

Examples include:

  • Removing Pub/Sub Subscriber roles
  • Replacing custom IAM roles
  • Disabling inherited permissions
  • Moving resources into different projects
  • Restricting service account access

If ingestion suddenly stops after security policy updates, reviewing recent IAM audit logs is often one of the fastest ways to identify the cause.

Firewall or Proxy Restrictions

Corporate environments often restrict outbound traffic through firewalls or proxy servers.

Potential issues include:

  • Blocked HTTPS traffic
  • SSL inspection interfering with certificates
  • Proxy authentication failures
  • Blocked Google Cloud endpoints
  • Network ACL restrictions

Even if authentication succeeds, blocked network traffic may prevent Wazuh from retrieving Pub/Sub messages.

When troubleshooting, verify that outbound connections to required Google Cloud services are permitted.

Version Compatibility Issues

Although Wazuh and Google Cloud maintain strong compatibility, older software versions may not support newer APIs, authentication methods, or integration features.

Potential compatibility issues include:

  • Deprecated authentication libraries
  • Older Google Cloud SDK components
  • Unsupported API versions
  • Wazuh integration bugs fixed in later releases

Before extensive troubleshooting, confirm that you’re running supported versions of:

  • Wazuh Manager
  • Wazuh Indexer
  • Wazuh Dashboard
  • Google Cloud SDK (if applicable)

Keeping the platform updated also improves stability and security.

If you’re already planning an upgrade, see How to Upgrade a Wazuh Agent for general upgrade guidance.


Verify Your Google Cloud Configuration

Before modifying Wazuh, verify that Google Cloud is exporting logs correctly.

Many ingestion problems originate within the Google Cloud environment rather than the Wazuh server.

Confirm Pub/Sub Topic Exists

First, verify that the expected Pub/Sub topic exists.

Using either the Google Cloud Console or the gcloud CLI, confirm:

  • Topic name is correct.
  • Topic has not been deleted.
  • The topic belongs to the expected project.
  • Cloud Logging is publishing messages.

If the topic is missing entirely, Wazuh has nothing to consume.

Verify Subscription Configuration

Next, inspect the Pub/Sub subscription attached to the topic.

Confirm:

  • The subscription exists.
  • It references the correct topic.
  • Messages are accumulating as expected.
  • The acknowledgment deadline is appropriate.
  • Message retention settings meet operational requirements.
  • No restrictive filters are applied.

Also review subscription metrics such as:

  • Unacknowledged messages
  • Delivery attempts
  • Backlog size
  • Oldest unprocessed message

Growing backlog values usually indicate downstream processing problems rather than publishing failures.

Check Service Account Roles

Open the IAM configuration and inspect the service account used by Wazuh.

Verify that it retains all required permissions.

Review for:

  • Recently removed roles
  • Project inheritance changes
  • Organization policy restrictions
  • Disabled service accounts

Google Cloud’s IAM troubleshooting guidance recommends reviewing effective permissions rather than only assigned roles, since inherited policies can affect access.

Ensure Required APIs Are Enabled

Open APIs & Services within the Google Cloud Console and verify that required APIs remain enabled.

Commonly required services include:

  • Pub/Sub API
  • Cloud Logging API
  • IAM API

Projects cloned from templates or restored from backups occasionally omit API enablement, preventing successful communication.

Validate Log Router Sinks

Cloud Logging exports logs using Log Router sinks.

Review each sink to ensure:

  • Destination is the correct Pub/Sub topic.
  • Sink filters match the desired log sources.
  • Export errors are not being reported.
  • Sink service account permissions remain valid.

A small mistake in a filter expression can unintentionally exclude entire categories of security logs.

Google provides detailed guidance on configuring Log Router exports and sink filters.

Test Pub/Sub Message Delivery

Before troubleshooting Wazuh itself, verify that Pub/Sub is actually receiving messages.

You can perform a simple validation by:

  1. Generating a known cloud event (for example, creating or modifying a test resource).
  2. Confirming the event appears in Cloud Logging.
  3. Verifying the event reaches the Pub/Sub topic.
  4. Pulling a message manually using the Google Cloud Console or the gcloud pubsub subscriptions pull command.

If messages never reach Pub/Sub, the issue exists within Google Cloud rather than Wazuh.


Inspect the Wazuh Configuration

Once Google Cloud has been verified, inspect the Wazuh configuration to ensure it is correctly configured to consume Pub/Sub messages.

Review ossec.conf Settings

The primary configuration file for Wazuh integrations is ossec.conf.

Review the cloud integration section carefully for:

  • Missing XML tags
  • Incorrect parameter names
  • Typographical errors
  • Disabled integration blocks
  • Invalid subscription references

Because XML is strict, even a missing closing tag can prevent the integration from loading successfully.

If Wazuh fails to start after editing this file, refer to How to Fix ossec.conf Syntax Errors in Wazuh Agents.

Validate GCP Integration Parameters

Confirm that every Google Cloud parameter matches your deployment.

Typical settings include:

  • Project ID
  • Pub/Sub subscription name
  • Credential file location
  • Integration interval
  • Authentication options

Pay particular attention to spelling and capitalization, as resource names must exactly match those configured in Google Cloud.

Confirm Credential File Paths

Verify that the configured credential file:

  • Exists on disk.
  • Is readable by the Wazuh user.
  • Contains valid JSON.
  • Matches the intended Google Cloud project.
  • Has not been replaced or removed.

Also ensure appropriate file permissions are configured to protect sensitive service account credentials from unauthorized access.

Check Module Status

After verifying the configuration, confirm that the cloud integration module is actually running.

Inspect:

  • Wazuh manager logs
  • Startup messages
  • Module initialization logs
  • Authentication messages
  • Pub/Sub connection attempts

You’re looking for indications that the module initialized successfully and established communication with Google Cloud.

If additional API-related errors appear during startup, you may also find Wazuh API Authentication Failed? Causes and Solutions helpful.

Restart Wazuh Services Safely

After making configuration changes, restart the appropriate Wazuh services so the new settings take effect.

A safe restart process should include:

  1. Validate the updated configuration.
  2. Restart the Wazuh manager.
  3. Monitor startup logs for errors.
  4. Verify the cloud integration initializes successfully.
  5. Generate a test Google Cloud event.
  6. Confirm the event appears in the Wazuh Dashboard.

Avoid making multiple configuration changes before restarting. Applying one change at a time makes it much easier to identify which modification resolved, or introduced, the problem.


Be First to Comment

    Leave a Reply

    Your email address will not be published. Required fields are marked *