Resolving OpenSearch Multi-Tenancy Mismatches in Wazuh Deployments

Multi-tenancy in Wazuh deployments backed by OpenSearch Dashboards is designed to enforce logical separation of observability data, dashboards, and saved objects across different users, teams, or environments.

In practice, however, this abstraction frequently breaks down when role mappings, tenant assignments, or index patterns drift out of alignment, resulting in what are commonly referred to as Wazuh OpenSearch Multi-Tenancy Mismatches.

At a high level, Wazuh relies on OpenSearch Dashboards’ security model to scope what a user can see.

When configured correctly, analysts in different roles (e.g., SOC Tier 1 vs. DevOps vs. compliance) should only see their assigned dashboards, indices, and visualizations.

However, even minor inconsistencies in RBAC configuration or tenant assignment can produce confusing and sometimes security-relevant anomalies.

Why tenant mismatches are a common operational issue

Tenant mismatches are not edge cases, they are a predictable outcome of how Wazuh and OpenSearch evolve independently across upgrades, plugin changes, and configuration drift. Common contributing factors include:

  • Role mapping updates not propagating correctly after upgrades
  • Saved objects being created in the wrong tenant context
  • Inconsistent security plugin configurations across nodes
  • Cached session artifacts preserving outdated tenant state

These issues are amplified in distributed environments where OpenSearch clusters and Wazuh managers are scaled independently.

Typical symptoms

Operators usually detect multi-tenancy mismatches through indirect signals rather than explicit errors:

  • Missing indices or dashboards despite confirmed ingestion pipelines
  • Inconsistent user views across roles (same query returns different dashboards)
  • “No data available” messages even when index health is green
  • Cross-tenant data leakage or unexpected visibility into restricted datasets

In many cases, the system appears “healthy” at the infrastructure level while the logical security layer is misaligned.


Understanding Wazuh + OpenSearch Multi-Tenancy Architecture

To understand tenant mismatches, it is essential to clarify how Wazuh leverages OpenSearch Dashboards’ multi-tenancy model and how it interacts with the security plugin layer.

How OpenSearch Dashboards implements tenants

OpenSearch Dashboards supports a tenant abstraction layer that isolates saved objects such as:

  • Dashboards
  • Visualizations
  • Index patterns
  • Saved searches

Each tenant represents a logical workspace.

This allows multiple user groups to operate on the same cluster without interfering with each other’s analytical context.

Default, global, and private tenant concepts

OpenSearch defines three primary tenant scopes:

  • Global tenant: Shared across all users with appropriate permissions
  • Private tenant: User-specific workspace tied to a single identity
  • Custom tenants: Organization-defined segments (e.g., SOC, Dev, Prod)

Misalignment often occurs when saved objects are created in one tenant but accessed from another due to role misconfiguration.

How Wazuh integrates with OpenSearch security plugin

Wazuh relies on the OpenSearch Security Plugin to enforce authentication, authorization, and tenant scoping.

The integration governs:

  • Role-based access control (RBAC)
  • Index-level permissions
  • Tenant visibility rules
  • API-level authorization checks

When Wazuh roles and OpenSearch roles diverge, tenant enforcement becomes inconsistent, leading to partial or broken UI experiences.

Relationship between Wazuh roles, OpenSearch roles, index patterns, and tenant-scoped saved objects

The architecture can be summarized as a layered dependency chain:

  • Wazuh roles define what security events and dashboards a user should access
  • OpenSearch roles enforce index-level and cluster-level permissions
  • Index patterns define how data streams are interpreted in Dashboards
  • Tenant-scoped saved objects determine what visualizations and dashboards are visible

A mismatch at any layer can cascade upward, producing symptoms such as missing dashboards or inconsistent query results.

📌 Related guides for deeper context:


Common Causes of Wazuh OpenSearch Multi-Tenancy Mismatches

Multi-tenancy mismatches almost always originate from configuration drift or inconsistent role propagation across the Wazuh–OpenSearch stack.

Misconfigured role-to-tenant mappings

One of the most frequent root causes is incorrect mapping between OpenSearch roles and tenants.

If a role is not explicitly bound to a tenant, users may unintentionally fall back to the global tenant or an unintended private workspace.

This typically results in:

  • Missing dashboards
  • Partial visibility of indices
  • Inconsistent UI state across users with similar roles

Default tenant inconsistencies after upgrades

Upgrades to Wazuh or OpenSearch can reset or alter default tenant assignments.

In distributed deployments, this leads to:

  • Users landing in different default tenants post-login
  • Saved objects appearing “lost” or “missing”
  • Role mappings no longer aligning with tenant scoping rules

This is especially common when upgrading across major OpenSearch versions where security plugin behavior changes.

Index pattern assigned to wrong tenant scope

Index patterns are tenant-scoped saved objects.

When created in the wrong tenant, they become invisible to users operating in other tenants, even if they have correct index-level permissions.

This produces a misleading scenario:

Data exists, but dashboards report “no matching indices found.”

For related debugging strategies, see Troubleshooting “No Matching Indices Found” Error in Wazuh Dashboard.

Conflicts between Wazuh RBAC and OpenSearch security roles

Wazuh RBAC defines application-level permissions, while OpenSearch security roles enforce backend access control.

Misalignment occurs when:

  • Wazuh grants dashboard access but OpenSearch denies index access
  • OpenSearch roles allow index access but restrict tenant visibility
  • Role inheritance is not consistently applied

This dual-layer model is powerful but fragile when not centrally governed.

Cached session or cookie persistence issues

OpenSearch Dashboards caches tenant selection in session cookies.

Stale cookies can result in:

  • Users being “stuck” in an incorrect tenant
  • Permissions not updating after role changes
  • Apparent inconsistencies that disappear after clearing browser state

This is often mistaken for RBAC failure when it is actually a client-side persistence issue.

Related internal troubleshooting reference:

  • Wazuh API Authentication Failed? Causes and Solutions

Cluster state divergence in multi-node setups

In multi-node OpenSearch clusters, inconsistent cluster state propagation can lead to:

  • Tenant metadata desynchronization
  • Partial visibility of saved objects across nodes
  • Inconsistent role enforcement depending on load balancer routing

This is more common in horizontally scaled deployments without strict cluster state coordination.

For architectural mitigation patterns: How to Build a Wazuh Indexer Cluster

References

Expert insight note: OpenSearch multi-tenancy design is heavily influenced by Kibana’s saved object model, which is widely discussed in security engineering literature as a trade-off between isolation and operational flexibility.


Symptoms and How to Recognize a Tenant Mismatch

Tenant mismatches in Wazuh–OpenSearch deployments rarely present as a single, explicit failure.

Instead, they surface as inconsistent UI behavior that suggests data loss or permission errors while the underlying ingestion pipeline remains healthy.

Dashboards loading but showing empty panels

A common early indicator is a fully rendered dashboard with empty visualizations.

Panels load structurally, but queries return no results despite confirmed ingestion into OpenSearch indices.

This typically points to a tenant-scoped index pattern mismatch rather than a data pipeline failure.

Saved searches missing or not visible

Users may report that previously created saved searches or dashboards have “disappeared.”

In reality, these objects often still exist but are stored under a different tenant scope (e.g., private vs global), making them invisible to the current session context.

Alerts appearing for admin but not standard users

Alerting discrepancies are a strong signal of tenant misalignment.

Administrators may see active alerts while standard users see none, even when both query the same index.

This usually indicates role-based tenant filtering or alert objects bound to a restricted tenant.

“No permission” errors in Dev Tools or Discover

Unexpected authorization failures in Discover or Dev Tools often indicate that the user has index-level access but lacks tenant-level visibility for the associated saved objects or index patterns.

This separation between data access and UI access is a core tenant isolation behavior.

Related troubleshooting reference: Wazuh Dev Tools 403 Forbidden Error in Wazuh Dashboard

Index patterns existing but not selectable per user

Another subtle symptom is when index patterns are visible in one account but not another, even though both have identical roles.

This almost always indicates that the index pattern was created under a different tenant scope.


Step-by-Step Diagnosis

A systematic diagnosis requires separating tenant context, RBAC configuration, and saved object scoping into discrete validation steps.

Verify Active Tenant

Checking current tenant in OpenSearch Dashboards UI

Start by confirming which tenant the user is currently operating in. OpenSearch Dashboards typically exposes tenant selection in the user menu. Ensure the user is not unintentionally operating in a private or empty tenant context.

Confirming user session tenant context via API

For deeper validation, inspect the session or authentication response via OpenSearch security endpoints. This confirms whether the backend is assigning the expected tenant scope or defaulting incorrectly.

Inspect Role Mappings

 

Reviewing roles in OpenSearch security plugin

Examine role definitions within the OpenSearch Security Plugin configuration. Focus on:

  • Tenant permissions (global, private, custom)
  • Index-level access rules
  • Backend role mappings

Mapping users → roles → tenants

Trace the full chain:
User → Assigned Roles → Role Permissions → Tenant Access

Breakdowns often occur when a role includes index access but omits tenant visibility, or vice versa.


Validate Index Patterns

 

Ensuring index patterns exist in correct tenant scope

Index patterns must be created within the same tenant context where they are consumed. If created under “Private tenant,” they will not appear in “Global tenant.”

Checking wildcard mismatches (e.g., wazuh-*)

Verify that index patterns correctly match your data streams. Common issues include:

  • Overly restrictive patterns (e.g., wazuh-alerts-2024.01 not covering newer indices)
  • Misaligned naming conventions after upgrades

Check Saved Objects Scope

 

Dashboards, visualizations, and alerts per tenant

Saved objects are strictly tenant-bound. Validate whether dashboards, visualizations, and alert rules were created in:

  • Global tenant (shared)
  • Private tenant (user-specific)
  • Custom tenant (team/environment-specific)

Identifying missing or orphaned objects

Objects may appear “missing” when:

  • They exist in another tenant
  • They were imported without correct tenant assignment
  • Migration processes did not preserve scope metadata

Review Security Logs

 

OpenSearch security audit logs for access denials

Audit logs provide the most reliable signal for tenant-related failures. Look for:

  • Permission denied events
  • Index access rejections
  • Tenant access violations

Wazuh manager logs for correlation issues

Wazuh logs can reveal upstream correlation issues where alerts are generated but not properly indexed or associated with the correct tenant context.


Fixing Wazuh OpenSearch Multi-Tenancy Mismatches

Once the mismatch source is identified, remediation involves standardizing tenant configuration, realigning RBAC, and repairing saved object scoping.

Correct Tenant Assignments

 

Reassign roles to correct tenants (global vs private)

Ensure each role explicitly defines its intended tenant scope. Avoid implicit defaults, as they often vary across OpenSearch versions.

Typical corrections include:

  • Moving SOC roles to a dedicated “SOC tenant”
  • Reserving global tenant for shared dashboards only
  • Restricting private tenant usage for debugging or personal views

Standardizing tenant strategy across environments

Consistency across dev, staging, and production is critical. Divergent tenant strategies are a primary cause of post-upgrade mismatches.

Rebuild Index Patterns

 

Recreating consistent index patterns per tenant

Recreate index patterns within each tenant that requires visibility. Ensure naming consistency (e.g., wazuh-alerts-* across environments).

Ensuring correct permissions on indices

Index access must align with tenant visibility. A valid pattern without permissions will still result in empty dashboards.

 Sync RBAC with OpenSearch Security Roles

 

Aligning Wazuh RBAC configuration with OpenSearch roles

Wazuh RBAC must not diverge from OpenSearch security definitions. Treat OpenSearch as the enforcement layer and Wazuh as the orchestration layer.

Avoiding conflicting permission layers

Conflicts occur when:

  • Wazuh grants access but OpenSearch restricts it
  • OpenSearch allows index access but blocks tenant visibility

For related RBAC troubleshooting patterns:

  • Troubleshooting Wazuh RBAC: Why Your Custom User Roles Aren’t Applying

Reset or Migrate Saved Objects

 

Export/import dashboards between tenants

Use controlled export/import workflows to migrate dashboards across tenants while preserving structure. Always verify tenant target during import.

Cleaning stale or duplicated objects

Remove outdated or duplicated saved objects that may be causing UI confusion or shadowing newer configurations.

Clear Session and Cache Issues

 

Forcing re-login to reset tenant context

Session-based tenant selection can persist incorrectly. A forced logout/login cycle often resolves mismatches caused by stale session state.

Clearing browser storage and cookies

OpenSearch Dashboards stores tenant context in browser storage. Clearing cookies and local storage ensures a clean tenant resolution state on next authentication.


Advanced Troubleshooting Techniques

When basic validation does not resolve Wazuh OpenSearch Multi-Tenancy Mismatches, you need to move into deeper inspection layers spanning OpenSearch Security APIs, internal indices, and cluster-level behavior.

At this stage, the goal is to eliminate ambiguity between UI-level issues and backend authorization state.

Using OpenSearch Security API to inspect roles and tenants

OpenSearch exposes Security Plugin APIs that allow direct inspection of role definitions, tenant mappings, and backend role assignments.

Key endpoints include:

  • _plugins/_security/api/roles
  • _plugins/_security/api/rolesmapping
  • _plugins/_security/api/internalusers

These APIs help confirm whether:

  • Roles are correctly bound to tenants
  • Backend roles are being mapped as expected
  • Permission inheritance is functioning correctly

This is the most reliable method to validate whether a mismatch is configuration-based or UI/session-based.

Querying security index directly (.opendistro_security)

OpenSearch stores security configuration in a dedicated system index (commonly .opendistro_security or .opensearch_security, depending on version).

Inspecting this index allows you to:

  • Verify persisted role definitions
  • Detect stale or conflicting mappings after upgrades
  • Confirm tenant metadata consistency across nodes

Direct index inspection is particularly useful when API responses and UI behavior diverge, indicating possible cluster state inconsistencies.

Comparing working vs broken user configurations

A highly effective diagnostic method is configuration diffing between:

  • A user with correct tenant visibility (baseline)
  • A user experiencing mismatch issues

Compare:

  • Assigned roles
  • Tenant permissions
  • Index pattern visibility
  • Backend role mappings

Even small discrepancies, such as missing “kibana_user” or equivalent role bindings, can produce significant tenant isolation failures.

Testing tenant isolation via curl/API calls

You can validate tenant isolation outside the UI using authenticated API requests.

Typical approach:

  • Authenticate as different users
  • Query same indices via _search
  • Compare response consistency across identities

If data is visible via API but missing in Dashboards, the issue is almost always:

  • Tenant-scoped saved objects
  • Missing index patterns
  • UI session context mismatch

If data is missing at API level, the issue is RBAC or index permission related.

Debugging in distributed cluster environments

In multi-node OpenSearch deployments, tenant mismatches can be amplified by cluster state inconsistency.

Key checks:

  • Ensure all nodes share identical security plugin configuration
  • Validate cluster state health via _cluster/state
  • Check for partial propagation of role/tenant updates
  • Confirm load balancer is not routing users to inconsistent nodes

In large-scale Wazuh deployments, even brief cluster state divergence can produce inconsistent tenant views across sessions.


Prevention: Avoiding Future Multi-Tenancy Issues

Preventing tenant mismatches requires treating tenant architecture as a first-class design constraint rather than a post-deployment configuration detail.

Defining a strict tenant strategy (per team, per environment, etc.)

Establish a deterministic tenant model such as:

  • Per team (SOC, DevOps, Compliance)
  • Per environment (Prod, Staging, Dev)
  • Hybrid models (Team × Environment separation)

Avoid ad-hoc tenant creation, which leads to uncontrolled saved object sprawl and inconsistent access patterns.

Automating role/tenant provisioning

Manual role and tenant configuration is a primary source of drift. Use automation via:

  • OpenSearch Security API scripts
  • Infrastructure-as-Code (Terraform, Ansible)
  • CI/CD-driven RBAC provisioning

This ensures role-to-tenant mappings remain consistent across environments and upgrades.

Version alignment between Wazuh and OpenSearch

Mismatched versions between Wazuh components and OpenSearch can introduce:

  • Changed tenant behavior defaults
  • Altered security plugin semantics
  • Incompatible saved object formats

Always validate compatibility matrices before upgrades to prevent silent tenant misalignment.

Monitoring role drift and configuration changes

Implement monitoring for:

  • Role modifications
  • Tenant mapping updates
  • Security plugin configuration changes

Audit logs should be continuously analyzed to detect unauthorized or accidental RBAC drift.

Related Guide: Fix Wazuh Active-Response Error 1204

Documentation of tenant and RBAC standards

Maintain a formalized governance document covering:

  • Tenant naming conventions
  • Role definitions and inheritance rules
  • Index pattern standards
  • Saved object ownership rules

This reduces ambiguity during onboarding, upgrades, and incident response.


Frequently Asked Questions (FAQ)

Question: Why do users see different dashboards in the same Wazuh cluster?

This typically occurs due to tenant-scoped saved objects. Even if users share identical roles, dashboards created in different tenants (private vs global) will not be visible across contexts.

Question: Can OpenSearch upgrades reset tenant configurations?

Yes. Major upgrades or security plugin changes can:

  • Reset default tenant assignments
  • Alter role mapping behavior
  • Require revalidation of saved object scope

This is one of the most common triggers for post-upgrade mismatches.

Question: What is the difference between global and private tenants?

  • Global tenant: Shared workspace for all authorized users
  • Private tenant: User-specific workspace isolated from others

Global tenants are used for standardized dashboards, while private tenants are typically used for experimentation or personal views.

Question: How do I migrate dashboards between tenants?

Dashboards can be exported and imported using OpenSearch Dashboards saved object management tools. During import, ensure:

  • Correct tenant is selected before import
  • All dependent objects (visualizations, index patterns) are included
  • No ID collisions exist across tenants

Question: Can RBAC override tenant settings in Wazuh?

No. RBAC and tenant scoping operate at different layers:

  • RBAC controls what a user can access
  • Tenants control where saved objects are visible

However, misaligned RBAC can appear to override tenants by blocking access to underlying data or saved objects.


Conclusion

Multi-tenancy mismatches in Wazuh–OpenSearch deployments are rarely caused by a single misconfiguration.

Instead, they emerge from systemic misalignment across RBAC definitions, tenant scoping rules, index patterns, and session state handling.

The most common root causes include:

  • Incorrect role-to-tenant mappings
  • Saved objects created in unintended tenant scopes
  • Post-upgrade configuration drift
  • Session or cache inconsistencies
  • Cluster state divergence in distributed environments

Resolving these issues requires both targeted debugging (APIs, security index inspection, role comparisons) and structural remediation (standardized tenant strategy, automated provisioning, and strict version alignment).

Ultimately, stable multi-tenant observability in Wazuh depends less on reactive troubleshooting and more on disciplined architecture design, where tenant boundaries, RBAC rules, and index access patterns are explicitly defined, consistently enforced, and continuously validated.

Be First to Comment

    Leave a Reply

    Your email address will not be published. Required fields are marked *