How to Clear Stuck Migration Indices in Wazuh Dashboard

The Wazuh Dashboard is a web-based interface that relies heavily on OpenSearch for indexing, search, and visualization of security data.

A “migration index stuck” issue typically refers to a scenario where the dashboard is unable to complete its internal saved object migrations.

These migrations are required whenever Wazuh Dashboard (or OpenSearch Dashboards) is upgraded, restarted, or when plugins change their schema.

When this process fails or freezes, the system remains in a perpetual “migration in progress” state.

In practical terms, this means the dashboard never fully loads. Instead, users often see a loading screen loop or a persistent message such as “Migrations in progress”, effectively rendering the UI unusable.

This issue is closely tied to backend consistency between dashboard plugins and OpenSearch indices, rather than being purely a frontend problem.

What Causes a Stuck Migration Index in Wazuh Dashboard?

A stuck migration process is almost always a symptom of underlying backend instability or incompatibility between components in the Wazuh stack.

The most common root causes include:

Interrupted Wazuh upgrade or rollback

If a Wazuh upgrade is interrupted (service crash, forced shutdown, or partial rollback), the saved object migration process may be left incomplete.

This results in mismatched index versions that the dashboard cannot reconcile.

OpenSearch cluster health issues (yellow/red status)

When the OpenSearch cluster is not in a healthy state, migrations may fail or pause indefinitely.

A cluster in yellow or red status often indicates shard allocation problems or missing replicas, both of which block index updates.

Failed saved objects migrations

The dashboard relies on saved object migrations to transform index schemas between versions. If a migration script fails (due to timeout, permission errors, or corrupted documents), the entire migration queue can stall.

Disk watermark issues blocking indexing

OpenSearch enforces disk watermarks to prevent node failure.

If disk usage exceeds thresholds (especially high or flood-stage watermark), indexing operations are blocked, which directly halts migration processes.

Plugin version mismatch between Wazuh Dashboard and OpenSearch

Version incompatibility between the Wazuh Dashboard plugin and OpenSearch backend is a frequent cause of migration failures. Even minor version mismatches can prevent proper schema translation.

Corrupted .kibana / .opensearch_dashboards indices

Corruption in system indices responsible for storing dashboard state (such as .kibana or .opensearch_dashboards) can completely break migration logic, leaving the system stuck in an initialization loop.

Additional Resources

https://opensearch.org/docs/latest/install-and-configure/configuring-opensearch/
https://documentation.wazuh.com/current/installation-guide/index.html

How to Identify the Problem

Diagnosing a stuck migration index requires correlating logs, cluster health metrics, and API responses.

Checking Wazuh Dashboard logs

The first step is inspecting dashboard logs for migration-related errors.

Look for patterns such as:

• “saved object migration failed”
• “migration lock not acquired”
• “timeout waiting for migrations”

These logs are typically found in /var/log/wazuh-dashboard/ or the OpenSearch Dashboards log directory depending on deployment.

Related Guide:

How to Fix Wazuh API Authentication Failed? Causes and Solutions

Inspecting OpenSearch cluster health

A quick way to confirm backend issues is checking cluster status:

• Green → Healthy, migrations should proceed normally
• Yellow → Replica issues; may slow or block migrations
• Red → Critical failure; migrations will not complete

You can query cluster health using OpenSearch API:

GET _cluster/health

If the cluster is not green, migration issues are likely a downstream symptom.

Verifying migration status via OpenSearch APIs

OpenSearch exposes index and migration-related endpoints that help identify stalled processes.

You can inspect system indices and check for long-running tasks or blocked writes.

Key things to look for:

• Indices in close or read-only state
• High number of pending tasks
• Repeated migration retries in task queue

Symptoms in browser UI

From the frontend perspective, a stuck migration index usually appears as:

• Endless loading spinner on dashboard startup
• “Migrations in progress” message that never resolves
• Inability to access visualization panels or alerts
• Partial UI rendering with missing dashboards

These symptoms confirm that the issue is backend-driven rather than a browser or cache problem.

Step-by-Step Fix for Stuck Migration Index

Verify OpenSearch Cluster Health

Before making any changes, confirm the state of the underlying OpenSearch cluster. Migration processes will fail or stall if the cluster is not fully healthy.

Check cluster status:

GET _cluster/health

Key interpretations:

• green → All primary and replica shards are allocated; migrations should proceed normally
• yellow → Replica shards missing; migrations may be degraded or delayed
• red → Critical issue; primary shards missing, migrations will stall completely

Next, inspect shard allocation issues:

GET _cat/shards?v

Look for:

• UNASSIGNED shards
• RELOCATING shards stuck indefinitely
• Repeated shard failures on specific nodes

If shard allocation is broken, fix this before attempting any migration recovery steps.

Restart Wazuh Stack in Correct Order

Service order matters because dependencies flow upward from storage to UI.

Correct restart sequence:

1. OpenSearch (backend indexer)
2. Wazuh Manager
3. Wazuh Dashboard

Improper ordering can cause:

• Partial index initialization
• Re-triggered failed migrations
• Locked saved object states

After restart, wait for cluster stabilization before accessing the dashboard UI.

Inspect and Reset Migration Status

If the dashboard remains stuck, inspect migration state directly through OpenSearch APIs.

Check indices related to dashboard state:

GET _cat/indices/.opensearch_dashboards*?v

Look for:

• “close” state indices
• unusually large document counts
• repeated write failures

You can also inspect task activity:

GET _tasks?actions=*migration*

If migration tasks are stuck in running or repeatedly retrying, this confirms a locked migration state.

At this stage, identify whether a specific index is blocking progress (usually .opensearch_dashboards_*).

Rebuild or Delete Corrupted Dashboard Indices

If migration state is irreparably stuck, the fastest recovery path is rebuilding dashboard indices.

Target indices:

• .opensearch_dashboards_*
• .kibana_* (only in legacy or mixed upgrade environments)

Delete carefully:

DELETE .opensearch_dashboards_*

On restart, the system will automatically:

• Recreate system indices
• Reinitialize saved object schema
• Re-run migrations cleanly

⚠️ This may remove saved dashboards, visualizations, and UI configurations unless backed up.

Fix Permission and Security Plugin Issues

Migration failures often occur when index permissions block schema updates.

Check:

• Role mappings for dashboard service user
• Index privileges for .opensearch_dashboards_*
• Authentication failures in logs

If using the OpenSearch Security Plugin:

• Reset internal users if credentials are corrupted
• Validate roles like opensearch_dashboards_user

Also ensure the dashboard user has:

• index read/write permissions
• create_index privileges during migration
• access to system indices

Increase Disk Space / Fix Watermarks

Disk saturation is a common hidden cause of migration stalls.

Check disk usage per node:

GET _cat/allocation?v

If disk watermark thresholds are exceeded:

• Delete old indices or logs
• Archive historical data
• Temporarily adjust watermark settings (only if safe)

Example (temporary adjustment):

PUT _cluster/settings
{
  "transient": {
    "cluster.routing.allocation.disk.watermark.low": "85%",
    "cluster.routing.allocation.disk.watermark.high": "90%",
    "cluster.routing.allocation.disk.watermark.flood_stage": "95%"
  }
}

Migration will resume only after write restrictions are lifted.

Advanced Recovery Methods

Force Migration Reset (Last Resort)

If standard fixes fail, you can force a reset of saved object migration state.

This involves:

• Clearing migration tracking metadata
• Resetting dashboard saved object version state
• Forcing reinitialization on next startup

Use with caution, as it may reset dashboard configuration state entirely.

Restore from Snapshot Backup

If the cluster is severely corrupted, restoring from a snapshot is often safer than manual repair.

Typical workflow:

• Identify last known good snapshot
• Stop index writes
• Restore OpenSearch snapshot
• Restart full Wazuh stack

This method is recommended when multiple system indices are corrupted beyond repair.

Reinstall Wazuh Dashboard (Clean State)

When configuration drift is too severe, a clean reinstall may be required.

Steps:

• Backup configuration files (opensearch_dashboards.yml, certificates, plugins)
• Remove dashboard package
• Reinstall matching version of Wazuh Dashboard
• Reconnect to existing OpenSearch cluster

This ensures a fresh migration baseline without legacy corruption.

Preventing Migration Index Issues in the Future

Preventing recurrence is largely about operational discipline across the stack:

• Always upgrade components in correct order: OpenSearch → Wazuh Manager → Wazuh Dashboard
• Monitor cluster health (ensure green status) before any upgrade or restart
• Maintain at least 20% free disk headroom on all data nodes
• Avoid abrupt shutdowns during migration or indexing processes
• Schedule regular snapshots of OpenSearch indices for fast rollback recovery

Additional Resources:

How to Build a Wazuh Indexer Cluster
How to Configure Wazuh Log Retention

Frequently Asked Questions (FAQ)

Question: Why is my Wazuh dashboard stuck on migration?

This usually happens when saved object migrations fail inside OpenSearch due to index corruption, shard allocation issues, or version mismatch between dashboard and backend components.

In most cases, the root cause is not the UI itself but instability in the underlying OpenSearch cluster or blocked system indices.

Question: Can restarting Wazuh fix a stuck migration index?

Restarting the stack can sometimes resolve the issue if the problem is transient (e.g., temporary node unavailability or short-lived cluster instability).

However, if the root cause is:

• corrupted system indices
• disk watermark enforcement
• persistent shard allocation failure

then a restart alone will not resolve the migration lock state.

Question: Is it safe to delete dashboard indices?

Yes, but with important trade-offs.

Deleting .opensearch_dashboards_* (or legacy .kibana_*) indices will:

• Reset all saved dashboards
• Remove visualizations and saved searches
• Force a clean migration reinitialization

It is safe from a cluster integrity standpoint, but it should be treated as a rebuild operation, not a minor fix.

Question: How long should migrations take in Wazuh?

Under normal conditions, migrations complete within a few minutes after startup of the Wazuh Dashboard.

If migrations exceed this window significantly, it typically indicates:

• OpenSearch performance degradation
• Disk watermark restrictions
• Corrupted or blocked system indices
• Cluster shard imbalance

Persistent delays are almost always a backend issue rather than a dashboard-level problem.

Conclusion

A stuck migration index in the Wazuh ecosystem is rarely an isolated dashboard issue.

It is usually a downstream symptom of problems within OpenSearch such as shard allocation failures, disk pressure, corrupted system indices, or version mismatches.

The most effective fixes follow a predictable pattern:

• Verify cluster health first
• Restart services in correct dependency order
• Inspect and reset migration state when necessary
• Rebuild corrupted dashboard indices if required
• Escalate to snapshot restore or reinstall only as last-resort recovery methods

Long-term stability depends on operational discipline: monitoring OpenSearch health, enforcing upgrade order, and maintaining adequate disk capacity.