How to Migrate Wazuh Indexer Data to New Storage Volumes

As Wazuh deployments grow, so does the amount of security data stored by the Wazuh Indexer. Every endpoint, firewall, cloud service, application, and network device continuously generates logs that are indexed for searching, correlation, and long-term analysis.

Over time, these indexes can consume hundreds of gigabytes, or even multiple terabytes, of storage, making it necessary to migrate the Indexer to larger or faster storage volumes.

Storage migration is a routine part of maintaining enterprise Wazuh environments.

Organizations often move their Indexer data when expanding storage capacity, replacing aging hardware, upgrading to SSD or NVMe drives, migrating virtual machines, or relocating workloads into cloud infrastructure.

When performed correctly, the migration is transparent to users and preserves all indexed security data.

However, migrating Wazuh Indexer storage is not as simple as copying files from one disk to another.

The Indexer manages distributed shards, transaction logs, cluster metadata, and persistent node information that must remain consistent throughout the migration.

Improper procedures can lead to shard corruption, cluster instability, unassigned indexes, or even permanent data loss.

Fortunately, careful planning dramatically reduces these risks.

By validating cluster health, creating reliable backups, using snapshots when appropriate, and following a structured migration process, administrators can relocate storage with minimal downtime while maintaining data integrity.

In this guide, you’ll learn how Wazuh Indexer stores data, when storage migration becomes necessary, how to prepare your environment, multiple migration methods, validation steps after migration, performance optimization techniques, common pitfalls to avoid, and best practices for keeping your Indexer healthy after the move.


Understanding How Wazuh Indexer Stores Data

Before migrating storage, it’s important to understand how the Wazuh Indexer organizes and stores security data.

Since Wazuh Indexer is based on OpenSearch, its storage architecture differs significantly from traditional databases.

Every alert, log, vulnerability result, and inventory record is stored inside distributed indexes that are automatically divided across cluster nodes.

Understanding this architecture helps prevent accidental data corruption during migration and makes troubleshooting much easier if problems occur afterward.

Wazuh Indexer Storage Architecture

The Wazuh Indexer is responsible for storing and indexing virtually all security events generated throughout your environment.

Rather than keeping information inside a single database file, the Indexer distributes data across multiple indexes, each consisting of one or more shards.

Each shard functions as an independent search engine containing a subset of the overall data.

The Indexer distributes these shards across cluster nodes to improve performance, scalability, and fault tolerance.

For example, a deployment monitoring 10,000 endpoints may generate millions of events daily.

Instead of storing everything in one location, Wazuh automatically spreads indexes across available nodes, allowing searches to execute in parallel.

This distributed architecture is one of the primary reasons why storage migrations require careful planning, moving one node incorrectly can affect shard allocation across the entire cluster.

Data Directories and Index Storage

All indexed security data is stored on disk inside the Indexer’s data directory.

This directory typically contains:

  • Index files
  • Lucene segments
  • Transaction logs (translogs)
  • Cluster metadata
  • Node metadata
  • Snapshot cache
  • Search indexes
  • Index State Management (ISM) metadata

The exact location depends on your installation method.

Common default paths include:

/var/lib/wazuh-indexer/
/var/lib/opensearch/
/usr/share/wazuh-indexer/data/

Administrators frequently relocate these directories onto dedicated storage volumes to increase capacity or improve disk performance.

Simply copying files while the Indexer is running is unsafe because indexes are constantly being updated.

Active writes may leave copied data in an inconsistent state, resulting in corrupted shards or failed node startups after migration.

If you’re also planning long-term storage optimization, see How to Manage Wazuh Storage.

Cluster Nodes and Shard Distribution

In clustered deployments, data is rarely stored entirely on a single server.

Instead, Wazuh distributes primary shards and replica shards across multiple Indexer nodes.

For example:

  • Node A stores primary shards
  • Node B stores replica shards
  • Node C stores additional indexes and replicas

This distribution provides several advantages:

  • High availability
  • Load balancing
  • Faster searches
  • Hardware failure tolerance
  • Automatic shard recovery

When migrating storage on one node, the cluster may temporarily relocate shards to healthy nodes if allocation remains enabled.

Many administrators temporarily disable shard allocation during planned maintenance to prevent unnecessary data movement and reduce migration time.

Understanding shard placement beforehand is essential.

If you are unfamiliar with shard planning, read How to Design a Wazuh OpenSearch Shard Allocation Strategy.

Persistent Volumes and Storage Paths

The underlying storage medium can vary considerably depending on the deployment environment.

Common storage options include:

  • Local SSDs
  • NVMe drives
  • RAID arrays
  • SAN storage
  • NAS storage
  • Cloud block storage
  • Kubernetes Persistent Volumes (PVs)
  • VMware virtual disks

Regardless of the storage technology, the Indexer references a configured data path where indexes are stored.

Changing this path typically involves:

  • Creating the new storage volume
  • Copying existing data safely
  • Updating Indexer configuration
  • Adjusting ownership and permissions
  • Restarting the Indexer
  • Verifying cluster health

In containerized deployments, persistent volumes make storage migrations easier because volumes can often be detached and reattached without reinstalling Wazuh.

OpenSearch documentation recommends using reliable, low-latency storage because indexing performance depends heavily on disk throughput and I/O latency.

SSDs generally provide significantly better indexing performance than mechanical hard drives, especially in environments with continuous write activity.


Common Reasons to Migrate Wazuh Indexer Storage

Storage migrations are usually driven by infrastructure growth, performance improvements, or hardware lifecycle management.

As organizations collect more security telemetry, the storage requirements of the Wazuh Indexer naturally increase, making migration an inevitable part of long-term maintenance.

Running Out of Disk Space

The most common reason for migrating Wazuh Indexer data is insufficient disk capacity.

Large environments continuously generate:

  • Security alerts
  • Endpoint telemetry
  • Vulnerability scan results
  • File Integrity Monitoring events
  • Cloud logs
  • Authentication logs
  • Network events

Without adequate retention policies, indexes can quickly consume available storage.

Low disk space can trigger problems such as:

  • Read-only indexes
  • Failed writes
  • Unassigned shards
  • Cluster health warnings
  • Indexing failures
  • “All Shards Failed” errors

If you’re already experiencing allocation failures caused by low storage, see How to Fix Wazuh All Shards Failed Error and How to Fix a Yellow Cluster Status in Wazuh Indexer.

Upgrading to Larger Drives

Many Wazuh deployments begin on modest virtual disks or small physical drives during testing or initial rollout.

As deployments mature, administrators often upgrade to:

  • Larger SSDs
  • Enterprise NVMe drives
  • RAID storage
  • High-capacity SAN volumes

Expanding storage allows organizations to:

  • Retain security data longer
  • Store additional indexes
  • Reduce emergency maintenance
  • Support more monitored endpoints
  • Handle future growth

Rather than rebuilding the entire Indexer, migrating existing data preserves historical alerts and minimizes operational disruption.

Moving from HDD to SSD/NVMe

Mechanical hard drives often become a performance bottleneck in busy SIEM environments.

The Wazuh Indexer performs frequent:

  • Disk writes
  • Segment merges
  • Index refreshes
  • Search operations
  • Background optimization

SSD and NVMe storage dramatically reduce latency for these operations, improving overall responsiveness.

Benefits commonly include:

  • Faster dashboard searches
  • Reduced indexing latency
  • Lower query response times
  • Faster shard recovery
  • Improved cluster stability during peak workloads

Performance gains depend on workload characteristics, but numerous benchmarking studies consistently show that low-latency solid-state storage significantly improves search engine performance compared to traditional spinning disks for write-intensive indexing workloads.

Migrating to Cloud Block Storage

Organizations moving workloads into cloud environments frequently migrate Indexer storage to managed block storage services.

Examples include:

  • Amazon EBS
  • Azure Managed Disks
  • Google Persistent Disk

Cloud block storage offers several operational advantages:

  • Easier capacity expansion
  • Built-in redundancy options
  • Snapshot capabilities
  • Simplified disaster recovery
  • Improved infrastructure automation

When planning cloud migrations, storage performance tiers (such as provisioned IOPS or premium SSD offerings) should be evaluated carefully to ensure they meet indexing and search workloads.

Improving Performance

Storage upgrades are not always about increasing capacity.

Many organizations migrate storage specifically to improve overall Indexer performance.

Common goals include:

  • Lower indexing latency
  • Faster dashboard loading
  • Improved search responsiveness
  • Reduced merge times
  • Faster snapshot operations
  • Better cluster recovery speeds

According to OpenSearch performance guidance, storage throughput and latency are among the most significant factors affecting indexing performance, particularly in clusters processing continuous log ingestion.

Replacing Failing Disks

Hardware eventually wears out.

Warning signs that indicate a storage migration may be necessary include:

  • Increasing SMART errors
  • Growing bad sector counts
  • Frequent filesystem errors
  • Disk controller warnings
  • Unusually slow disk I/O
  • Kernel storage-related error messages

Replacing aging drives proactively reduces the likelihood of unexpected failures and emergency recovery procedures.

For clustered deployments, replacing one node’s storage at a time while maintaining healthy replicas helps minimize downtime and protects against data loss.

Before beginning any migration, ensure recent snapshots or backups are available so that data can be restored if unexpected hardware issues occur.


Prerequisites Before Migrating

Proper preparation is the difference between a routine storage migration and an extended outage.

Before moving any Wazuh Indexer data, verify that the cluster is healthy, create recoverable backups, and ensure the destination storage is ready.

Spending time on these checks greatly reduces the likelihood of shard corruption, failed node startups, or prolonged recovery efforts.

Verify Cluster Health

Never begin a storage migration if the cluster is already experiencing issues.

Start by checking the cluster health using the OpenSearch Cluster Health API:

curl -k -u admin:password https://<INDEXER_IP>:9200/_cluster/health?pretty

A healthy cluster should ideally report:

  • Green — All primary and replica shards are allocated.
  • Yellow — Primary shards are available, but one or more replica shards are unassigned.
  • Red — One or more primary shards are unavailable.

While migrating from a green cluster is strongly recommended, a yellow cluster may be acceptable in some single-node deployments where replicas cannot be allocated.

A red cluster should always be investigated before proceeding.

Review the following information carefully:

  • Cluster status
  • Number of active shards
  • Unassigned shards
  • Pending tasks
  • Initializing shards
  • Relocating shards

If shard allocation problems already exist, resolve them first.

See How to Fix a Yellow Cluster Status in Wazuh Indexer and How to Fix Wazuh All Shards Failed Error.

Check Disk Utilization

Evaluate both the current storage volume and the destination volume before migrating.

Useful Linux commands include:

df -h
lsblk
du -sh /var/lib/wazuh-indexer

Look for:

  • Current disk usage
  • Remaining free space
  • Filesystem type
  • Mounted volumes
  • Storage utilization trends

If the existing disk is already above 85–90% utilization, prioritize creating a snapshot before performing any migration.

Create Full Snapshots

Snapshots provide the safest recovery option if something goes wrong during migration.

Unlike copying raw files, snapshots create a consistent backup of indexes while preserving metadata and shard information.

Typical snapshot workflow:

  1. Register a snapshot repository.
  2. Create a snapshot.
  3. Wait until it completes successfully.
  4. Verify snapshot status.
  5. Record the repository location.

OpenSearch recommends snapshots as the primary backup mechanism for disaster recovery because filesystem-level backups of active nodes are not guaranteed to produce a consistent cluster state.

Back Up Configuration Files

Storage migration normally involves more than just moving data.

Back up important configuration files such as:

/etc/wazuh-indexer/opensearch.yml
/etc/wazuh-indexer/jvm.options
/etc/wazuh-indexer/log4j2.properties
/etc/systemd/system/

Also preserve:

  • TLS certificates
  • Security configuration
  • Cluster settings
  • Custom plugins
  • Index templates
  • ISM policies

These files simplify recovery if configuration changes need to be rolled back after the migration.

Ensure Sufficient Free Storage

The destination storage should comfortably accommodate both current and future data growth.

As a best practice, plan for:

  • Existing index size
  • Transaction logs
  • Future index growth
  • Replica shards
  • Snapshot cache
  • Temporary migration overhead

Avoid migrating to a volume that will immediately approach capacity limits.

Maintaining at least 20–30% free space after migration helps sustain indexing performance and reduces the risk of disk watermark issues.

If you’re planning long-term storage growth, see How to Manage Wazuh Storage.

Schedule a Maintenance Window

Even carefully planned migrations introduce some level of operational risk.

Schedule maintenance during periods of low log ingestion whenever possible.

A maintenance window allows you to:

  • Stop indexing safely
  • Validate copied data
  • Perform integrity checks
  • Correct unexpected issues
  • Roll back if necessary

For clustered environments, downtime may be minimal or avoided altogether with rolling migrations, but administrators should still notify users of potential performance impacts.

Document Existing Storage Configuration

Before changing anything, document the current environment.

Capture information such as:

  • Current data path
  • Mount points
  • Disk UUIDs
  • Filesystem type
  • Device names
  • Cluster node names
  • Current storage capacity
  • Index sizes
  • Shard allocation

Useful commands include:

mount
blkid
cat /etc/fstab
curl -k -u admin:password https://<INDEXER_IP>:9200/_cat/allocation?v

Having this information readily available makes troubleshooting and rollback significantly easier if unexpected issues arise.


Migration Planning Best Practices

A successful migration starts long before any data is copied.

Careful planning reduces downtime, minimizes risk, and helps ensure that the new storage environment supports future growth.

The following best practices can help you avoid common pitfalls and make the migration process more predictable.

Estimate Required Storage Capacity

Avoid sizing the new storage volume based only on current disk usage.

Consider future growth by accounting for:

  • Daily log ingestion
  • Expected endpoint growth
  • Log retention requirements
  • Replica shards
  • Snapshots
  • Temporary maintenance space

For example, if your Indexer currently stores 800 GB of data and grows by approximately 20 GB per week, deploying a 1 TB volume may leave little room for expansion.

Selecting a larger volume now can prevent another migration in the near future.

Verify Filesystem Compatibility

Most Linux filesystems work well with Wazuh Indexer, but consistency across cluster nodes simplifies management and troubleshooting.

Common choices include:

  • XFS
  • ext4

Verify:

  • Filesystem supports large files
  • Mount options are appropriate
  • Ownership and permissions are preserved
  • Performance characteristics meet workload requirements

After mounting the new storage, confirm that the Indexer service account has full read and write access to the data directory before restarting the service.

Plan for Downtime or Rolling Migration

The migration approach depends largely on your deployment architecture.

Single-node deployments

  • Require planned downtime.
  • Indexing stops while data is copied.
  • Simpler migration process.

Multi-node clusters

  • May support rolling migrations.
  • Nodes are migrated individually.
  • Cluster continues serving requests using remaining healthy nodes.
  • Downtime can often be minimized.

Choosing the correct strategy depends on business requirements, maintenance windows, and cluster size.

Test Migration in a Staging Environment

If possible, perform a complete test migration in a non-production environment.

A staging test helps verify:

  • Copy procedures
  • Mount configuration
  • Startup behavior
  • Cluster discovery
  • Permissions
  • Performance
  • Recovery procedures

Testing also provides realistic estimates for migration duration, allowing maintenance windows to be planned more accurately.

Prepare Rollback Procedures

Every migration should include a documented rollback plan.

Typical rollback steps include:

  1. Stop the Indexer.
  2. Restore the original storage mount.
  3. Restore configuration files if changed.
  4. Restart the Indexer.
  5. Verify cluster health.
  6. Restore snapshots if required.

Having a clearly documented rollback procedure allows administrators to recover quickly if the new storage experiences unexpected problems or performance issues.


Method 1: Migrate a Single-Node Wazuh Indexer

Migrating a single-node Wazuh Indexer is generally straightforward because there are no replica shards or additional cluster members to coordinate.

However, since all indexed data resides on one server, careful handling is essential to avoid corruption or data loss.

The following example assumes the Indexer data directory is located at:

/var/lib/wazuh-indexer

Adjust paths as needed for your environment.

Stop Wazuh Indexer Safely

Before copying any data, stop the Indexer service to ensure all pending writes are flushed to disk.

For most Linux installations:

sudo systemctl stop wazuh-indexer

Verify that the service has stopped completely:

sudo systemctl status wazuh-indexer

You should also confirm that no Java processes associated with the Indexer are still running:

ps -ef | grep opensearch

Copying files while the Indexer is actively writing data can result in incomplete indexes or corrupted shard segments.

Create a Backup

Even if snapshots have already been created, maintain an additional backup before making changes.

For example:

sudo tar -czf wazuh-indexer-backup.tar.gz /var/lib/wazuh-indexer

Or create a filesystem-level backup using your preferred backup software.

Store backups on separate storage whenever possible.

This extra layer of protection can significantly reduce recovery time if hardware or migration errors occur.

Copy Existing Data

Mount the new storage volume and copy the complete data directory while preserving permissions, timestamps, symbolic links, and extended attributes.

A common approach is using rsync:

sudo rsync -aHAX --progress /var/lib/wazuh-indexer/ /mnt/new-storage/

The options preserve:

  • File ownership
  • Permissions
  • Hard links
  • ACLs
  • Extended attributes
  • Modification timestamps

After the copy completes, compare directory sizes to ensure the transfer finished successfully:

du -sh /var/lib/wazuh-indexer
du -sh /mnt/new-storage

For additional verification, some administrators compare file counts or generate checksums for critical directories before switching to the new storage.

Update Storage Mount

Once the data has been copied successfully, configure the new volume to replace the existing storage location.

Typical steps include:

  1. Unmount the old storage.
  2. Mount the new volume.
  3. Update /etc/fstab if the mount should persist across reboots.
  4. Verify the mount.

Example:

mount | grep wazuh-indexer

Ensure that the mounted directory now points to the new storage device before restarting the service.

Modify Configuration if Required

If the data directory has changed, update the Indexer configuration accordingly.

For example, review the path.data setting in:

/etc/wazuh-indexer/opensearch.yml

Example:

path.data: /mnt/new-storage

If the new volume is mounted at the original path, configuration changes may not be necessary.

After modifying configuration files, double-check:

  • File permissions
  • Ownership
  • YAML syntax
  • Correct mount paths

Configuration mistakes are a common cause of failed Indexer startups after migration.

Restart the Indexer

After verifying the storage configuration, start the Indexer:

sudo systemctl start wazuh-indexer

Monitor startup logs for warnings or errors:

sudo journalctl -u wazuh-indexer -f

Look for messages indicating:

  • Successful node startup
  • Cluster initialization
  • Shard recovery
  • Index loading
  • Filesystem access

Address any reported errors before allowing production log ingestion to resume.

Verify Data Integrity

Once the Indexer is running, verify that the migration completed successfully.

Recommended validation steps include:

  • Check cluster health.
  • Confirm all indexes are present.
  • Verify document counts.
  • Ensure dashboards load normally.
  • Run several representative searches.
  • Monitor logs for filesystem or shard errors.
  • Verify new alerts are being indexed.

Useful API calls include:

curl -k -u admin:password https://<INDEXER_IP>:9200/_cluster/health?pretty
curl -k -u admin:password https://<INDEXER_IP>:9200/_cat/indices?v

Continue monitoring the Indexer over the next several hours to ensure indexing remains stable and no delayed shard or storage-related issues appear.


Method 2: Migrate a Multi-Node Wazuh Indexer Cluster

Migrating a multi-node Wazuh Indexer cluster requires more coordination than a single-node deployment, but it also offers a major advantage: the cluster can often remain available throughout the migration.

By moving one node at a time and allowing the remaining nodes to continue serving requests, administrators can significantly reduce or even eliminate downtime.

The key is to migrate each node in a controlled sequence while maintaining cluster health and ensuring that shard allocation remains consistent.

Migrate One Node at a Time

Never migrate every Indexer node simultaneously.

Instead:

  1. Select one node.
  2. Remove it from service.
  3. Migrate its storage.
  4. Rejoin it to the cluster.
  5. Verify cluster health.
  6. Repeat for the next node.

This rolling approach maintains cluster availability while minimizing operational risk.

Before starting, verify that:

  • Replica shards are healthy.
  • Remaining nodes have sufficient capacity.
  • Recent snapshots are available.
  • No ongoing shard recoveries are in progress.

If your deployment is still being expanded, see How to Set Up a Multi-Node Wazuh Cluster.

Relocate Shards Safely

Before shutting down a node, prevent OpenSearch from performing unnecessary shard reallocations.

Many administrators temporarily disable shard allocation during planned maintenance:

PUT /_cluster/settings
{
  "persistent": {
    "cluster.routing.allocation.enable": "primaries"
  }
}

This allows primary shards to remain available while avoiding excessive network traffic caused by automatic replica movement.

After the migration is complete, restore normal shard allocation:

PUT /_cluster/settings
{
  "persistent": {
    "cluster.routing.allocation.enable": "all"
  }
}

Temporarily disabling allocation during maintenance is a commonly recommended practice for planned node restarts because it reduces unnecessary shard recovery and shortens maintenance windows.

Remove the Node from Service

Once shard allocation has been configured appropriately, stop the Wazuh Indexer service on the selected node:

sudo systemctl stop wazuh-indexer

Confirm that:

  • The service has stopped.
  • No Java processes remain.
  • Disk activity has ceased.
  • No active writes are occurring.

Also verify from another cluster node that the remaining cluster members remain healthy.

Monitor:

  • Cluster status
  • Active shards
  • Pending tasks
  • Node availability

Move Storage Volumes

With the node offline, migrate the storage using your preferred method.

Typical steps include:

  • Attach the new storage device.
  • Format the filesystem if necessary.
  • Mount the new volume.
  • Copy Indexer data.
  • Verify copied files.
  • Update mount configuration.
  • Preserve permissions.

Since the node is offline, no index modifications will occur during the copy process.

Rejoin the Node to the Cluster

After the storage migration is complete, restart the Indexer:

sudo systemctl start wazuh-indexer

Monitor startup logs to verify:

  • Successful node startup
  • Cluster discovery
  • Shard recovery
  • No filesystem errors
  • No permission issues

Once the node rejoins the cluster, check that it appears in the node list:

curl -k -u admin:password https://<INDEXER_IP>:9200/_cat/nodes?v

The node should report a healthy status before proceeding to the next migration.

Repeat for Remaining Nodes

After confirming that the first migrated node is fully operational:

  • Select the next node.
  • Repeat the migration procedure.
  • Validate cluster health.
  • Continue until every storage volume has been migrated.

Avoid migrating multiple nodes simultaneously unless you have thoroughly tested the procedure and understand the impact on replica availability.

Large production clusters often migrate one node per maintenance window to minimize operational risk.

Validate Cluster Health

After every node has been migrated, perform a complete cluster validation.

Recommended checks include:

  • Cluster health is green.
  • No unassigned shards exist.
  • Replica shards are allocated.
  • Node count matches expectations.
  • Index counts are unchanged.
  • Searches return expected results.
  • Dashboard loads correctly.
  • New alerts are indexed successfully.

Useful commands include:

curl -k -u admin:password https://<INDEXER_IP>:9200/_cluster/health?pretty
curl -k -u admin:password https://<INDEXER_IP>:9200/_cat/shards?v
curl -k -u admin:password https://<INDEXER_IP>:9200/_cat/allocation?v

If shards fail to recover correctly after migration, consult How to Fix Wazuh All Shards Failed Error and How to Design a Wazuh OpenSearch Shard Allocation Strategy.


Moving Data to New Storage Volumes

Regardless of whether you’re migrating a single node or an entire cluster, the actual movement of data follows the same core process.

The objective is to copy every Indexer file accurately while preserving metadata, permissions, and filesystem attributes so the Indexer can resume normal operation without rebuilding indexes.

Create and Mount the New Volume

Provision the new storage before beginning the migration.

Typical steps include:

  1. Attach the new disk.
  2. Create a partition if necessary.
  3. Format the filesystem.
  4. Create a mount point.
  5. Mount the volume.
  6. Verify available capacity.

Example:

sudo mkdir -p /mnt/wazuh-indexer
sudo mount /dev/sdb1 /mnt/wazuh-indexer

Confirm the mount:

df -h

Before copying data, ensure the new volume provides sufficient free space for current indexes and projected growth.

Copy Data Using rsync

The preferred tool for migrating Wazuh Indexer data is rsync.

Example:

sudo rsync -aHAX --info=progress2 /var/lib/wazuh-indexer/ /mnt/wazuh-indexer/

The options preserve:

  • Ownership
  • Permissions
  • ACLs
  • Hard links
  • Extended attributes
  • Symbolic links
  • Modification timestamps

Compared to a basic cp command, rsync offers better reliability, verification, and resumable transfers for large datasets.

Preserve Ownership and Permissions

Incorrect permissions are among the most common causes of startup failures after storage migrations.

After copying the files, verify ownership:

ls -ld /mnt/wazuh-indexer

If necessary:

sudo chown -R wazuh-indexer:wazuh-indexer /mnt/wazuh-indexer

Also verify:

  • Directory permissions
  • Parent directory access
  • SELinux contexts (if enabled)
  • AppArmor policies (if applicable)

Failure to preserve permissions may prevent the Indexer from reading shard data during startup.

Verify Copied Files

Before deleting or unmounting the original storage, compare the copied data.

Useful verification methods include:

  • Directory size comparison
  • File count comparison
  • Spot-checking critical directories
  • Generating checksums for important files

Examples:

du -sh /var/lib/wazuh-indexer
du -sh /mnt/wazuh-indexer

For especially large environments, consider running a final rsync pass after the initial copy to ensure no files were missed before switching storage.

Update Mount Points

Once verification is complete, configure the new storage to mount automatically after reboot.

Review:

cat /etc/fstab

Update the entry to reference the new filesystem UUID whenever possible rather than relying on device names such as /dev/sdb1, which may change after reboots.

Retrieve the UUID using:

blkid

Using UUIDs generally provides more reliable mount behavior across hardware changes.

Test the New Storage

Before declaring the migration complete, verify that the Indexer is actively using the new storage.

Check:

mount
df -h
lsblk

Then confirm:

  • New indexes are created.
  • Existing indexes remain searchable.
  • Disk usage increases on the new volume.
  • No writes occur on the old storage.
  • System logs contain no filesystem errors.

Monitor performance during the first several hours after migration to ensure normal indexing rates and acceptable storage latency.


Updating Wazuh Indexer Configuration

Most storage migrations require little or no configuration changes if the new storage is mounted at the same location.

However, if the data path changes or new mount points are introduced, administrators should carefully review the Indexer’s configuration before returning the node to production.

Verify Data Path Settings

The most important configuration value is the Indexer’s data directory.

Review the path.data setting in:

/etc/wazuh-indexer/opensearch.yml

Example:

path.data: /var/lib/wazuh-indexer

Or:

path.data: /mnt/wazuh-indexer

Ensure the configured directory matches the location of the migrated data.

A mismatched path may cause the Indexer to start with an empty data directory or fail to locate existing indexes.

Check OpenSearch Configuration

Review the remainder of the OpenSearch configuration for settings affected by the migration.

Verify items such as:

  • Node name
  • Cluster name
  • Discovery settings
  • Network configuration
  • TLS certificates
  • Snapshot repositories
  • Logging paths

Avoid making unrelated configuration changes during the storage migration, as this can complicate troubleshooting if problems occur afterward.

For additional information about Index State Management and storage optimization, see How to Configure Wazuh Index State Management (ISM).

Confirm systemd Service Settings

If custom service overrides have been configured, verify that they remain valid.

Review:

systemctl cat wazuh-indexer

Check for:

  • Custom environment variables
  • Startup parameters
  • Resource limits
  • Custom data directory references

After any modifications, reload the systemd configuration:

sudo systemctl daemon-reload

This ensures that any updated service definitions are recognized before restarting the Indexer.

Validate Filesystem Permissions

Before starting the Indexer, confirm that the service account has complete access to the new storage location.

Verify:

  • Ownership
  • Read permissions
  • Write permissions
  • Execute permissions on directories

Example:

sudo -u wazuh-indexer ls /mnt/wazuh-indexer

Successfully listing the directory helps confirm that the service account can access the migrated data.

Review Storage Mount Persistence

Finally, verify that the new storage mounts automatically after every reboot.

Check:

  • /etc/fstab
  • Filesystem UUID
  • Mount options
  • Boot-time dependencies

A simple reboot test in a maintenance window can confirm that:

  • The volume mounts successfully.
  • The Indexer starts automatically.
  • Existing indexes are detected.
  • Cluster health remains normal.

Completing these verification steps helps ensure that the migration remains stable long after the maintenance window has ended.


Verifying a Successful Migration

Completing the migration does not necessarily mean the job is finished.

Before returning the Wazuh environment to production, verify that the cluster is healthy, all indexes are available, shard allocation is complete, and new security events are being indexed correctly.

A thorough post-migration validation helps detect issues before they affect security monitoring.

Confirm Cluster Health

The first step is verifying the overall health of the Indexer cluster.

Run:

curl -k -u admin:password https://<INDEXER_IP>:9200/_cluster/health?pretty

Review the following fields:

  • Cluster status
  • Number of nodes
  • Active primary shards
  • Active replica shards
  • Unassigned shards
  • Pending tasks
  • Relocating shards

Ideally, the cluster should report a green status.

A yellow status may be expected temporarily while replica shards recover or in single-node deployments where replicas cannot be assigned.

A red status indicates missing primary shards and should be investigated immediately.

Verify Index Availability

Ensure that every expected index is present after the migration.

Use:

curl -k -u admin:password https://<INDEXER_IP>:9200/_cat/indices?v

Verify:

  • Expected index names
  • Index health
  • Primary shard count
  • Replica count
  • Document totals
  • Storage size

Compare the output with records collected before the migration.

If indexes are missing, investigate before resuming production indexing.

Check Shard Allocation

Confirm that every shard has been assigned successfully.

Run:

curl -k -u admin:password https://<INDEXER_IP>:9200/_cat/shards?v

Review:

  • Primary shard allocation
  • Replica allocation
  • Shard states
  • Node placement

If shards remain unassigned, review cluster allocation decisions:

curl -k -u admin:password https://<INDEXER_IP>:9200/_cluster/allocation/explain?pretty

This API helps identify why specific shards cannot be allocated.

Validate Document Counts

A successful migration should preserve all indexed documents.

Compare document counts before and after migration using:

curl -k -u admin:password https://<INDEXER_IP>:9200/_cat/count?v

Also perform spot checks against several important indexes to verify that:

  • Historical alerts exist.
  • Recent alerts are present.
  • Search queries return expected results.
  • Dashboards display historical data correctly.

Unexpected differences may indicate an incomplete copy or index recovery issue.

Confirm Dashboard Functionality

Open the Wazuh Dashboard and verify that it operates normally.

Test:

  • Dashboard login
  • Alert searches
  • Index patterns
  • Visualizations
  • Discover view
  • Saved dashboards
  • Security events
  • API communication

Also generate several test alerts to confirm that new events appear within the expected timeframe.

If the Dashboard fails to display data, review Wazuh Dashboard Not Loading? Complete Troubleshooting Guide and Troubleshooting “No Matching Indices Found” Error in Wazuh Dashboard.

Monitor Indexing Performance

Continue monitoring the Indexer for several hours after migration.

Watch for:

  • Indexing throughput
  • Search latency
  • CPU utilization
  • Heap usage
  • Disk I/O
  • Merge operations
  • Filesystem errors

Useful APIs include:

curl -k -u admin:password https://<INDEXER_IP>:9200/_nodes/stats?pretty
curl -k -u admin:password https://<INDEXER_IP>:9200/_cat/thread_pool?v

If indexing latency increases noticeably after migration, investigate storage performance before concluding the migration is complete.


Performance Optimization After Migration

Migrating to new storage often presents an opportunity to improve the overall performance of your Wazuh deployment.

After verifying that the migration succeeded, review your Indexer configuration to ensure the new hardware is being used efficiently.

Tune JVM Heap

The Java Virtual Machine (JVM) heap directly affects Indexer performance.

General recommendations include:

  • Allocate approximately 50% of available RAM.
  • Avoid exceeding roughly 32 GB of heap because larger heaps may reduce JVM efficiency.
  • Leave sufficient memory for the operating system’s filesystem cache.

Monitor:

  • Garbage collection frequency
  • Heap utilization
  • Memory pressure

For detailed guidance, see How to Tune OpenSearch Heap Size to Stop Wazuh High Memory Crashes.

Optimize Filesystem Settings

Filesystem configuration can significantly affect indexing performance.

Review:

  • Mount options
  • Filesystem type
  • Read-ahead settings
  • Scheduler configuration
  • Reserved blocks

Modern Linux filesystems such as XFS and ext4 generally provide excellent performance for Indexer workloads when configured appropriately.

Enable SSD Optimizations

If you’ve migrated to SSD or NVMe storage, verify that the operating system is optimized accordingly.

Common optimizations include:

  • Enable TRIM where appropriate.
  • Verify I/O scheduler settings.
  • Confirm storage alignment.
  • Disable unnecessary defragmentation.

These optimizations help maintain consistent performance over time.

Monitor Disk Latency

Low disk latency is one of the most important factors influencing Indexer responsiveness.

Monitor metrics such as:

  • Average read latency
  • Average write latency
  • Queue depth
  • IOPS
  • Disk utilization

Tools such as iostat, iotop, and sar can help identify storage bottlenecks.

High write latency often causes delayed indexing and slower dashboard searches.

Balance Shards Across Nodes

In clustered environments, evenly distributed shards improve both performance and resilience.

Review allocation using:

curl -k -u admin:password https://<INDEXER_IP>:9200/_cat/allocation?v

Verify:

  • Even disk utilization
  • Balanced shard counts
  • Comparable storage usage
  • No overloaded nodes

If necessary, rebalance shards according to your storage strategy.

For additional guidance, see How to Design a Wazuh OpenSearch Shard Allocation Strategy.

Adjust Replica Settings

Replica shards improve fault tolerance but also consume storage and indexing resources.

Evaluate:

  • Number of replica shards
  • Recovery speed
  • Storage capacity
  • Cluster size
  • Availability requirements

Smaller environments may require fewer replicas, while production clusters should maintain sufficient redundancy to tolerate node failures.


Common Migration Problems and Solutions

Although most storage migrations proceed without incident, configuration mistakes, permission issues, and storage inconsistencies can prevent the Indexer from operating correctly.

The following are some of the most common problems administrators encounter after migrating Wazuh Indexer data.

Wazuh Indexer Fails to Start

If the Indexer does not start after migration, review:

  • Startup logs
  • Configuration syntax
  • Data path
  • Filesystem permissions
  • Disk availability
  • Mounted volumes

Useful commands:

sudo journalctl -u wazuh-indexer
sudo systemctl status wazuh-indexer

Incorrect Data Directory

A common mistake is pointing path.data to the wrong location.

Verify:

  • The configured directory exists.
  • Migrated files are present.
  • The mount point is correct.
  • The volume is mounted.

An incorrect data path may cause the Indexer to create a new empty data directory instead of using the migrated indexes.

Permission Problems

Incorrect permissions often prevent the Indexer from reading existing shard data.

Check:

  • Directory ownership
  • Read permissions
  • Write permissions
  • Parent directory access

Correct ownership if necessary:

sudo chown -R wazuh-indexer:wazuh-indexer /var/lib/wazuh-indexer

Invalid Configuration

Configuration errors frequently occur when editing opensearch.yml.

Common examples include:

  • Invalid YAML indentation
  • Incorrect paths
  • Missing quotation marks
  • Typographical errors

Validate configuration carefully before restarting the service.

Unassigned Shards After Migration

If shards remain unassigned:

  • Verify available disk space.
  • Confirm every node has rejoined the cluster.
  • Review allocation decisions.
  • Check replica settings.

See How to Fix Wazuh All Shards Failed Error.

Cluster Allocation Issues

Allocation problems may result from:

  • Disabled allocation settings
  • Disk watermarks
  • Missing nodes
  • Allocation filters
  • Node attribute mismatches

Review cluster settings before manually reallocating shards.

Disk Watermark Limits

OpenSearch prevents shard allocation when disk usage exceeds configured watermarks.

Check:

  • Low watermark
  • High watermark
  • Flood-stage watermark

If storage utilization remains high after migration, consider increasing capacity or removing unnecessary indexes.

Node Communication Problems

Nodes that cannot communicate may fail to join the cluster.

Verify:

  • Network connectivity
  • Firewall rules
  • TLS certificates
  • Cluster discovery settings
  • Hostname resolution

Missing Indexes

If expected indexes are absent:

  • Confirm the copy completed successfully.
  • Verify snapshot restoration if used.
  • Compare document counts.
  • Review startup logs.

Incomplete File Copy

Interrupted copy operations can leave shard files incomplete.

Compare:

  • Directory sizes
  • File counts
  • Checksums

If necessary, rerun rsync, which efficiently copies only missing or changed files.

Snapshot Restoration

If migration fails or data becomes corrupted, restore the most recent verified snapshot rather than attempting to repair damaged shard files manually.

This is the recovery approach recommended by the OpenSearch project for preserving cluster consistency.

Data Path Errors

Incorrect mount points or stale symbolic links can cause the Indexer to read from the wrong location.

Confirm:

  • Mounted filesystem
  • path.data
  • /etc/fstab
  • Storage UUID

Slow Indexing Performance

If indexing slows after migration:

  • Measure disk latency.
  • Review JVM utilization.
  • Check CPU usage.
  • Monitor merge operations.
  • Verify SSD performance.

Storage Bottlenecks

Performance problems may originate from:

  • Shared SAN storage
  • Network-attached storage latency
  • Oversubscribed cloud volumes
  • Slow HDD arrays

Benchmark storage performance before blaming the Indexer.

JVM Configuration

An oversized or undersized JVM heap can reduce indexing throughput.

Review:

  • Heap allocation
  • Garbage collection
  • Memory pressure

Adjust JVM settings only after confirming storage performance is healthy.

Filesystem Tuning

Poor filesystem configuration may increase write latency.

Review:

  • Filesystem type
  • Mount options
  • I/O scheduler
  • Read-ahead values

Permission Denied Errors

If logs report “Permission denied”:

  • Verify directory ownership.
  • Check ACLs.
  • Review parent directories.
  • Confirm service account access.

Incorrect Ownership

Ownership frequently changes when copying data as the root user.

Always verify ownership before restarting the Indexer.

SELinux or AppArmor Restrictions

Mandatory access control systems can block the Indexer from accessing the migrated storage.

If enabled:

  • Verify security contexts.
  • Restore labels where necessary.
  • Review audit logs for denied operations.

Mount Option Problems

Incorrect mount options can cause:

  • Read-only filesystems
  • Performance degradation
  • Startup failures

Verify mount options after every reboot.

Cluster Health Remains Yellow or Red

Persistent health warnings usually indicate:

  • Missing replicas
  • Offline nodes
  • Allocation failures
  • Disk watermarks
  • Recovery still in progress

Review cluster health continuously until recovery completes.

Replica Allocation

If replica shards cannot be assigned:

  • Ensure additional nodes are available.
  • Verify disk capacity.
  • Confirm allocation rules.

Single-node clusters will normally remain yellow if replicas are configured.

Missing Nodes

If a migrated node does not return:

  • Review startup logs.
  • Verify networking.
  • Confirm TLS configuration.
  • Validate cluster discovery settings.

Recovery Monitoring

Monitor recovery until all shard operations have completed:

curl -k -u admin:password https://<INDEXER_IP>:9200/_cat/recovery?v

Only consider the migration complete once recovery has finished and the cluster is stable.


Best Practices for Future Storage Migrations

Storage migrations become much easier when they are planned as part of a long-term operational strategy rather than as emergency maintenance.

The following practices can help reduce downtime, simplify future migrations, and improve the overall reliability of your Wazuh deployment.

Use Snapshots Before Every Migration

Always create and verify snapshots before modifying storage.

Snapshots provide:

  • Reliable recovery
  • Faster rollback
  • Protection against corruption
  • Disaster recovery capability

Never rely solely on filesystem copies as your only backup.

Monitor Storage Growth Proactively

Track storage utilization continuously instead of waiting until disks approach capacity.

Monitor:

  • Disk usage trends
  • Daily index growth
  • Retention periods
  • Shard sizes
  • Available capacity

Early planning allows migrations to be scheduled during routine maintenance instead of emergency outages.

Implement Index State Management (ISM)

Index State Management automatically manages index lifecycles, helping prevent uncontrolled storage growth.

Typical ISM policies can:

  • Roll over indexes
  • Delete expired data
  • Move older indexes between storage tiers
  • Automate retention policies

See How to Configure Wazuh Index State Management (ISM).

Automate Health Verification

Automate routine validation using scripts or monitoring systems that check:

  • Cluster health
  • Node availability
  • Disk utilization
  • Shard allocation
  • Snapshot status
  • Index growth

Automated monitoring helps identify problems before they become critical.

Regularly Test Disaster Recovery

Backups are only valuable if they can be restored successfully.

Regularly test:

  • Snapshot restoration
  • Cluster recovery
  • Configuration restoration
  • Node replacement
  • Storage recovery

Periodic testing increases confidence that recovery procedures will work during an actual incident.

Maintain Documentation

Keep documentation current after every infrastructure change.

Document:

  • Storage layouts
  • Mount points
  • Filesystem types
  • Configuration changes
  • Cluster topology
  • Recovery procedures
  • Maintenance history

Accurate documentation reduces troubleshooting time and helps ensure consistent operational practices.

Use High-Performance Storage

The Wazuh Indexer is highly dependent on storage performance.

For production environments, prioritize:

  • Enterprise SSDs or NVMe drives
  • Reliable RAID configurations where appropriate
  • Low-latency cloud block storage
  • Redundant storage infrastructure
  • Sufficient capacity for future growth

Selecting high-performance, reliable storage not only improves search and indexing performance but also reduces the frequency of future migrations, contributing to a more stable and scalable Wazuh deployment.


Real-World Example

Scenario

A security operations team manages a three-node Wazuh Indexer cluster responsible for storing approximately 6 TB of security events.

The environment monitors thousands of endpoints across multiple business units, collecting authentication logs, endpoint telemetry, vulnerability data, and network security events.

Over time, the original SATA-based storage infrastructure begins approaching capacity limits.

The team notices increasing disk utilization, slower dashboard searches, and higher indexing latency during periods of peak log ingestion.

To improve performance and prepare for future growth, the administrators decide to migrate each Indexer node to larger NVMe-backed storage volumes using a rolling migration approach.

Before beginning the migration, the team performs several preparation steps:

  • Creates full OpenSearch snapshots.
  • Verifies that cluster health is green.
  • Records existing storage configurations.
  • Confirms sufficient capacity on the new NVMe volumes.
  • Schedules maintenance windows for each node.

The administrators then migrate the cluster one node at a time.

For each Indexer node, they:

  1. Temporarily disable shard allocation changes.
  2. Stop the Wazuh Indexer service.
  3. Detach the existing SATA storage.
  4. Attach and mount the new NVMe volume.
  5. Copy existing Indexer data using rsync.
  6. Preserve ownership and filesystem permissions.
  7. Update mount configuration.
  8. Restart the Indexer service.
  9. Verify shard recovery and cluster membership.

After each node rejoins the cluster, the team confirms:

  • Node availability.
  • Successful shard allocation.
  • Normal indexing operations.
  • No missing indexes.
  • No unexpected cluster warnings.

Once all three nodes have been migrated, the results show significant improvements:

  • Indexing throughput increases.
  • Dashboard searches return faster.
  • Disk latency decreases.
  • Shard distribution remains balanced.
  • Cluster health remains green throughout the migration.

The team continues monitoring the environment after the migration and updates storage capacity planning procedures to prevent future storage exhaustion.

This example demonstrates why storage migration should be treated as a planned infrastructure operation rather than an emergency response.

Proper preparation, backups, and validation allow organizations to upgrade storage while maintaining security visibility and data integrity.


Frequently Asked Questions (FAQ)

Question: Can I migrate Wazuh Indexer without downtime?

Yes, but it depends on your deployment architecture.

A single-node Wazuh Indexer usually requires downtime because the entire indexing service must be stopped while data is copied.

A multi-node cluster can often be migrated with minimal downtime by moving one node at a time. Remaining nodes continue handling indexing and search requests while the migrated node is offline.

Rolling migrations are generally the preferred approach for production clusters because they reduce service interruption and maintain availability.

Question: Should I use snapshots before migrating?

Yes.

Creating snapshots before migration is one of the most important safety steps. Snapshots provide a recovery point if:

  • Data becomes corrupted.
  • The migration fails.
  • Storage hardware has problems.
  • Configuration changes prevent startup.

A filesystem copy alone should not be considered a complete backup strategy because active Indexer files may not represent a consistent cluster state.

Always verify that snapshots complete successfully before starting the migration.

Question: Can I move only one node at a time?

Yes.

Migrating one node at a time is the recommended method for multi-node Wazuh Indexer clusters.

A rolling migration allows you to:

  • Maintain cluster availability.
  • Reduce risk.
  • Validate each node before continuing.
  • Quickly identify problems.

After each node migration, confirm that the cluster returns to a healthy state before moving to the next server.

Question: What is the safest way to copy index data?

The safest approach depends on your migration strategy.

For offline filesystem migrations, rsync is commonly used because it preserves:

  • File permissions.
  • Ownership.
  • Timestamps.
  • Extended attributes.
  • Hard links.

Example:

sudo rsync -aHAX --progress /var/lib/wazuh-indexer/ /mnt/new-storage/

For disaster recovery or cross-environment migrations, OpenSearch snapshots are usually the preferred method because they preserve index metadata and cluster consistency.

Question: How do I verify all data migrated successfully?

After migration, validate the environment using multiple checks.

Recommended validation steps include:

  1. Confirm cluster health:
curl -k -u admin:password https://<INDEXER_IP>:9200/_cluster/health?pretty
  1. Verify indexes exist:
curl -k -u admin:password https://<INDEXER_IP>:9200/_cat/indices?v
  1. Compare document counts:
curl -k -u admin:password https://<INDEXER_IP>:9200/_cat/count?v
  1. Check shard allocation:
curl -k -u admin:password https://<INDEXER_IP>:9200/_cat/shards?v

Also verify:

  • Wazuh Dashboard displays historical alerts.
  • New events are indexed.
  • No shard recovery errors appear.
  • Storage usage matches expectations.

Question: Can I migrate from HDD to SSD?

Yes.

Migrating from HDD storage to SSD or NVMe storage is a common Wazuh Indexer improvement strategy.

Benefits include:

  • Lower disk latency.
  • Faster indexing.
  • Faster searches.
  • Improved shard recovery.
  • Better dashboard responsiveness.

Because the Wazuh Indexer performs continuous writes and searches, storage latency has a significant impact on overall performance.

However, ensure that the new SSD storage provides enough capacity and reliability for long-term production usage.

Question: Why are shards unassigned after migration?

Unassigned shards usually indicate that the cluster cannot place one or more shards on available nodes.

Common causes include:

  • Insufficient disk space.
  • Disabled shard allocation.
  • Missing cluster nodes.
  • Incorrect node configuration.
  • Replica allocation problems.
  • Disk watermark limits.

Investigate the allocation reason using:

curl -k -u admin:password https://<INDEXER_IP>:9200/_cluster/allocation/explain?pretty

This response provides the specific reason why a shard cannot be allocated.

Question: What should I do if the Indexer won’t start after moving data?

If the Wazuh Indexer fails to start after migration, check the following areas:

  1. Review service logs
sudo journalctl -u wazuh-indexer -f
  1. Verify the data path

Confirm that path.data points to the migrated storage location.

  1. Check permissions

Ensure the Indexer service account owns the migrated files.

  1. Confirm the storage mount

Verify that the new volume is mounted correctly.

  1. Validate configuration syntax

Check for errors in:

/etc/wazuh-indexer/opensearch.yml

If recovery is unsuccessful, restore from the latest verified snapshot rather than attempting risky manual repairs.


Conclusion

Migrating Wazuh Indexer data to new storage volumes is an important maintenance task for growing security environments.

As log volume increases, organizations eventually need additional capacity, faster storage, or improved infrastructure reliability.

The migration process involves several critical stages:

  • Assessing the current storage environment.
  • Verifying cluster health.
  • Creating snapshots and backups.
  • Planning the migration approach.
  • Moving data safely.
  • Updating storage configuration.
  • Validating indexes and shard allocation.
  • Monitoring performance after completion.

For single-node deployments, migration typically requires planned downtime because the Indexer must be stopped while data is transferred.

Multi-node clusters provide more flexibility by allowing administrators to migrate nodes individually through a rolling migration strategy.

Backups and snapshots should always be created before modifying storage.

They provide a reliable recovery option if unexpected problems occur, such as corrupted files, failed hardware, or configuration issues.

After migration, verification is just as important as the data transfer itself. Administrators should confirm:

  • Cluster health is stable.
  • All indexes are available.
  • Shards are allocated correctly.
  • Dashboard functionality works.
  • New security events continue indexing normally.
  • Storage performance meets expectations.

Storage migration should not only solve immediate capacity problems but also support future growth.

By monitoring storage usage proactively, implementing Index State Management policies, selecting high-performance storage, and maintaining tested recovery procedures, organizations can keep their Wazuh deployments scalable and reliable.

A well-planned storage migration allows security teams to expand monitoring capabilities without sacrificing availability, performance, or historical security visibility.

Be First to Comment

    Leave a Reply

    Your email address will not be published. Required fields are marked *