zpool: Add zpool status -vv error ranges #17864

tonyhutter · 2025-10-22T18:34:15Z

Motivation and Context

Print error byte ranges with zpool status -vv

Description

Print the byte error ranges with 'zpool status -vv'. This works with all the normal zpool status formatting flags: -p, -j, --json-int

In addition:

Move range_tree/btree to common userspace/kernel code.
Modify ZFS_IOC_OBJ_TO_STATS ioctl to optionally return "extended" object stats.
Let zinject corrupt zvol data.
Add test case.

This commit takes code from these PRs: #17502 #9781 #8902

How Has This Been Tested?

Test case added

Types of changes

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Performance enhancement (non-breaking change which improves efficiency)
Code cleanup (non-breaking change which makes code smaller or more readable)
Quality assurance (non-breaking change which makes the code more robust against bugs)
Breaking change (fix or feature that would cause existing functionality to change)
Library ABI change (libzfs, libzfs_core, libnvpair, libuutil and libzfsbootenv)
Documentation (a change to man pages or other documentation)

Checklist:

My code follows the OpenZFS code style requirements.
I have updated the documentation accordingly.
I have read the contributing document.
I have added tests to cover my changes.
I have run the ZFS Test Suite with this change applied.
All commit messages are properly formatted and contain Signed-off-by.

tonyhutter · 2025-10-22T18:35:33Z

Sample output:

$ zpool status -vv

  pool: testpool                                                     
 state: ONLINE                                                       
status: One or more devices has experienced an error resulting in data
    corruption.  Applications may be affected.                       
action: Restore the file in question if possible.  Otherwise restore the
    entire pool from backup.                                         
   see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-8A       
  scan: scrub repaired 0B in 00:00:00 with 3 errors on Tue Oct 21 17:22:20 2025
config:                                                              
                                                                     
    NAME        STATE     READ WRITE CKSUM                           
    testpool    ONLINE       0     0     0                           
      loop0     ONLINE       3     0    18                           
                                                                     
errors: Permanent errors have been detected in the following files:  
                                                                     
        <metadata>:<0x1> (no ranges)                                 
        /testpool/4k/4k_file1 0-4.00K                                
        /testpool/4k/4k_file2 0-4.00K,16K-20.0K,24K-28.0K            
        /testpool/1m/1m_file 1M-2.00M                                
        testpool/testvol:<0x1> 3.91M-3.91M,4.30M-4.30M

$ zpool status -vvjp --json-int | jq
...
      "errors": {
        "<metadata>:<0x1>": {
          "name": "<metadata>:<0x1>",
          "object": 1,
          "dataset": 0
        },
        "/testpool/4k/4k_file1": {
          "object_type": "ZFS plain file",
          "ranges": [
            {
              "start_byte": 0,
              "end_byte": 4095
            }
          ],
          "name": "/testpool/4k/4k_file1",
          "object": 2,
          "dataset": 262,
          "block_size": 4096
        },
        "/testpool/4k/4k_file2": {
          "object_type": "ZFS plain file",
          "ranges": [
            {
              "start_byte": 0,
              "end_byte": 4095
            },
            {
              "start_byte": 16384,
              "end_byte": 20479
            },
            {
              "start_byte": 24576,
              "end_byte": 28671
            }
          ],
          "name": "/testpool/4k/4k_file2",
          "object": 128,
          "dataset": 262,
          "block_size": 4096
        },
        "/testpool/1m/1m_file": {
          "object_type": "ZFS plain file",
          "ranges": [
            {
              "start_byte": 1048576,
              "end_byte": 2097151
            }
          ],
          "name": "/testpool/1m/1m_file",
          "object": 2,
          "dataset": 270,
          "block_size": 1048576
        },
        "testpool/testvol:<0x1>": {
          "object_type": "zvol",
          "ranges": [
            {
              "start_byte": 4096000,
              "end_byte": 4100095
            },
            {
              "start_byte": 4505600,
              "end_byte": 4509695
            }
          ],
          "name": "testpool/testvol:<0x1>",
          "object": 1,
          "dataset": 278,
          "block_size": 4096
        }
      }
    }
  }
}

tonyhutter · 2025-10-22T18:42:56Z

Just the filenames (JSON):

$ zpool status -vj
...
      "errors": {
        "<metadata>:<0x1>": {
          "name": "<metadata>:<0x1>"
        },
        "/testpool/4k/4k_file1": {
          "name": "/testpool/4k/4k_file1"
        },
        "/testpool/4k/4k_file2": {
          "name": "/testpool/4k/4k_file2"
        },
        "/testpool/1m/1m_file": {
          "name": "/testpool/1m/1m_file"
        },
        "testpool/testvol:<0x1>": {
          "name": "testpool/testvol:<0x1>"
        }
      }
    }
  }

cmd/zinject/translate.c

lib/libzfs/libzfs_pool.c

lib/libzfs/Makefile.am

module/zcommon/zfs_comutil.c

module/zfs/zfs_znode.c

module/zcommon/range_tree.c

Break out the range_tree, btree, and highbit64 code from kernel space into shared kernel and userspace code. This is needed for the updated `zpool status -vv` error byte range reporting in the next commit. That commit needs the range_tree code in kernel and userspace. Signed-off-by: Tony Hutter <[email protected]>

Print the byte error ranges with 'zpool status -vv'. This works with the normal zpool status formatting flags (-p, -j, --json-int). In addition: - Move range_tree/btree to common userspace/kernel code. - Modify ZFS_IOC_OBJ_TO_STATS ioctl to optionally return "extended" object stats. - Let zinject corrupt zvol data. - Add test case. This commit takes code from these PRs: openzfs#17502 openzfs#9781 openzfs#8902 Signed-off-by: Tony Hutter <[email protected]> Co-authored-by:: Alan Somers <[email protected]> Co-authored-by: TulsiJain <[email protected]>

tonyhutter · 2025-12-29T17:29:25Z

This is now passing all CI test. Reviewers - would you mind taking another look?

behlendorf

I only managed to make it through the first patch but I wanted to post a first round of feedback. I like the direction this is going!

behlendorf · 2026-01-08T23:47:55Z

lib/libspl/include/sys/sysmacros.h

+		return (0);
+
+	return (8 * sizeof (uint64_t) - __builtin_clzll(i));
+}


We don't want to split up highbit64 and lowbit64 so let's move them both here.
While you're at it we can also define them a little more concisely. Unfortunately, gcc documentation for __builtin_clzll states the result is undefined when the value is zero so we need to keep that check.

#define highbit64(x) (((x) == 0) ? 0 : CHAR_BIT * sizeof(x) - __builtin_clzll(x)) #define lowbit64(x) (__builtin_ffsll(x))

This is a purely mechanical bit of cleanup so if you can move the highbit64/lowbit64 change in to its own PR we could get that chunk merged more quickly.

Let's also pull btree and range_tree refactoring in to its own PR. That way we can test it in isolation.

behlendorf · 2026-01-08T23:50:54Z

lib/librange_tree/Makefile.am

@@ -0,0 +1,7 @@
+librange_tree_la_CFLAGS  = $(AM_CFLAGS) $(KERNEL_CFLAGS) $(LIBRARY_CFLAGS)
+# librange_tree_la_CFLAGS += -fvisibility=hidden


Shouldn't be commented out.

behlendorf · 2026-01-08T23:51:53Z

lib/librange_tree/Makefile.am

+noinst_LTLIBRARIES += librange_tree.la
+
+nodist_librange_tree_la_SOURCES = module/zfs/range_tree.c
+nodist_EXTRA_librange_tree_la_SOURCES = module/zfs/btree.c


Why is this needed?

behlendorf · 2026-01-08T23:55:04Z

lib/libspl/include/sys/sysmacros.h


 #include <stdint.h>
-
+#include <sys/param.h>


I believe you'll want <limits.h> for CHAR_BIT and the highbit64/lowbit64 changes.

behlendorf · 2026-01-09T00:08:58Z

lib/libzfs/Makefile.am

-	module/zcommon/zprop_common.c
+	module/zcommon/zprop_common.c \
+	module/zfs/btree.c \
+	module/zfs/range_tree.c


We don't want to include these as sources. They should be added to the _LIBADD section as you did for libzpool.

behlendorf · 2026-01-09T00:11:22Z

lib/libzfs/Makefile.am


 libzfs_la_LIBADD += -lrt -lm $(LIBCRYPTO_LIBS) $(ZLIB_LIBS) $(LIBFETCH_LIBS) $(LTLIBINTL)

+# libzfs_la_LDFLAGS = -pthread -lrange_tree -lbtree


Can be dropped.

behlendorf · 2026-01-09T00:13:03Z

lib/Makefile.am

+include $(srcdir)/%D%/libbtree/Makefile.am
 include $(srcdir)/%D%/libicp/Makefile.am
 include $(srcdir)/%D%/libnvpair/Makefile.am
+include $(srcdir)/%D%/librange_tree/Makefile.am


Can you also update the diagram above.

behlendorf · 2026-01-09T00:13:26Z

module/zcommon/zfs_comutil.c

 #include <sys/nvpair.h>
 #include "zfs_comutil.h"
 #include <sys/zfs_ratelimit.h>
+#include <sys/dmu.h>


It looks like this can be dropped.

behlendorf · 2026-01-09T00:22:56Z

tests/zfs-tests/cmd/Makefile.am

-	libzfs_core.la
-
+	libzfs_core.la \
+	libbtree.la


You should be able to drop libzpool.la and libzfs_core.la here. I'm pretty sure they were only being added to pull in the btree implementation. Plus dropping them will let us verify the new library can be used stand alone in user space.

behlendorf · 2026-01-09T00:23:38Z

module/zfs/btree.c

-kmem_cache_t *zfs_btree_leaf_cache;
+#ifndef _KERNEL
+#include <stdio.h>
+#include <stdlib.h>


Do you recall what you need to pull these two headers in for?

tonyhutter · 2026-01-10T01:00:28Z

@behlendorf let me move over the btree/range_tee commit to another PR as you suggested, and I'll address your comments in there.

behlendorf added the Status: Code Review Needed Ready for review and testing label Oct 22, 2025

tonyhutter force-pushed the zpool_err_range branch from 45b19a7 to eb6a882 Compare October 22, 2025 22:46

behlendorf reviewed Oct 22, 2025

View reviewed changes

tonyhutter force-pushed the zpool_err_range branch from eb6a882 to 6c63c6a Compare November 6, 2025 00:34

tonyhutter force-pushed the zpool_err_range branch 3 times, most recently from 1c4134d to b939af9 Compare November 14, 2025 01:43

tonyhutter force-pushed the zpool_err_range branch 8 times, most recently from 11fbd66 to 6a2cf00 Compare December 12, 2025 22:24

tonyhutter force-pushed the zpool_err_range branch 3 times, most recently from f33ab4e to 49eb048 Compare December 19, 2025 17:52

tonyhutter and others added 2 commits December 23, 2025 09:19

tonyhutter force-pushed the zpool_err_range branch from 49eb048 to 0e11fb1 Compare December 23, 2025 17:19

behlendorf reviewed Jan 9, 2026

View reviewed changes

		@@ -0,0 +1,7 @@
		librange_tree_la_CFLAGS = $(AM_CFLAGS) $(KERNEL_CFLAGS) $(LIBRARY_CFLAGS)
		# librange_tree_la_CFLAGS += -fvisibility=hidden


		libzfs_la_LIBADD += -lrt -lm $(LIBCRYPTO_LIBS) $(ZLIB_LIBS) $(LIBFETCH_LIBS) $(LTLIBINTL)

		# libzfs_la_LDFLAGS = -pthread -lrange_tree -lbtree

zpool: Add zpool status -vv error ranges #17864

Are you sure you want to change the base?

zpool: Add zpool status -vv error ranges #17864

Conversation

tonyhutter commented Oct 22, 2025

Motivation and Context

Description

How Has This Been Tested?

Types of changes

Checklist:

Uh oh!

tonyhutter commented Oct 22, 2025

Uh oh!

tonyhutter commented Oct 22, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

tonyhutter commented Dec 29, 2025

Uh oh!

behlendorf left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tonyhutter commented Jan 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants