NetBSD-Bugs archive
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]

Re: port-xen/56782: panic: /: bad dir ino null entry

Guest disk (one with problem). This is maximum disk size of 2TB that XCP-NG (Hypervisor/Domo) can create. There is still 0,5TB free space.

# dmesg | grep xbd
[     1.290057] xbd0 at xenbus0 id 51712: Xen Virtual Block Device Interface
[     1.290057] xbd0: using event channel 8
[     1.310078] xbd0: 2000 GB, 512 bytes/sect x 4194304000 sectors
[     1.320070] boot device: xbd0
[     1.320070] root on xbd0a dumps on xbd0b
[    11.490066] xbd0: autoconfiguration error: WARNING: cache flush not supported by backend


# disklabel xbd0                                                        
# /dev/rxbd0d:
type: unknown
disk: XenServer
label:
flags:
bytes/sector: 512
sectors/track: 63
tracks/cylinder: 16
sectors/cylinder: 1008
cylinders: 20805
total sectors: 4194304000
rpm: 3600
interleave: 1
trackskew: 0
cylinderskew: 0
headswitch: 0# microseconds
track-to-track seek: 0# microseconds
drivedata: 0

16 partitions:
#        size    offset     fstype [fsize bsize cpg/sgs]
 a: 4192206848         0     4.2BSD   2048 16384     0  # (Cyl.      0 - 4158935*)
 b:   2097152 4192206848       swap                     # (Cyl. 4158935*- 4161015*)
 c: 4194304000         0     unused      0     0        # (Cyl.      0 - 4161015*)
 d: 4194304000         0     unused      0     0        # (Cyl.      0 - 4161015*)

# cat /etc/fstab                                                       
/dev/xbd0a    /    ffs rw,log 1 1
/dev/xbd0b    none    swap sw,dp
kernfs        /        kernkernfs rw
procfs        /        procprocfs rw

tmpfs        /var/shm    tmpfsrw,-m1777,-sram%25

DF:

# df -h
Filesystem         Size       Used      Avail %Cap Mounted on
/dev/xbd0a         1.9T       1.3T       492G  73% /
kernfs             1.0K       1.0K         0B 100% /kern
procfs             4.0K       4.0K         0B 100% /proc
tmpfs              1.0G       4.0K       1.0G   0% /var/shm

As for Dom0. It is XCP-NG 8.2.1 running on Supermicro A1SRM-2758F

Disk of NetBSD is an image on mirrored with mdraid 10TB disks with EXT FS.

Disks:

# dmesg | grep sd[a-b]
[    3.649451] sd 0:0:0:0: [sda] 19532873728 512-byte logical blocks: (10.0 TB/9.10 TiB)
[    3.649457] sd 0:0:0:0: [sda] 4096-byte physical blocks
[    3.649490] sd 0:0:0:0: [sda] Write Protect is off
[    3.649494] sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00
[    3.649551] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[    3.650532] sd 1:0:0:0: [sdb] 19532873728 512-byte logical blocks: (10.0 TB/9.10 TiB)
[    3.650538] sd 1:0:0:0: [sdb] 4096-byte physical blocks
[    3.650824] sd 1:0:0:0: [sdb] Write Protect is off
[    3.650828] sd 1:0:0:0: [sdb] Mode Sense: 00 3a 00 00
[    3.650888] sd 1:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[    3.669140] sd 1:0:0:0: [sdb] Attached SCSI disk
[    3.669803] sd 0:0:0:0: [sda] Attached SCSI disk

lsblk:

# lsblk
NAME                                                                                                      MAJ:MIN RM  SIZE RO TYPE  MOUNTPOINT
sdd                                                                                                         8:48   0 16.4T  0 disk
├─sdd2                                                                                                      8:50   0 16.4T  0 part
└─sdd1                                                                                                      8:49   0    2G  0 part
tdg                                                                                                       254:6    0   10G  0 disk
sdb                                                                                                         8:16   0  9.1T  0 disk
└─md0                                                                                                       9:0    0  9.1T  0 raid1
  └─XSLocalEXT--f5c40583--1708--0f1d--cdfc--d3459226e19b-f5c40583--1708--0f1d--cdfc--d3459226e19b         253:1    0  9.1T  0 lvm   /run/sr-mount/f5c40583-1708-0f1d-cdfc-d3459226e19b
tde                                                                                                       254:4    0  100G  0 disk
tdc                                                                                                       254:2    0   16G  0 disk
tda                                                                                                       254:0    0 16.4T  0 disk
sde                                                                                                         8:64   0 16.4T  0 disk
├─sde2                                                                                                      8:66   0 16.4T  0 part
└─sde1                                                                                                      8:65   0    2G  0 part
tdh                                                                                                       254:7    0  100G  0 disk
sdc                                                                                                         8:32   0 59.6G  0 disk
├─sdc2                                                                                                      8:34   0   18G  0 part
├─sdc5                                                                                                      8:37   0    4G  0 part  /var/log
├─sdc3                                                                                                      8:35   0 18.1G  0 part
│ ├─VG_XenStorage--3978d961--590c--7cb4--25d5--2493cddaf526-VHD--b4c88038--ba88--460b--82e4--ac33b58197c3 253:2    0   16G  0 lvm
│ └─VG_XenStorage--3978d961--590c--7cb4--25d5--2493cddaf526-MGT                                           253:0    0    4M  0 lvm
├─sdc1                                                                                                      8:33   0   18G  0 part  /
├─sdc6                                                                                                      8:38   0    1G  0 part  [SWAP]
└─sdc4                                                                                                      8:36   0  512M  0 part
tdf                                                                                                       254:5    0    2T  0 disk
sda                                                                                                         8:0    0  9.1T  0 disk
└─md0                                                                                                       9:0    0  9.1T  0 raid1
  └─XSLocalEXT--f5c40583--1708--0f1d--cdfc--d3459226e19b-f5c40583--1708--0f1d--cdfc--d3459226e19b         253:1    0  9.1T  0 lvm   /run/sr-mount/f5c40583-1708-0f1d-cdfc-d3459226e19b
tdd                                                                                                       254:3    0   16G  0 disk
tdb                                                                                                       254:1    0 16.4T  0 disk

Content of /run/sr-mount/f5c40583-1708-0f1d-cdfc-d3459226e19b (higlighted is NetBSD disk)

# ls -la /run/sr-mount/f5c40583-1708-0f1d-cdfc-d3459226e19b
total 3413300344
drwxr-xr-x 3 root root          4096 Apr  8 02:07 .
drwxr-xr-x 3 root root            60 Apr  4 14:21 ..
-rw-r--r-- 1 root root   10758410752 Apr  7 17:03 1142a650-3500-4909-acaf-c54b6348e2da.vhd
-rw-r--r-- 1 root root    2597167104 Apr  8 09:35 1ad40581-3211-4cf4-8f69-03c69da3a43a.vhd
-rw-r--r-- 1 root root    7978459648 Mar 22 17:22 1b010f52-34cc-4ae2-952f-cf8a56d4f316.vhd
-rw-r--r-- 1 root root   23758897664 Feb  2 07:19 3af9068f-5f80-40a3-9691-3168e25b86cf.vhd
-rw-r--r-- 1 root root 1580913943040 Apr  8 09:35 3d41c559-7edb-4fe7-8f98-bd50e21d0e3a.vhd
-rw-r--r-- 1 root root    4082786816 Apr  7 17:13 47e07dd4-a737-46ec-8689-779edb6c4d62.vhd
-rw-r--r-- 1 root root   79704625152 Apr  8 09:35 4abfa868-a86e-4195-b9f5-200099233d11.vhd
-rw-r--r-- 1 root root    1197797888 Apr  7 09:15 5329c0d0-bdf4-47f7-a040-297d8f578649.vhd
-rw-r--r-- 1 root root    5957070848 Apr  8 09:27 5d354c3a-abd1-4eee-8882-e3662a4ebfd4.vhd
-rw-r--r-- 1 root root         88576 Apr  6 17:44 67d10b75-ba10-4d7e-8121-453bbda4d4e0.vhd
-rw-r--r-- 1 root root    8136053248 Nov  2 19:33 6c11c197-d381-470d-bc00-2caeaec9eb7e.vhd
-rw-r--r-- 1 root root 1081619558912 Aug 21  2021 72cbcdb1-ed9f-457c-b5d5-61a75ebbeab5.vhd
-rw-r--r-- 1 root root         25600 Apr  7 17:03 7c356778-2b7b-42ee-b717-39dfa401100d.vhd
-rw-r--r-- 1 root root    4070138368 Nov  2 19:17 7fdcc88b-0e09-48ca-a46f-b24064a67e8e.vhd
-rw-r--r-- 1 root root  106581770240 Apr  8 09:35 94b5cbd7-a02d-4220-b261-85c83010202d.vhd
-rw-r--r-- 1 root root    7480865280 Nov 21 19:16 a9ff4d69-bb71-4411-ace8-c670bfd7bf93.vhd
-rw-r--r-- 1 root root   81938334208 Apr  8 02:07 aaddab77-0878-451b-b16f-291079560549.vhd
-rw-r--r-- 1 root root   30293860352 Jan 30 00:15 ba08c5b7-83ad-4e35-b93f-abdf9fd05b16.vhd
-rw-r--r-- 1 root root     311009792 Apr  7 17:05 bb3a781b-810f-4f81-b03b-e04a55f275e4.vhd
-rw-r--r-- 1 root root  457834684928 Apr  7 17:02 cfd8f64c-e912-4e94-95f4-5daac3a82c80.vhd
-rw-r--r-- 1 root root       2166784 Apr  7 17:02 df19e83e-e130-4db5-b0b6-f94554168252.vhd
-rw-r--r-- 1 root root         25600 Mar 22 17:22 f17df388-64ac-4029-a2de-d6afa02e1b88.vhd
-rw-r--r-- 1 root root        288692 Apr  7 17:03 filelog.txt
drwx------ 2 root root         16384 Jul 17  2021 lost+found

3d41c559-7edb-4fe7-8f98-bd50e21d0e3aStorage is mirrored with mdraid:

# cat /proc/mdstat Personalities : [raid1] md0 : active raid1 sda[1] sdb[0] 9766304768 blocks super 1.2 [2/2] [UU] bitmap: 11/73 pages [44KB], 65536KB chunk 3d41c559-7edb-4fe7-8f98-bd50e21d0e3a

# cat /proc/mdstat
Personalities : [raid1]
md0 : active raid1 sda[1] sdb[0]
      9766304768 blocks super 1.2 [2/2] [UU]
      bitmap: 11/73 pages [44KB], 65536KB chunk

Hope this helps.

W dniu 07.04.2022 o 20:55, Robert Elz pisze:
The following reply was made to PR port-xen/56782; it has been noted by GNATS.

From: Robert Elz <kre%munnari.OZ.AU@localhost>
To: Bartosz Maciejewski <bartosz%maciejewski.org@localhost>
Cc: gnats-bugs%netbsd.org@localhost, port-xen-maintainer%netbsd.org@localhost,
        gnats-admin%netbsd.org@localhost, netbsd-bugs%netbsd.org@localhost
Subject: Re: port-xen/56782: panic: /: bad dir ino null entry
Date: Fri, 08 Apr 2022 01:18:04 +0700

     Date:        Thu, 7 Apr 2022 18:49:59 +0200
     From:        Bartosz Maciejewski <bartosz=40maciejewski.org>
     Message-ID:  <56786d2f-90f0-6388-21ae-3410cc735973=40maciejewski.org>=
 
 
   =7C Guest is running Nagios. Will try to rebuild, or delete and install=
  it=20
   =7C again. Maybe something went off during updates of NetBSD and Nagios=
 .
 
 Unlikely, there's nothing Nagios can do to overwrite directory blocks,
 no matter how hard it tried.
 
 But it is easy for a drive misconfiguration to cause almost
 anything.
 
 Show us (on the guest with the problems) the labels of all drives (or
 pseudo-drives) - all the xbd devices that exist on the system - if they
 are GPT partitioned, show that, otherwise the disklabel.
 
 Also show the contents of /etc/fstab
 
 If you have access to the Dom0 also show the configuration of the guest,
 and any other guests that are running (the drive configuration section is=
 
 what matters) and the labels from any (real) drives connected to that
 system.
 
 Somewhere in all of that is likely to be the cause of the problem.
 Until it is fixed, you want to be very careful using that guest
 system - almost anything might be corrupted next.
 
 kre
Home | Main Index | Thread Index | Old Index