Subject: Re: bin/28492 (more(1) doesn't handle NUL data blocks well)
To: None <tron@NetBSD.org, gnats-admin@netbsd.org, netbsd-bugs@netbsd.org,>
From: Bob Kemp <bsd@allegory.demon.co.uk>
List: netbsd-bugs
Date: 09/25/2006 22:20:02
The following reply was made to PR bin/28492; it has been noted by GNATS.
From: Bob Kemp <bsd@allegory.demon.co.uk>
To: gnats-bugs@NetBSD.org
Cc: tron@NetBSD.org, gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Subject: Re: bin/28492 (more(1) doesn't handle NUL data blocks well)
Date: Mon, 25 Sep 2006 23:16:54 +0100
--BOKacYhQ+x31HxR3
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
On Mon, Sep 25, 2006 at 09:20:03PM +0000, Matthias Scheler wrote:
> > Since I use Java, the memory limits are set higher and the box only has 512M.
> > If you have more memory, it might not feel so bad to you.
> [...]
> > $ ulimit -a
> > time(cpu-seconds) unlimited
> > file(blocks) unlimited
> > coredump(blocks) unlimited
> > data(kbytes) 3145728 <---
> > stack(kbytes) 2048
> > lockedmem(kbytes) 169933
> > memory(kbytes) 509800
> > nofiles(descriptors) 500
> > processes 160
> > sbsize(bytes) unlimited
>
> [...]
>
> Your datasize exceeds the hard limit of my box:
>
> $ ulimit -a
> time(seconds) unlimited
> file(blocks) unlimited
> data(kbytes) 131072 <---
> stack(kbytes) 2048
> coredump(blocks) 0
> memory(kbytes) 2045980
> locked memory(kbytes) 681994
> process(processes) 160
> nofiles(descriptors) 64
> sbsize(bytes) unlimited
In my .tcshrc, I have
limit datasize unlimited
but beyond that I don't think there's anything special about my setup.
> > If you want more info, or for me to try something, just let me know.
>
> Can you please provide your kernel configuration and more importantly
> try with a standard kernel configuration? It looks to me like you
> extended certain limits above a save boundary.
The config file is attached.
> And could please also try to run "more" on a very large (e.g. 4GB) file?
> I bet your problem has nothing to do with a file full of zeros but
> with the VM limits and the large amount of input data.
Interesting idea but...
$ ls -ls massive*
1485872 -rw-r--r-- 1 rob wheel 1520772480 Sep 25 22:37 massiveFile
922064 -rw-r--r-- 1 rob wheel 943718400 Sep 25 22:39 massiveZeroFile
where they were created by (in effect)
dd if=/dev/zero bs=100m of=massiveZeroFile
and
cat /etc/hosts massiveFile >> massiveFile
As you can see the file of zeroes is smaller than the text file but
the text file works just fine and the file of zeroes behaves as reported.
I don't have enough free filespace to create 4Gb files.
GENERIC_LAPTOP doesn't run on my laptop, hangs at RaidFrame init,
so I can't try that. I'll try supping and updating my system.
Thanks for your help!
Bob
PS: I've just tried 'less' from pkgsrc of a few days ago.
It behaves differently on the zeroes file.
It burns 100% of the CPU but uses relatively little memory.
It is however unresponsive, eg I cannot ^C it or suspend it.
On the large text file all works fine, just like the NetBSD version.
--BOKacYhQ+x31HxR3
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment; filename=TENEBRES
include "arch/i386/conf/std.i386"
options INCLUDE_CONFIG_FILE # embed config file in kernel binary
ident "TENEBRES"
maxusers 32 # estimated number of users
# CPU support. At least one is REQUIRED.
options I686_CPU
# CPU-related options.
options MATH_EMULATE # floating point emulation
options VM86 # virtual 8086 emulation
options USER_LDT # user-settable LDT; used by WINE
# Enhanced SpeedStep Technology in the Pentium M
options ENHANCED_SPEEDSTEP
options EST_FREQ_USERWRITE # any user can set frequency
options MTRR # memory-type range register syscall support
# delay between "rebooting ..." message and hardware reset, in milliseconds
#options CPURESET_DELAY=2000
# This option allows you to force a serial console at the specified
# I/O address. see console(4) for details.
#options CONSDEVNAME="\"com\"",CONADDR=0x2f8,CONSPEED=57600
# you don't want the option below ON iff you are using the
# serial console option of the new boot strap code.
#options CONS_OVERRIDE # Always use above! independent of boot info
# Standard system options
options INSECURE # disable kernel security levels - X needs this
## options RTC_OFFSET=0 # hardware clock is this many mins. west of GMT
options NTP # NTP phase/frequency locked loop
options KTRACE # system call tracing via ktrace(1)
options SYSTRACE # system call vetting via systrace(1)
options SYSVMSG # System V-like message queues
options SYSVSEM # System V-like semaphores
#options SEMMNI=10 # number of semaphore identifiers
#options SEMMNS=60 # number of semaphores in system
#options SEMUME=10 # max number of undo entries per process
#options SEMMNU=30 # number of undo structures in system
options SYSVSHM # System V-like memory sharing
options SHMMAXPGS=32768
options P1003_1B_SEMAPHORE # p1003.1b semaphore support
options LKM # loadable kernel modules
options KLOADER
options KLOADER_KERNEL_PATH="\"/netbsd\""
options USERCONF # userconf(4) support
options SYSCTL_INCLUDE_DESCR # Include sysctl descriptions in kernel
# Diagnostic/debugging support options
#options DIAGNOSTIC # expensive kernel consistency checks
#options DEBUG # expensive debugging checks/support
#options KMEMSTATS # kernel memory statistics (vmstat -m)
options DDB # in-kernel debugger
options DDB_ONPANIC=1 # see also sysctl(8): `ddb.onpanic'
options DDB_HISTORY_SIZE=512 # enable history editing in DDB
#options KGDB # remote debugger
#options KGDB_DEVNAME="\"com\"",KGDB_DEVADDR=0x3f8,KGDB_DEVRATE=9600
makeoptions DEBUG="-g" # compile full symbol table
# Compatibility options
options COMPAT_20 # NetBSD 2.0
options COMPAT_21 # NetBSD 2.1
options COMPAT_30 # NetBSD 3.0
options COMPAT_40 # NetBSD 4.0
options COMPAT_50 # NetBSD 4.0
options COMPAT_OSSAUDIO # OSS (Voxware) audio driver compatibility
#options COMPAT_SVR4 # binary compatibility with SVR4
#options COMPAT_IBCS2 # binary compatibility with SCO and ISC
options COMPAT_LINUX # binary compatibility with Linux
#options COMPAT_FREEBSD # binary compatibility with FreeBSD
#options COMPAT_PECOFF # kernel support to run Win32 apps
options COMPAT_BSDPTY # /dev/[pt]ty?? ptys.
# File systems
file-system FFS # UFS
#file-system EXT2FS # second extended file system (linux)
#file-system LFS # log-structured file system
file-system MFS # memory file system
file-system NFS # Network File System client
#file-system NTFS # Windows/NT file system (experimental)
file-system CD9660 # ISO 9660 + Rock Ridge file system
file-system MSDOSFS # MS-DOS file system
#file-system FDESC # /dev/fd
file-system KERNFS # /kern
file-system NULLFS # loopback file system
#file-system OVERLAY # overlay file system
#file-system PORTAL # portal filesystem (still experimental)
file-system PROCFS # /proc
#file-system UMAPFS # NULLFS + uid and gid remapping
file-system UNION # union file system
#file-system CODA # Coda File System; also needs vcoda (below)
file-system SMBFS # experimental - CIFS; also needs nsmb (below)
#file-system PTYFS # New pty filesystem -- UNOFFICIAL
file-system TMPFS # experimental - Efficient memory file-system
# File system options
#options FFS_EI # FFS Endian Independent support
options SOFTDEP # FFS soft updates support.
options NFSSERVER # Network File System server
# Networking options
#options GATEWAY # packet forwarding
options INET # IP + ICMP + TCP + UDP
options INET6 # IPV6
options IPSEC # IP security
options IPSEC_ESP # IP security (encryption part; define w/IPSEC)
options IPSEC_DEBUG # debug for IP security
options PPP_BSDCOMP # BSD-Compress compression support for PPP
options PPP_DEFLATE # Deflate compression support for PPP
options PPP_FILTER # Active filter support for PPP (requires bpf)
options PFIL_HOOKS # pfil(9) packet filter hooks
options IPFILTER_LOG # ipmon(8) log support
options IPFILTER_DEFAULT_BLOCK # block all packets by default
#
# wscons options
#
# builtin terminal emulations
#options WSEMUL_SUN # sun terminal emulation
options WSEMUL_VT100 # VT100 / VT220 emulation
# different kernel output - see dev/wscons/wsdisplayvar.h
options WS_KERNEL_FG=WSCOL_GREEN
#options WS_KERNEL_BG=WSCOL_BLACK
# compatibility to other console drivers
options WSDISPLAY_COMPAT_PCVT # emulate some ioctls
options WSDISPLAY_COMPAT_SYSCONS # emulate some ioctls
options WSDISPLAY_COMPAT_USL # VT handling
options WSDISPLAY_COMPAT_RAWKBD # can get raw scancodes
# allocate a number of virtual screens at autoconfiguration time
options WSDISPLAY_DEFAULTSCREENS=8
# use a large software cursor that doesn't blink
options PCDISPLAY_SOFTCURSOR
# modify the screen type of the console; defaults to "80x25"
#options VGA_CONSOLE_SCREENTYPE="\"80x24\""
# Kernel root file system and dump configuration.
config netbsd root on ? type ?
#config netbsd root on sd0a type ffs
#config netbsd root on ? type nfs
#
# Device configuration
#
mainbus0 at root
cpu0 at mainbus0
apm0 at mainbus0 # Advanced power management
# Tuning for power management, see apm(4) for more details.
#options APM_NO_IDLE # Don't call BIOS CPU idle function
#options APM_V10_ONLY # Use only the APM 1.0 calls
#options APM_NO_POWEROFF # Don't power off on halt(8)
#options APM_POWER_PRINT # Print stats on the console
options APM_DISABLE_INTERRUPTS=1 # Disable interrupts
options APMDEBUG
options APMCALLDEBUG
# Basic Bus Support
# PCI bus support
pci0 at mainbus? bus ?
pci* at mainbus? bus ?
pci* at pchb? bus ?
pci* at ppb? bus ?
# Configure PCI using BIOS information
options PCIBIOS # PCI BIOS support
options PCIBIOSVERBOSE # PCI BIOS verbose info
#options PCIBIOS_IRQS_HINT=0x0a00 # PCI interrupts hint. IRQ 9 or 11
options PCIBIOS_INTR_GUESS # see pcibios(4)
#options PCIINTR_DEBUG # super-verbose PCI interrupt fixup
options PCI_ADDR_FIXUP # fixup PCI I/O addresses
options PCI_BUS_FIXUP # fixup PCI bus numbering
options PCI_INTR_FIXUP # fixup PCI interrupt routing
# PCI bridges
pchb* at pci? dev ? function ? # PCI-Host bridges
pcib* at pci? dev ? function ? # PCI-ISA bridges
ppb* at pci? dev ? function ? # PCI-PCI bridges
# XXX 'puc's aren't really bridges, but there's no better place for them here
puc* at pci? dev ? function ? # PCI "universal" comm. cards
agp* at pchb?
# ISA bus support
isa0 at mainbus?
isa0 at pcib?
# PCMCIA bus support
pcmcia* at pcic? controller ? socket ?
pcmcia* at tcic? controller ? socket ?
# ISA PCMCIA controllers
pcic0 at isa? port 0x3e0 iomem 0xd0000 iosiz 0x10000
pcic1 at isa? port 0x3e2 iomem 0xe0000 iosiz 0x4000
pcic2 at isa? port 0x3e4 iomem 0xe0000 iosiz 0x4000
tcic0 at isa? port 0x240 iomem 0xd0000 iosiz 0x10000
# PCI PCMCIA controllers
pcic0 at pci? dev? function ?
# CardBus bridge support
cbb* at pci? dev ? function ?
cardslot* at cbb?
# CardBus bus support
cardbus* at cardslot?
pcmcia* at cardslot?
# Coprocessor Support
# Math Coprocessor support
npx0 at isa? port 0xf0 irq 13 # x86 math coprocessor
# Console Devices
# ISA console
#pc0 at isa? port 0x60 irq 1 # pccons generic PC console driver
# Keyboard layout configuration for pccons
#options FRENCH_KBD
#options FINNISH_KBD
#options GERMAN_KBD
#options NORWEGIAN_KBD
# pccons-specific options:
#options XSERVER_DDB # PF12 gets you into DDB when X is running
#options XSERVER # X server support
# wscons
pckbc0 at isa? # pc keyboard controller
pckbd* at pckbc? # PC keyboard
pms* at pckbc? # PS/2 mouse for wsmouse
vga0 at isa?
vga* at pci? dev ? function ?
pcdisplay0 at isa? # CGA, MDA, EGA, HGA
wsdisplay* at vga? console ?
wsdisplay* at pcdisplay? console ?
wskbd* at pckbd? console ?
wsmouse* at pms? mux 0
pcppi0 at isa?
sysbeep0 at pcppi?
# Serial Devices
# PCI serial interfaces
com* at puc? port ? # 16x50s on "universal" comm boards
# PCMCIA serial interfaces
com* at pcmcia? function ? # Modems and serial cards
pcmcom* at pcmcia? function ? # PCMCIA multi-port serial cards
com* at pcmcom? slave ? # ...and the slave devices
# CardBus serial interfaces
com* at cardbus? function ? # Modems and serial cards
# ISA serial interfaces
com0 at isa? port 0x3f8 irq 4 # Standard PC serial ports
com1 at isa? port 0x2f8 irq 3
com2 at isa? port 0x3e8 irq 5
# Parallel Printer Interfaces
# ISA parallel printer interfaces
lpt0 at isa? port 0x378 irq 7 # standard PC parallel ports
lpt1 at isa? port 0x278
lpt2 at isa? port 0x3bc
# SCSI Controllers and Devices
# SCSI bus support
scsibus* at scsi?
scsibus0 at scsi?
# SCSI devices
sd0 at scsibus0 target 0 lun ? # SCSI disk drives
sd1 at scsibus0 target 1 lun ? # SCSI disk drives
sd2 at scsibus0 target 2 lun ? # SCSI disk drives
sd3 at scsibus0 target 3 lun ? # SCSI disk drives
sd4 at scsibus0 target 4 lun ? # SCSI disk drives
sd* at scsibus? target ? lun ? # SCSI disk drives
cd* at scsibus? target ? lun ? # SCSI CD-ROM drives
# IDE drives
# Flags are used only with controllers that support DMA operations
# and mode settings (e.g. some pciide controllers)
# The lowest order four bits (rightmost digit) of the flags define the PIO
# mode to use, the next set of four bits the DMA mode and the third set the
# UltraDMA mode. For each set of four bits, the 3 lower bits define the mode
# to use, and the last bit must be 1 for this setting to be used.
# For DMA and UDMA, 0xf (1111) means 'disable'.
# 0x0fac means 'use PIO mode 4, DMA mode 2, disable UltraDMA'.
# (0xc=1100, 0xa=1010, 0xf=1111)
# 0x0000 means "use whatever the drive claims to support".
# ATA (IDE) bus support
atabus* at ata?
wd* at atabus? drive ? flags 0x0000
# ATAPI bus support
atapibus* at atapi?
# ATAPI devices
# flags have the same meaning as for IDE drives.
cd* at atapibus? drive ? flags 0x0000 # ATAPI CD-ROM drives
sd* at atapibus? drive ? flags 0x0000 # ATAPI disk drives
piixide* at pci? dev ? function ? # Intel IDE controllers
# Miscellaneous mass storage devices
# ISA floppy
fdc0 at isa? port 0x3f0 irq 6 drq 2 # standard PC floppy controllers
fd* at fdc? drive ? # the drives themselves
# Network Interfaces
# PCI network interfaces
fxp* at pci? dev ? function ? # Intel EtherExpress PRO 10+/100B
ne* at pci? dev ? function ? # NE2000-compatible Ethernet
rtk* at cardbus? function ? # Realtek 8129/8139
ral* at cardbus? function ? # Ralink Technology, Inc., RT2500, 802.11
an* at pcmcia? function ? # Aironet PC4500/PC4800 (802.11)
# PCMCIA network interfaces
ne* at pcmcia? function ? # NE2000-compatible Ethernet
# CardBus network cards
fxp* at cardbus? function ? # Intel i8255x
# MII/PHY support
inphy* at mii? phy ? # Intel 82555 PHYs
# USB Controller and Devices
# PCI USB controllers
ohci* at pci? dev ? function ? # Open Host Controller
uhci* at pci? dev ? function ? # Universal Host Controller (Intel)
# USB bus support
usb* at ohci?
usb* at uhci?
# USB Hubs
uhub* at usb?
uhub* at uhub? port ? configuration ? interface ?
# USB HID device
uhidev* at uhub? port ? configuration ? interface ?
# USB Generic HID devices
uhid* at uhidev? reportid ?
# USB Printer
ulpt* at uhub? port ? configuration ? interface ?
# USB Modem
umodem* at uhub? port ? configuration ?
ucom* at umodem?
# USB Mass Storage
umass* at uhub? port ? configuration ? interface ?
# USB audio
uaudio* at uhub? port ? configuration ?
# USB MIDI
umidi* at uhub? port ? configuration ?
# USB IrDA
# USB-IrDA bridge spec
uirda* at uhub? port ? configuration ? interface ?
irframe* at uirda?
# SigmaTel STIr4200 USB/IrDA Bridge
ustir* at uhub? port ?
irframe* at ustir?
# USB Generic driver
ugen* at uhub? port ?
# IrDA and Consumer Ir devices
# Toshiba Oboe
## oboe* at pci0 dev 31 function 0
## irframe* at oboe?
# Audio Devices
# PCI audio devices
auich* at pci? dev ? function ? # Intel ICH integrated AC'97 Audio
# Audio support
audio* at audiobus?
# MIDI support
midi* at midibus?
midi* at pcppi? # MIDI interface to the PC speaker
# Pseudo-Devices
# disk/mass storage pseudo-devices
pseudo-device cgd 8 # cryptographic disk devices
## pseudo-device ptm
pseudo-device md 1 # memory disk device (ramdisk)
pseudo-device vnd 8 # disk-like interface to files
# network pseudo-devices
pseudo-device bpfilter 8 # Berkeley packet filter
pseudo-device ipfilter # IP filter (firewall) and NAT
pseudo-device loop 2 # network loopback
pseudo-device ppp 2 # Point-to-Point Protocol
pseudo-device pppoe # PPP over Ethernet (RFC 2516)
pseudo-device irframetty # IrDA frame line discipline
pseudo-device tun 5 # network tunneling over tty
pseudo-device tap 5 # virtual Ethernet
# miscellaneous pseudo-devices
pseudo-device pty # pseudo-terminals
pseudo-device rnd # /dev/random and in-kernel generator
pseudo-device clockctl # user control of clock subsystem
# a pseudo device needed for SMBFS
pseudo-device nsmb # experimental - SMB requester
# wscons pseudo-devices
pseudo-device wsmux # mouse & keyboard multiplexor
pseudo-device wsfont
--BOKacYhQ+x31HxR3--