Re: GPT questions - gpt reliability, wedge naming, and filesystem scaling.

[Robert Elz <kre%munnari.OZ.AU@localhost>]

    Date:        Fri, 20 Jun 2014 01:20:03 +0100
    From:        Gerard Lally <lists+netbsd.users%netmail.ie@localhost>
    Message-ID:  <20140620012000.2C49.280FC639%netmail.ie@localhost>
 | 
  | 1) Is it safe to use GPT on NetBSD?

Yes, though it is a little tricky to get boot from gpt to work properly
(you might have fluked onto the technique for that, unless it has been
recently fixed, you need to first make a bootable MBR, then convert to
to GPT, and not try to simply do the GPT boot process on a virgin disk).
(No need for the MBR partitions and GPT ones to be related at all, it is
the MBR init, which becomes PMBR init, that is important here).

  | 3) Using "NAME=dk0" in /etc/fstab didn't work for me; I had to specify
  | /dev/dk0, /dev/dk1, etc.
  | dk names also do not persist across reboots. For example, if I create a
  | wedge as follows the dk_swap name reverts to dk1 after rebooting.

That stuff really doesn't work in NetBSD 6, you need a -current
kernel (or something from the past 6-9 months on the current stream)
to get this functioning the way it should.  It would be nice for all
the wedge labeling, and auto-discovery, to get pulled up...

  | dkctl wd0 addwedge dk_swap 64 2097152 swap
  | 
  | This is not a big deal but it leaves me wondering how NAME=xxx in fstab
  | is supposed to work. Does it work with GPT labels instead?

Yes, the GPT label is the name value.  But the label in addwedge is the
same thing, but I think only applied n ram, not written back to the filesys
(not sure about that, but it certainly gave me some weirdness when I
started).

  | 4) To get the sector offsets and sizes right I first created a
  | traditional MBR + disklabel setup, sizing partitions in MB and taking
  | note of the sector offsets and sector sizes this produced. I started at
  | 2048.

Sounds OK.

  | After destroying the MBR + disklabel setup I then used this
  | information to create GPT partitions. I assume this is a safe way to do
  | it?

Safe, if a little over cautious.

  | I am not really familiar with partition alignment, and even less so
  | since the new disks came out.

As Greg said, just avoid splitting things so that one write requires
a read/modify/write on the drive (so your writes should be whole drive
sectors).  For some big drives that's 2K or 4K, so everything (even
perhaps filesys fragment size) should be multiples of that.

Most important is to forget that the magic number "63" (or anything like
it) ever existed...

I'll append a script I use to make GPT partitions and do most of the
rest of the work (it uses NAME=... entries in data it adds to fstab if
fstab already contains any of those, otherwise not - so if they're to
work, you need to add one NAME= entry manually.)

kre


#! /bin/sh

case "$1" in
-[h?])  echo "Usage: $0 base-drive options ..."
        echo ""
        echo "  Options:"
        echo "     -s size   (mandatory option)"
        echo "     -b base   (starting blockno)"
        echo "     -l label  (GPT part label / wedge name)"
        echo "     -t filesys-type  (ffs, msdos, ...)  (default ffs)"
        echo ""
        echo "  for -t ffs (which is the default):"
        echo "     -m mount-point   (created if required)"
        echo "     -f fstab-options   (ro,noauto,...) (requires -m)"
        echo "           (-f can be repeated, or value can contain commas)"
        echo "     -B blocksize   (-b opt to newfs)"
        echo "     -F fragsize    (-f opt to newfs)"
        echo "     -N num-inodes  (-n opt to newfs)"
        echo "     -O ffs_version (-O opt to newfs) (default 2)"
        echo ""
        echo "  filesystem will be created (newfs) for -t ffs"
        echo "  if either (or both) -m or -N is given, not otherwise"
        echo "  filesystem must be created to be mounted (if -m && ! -f noauto)"
        echo "  filesystem must be mounted (or -f noauto) to be added to fstab"
        echo "  fstab options (-f) must be given to add entry to fstab"
        exit 0
        ;;

-*)     echo >&2 "Usage: $0 base-drive options ..."; exit 1;;
esac

case "$#" in
0|1)    echo >&2 "Usage: $0 base-drive options ..."; exit 1;;
esac

# Option default values... anything not set here has no default
TYPE=ffs
FFS=2

# These options can be issued more than once, and values accumulate
# (value of APP_xxx variable is inserted between values as separator)
APP_FSTAB=,
APP_LABEL=_
# For other options, if used more than once, last value set wins.
# This allows funcs/aliases then values to be overridden easily

D=$1; shift

VAR=
for arg
do
        test -n "${VAR}" && {
                V=APP_${VAR}
                eval VV=\$"${V}"
                if test -n "${VV}"
                then
                        eval 
${VAR}='$'"${VAR}"'${'"${VAR}"':+'"'${VV}'"'}'"'${arg}'"
                else
                        eval ${VAR}="'${arg}'"
                fi
                VAR=
                continue
        }

        F=${arg}
        case "${arg}" in

        -b)     VAR=BASE;;
        -f)     VAR=FSTAB;;
        -l)     VAR=LABEL;;
        -m)     VAR=MOUNT;;
        -s)     VAR=SIZE;;
        -t)     VAR=TYPE;;

        -B)     VAR=BLKSIZE;;
        -F)     VAR=FRAGSIZE;;
        -N)     VAR=INODES;;
        -O)     VAR=FFS;;

        -?)     echo >&2 "Unrecognised option '${arg}'"; exit 1;;
        *)      echo >&2 "Don't know what to do with '${arg}'"; exit 1;;

        esac
done

test -n "${VAR}" && {
        echo >&2 "No value given for ${F}"
        exit 1
}

test -z "${SIZE}" && {
        echo >&2 "Need a size (-s)"
        exit 1
}

# More recent gpt commands would do this internally, but not all, so
# convert from human readable size in GB or MB into a sector count...
case "${SIZE}" in
[0-9]*[Gg]|[0-9]*[Gg][Bb])      SIZE=$(( ${SIZE%[Gg]*} * 2097152 ));;
[0-9]*[Mm]|[0-9]*[Mm][Bb])      SIZE=$(( ${SIZE%[Mm]*} * 2048 ));;
*[^0-9]*)       echo >&2 "Unrecognised size (-s) value: ${SIZE}"; exit1;;
esac

if [ -n "${FSTAB}" ] && [ -z "${MOUNT}" ]
then
        echo >&2 "Cannot add to fstab (-f option) without mount point (-m)"
        exit 1
fi

if [ -n "${BLKSIZE}${FRAGSIZE}" ] && [ -z "${MOUNT}${INODES}" ]
then
        echo >&2 "Warning: -B or -F with neither -m nor -N is useless"
fi

if [ -n "${MOUNT}" ] && ! [ -d "${MOUNT}" ]
then
        echo "Attempting to create mount point: ${MOUNT}"
        mkdir -p "${MOUNT}" || exit 1
fi

set -- $( gpt add -s "${SIZE}" ${TYPE:+-t} "${TYPE}" "${D}" ) ||
{
        echo >&2 "gpt add failed!"
        exit 1
}
if [ "$9" -ne "${SIZE}" ]
then
        echo >&2 "Partition created with incorect size"
        exit 1
fi
B=$8

if [ -n "${LABEL}" ]
then
        gpt label -l "${LABEL}" -b "${B}" "${D}"
        # Don't panic if label fails...
else
        set -- $( dkctl "${D}" addwedge NO-NAME "${B}" "${SIZE}" "${TYPE}" ) ||
        {
                echo >&2 "Unable to make temporary wedge"
                gpt remove -b "${B}" "${D}"
                exit 1
        }
        DK="$1"
        test -n "${DK}" ||
        {
                echo >&2 "Unable get wedge id of temporary wedge"
                gpt show "${D}"
                dkctl "${D}" listwedges
                exit 1
        }
        dkctl "${D}" delwedge "${DK}" ||
        {
                echo >&2 "Unable to remove temporary wedge"
                gpt remove -b "${B}" "${D}"
                exit 1
        }
        LABEL="${DK}"
fi

set -- $( dkctl "${D}" addwedge "${LABEL}" "${B}" "${SIZE}" "${TYPE}" ) ||
{
        echo >&2 "Unable to make wedge for ${LABEL}"
        gpt remove -b "${B}" "${D}"
        exit 1
}
DK="$1"
test -n "${DK}" ||
{
        echo >&2 "Unable to extract name of created wedge"

        gpt show "${D}"
        dkctl "${D}" listwedges
        exit 1
}

test -e "/dev/${DK}" && test -e "/dev/r${DK}" ||
{
        cd /dev &&
        ./MAKEDEV "${DK}"
} || {
        echo >&2 "${DK} created does not exist in /dev and could not be made"
        exit 1

        # Or perhaps ...
        dkctl "${D}" delwedge "${DK}"
        gpt remove -b "${B}" "${D}"
        exit 1
}

# This is purely because there is no code below to create any
# other type of filesystem, except a ffs filesystem.  We do
# not want to attempt to mount uninitialised trash, nor do
# we want that to happen at next boot (via fstab), hence if the
# filesystem type is not ffs, we just stop here, and allow
# everything else to be setup manually.
# Add the code for other newfs types, then exempt that type from the exit 0
case "${TYPE}" in
ffs)    ;;
*)      exit 0;;
esac

if [ -n "${MOUNT}" ] || [ -n "${INODES}" ]
then
        newfs \
                ${FFS:+-O} ${FFS}               \
                ${BLKSIZE:+-b} ${BLKSIZE}       \
                ${FRAGSIZE:+-f} ${FRAGSIZE}     \
                ${INODES:+-n} ${INODES}         \
                        "/dev/r${DK}"                   || {
                                echo >&2 "Newfs failed on /dev/r${DK}"
                                dkctl "${D}" delwedge "${DK}"
                                gpt remove -b "${B}" "${D}"
                                exit 1
                        }
fi

if [ -n "${MOUNT}" ]
then
        M=mount
        MOPTS=log
        case "${FSTAB}" in
        *ro*,log*|*log*,ro*)
                # The log option makes no sense (and is not permitted) with ro
                MOPTS=ro
                FSTAB=$(echo "${FSTAB}" | sed -e "s/log//" -e "s/,,/,/g")
                                ;;
        *ro*)   MOPTS=ro        ;;
        *rw,*log|*log*,rw)      ;;
        *rw*)   MOPTS="rw"      ;;
        *log*)  FSTAB="rw,${FSTAB}" ;;
        ?*)     FSTAB="rw,${FSTAB}" MOPTS="rw" ;;
        esac
        case "${FSTAB}" in
        *noauto*)       M=:     ;;
        esac

        ${M} -o ${MOPTS} /dev/"${DK}" "${MOUNT}" || exit 1

        if [ -n "${FSTAB}" ]
        then
                if test -n "${LABEL}" && grep -s >/dev/null "^NAME=" /etc/fstab
                then
                        F1=NAME="${LABEL}"
                else
                        F1=/dev/"${DK}"
                fi
                case "${FSTAB}" in
                *noauto*)       F6=0;;
                *)              F6=2;;
                esac
                echo "${F1}     ${MOUNT}        ${TYPE} ${FSTAB}         1 
${F6}" >> /etc/fstab
        fi
fi