Re: st(4) and mt eom

[John Nemeth <jnemeth%cue.bc.ca@localhost>]

On Mar 20,  8:08pm, Frank Kardel wrote:
} 
} This seems to be a long standing deficiency of the driver. Looking at 
} the SCSI spec it is recommended to issue a READ POSITION command

     I did read a book about SCSI sometime ago, so I have a good
overview of CCBs and the protocol, but it has been some time since
I've looked at it in any depth.  And, although I've made minor
changes to our SCSI code, I can't say that I know it in any depth.

} get the current position. Looking at the spec and code it should be 
} possible to handle the SP_EOM case better with respect to the position 
} information
} 
} by issuing READ POSITION (service action code 6 = LONG FORM) at that 
} point to set the correct position.
} 
} I also ran bacula (now I run freshly ported bareos 18.2.5 for HW 
} encryption, tapealert) with HW EOM set to false. btape test made those 
} recommendation a long time ago.

     btape test recommended:

Hardware End of Medium = No
Fast Forward Space File = No

Apparently it doesn't try all combinations.  This combination made
restores ridiculously slow (I'm currently using LTO-4, 800GB drives).
I spent a fair bit of time reading the source code, running ktrace,
instrumenting things, and finally reading the kernel source code.
The kernel source code explained everything.

} On 03/20/19 19:02, John Nemeth wrote:
} >       If you issue an "mt eom" (forward to end of media), the driver
} > loses track of the tape position.  This seriously messes with
} > Bacula's tape handling.  Since Bacula expects the driver not to
} > lose the tape position I get the feeling there are other operating
} > systems that don't.  I found this code in st.c:st_space():
} >
} >          error = scsipi_command(st->sc_periph, (void *)&cmd, sizeof(cmd), 0, 0,
} >              0, ST_SPC_TIME, NULL, flags);
} >
} >          if (error == 0 && (st->flags & ST_POSUPDATED) == 0) {
} >                  number = number - st->last_ctl_resid;
} >                  if (what == SP_BLKS) {
} >                          if (st->blkno != -1)
} >                                  st->blkno += number;
} >                  } else if (what == SP_FILEMARKS) {
} >                          if (st->fileno != -1) {
} >                                  st->fileno += number;
} >                                  if (number > 0)
} >                                          st->blkno = 0;
} >                                  else if (number < 0)
} >                                          st->blkno = -1;
} >                          }
} >                  } else if (what == SP_EOM) {
} >                          /* This loses us relative position. */
} >                          st->fileno = st->blkno = -1;
} >                  }
} >          }
} >          return error;
} > }
} >
} > Notice the SP_EOM case.  Can any SCSI experts, in particular SCSI
} > tape experts, shed some light on this and what can be done about
} > it?
} >
} >       I have found a workaround for Bacula which is to tell it about
} > this problem.  If you do that, Bacula will do "mt fsf 65535" (and
} > pray that there aren't more files then that on the tape).  The tape
} > I have with the largest number of files is at 1186, so this will
} > do for now.  Still, it would be nice to fix the underlying problem.
} >
} >       For those wondering, my bacula-sd.conf contains:
} >
} > Device {
} >    Name = LTO-4a
} >    Archive Device = /dev/nrst0
} >    Device Type = Tape
} >    Media Type = LTO-4
} >    AutoChanger = yes
} >    LabelMedia = no
} >    Drive Index = 0
} >    AlwaysOpen = yes;
} >    Removable Media = yes;
} >    Random Access = no;
} >    Maximum File Size = 2GB
} >    Automatic Mount = yes;               # when device opened, read it
} >    Spool Directory = /bacula/spool-sd
} >    Hardware End of Medium = No
} >    #Fast Forward Space File = No
} >    BSF at EOM = yes
} >    #
} >    # New alert command in Bacula 9.0.0
} >    #  Note: you must have the sg3_utils (rpms) or the
} >    #        sg3-utils (deb) installed on your system.
} >    #        and you must set the correct control device that
} >    #        corresponds to the Archive Device
} > #  Control Device = /dev/sg??  # must be SCSI ctl for /dev/nrst0
} > #  Alert Command = "/usr/pkg/libexec/bacula/tapealert %l"
} >
} >    # Enable the Alert command only if you have the mtx package loaded
} > # Alert Command = "sh -c 'tapeinfo -f %c |grep TapeAlert|cat'"
} > # If you have smartctl, enable this, it has more info than tapeinfo
} > # Alert Command = "sh -c 'smartctl -H -l error %c'"
} > }
} 
}-- End of excerpt from Frank Kardel