On 02/27/17 14:27, Brook Milligan wrote:
On Feb 27, 2017, at 9:50 AM, Jason Bacon <bacon4000%gmail.com@localhost> wrote:It is off-topic but I gave up on R packages due to lack of PLIST files. This made my work on R uncontrollable and unmaintainable. That said probably it's just better to rely on packages from CRAN directly without pkgsrc...I've been leaning in this direction as well. It seems impossible to keep the R-cran packages in any package manager up-to-date and complete. Our users inevitably have to build some of their own, which is easy within an R script.On the other hand, I have run into cases where R complains that a package version is not available via install.packages(), but it can be installed separately through pkgsrc just fine. Are you aware of pkgtools/R2pkg? That is a little package I wrote to create and manage CRAN packages. I have found that it works quite well, although I need to push out some of my recent additions. There is some tension between CRAN and package managers, but I have found having packages in pkgsrc to be more useful than not. In fact, the problem that triggered this thread is the first time I have run into direct conflict and that is a problem with R dictating/recommending an out-of-date CRAN package. Cheers, Brook
Some good points.The main reason we gave up on it here is that much of our R analysis is done on our HTCondor grid, where most of the compute hosts are virtual machines running on campus lab PCs. Adding new packages can presently only be done by rolling out a new guest image. I'd thought about ways to rig them to self-update periodically, but the main problem is that they would then be out of sync much of the time, since we can't predict when they will be running. They are shut down automatically whenever someone is using the PC.
For this env, I wrote an HTCondor script that users can use to build and cache R packages. ( Attached and part of my cluster-admin tools: http://acadix.biz/cluster-admin.php )
On our HPC clusters, where we run user software from an NFS share, it would be much easier to keep CRAN packages up-to-date. I'll re-explore this approach using your nifty R2pkg tool.
Thanks!
Jason
--
Earth is a beta site.
#!/bin/sh -e
##########################################################################
# Script description:
#
# Arguments:
#
# Returns:
#
# History:
# Date Name Modification
# 2015-11-04 Jason Bacon Begin
##########################################################################
usage()
{
cat << EOM
Usage: $0 opsys arch R-lib [R-lib ...]
Example:
$0 FreeBSD X86_64 magic
EOM
exit 1
}
##########################################################################
# Main
##########################################################################
if [ $# -lt 3 ]; then
usage
fi
opsys=$1
arch=$2
shift
shift
printf "Remove and replace old R packages? y/[n] "
read remove
if [ 0$remove = 0y ]; then
rm -rf R R-packages.log
fi
##########################################################################
# Generate HTCondor executable file
##########################################################################
cat << EOM > R-packages.sh
#!/bin/sh -e
hostname
pwd
# Condor nodes have no home directory
release=\$(uname -r)
os=\$(uname -s)-\${release%-*}-\$(uname -m)
export R_LIBS="\$(pwd)/R/lib/\$os"
echo \$R_LIBS
mkdir -p \$R_LIBS
# Caution:
# The line below is modified by install.sh. Be careful not to replace
# %%PREFIX%% with a hard-coded prefix from an installed script.
prefix=%%PREFIX%%
# Help script find Rscript
export PATH=\${PATH}:/usr/local/bin:$prefix/bin
# HTCondor process IDs begin at 0.
Rscript R-packages.r
EOM
##########################################################################
# Generate R script for building R libraries
##########################################################################
cat << EOM > R-packages.r
# Alternative to specifying in install.packages()
# old <- getOption("defaultPackages");
# r <- getOption("repos")
# r["CRAN"] <- "http://cran.mtu.edu";
# r["CRAN"] <- "http://cran.fhcrc.org";
# r["CRAN"] <- "http://rweb.crmda.ku.edu";
# options(defaultPackages = c(old, "MASS"), repos = r)
EOM
for package in $@; do
printf \
"install.packages(\"$package\", repos=\"http://cran.us.r-project.org\";)\n" \
>> R-packages.r
done
##########################################################################
# Generate HTCondor submit file
##########################################################################
cat << EOM > R-packages.condor
universe = vanilla
executable = R-packages.sh
output = R-packages.out
error = R-packages.err
log = R-packages.log
request_memory = 1000
requirements = (target.arch == "$arch") && (target.opsys == "$opsys")
transfer_executable = true
should_transfer_files = if_needed
transfer_input_files = R-packages.r
transfer_output_files = R
when_to_transfer_output = on_exit
queue 1
EOM
condor_submit R-packages.condor
condor_wait R-packages.log
tar zcvf R-packages.tgz R
cat << EOM
In your HTCondor submit file:
Make sure R-packages.tgz is listed as an input
In your HTCondor executable script:
Add the following commands before your R/Rscript commands:
tar zxvf R-packages.tgz
release=\$(uname -r)
os=\$(uname -s)-\${release%-p*}-\$(uname -m)
export R_LIBS="\$(pwd)/R/lib/\$os"
EOM