Subject: Re: Ultra5?
To: Murray Stokely <murray@cdrom.com>
From: Eduardo E. Horvath <eeh@one-o.com>
List: port-sparc64
Date: 03/10/2000 14:51:48
>   This list is dissappointingly quiet.  Is there any other forum where
> actual development is taking place on sparc64 / PCI machines?  I'd
> like to help in anyway I can to get NetBSD running on Ultra 5 and
> Ultra 10's.  Does anyone in the project have access to the relevant
> hardware documentation?

Wow.  Someone is subscribed to this list.  I thought I was the only one.

Well, most of the public stuff tends to go on port-sparc, tech-kern,
and tech-toolchain.  But since I'm probably the only one doing any
work on sparc64, there's not an awful lot of traffic.

Here's a quick description about how to work on sparc64.  I should try
to get someone to HTML-ify this.

First of all you need a build platform.  This can be just about
anything, but certain configurations are easier to put together than
others.  Ideally you would have a supported Ultra1 machine and run
native.  The second best would be to have a NetBSD/sparc machine with
an ELF snapshot.  Third would be any NetBSD machine.  Finally, it is
possible to do this on a machine not running NetBSD, but that's much
more difficult.

But remember the most important rule: do not mix 32-bit and 64-bit
kernels and userlands.  A 64-bit kernel may limp along for a short
time on a 32-bit userland before it dies, but a 32-bit kernel will
immediately fail on a 64-bit userland.


Building in 32-bit Land.

If you have a working Ultra1, grab a 32-bit kernel and recent
NetBSD/sparc ELF snapshot and install them.  Then follow the
instructions for a NetBSD/sparc machine.

Once you have a machine running NetBSD/sparc ELF, you need to make
sure you have a 64-bit enabled assembler.  I believe that the one in a
recent snapshot should work.  

Anyway, extract a full source tree and test out the assembler by
building src/sys/arch/sparc64/stand/ofwboot/ofwboot.  You should be
able to go into that directory any type `make'.  If that works you
should be able to go on to building a kernel.

If the assembler complains about unknown or unsupported instructions,
you need to build a new assembler.  The in-tree version is supposed to
work if you go to src/gnu/dist and type `./configure
--enable-64-bit-bfd'.  I usually grab binutils-2.9.1, copy
src/gnu/dist/opcodes/sparc-opc.c to opcodes/sparc-opc.c (there is a
bug in one of the instructions that we fixed) and configure with
`--enable-64-bit-bfd' to get the 64-bit instructions.  Try replacing
the system assembler with that one (but keep a copy of the original
around in case anything breaks) and build owfboot with it.  If it
works you can now go on to building a kernel.

On a NetBSD box of a different architecture, you need to build a
cross-compiler setup.  First build a cross toolchain.  Either grab
binutils-2.9.1 and update opcodes/sparc-opc.c or go to the in-tree
toolchain and `./configure --target=sparc-sun-netbsd
--enable-64-bit-bfd', and install the results in `/usr/local'.

Next, build a recent version of gcc or egcs configured for
`--target=sparc-sun-netbsd' and install this in `/usr/local' too.  The
compiler will try to build itself and fail with an error, but that's
normal.

Now you need to put together an /etc/mk.conf that grabs the cross
tools.  Try this:

.ifdef SPARC_BINARIES
OBJECT_FMT=ELF
NOGCCERROR=1
CC=/usr/local/sparc-sun-elf/bin/gcc -D__NetBSD__ -Wno-format
CXX=/usr/local/sparc-sun-elf/bin/gcc -x c++ -D__NetBSD__ -Wno-format
LD=/usr/local/sparc-sun-elf/bin/ld
AR=/usr/local/sparc-sun-elf/bin/ar
AS=/usr/local/sparc-sun-elf/bin/as
RANLIB=/usr/local/sparc-sun-elf/bin/ranlib
NM=/usr/local/sparc-sun-elf/bin/nm
SIZE=/usr/local/bin/sparc-sun-elf-size
STRIP=/usr/local/sparc-sun-elf/bin/strip
MACHINE_ARCH=sparc
CPPFLAGS+=-D__ELF__
.endif

Now you can try building ofwboot.  Remember you need to do a `make
SPARC_BINARIES=1' to enable the toolchain.

If you're not using a NetBSD build platform, you need to build a
toolchain as described above, as well as a native NetBSD make and
config, and install the required /usr/share/mk files.  This one you
need to figure out yourself.

Building a 32-bit Kernel.

If you have a working 32-bit toolchian and config, you should be able
to build a kernel in the normal way.  Edit a copy of
src/sys/arch/sparc64/conf/GENERIC to reflect your preferences, run
config on it, cd to the build directory, do a `make depend' and a make
and you should get a kernel.

Building in 64-bit Land.

If you're a real glutton for punishment you can try to build in 64-bit
land.  The compiler is still quite buggy, so this will not be easy.
First build binutils-2.9.1 or later with `--target=sparc64-sun-elf'
and install it.  Then build gcc 2.95.2 with `--target=sparc64-sun-elf'
and install it.  Now try to build src/sys/arch/sparc64/stand/ofwboot.

If that works, you can give the kernel a try.  You want to start with
src/sys/arch/sparc64/conf/GENERIC64 instead of GENERIC which is for
32-bit kernels.  Go through the normal kernel build process.  Several
files in the networking area will cause gcc assertion failures.  Those
will need to be built by hand with optimization turned off.

Finally, you can give userland a try, but don't even try to build
shared libraries.  This will also fail for a number of packages, so
you will want make sure make has the `-k' flag set.

You can try installing the 64-bit kernel and 64-bit userland on a disk
and booting from it.  Results vary, but things seem to work reasonably
well, and you may even get a compiler and toolchain out of it.  You
may be able to get a kernel out of that toolchain, but userland will
fail because the 64-bit C++ compiler simply does not work.


Installing.

To boot you need 6 things:

	A bootblock.

	A disklabel, NetBSD or Sun.

	A UFS filesystem, NetBSD or Sun.

	A second stage bootloader, 32- or 64-bit

	A kernel, 32- or 64-bit.

	A userland 32- and/or 64-bit.

The kernel and userland must match.  Both 32-bit and 64-bit versions
of the second stage bootloader should be able to load all kernels, so
it should not matter what you use.  The bootblock is in FORTH, so one
size fits all, but it will only load and execute a file called
`/ofwboot'.

The easiest way to get started is to use a NetBSD-sparc miniroot with
a 32-bit NetBSD/sparc64 kernel.  If you have multiple disks and one of
them has Solaris on it you can copy the netbsd kernel to /netbsd, `dd'
the miniroot to your swap partition (after disabling it) and then type:

	reboot -- '<disk> /netbsd -a' 

(replace `<disk>' with the name of the disk for your Solaris partition.)

Otherwise boot Solaris from CDROM, and partition a disk.  `dd' the
miniroot into the swap partition on any disk.  Then create an empty
filesystem in the root partition and copy `ofwboot' and the NetBSD
kernel to it.  Finally, use either the Solaris or NetBSD `installboot'
to install the NetBSD bootblock to that disk.  Then do a:

	reboot -- '<disk> -a'

The kernel should load and it should ask you for a boot device.  You
should type `sdX*' where X is the SCSI ID.  Sysinst should start
running from the miniroot and fail almost immediately since the kernel
doesn't match.  You should now have use of all the standard commands.

Use `disklabel' to create a disklabel for the disk, newfs to create a
root filesystem, ftp to grab files, tar to extract an archive, etc.

=========================================================================
Eduardo Horvath				eeh@netbsd.org
	"I need to find a pithy new quote." -- me