Re: evbarm-earmv6hf in qemu

[Brad Spencer <brad%anduin.eldar.org@localhost>]

Benny Siegert <bsiegert%gmail.com@localhost> writes:

> Hi!
>
> I am trying to get an emulated VM with two or more cores running NetBSD evbarm-earmv6hf.
>
> I tried adding earmv6hf support to anita (https://github.com/bsiegert/anita/tree/earmv6hf) by adding the following:
>
> - qemu with machine set to raspi2b
> - kernel as https://cdn.netbsd.org/pub/NetBSD/NetBSD-10.0/evbarm-earmv6hf/binary/kernel/netbsd-RPI2.gz
> - root FS based on https://cdn.netbsd.org/pub/NetBSD/NetBSD-10.0/evbarm-earmv6hf/binary/gzimg/rpi.img.gz
>
> This results in  the following qemu command line:
>
> qemu-system-arm \
>     -m 1024 \
>     -drive file=work-https---cdn.netbsd.org-pub-NetBSD-NetBSD-10.0-evbarm-earmv6hf-+a5c40c52/wd0.img,format=raw,media=disk,snapshot=off \
>     -nographic \
>     -M raspi2b \
>     -append root=ld0a \
>     -kernel work-https---cdn.netbsd.org-pub-NetBSD-NetBSD-10.0-evbarm-earmv6hf-+a5c40c52/netbsd-RPI2
>
> However, this doesn’t work at all. There is no output, or a black screen when running the same without -nographic.
>
> Does anyone have a working setup to test earmv6hf? If so, how?
>
> — 
> Benny


This can be made to work, but there is a lot of manual to it (at least
the way I did it).  Following is a message I posted on tech-pkg some
time back that describes how I did it.  I use such a set up all of the
time to build earmv6hf packages.

Short explanation:
What the following describes is building a GENERIC kernel for armv6,
which is not something that is usually done.  Further, it builds the
kernel to be compatible with armv7 which is exactly how you use a armv6
kernel on a RPI2 or RPI3.  The rest is about using UEFI to boot the
thing using parts of the armv7.img.

I do understand that QEMU does have some support for the RPI family in a
more direct sense....  you used that in your post.  This method does not
use any of that and I have have never tried to see how that support
exactly works.



------------------posted to tech-pkg--------------------------------
In a previous message I replied that I wondered if a GENERIC armv6
kernel / image could exist (in the original message, for the purposes of
building a armv6 rust version).  By default it is not made, but it
appears that it is possible to create one manually that functions with
QEMU.

1) Build the earmv6hf release.  You will need to do this to get the TOOL
chain.  You can probably just build the tools if you like and pull the
rest of the parts from precompiled images.  The goal is to create a
image that will be bootable with UEFI and contains a armv6 kernel and
armv6 userland binaries.

2) Assuming your source tree is in /usr/src.  Create
/usr/src/sys/arch/evbarm/conf/GENERICV6 that contains the following:

----
include "arch/evbarm/conf/GENERIC"

makeoptions     CPUFLAGS="-mcpu=cortex-a7 -mfpu=neon"
-----

3) set your $PATH to have the TOOL chain for earmv6hf first and do this
to build the GENERICV6 kernel using the earmv6hf tools:

nbconfig GENERICV6
cd ../compile/GENERICV6
nbmake-evbarm depend && nbmake-evbarm

If everything worked as expected you will have a netbsd kernel built
with the desired tools.

4) Pull a copy down of the armv7.img.gz or build the earmv7hf release.

5) Pull down a copy of rpi.img.gz or use the one from your earmv6hf
build if you did a full release.

6) There are a couple of ways to proceed at this point..  if you can get
the FFS mounted from the armv7.img and rpi.img images you can create the
needed parts from those (I was not able to do this from a amd64 box,
BTW, but YMMV).  If that doesn't work for whatever reason, create a
image with a MSDOS MBR on it that has a 80MB 32bit FAT and a FFS format
1 filesystem for the rest of the disk (newfs -O1 ....).  Don't forget to
put a MBR on it with fdisk and make the MS-DOS filesystem active.

7) What you will want to create is a image with the following in it:

On the MS-DOS filesystem:
mkdir -p EFI/BOOT
copy to EFI/BOOT the bootarm.efi from the MSDOS filesystem from armv7.img

On the FFS:
Put the entire contents of the FFS from rpi.img or the entire contents
from a earmv6hf release build.  If you do the second, make sure that you
fix up the fstab and rc.conf files and MAKEDEV the devices.

Also put the GENERICV6 netbsd kernel that you built in #3 in the root of
this FFS.  Note that this is the real kernel, not the .img or .ub
version.

How you do #7 is up to you. The simplest may be to get the rpi.img
filesystems mounted and just add EFI/BOOT to the MSDOS filesystem and
the GENERICV6 kernel to the root of the FFS.  (Again, I had trouble
mounting the FFS of these images on a amd64 box ...  it didn't like the
superblock and I can't figure out why).


If the image was created as expected you should be able to invoke it
like this with QEMU:


qemu-system-arm -M virt -cpu cortex-a7 -smp 1 -m 1g -drive if=none,file=armv6.img,format=raw,id=hd0 -device virtio-blk-device,drive=hd0 -netdev tap,id=net0,ifname=tap0,script=no,downscript=no -device virtio-net-device,netdev=net0,mac=00:11:22:33:44:77 -bios QEMU_EFI.fd -nographic

The location of QEMU_EFI.fd is:

https://snapshots.linaro.org/components/kernel/leg-virt-tianocore-edk2-upstream/4480/QEMU-ARM/RELEASE_GCC5/QEMU_EFI.fd 

Note that the above qmeu-system-arm command creates a bridged network
arrangement for the guest.  If you do not wish to do that, adjust
accordingly.  See https://wiki.netbsd.org/ports/evbarm/qemu_arm/ for
details.

What this system appears to be is something like a Raspberry PI 2
running a armv6 release.  Not strictly speaking a v6 processor, but the
closest thing you can get to one with QEMU.  QEMU does not appear to
support the virt machine type with a real v6 CPU (only the cortex-a*
CPUs can be a virt machine, it seems) and I am not sure what the EFI
firmware would think of a true v6 cpu anyway.

armv6# dmesg|head -30
[     1.000000] Copyright (c) 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005,
[     1.000000]     2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016, 2017,
[     1.000000]     2018, 2019, 2020, 2021, 2022
[     1.000000]     The NetBSD Foundation, Inc.  All rights reserved.
[     1.000000] Copyright (c) 1982, 1986, 1989, 1991, 1993
[     1.000000]     The Regents of the University of California.  All rights reserved.

[     1.000000] NetBSD 9.99.101 (GENERICV6) #0: Fri Oct 21 12:48:46 EDT 2022
[     1.000000]         brad%samwise.nat.eldar.org@localhost:/usr/src/sys/arch/evbarm/compile/GENERICV6
[     1.000000] total memory = 1024 MB
[     1.000000] avail memory = 983 MB
[     1.000000] timecounter: Timecounters tick every 10.000 msec
[     1.000000] Kernelized RAIDframe activated
[     1.000000] armfdt0 (root)
[     1.000000] armfdt0: using EFI runtime services for RTC
[     1.000000] simplebus0 at armfdt0
[     1.000000] cpus0 at simplebus0
[     1.000000] simplebus1 at simplebus0
[     1.000000] simplebus2 at simplebus0
[     1.000000] cpu0 at cpus0: Cortex-A7 r0p5 (Cortex V7A core)
[     1.000000] cpu0: DC enabled IC enabled WB enabled LABT branch prediction enabled
[     1.000000] cpu0: L1 32KB/64B 2-way (256 set) VIPT Instruction cache
[     1.000000] cpu0: L1 32KB/64B 2-way (256 set) write-back-locking-C PIPT Data cache
[     1.000000] cpu0: L2 2304KB/64B 16-way (2304 set) write-through PIPT Unified cache
[     1.000000] vfp0 at cpu0: NEON MPE (VFP 3.0+), rounding, NaN propagation, denormals
[     1.000000] fclock0 at simplebus0: 24000000 Hz fixed clock (clk24mhz)
[     1.000000] gic0 at simplebus0: GIC
[     1.000000] armgic0 at gic0: Generic Interrupt Controller, 288 sources (288 valid)
[     1.000000] armgic0: 256 Priorities, 256 SPIs, 16 PPIs, 16 SGIs
[     1.000000] armgic0: GICv2m @ 0x8020000, SPIs 80-143
armv6# 
armv6# 
armv6# uname -p
earmv6hf
armv6# 
armv6# 

One should be able to add more than one CPU and add more memory.
Package builds should be possible from such an arrangement that should
work on any real armv6 board.  A problem that might be encountered is if
a software build probed the CPU directly to see what features were
supported and built itself according to what it found.  This sort of
thing would be a problem even on real hardware too, of course.  As long
as stuff uses the same information as "uname -p" everything should be
fine.
------------------posted to tech-pkg--------------------------------







-- 
Brad Spencer - brad%anduin.eldar.org@localhost - KC8VKS - http://anduin.eldar.org