cpu_fdt.c.diff take 2: hangs on mpstart

[Christoph Badura <bad%bsd.de@localhost>]

I'd like to propose the following change to fix the hangs when starting
secondary CPUs.  As seen on my NanoPi R1 and likely other Allwinner H3
based boards.

I'd appreciate feedback and testing on boards I don't have access to.

This is basically the same change as in r1.26 that I backed out because
it caused e.g. Pinebooks to hang on boot.

The reason for those hangs is that cache operations can't be used that
early on ARM v8/AArch64.  There the cache ops depend on curcpu()'s
ci_cacheinfo which requires ci_package_id being initialized before
aarch_getcacheinfo() is called, which isn't done until cpu_attach() runs
much later during autoconfigure.

Tested on both Pinebook and NanoPi R1.

Thanks to Jared for the help.
--chris

Index: cpu_fdt.c
===================================================================
RCS file: /cvsroot/src/sys/arch/arm/fdt/cpu_fdt.c,v
retrieving revision 1.28
diff -u -r1.28 cpu_fdt.c
--- cpu_fdt.c	19 Oct 2019 18:04:26 -0000	1.28
+++ cpu_fdt.c	27 Oct 2019 19:45:59 -0000
@@ -347,6 +347,10 @@
 
 	fdtbus_get_reg64(phandle, 0, &mpidr, NULL);
 
+#ifndef AARCH64
+	/* on AARCH64 cache ops are functional only after cpu_attach(). */
+	cpu_dcache_wbinv_all();
+#endif
 	ret = psci_cpu_on(mpidr, cpu_fdt_mpstart_pa(), 0);
 	if (ret != PSCI_SUCCESS)
 		return EIO;