tech-kern archive
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]
AES leaks, cgd ciphers, and vector units in the kernel
[bcc tech-crypto, tech-security; followups to tech-kern]
It's been well-known since 2005[1] that naive AES software, like we
use today in the NetBSD kernel, is vulnerable to cache-timing attacks
(CVE-2005-1797). These attacks have gotten progressively better over
time, and over a decade ago were even applied to Linux dm-crypt disk
encryption[2].
Timing side channel attacks are not theoretical: shared virtual hosts
and JavaScript engines in web browsers provide adversaries with
abundant attack surfaces to trigger disk I/O, prime/probe/flush/reload
caches, and measure high-resolution timings.
We already replaced NIST CTR_DRBG-AES by NIST Hash_DRBG-SHA256 for
/dev/u?random in part because of AES timing side channel attacks.
It's long since overdue for us to address them in cgd(4), and anything
else in the kernel that uses AES.
The attached patch set provides a three-pronged approach to addressing
the problem:
1. Replace the variable-time AES reference implementation we've been
using by constant-time AES software from Thomas Pornin's
high-quality BearSSL libary.
Security impact: This essentially plugs the leak on all NetBSD
platforms for all existing disk setups (and anything else in the
kernel like IPsec) that already use AES as long as they run an
updated kernel.
(In principle a C compiler could compile the BearSSL logic gates
into secret-dependent branches and memory references, and in
principle a machine could implement logic gates in variable time,
but realistically this goes a long way to plugging the leak.)
Performance impact: The cost is that constant-time AES software is
much slower -- cgd AES-CBC encryption throughput is reduced to
about 1/3, and decryption to about 1/2 (very roughly). This is
bad, obviously, but it is mostly addressed by the next two parts.
2. Add support for CPU AES instructions on Intel, AMD, VIA, and
aarch64 CPUs to implement the kernel's synchronous AES API,
including machinery to allow the kernel to use the CPU's vector
unit.
Security impact: This generally plugs the leak (except perhaps in
software CPU emulators like qemu) on all relevant hardware just
by updating the kernel.
Performance impact: This significantly improves performance over
what it was before with variable-time AES software, on CPUs that
have AES instructions we can use -- cgd AES-CBC throughput very
roughly doubles on a VIA laptop I tried, for instance.
So on ~all amd64 and aarch64 CPUs of the last decade (and VIA
CPUs), this patch set improves security _and_ performance.
3. Add an alternative cgd cipher Adiantum[3], which is built out of
AES (used only once per disk sector), Poly1305, NH, and XChaCha12,
and has been deployed by Google for disk encryption on lower-end
ARM systems.
Security impact: Adiantum generally provides better disk
encryption security than AES-CBC or AES-XTS because it encrypts
an entire disk sector at a time, rather than individual cipher
blocks independently like AES-XTS does or suffixes in units of
cipher blocks like AES-CBC does, so two snapshots of a disk
reveal less information with Adiantum than with AES-CBC or
AES-XTS. Of course, Adiantum is a different cipher so you have
to create new cgd volumes if you want to use it.
(The Adiantum implementation uses the same AES logic as the rest
of the kernel for the one invocation per disk sector it needs, so
it will take advantage of constant-time software or hardware
support.)
Performance impact: Adiantum provides much better software
performance than AES-CBC, AES-XTS, or generally anything that
feeds all the data through AES. (The one AES invocation per disk
sector accounts for only a small fraction of Adiantum's time,
<10%.) This should generally provide performance that is at
least as good as the leaky AES software was on machines that
don't have CPU support for AES.
The net effect is:
(a) there is no more variable-time AES software in the kernel at all,
(b) on most machines of the past decade, AES is (a lot) faster, and
(c) there's an alternative to AES-CBC/AES-XTS in cgd for machines
where fixing the security vulnerability made it slower.
Some additional notes:
* Vector unit in the kernel.
All the CPU AES instructions I dealt with (AES-NI, VIA Padlock,
ARMv8.0-AES) require using the CPU's vector unit. The mechanism is
that we disable interrupts and save any user lwp vector unit state
before computing AES, and then zero the vector registers afterward
to prevent any Spectre-class attacks:
- If the kernel is using the vector unit while in a user lwp, we
have to disable preemption because there's nowhere to save the
kernel's vector registers alongside the user's vector registers.
- If we ever want to compute AES in interrupt context we also need
to disable interrputs, but if we decide never to do AES in
interrupt context (which would be reasonable, just not a
proposition I'm committing to at the moment) then disabling
preemption instead of disabling interrupts would be sufficient.
As future work, in kthreads, we don't need to disable preemption at
all since there's no user lwp state so we can save the kernel's
vector registers in the lwp pcb. Also, in kthreads, we can avoid
zeroing the vector registers after every AES subroutine, since user
code can't even run until after switching to another lwp anyway.
I experimented with doing this in cgd -- adding fpu_kthread_enter
and fpu_kthread_leave around cgd_cipher to set a bit MDL_SYSTEM_FPU
in the lwp, and teaching fpu save/restore to allow saving and
restoring to kthreads with MDL_SYSTEM_FPU set -- and cgd throughput
on my VIA laptop improved by about 1.2x on top of the already huge
throughput increase from using the CPU instructions in the first
place.
I'm not settled on how this should manifest in an MI API yet,
though, so the experiment is not included in the patch set other
than to define fpu_kthread_enter/leave for experimentation.
* Other CPUs' AES instructions.
With a little more effort we could:
- adapt the x86 AES-NI logic to 32-bit mode
- add support for Cavium MIPS CPUs
- adopt vectorized MD constant-time software for CPUs with vector
units like Altivec, NEON, VFP, &c., even if they don't have AES
instructions per se
I didn't do any of that because I was going for low-hanging fruit,
but I would be happy to help if you want to adopt other
implementations.
We could also use a similar mechanism for, e.g., synchronous SHA-256
instructions, to make /dev/urandom (which uses NIST Hash_DRBG with
SHA-256) faster on, e.g., aarch64 and Cavium MIPS CPUs. Also not a
high priority for me because SHA-256 does not invite side channel
threats like AES does, but happy to help if you want to work on it.
* Adiantum components.
Adiantum is built out of components that are useful in their own
right for other applications like Wireguard, notably Poly1305 and
ChaCha, so we could fruitfully factor them out into their own
modules and provide vectorized MD implementations of them from (say)
SUPERCOP to further improve performance.
This makes Adiantum more attractive than, e.g., Threefish as I
suggested some years ago, which is a primitive that almost nobody
uses in the real world.
* cgd disk sector sizes and Adiantum.
cgd currently uses the underlying disk's sector size as advertised
by getdisksize (i.e., DIOCGWEDGEINFO or DIOCGPARTINFO). On almost
all disks today that's 512 bytes, even if the disk actually uses
4096-byte sectors and requires r/m/w cycles to do 512-byte writes.
The sector size should really be a parameter to cgd like the name of
the cipher, because it qualitatively changes the cipher that cgd
computes -- and if some chain of adapters causes a disk with
4096-byte sectors to be presented with 512 bytes or vice versa,
you'll see garbage on your disk.
Unlike AES-CBC or AES-XTS (which don't really care what the sector
size is), Adiantum also takes better advantage of larger sectors --
cursory measurements suggest that it's about 1.5x throughput for
4096-byte sectors over 512-byte sectors.
I did not add any mechanism for configuring the sector size, but it
would be good if we taught cgd to do that (and an upgrade path for
storing it in the parameters file).
* Other existing ciphers.
Our 3DES, Blowfish, CAST128, Camellia, and Skipjack software in the
kernel also obviously relies on secret-dependent array indices.
These are not as high a priority because frankly I don't think
anyone should be using these, and I'd rather get rid of them -- or
maybe reduce 3DES and Blowfish to decryption only, to read old cgd
disks -- than spend any other effort on them.
* Performance measurement.
Most of the performance measurement I did -- which was very rough,
enough to convince me that hardware AES as implemented here clearly
wins in practice over even variable-time software AES, and that my
totally untuned first draft of Adiantum is not worse than
variable-time software AES -- was with:
dd if=/dev/zero of=/tmp/disk bs=1m count=512 progress=$((512/80))
vnconfig -cv vnd0 /tmp/disk
cgdconfig -s cgd0 /dev/vnd0 aes-cbc 256 < /dev/zero
# measure decryption throughput
dd if=/dev/rcgd0d of=/dev/null bs=64k progress=$((512*1024/64/80))
# measure encryption throughput
dd if=/dev/zero of=/dev/rcgd0d bs=64k progress=$((512*1024/64/80))
(Substitute `aes-xts 512' or `adiantum 256' in the cgdconfig
incantation for a fair comparison.)
Thoughts? Comments? Objections? Musical numbers by Groucho Marx on
the nature of consensus?
[1] Daniel J. Bernstein, `Cache-timing attacks on AES', 2004-11-11.
https://cr.yp.to/papers.html#cachetiming
[2] Eran Tromer, Dag Arne Osvik, and Adi Shamir, `Efficient cache
attacks on AES, and countermeasures', Journal of Cryptology 23(1),
pp. 37--71, Springer, 2010. DOI: 10.1007/s00145-009-9049-y
http://www.cs.tau.ac.il/~tromer/papers/cache-joc-official.pdf
[3] Paul Crowley and Eric Biggers, `Adiantum: length-preserving
encryption for entry-level processors', IACR Transactions on
Symmetric Cryptology 2018(4), pp. 39--61.
https://doi.org/10.13154/tosc.v2018.i4.39-61
# HG changeset patch
# User Taylor R Campbell <riastradh%NetBSD.org@localhost>
# Date 1592424014 0
# Wed Jun 17 20:00:14 2020 +0000
# Branch trunk
# Node ID 4a0394d9dc15ee6e51a1f1d6ec158d6f172bb9e0
# Parent 9d717769d8e9978731b1dc571cacd36aa44c7d3d
# EXP-Topic riastradh-kernelcrypto
Spell `blowfish-cbc' as such, not like `bf-cbc'.
Gotta match the name we actually use for this to work!
diff -r 9d717769d8e9 -r 4a0394d9dc15 sys/dev/cgd.c
--- a/sys/dev/cgd.c Mon Jun 15 01:24:20 2020 +0000
+++ b/sys/dev/cgd.c Wed Jun 17 20:00:14 2020 +0000
@@ -1298,7 +1298,7 @@ cgd_ioctl_set(struct cgd_softc *sc, void
if (encblkno[i].v != CGD_CIPHER_CBC_ENCBLKNO1) {
if (strcmp(sc->sc_cfuncs->cf_name, "aes-cbc") &&
strcmp(sc->sc_cfuncs->cf_name, "3des-cbc") &&
- strcmp(sc->sc_cfuncs->cf_name, "bf-cbc")) {
+ strcmp(sc->sc_cfuncs->cf_name, "blowfish-cbc")) {
log(LOG_WARNING, "cgd: %s only makes sense for cbc,"
" not for %s; ignoring\n",
encblkno[i].n, sc->sc_cfuncs->cf_name);
# HG changeset patch
# User Taylor R Campbell <riastradh%NetBSD.org@localhost>
# Date 1591241685 0
# Thu Jun 04 03:34:45 2020 +0000
# Branch trunk
# Node ID 08a86cf7e9ffdc8949751596b2f93934c8f3b692
# Parent 4a0394d9dc15ee6e51a1f1d6ec158d6f172bb9e0
# EXP-Topic riastradh-kernelcrypto
Draft fpu_kthread_enter/leave on x86.
Only fit for kthreads, not user lwps. Preemptible, nestable.
diff -r 4a0394d9dc15 -r 08a86cf7e9ff sys/arch/amd64/include/proc.h
--- a/sys/arch/amd64/include/proc.h Wed Jun 17 20:00:14 2020 +0000
+++ b/sys/arch/amd64/include/proc.h Thu Jun 04 03:34:45 2020 +0000
@@ -55,6 +55,7 @@ struct mdlwp {
#define MDL_COMPAT32 0x0008 /* i386, always return via iret */
#define MDL_IRET 0x0010 /* force return via iret, not sysret */
#define MDL_FPU_IN_CPU 0x0020 /* the FPU state is in the CPU */
+#define MDL_SYSTEM_FPU 0x0040 /* system thread is allowed FPU use */
struct mdproc {
int md_flags;
diff -r 4a0394d9dc15 -r 08a86cf7e9ff sys/arch/i386/include/proc.h
--- a/sys/arch/i386/include/proc.h Wed Jun 17 20:00:14 2020 +0000
+++ b/sys/arch/i386/include/proc.h Thu Jun 04 03:34:45 2020 +0000
@@ -44,6 +44,7 @@ struct pmap;
struct vm_page;
#define MDL_FPU_IN_CPU 0x0020 /* the FPU state is in the CPU */
+#define MDL_SYSTEM_FPU 0x0040 /* system thread is allowed FPU use */
struct mdlwp {
volatile uint64_t md_tsc; /* last TSC reading */
diff -r 4a0394d9dc15 -r 08a86cf7e9ff sys/arch/x86/include/fpu.h
--- a/sys/arch/x86/include/fpu.h Wed Jun 17 20:00:14 2020 +0000
+++ b/sys/arch/x86/include/fpu.h Thu Jun 04 03:34:45 2020 +0000
@@ -33,6 +33,9 @@ void fpu_lwp_abandon(struct lwp *l);
void fpu_kern_enter(void);
void fpu_kern_leave(void);
+int fpu_kthread_enter(void);
+void fpu_kthread_leave(int);
+
void process_write_fpregs_xmm(struct lwp *, const struct fxsave *);
void process_write_fpregs_s87(struct lwp *, const struct save87 *);
diff -r 4a0394d9dc15 -r 08a86cf7e9ff sys/arch/x86/x86/fpu.c
--- a/sys/arch/x86/x86/fpu.c Wed Jun 17 20:00:14 2020 +0000
+++ b/sys/arch/x86/x86/fpu.c Thu Jun 04 03:34:45 2020 +0000
@@ -137,7 +137,8 @@ fpu_lwp_area(struct lwp *l)
struct pcb *pcb = lwp_getpcb(l);
union savefpu *area = &pcb->pcb_savefpu;
- KASSERT((l->l_flag & LW_SYSTEM) == 0);
+ KASSERT((l->l_flag & LW_SYSTEM) == 0 ||
+ (l->l_md.md_flags & MDL_SYSTEM_FPU));
if (l == curlwp) {
fpu_save();
}
@@ -154,7 +155,8 @@ fpu_save_lwp(struct lwp *l)
kpreempt_disable();
if (l->l_md.md_flags & MDL_FPU_IN_CPU) {
- KASSERT((l->l_flag & LW_SYSTEM) == 0);
+ KASSERT((l->l_flag & LW_SYSTEM) == 0 ||
+ (l->l_md.md_flags & MDL_SYSTEM_FPU));
fpu_area_save(area, x86_xsave_features);
l->l_md.md_flags &= ~MDL_FPU_IN_CPU;
}
@@ -343,6 +345,75 @@ fpu_lwp_abandon(struct lwp *l)
/* -------------------------------------------------------------------------- */
+static const union savefpu zero_fpu __aligned(64);
+
+/*
+ * s = fpu_kthread_enter()
+ *
+ * Allow the current kthread to use the FPU without disabling
+ * preemption as fpu_kern_enter/leave do. Must not be used in a
+ * user lwp. When done, call fpu_kthread_leave(s). May be
+ * recursively nested.
+ *
+ * Must not be invoked while in a fpu_kern_enter/leave block.
+ */
+int
+fpu_kthread_enter(void)
+{
+ struct lwp *l = curlwp;
+ int system_fpu = l->l_md.md_flags & MDL_SYSTEM_FPU;
+
+ KASSERTMSG(l->l_flag & LW_SYSTEM,
+ "fpu_kthread_enter is allowed only in kthreads");
+ KASSERTMSG(curcpu()->ci_kfpu_spl == -1,
+ "fpu_kthread_enter is not allowed between fpu_kern_enter/leave");
+
+ if (!system_fpu) {
+ /*
+ * Notify the FPU fault handler to save the FPU state
+ * for us.
+ */
+ l->l_md.md_flags |= MDL_SYSTEM_FPU;
+
+ /* Clear CR0_TS to enable the FPU. */
+ clts();
+ }
+
+ return system_fpu;
+}
+
+/*
+ * fpu_kthread_leave(s)
+ *
+ * Return to the previous state of whether the current kthread can
+ * use the FPU without disabling preemption.
+ */
+void
+fpu_kthread_leave(int system_fpu)
+{
+ struct lwp *l = curlwp;
+
+ KASSERTMSG(l->l_flag & LW_SYSTEM,
+ "fpu_kthread_leave is allowed only in kthreads");
+ KASSERTMSG(l->l_md.md_flags & MDL_SYSTEM_FPU,
+ "fpu_kthread_leave without fpu_kthread_enter");
+
+ if (!system_fpu) {
+ /*
+ * Zero the fpu registers; otherwise we might leak
+ * secrets through Spectre-class attacks to userland,
+ * even if there are no bugs in fpu state management.
+ */
+ fpu_area_restore(&zero_fpu, x86_xsave_features);
+
+ /* Set CR0_TS to disable use of the FPU. */
+ stts();
+
+ /* Stop asking to save our FPU state. */
+ l->l_md.md_flags &= ~MDL_SYSTEM_FPU;
+ }
+}
+
/*
* fpu_kern_enter()
*
@@ -359,6 +430,10 @@ fpu_kern_enter(void)
struct cpu_info *ci;
int s;
+ /* Nothing needed if we're in a kthread with FPU enabled. */
+ if (l->l_md.md_flags & MDL_SYSTEM_FPU)
+ return;
+
s = splhigh();
ci = curcpu();
@@ -392,10 +467,14 @@ fpu_kern_enter(void)
void
fpu_kern_leave(void)
{
- static const union savefpu zero_fpu __aligned(64);
+ struct lwp *l = curlwp;
struct cpu_info *ci = curcpu();
int s;
+ /* Nothing needed if we're in a kthread with FPU enabled. */
+ if (l->l_md.md_flags & MDL_SYSTEM_FPU)
+ return;
+
KASSERT(ci->ci_ilevel == IPL_HIGH);
KASSERT(ci->ci_kfpu_spl != -1);
# HG changeset patch
# User Taylor R Campbell <riastradh%NetBSD.org@localhost>
# Date 1591240980 0
# Thu Jun 04 03:23:00 2020 +0000
# Branch trunk
# Node ID e7941432a3cd362134c7e5195b5c9725e332de7f
# Parent 08a86cf7e9ffdc8949751596b2f93934c8f3b692
# EXP-Topic riastradh-kernelcrypto
Draft fpu_kern_enter/leave on aarch64.
diff -r 08a86cf7e9ff -r e7941432a3cd sys/arch/aarch64/aarch64/cpu.c
--- a/sys/arch/aarch64/aarch64/cpu.c Thu Jun 04 03:34:45 2020 +0000
+++ b/sys/arch/aarch64/aarch64/cpu.c Thu Jun 04 03:23:00 2020 +0000
@@ -133,6 +133,8 @@ cpu_attach(device_t dv, cpuid_t id)
ci->ci_dev = dv;
dv->dv_private = ci;
+ ci->ci_kfpu_spl = -1;
+
arm_cpu_do_topology(ci);
cpu_identify(ci->ci_dev, ci);
diff -r 08a86cf7e9ff -r e7941432a3cd sys/arch/aarch64/aarch64/fpu.c
--- a/sys/arch/aarch64/aarch64/fpu.c Thu Jun 04 03:34:45 2020 +0000
+++ b/sys/arch/aarch64/aarch64/fpu.c Thu Jun 04 03:23:00 2020 +0000
@@ -38,6 +38,8 @@
#include <sys/lwp.h>
#include <sys/evcnt.h>
+#include <aarch64/fpu.h>
+#include <aarch64/locore.h>
#include <aarch64/reg.h>
#include <aarch64/pcb.h>
#include <aarch64/armreg.h>
@@ -172,3 +174,68 @@ fpu_state_release(lwp_t *l)
reg_cpacr_el1_write(CPACR_FPEN_NONE);
__asm __volatile ("isb");
}
+
+void
+fpu_kern_enter(void)
+{
+ struct lwp *l = curlwp;
+ struct cpu_info *ci;
+ int s;
+
+ /*
+ * Block all interrupts. We must block preemption since -- if
+ * this is a user thread -- there is nowhere to save the kernel
+ * fpu state, and if we want this to be usable in interrupts,
+ * we can't let interrupts interfere with the fpu state in use
+ * since there's nowhere for them to save it.
+ */
+ s = splhigh();
+ ci = curcpu();
+ KASSERT(ci->ci_kfpu_spl == -1);
+ ci->ci_kfpu_spl = s;
+
+ /*
+ * If we are in a softint and have a pinned lwp, the fpu state
+ * is that of the pinned lwp, so save it there.
+ */
+ if ((l->l_pflag & LP_INTR) && (l->l_switchto != NULL))
+ l = l->l_switchto;
+ if (fpu_used_p(l))
+ fpu_save(l);
+
+ /*
+ * Enable the fpu, and wait until it is enabled before
+ * executing any further instructions.
+ */
+ reg_cpacr_el1_write(CPACR_FPEN_ALL);
+ arm_isb();
+}
+
+void
+fpu_kern_leave(void)
+{
+ static const struct fpreg zero_fpreg;
+ struct cpu_info *ci = curcpu();
+ int s;
+
+ KASSERT(ci->ci_cpl == IPL_HIGH);
+ KASSERT(ci->ci_kfpu_spl != -1);
+
+ /*
+ * Zero the fpu registers; otherwise we might leak secrets
+ * through Spectre-class attacks to userland, even if there are
+ * no bugs in fpu state management.
+ */
+ load_fpregs(&zero_fpreg);
+
+ /*
+ * Disable the fpu so that the kernel can't accidentally use
+ * it again.
+ */
+ reg_cpacr_el1_write(CPACR_FPEN_NONE);
+ arm_isb();
+
+ s = ci->ci_kfpu_spl;
+ ci->ci_kfpu_spl = -1;
+ splx(s);
+}
diff -r 08a86cf7e9ff -r e7941432a3cd sys/arch/aarch64/include/cpu.h
--- a/sys/arch/aarch64/include/cpu.h Thu Jun 04 03:34:45 2020 +0000
+++ b/sys/arch/aarch64/include/cpu.h Thu Jun 04 03:23:00 2020 +0000
@@ -89,6 +89,8 @@ struct cpu_info {
volatile u_int ci_astpending;
volatile u_int ci_intr_depth;
+ int ci_kfpu_spl;
+
/* event counters */
struct evcnt ci_vfp_use;
struct evcnt ci_vfp_reuse;
diff -r 08a86cf7e9ff -r e7941432a3cd sys/arch/aarch64/include/fpu.h
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/sys/arch/aarch64/include/fpu.h Thu Jun 04 03:23:00 2020 +0000
@@ -0,0 +1,35 @@
+/* $NetBSD$ */
+
+/*
+ * Copyright (c) 2020 The NetBSD Foundation, Inc.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND CONTRIBUTORS
+ * ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
+ * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+ * POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _AARCH64_FPU_H_
+#define _AARCH64_FPU_H_
+
+void fpu_kern_enter(void);
+void fpu_kern_leave(void);
+
+#endif /* _AARCH64_FPU_H_ */
diff -r 08a86cf7e9ff -r e7941432a3cd sys/arch/aarch64/include/machdep.h
--- a/sys/arch/aarch64/include/machdep.h Thu Jun 04 03:34:45 2020 +0000
+++ b/sys/arch/aarch64/include/machdep.h Thu Jun 04 03:23:00 2020 +0000
@@ -142,8 +142,11 @@ void aarch64_setregs_ptrauth(struct lwp
/* fpu.c */
void fpu_attach(struct cpu_info *);
struct fpreg;
-void load_fpregs(struct fpreg *);
+void load_fpregs(const struct fpreg *);
void save_fpregs(struct fpreg *);
+void fpu_kern_enter(void);
+void fpu_kern_leave(void);
+
#ifdef TRAP_SIGDEBUG
#define do_trapsignal(l, signo, code, addr, trap) \
# HG changeset patch
# User Taylor R Campbell <riastradh%NetBSD.org@localhost>
# Date 1592418582 0
# Wed Jun 17 18:29:42 2020 +0000
# Branch trunk
# Node ID 5f0c9efc2bac72063928aca09b8529df4e63e77a
# Parent e7941432a3cd362134c7e5195b5c9725e332de7f
# EXP-Topic riastradh-kernelcrypto
Draft fpu_kthread_enter/leave for aarch64.
diff -r e7941432a3cd -r 5f0c9efc2bac sys/arch/aarch64/aarch64/fpu.c
--- a/sys/arch/aarch64/aarch64/fpu.c Thu Jun 04 03:23:00 2020 +0000
+++ b/sys/arch/aarch64/aarch64/fpu.c Wed Jun 17 18:29:42 2020 +0000
@@ -175,6 +175,59 @@ fpu_state_release(lwp_t *l)
__asm __volatile ("isb");
}
+static const struct fpreg zero_fpreg;
+
+int
+fpu_kthread_enter(void)
+{
+ struct lwp *l = curlwp;
+ int system_fpu = l->l_md.md_flags & MDL_SYSTEM_FPU;
+
+ KASSERTMSG(l->l_flag & LW_SYSTEM,
+ "fpu_kthread_enter is allowed only in kthreads");
+ KASSERTMSG(curcpu()->ci_kfpu_spl == -1,
+ "fpu_kthread_enter is not allowed between fpu_kern_enter/leave");
+
+ if (!system_fpu) {
+ /*
+ * Notify the FPU fault handler to save the FPU state
+ * for us.
+ */
+ l->l_md.md_flags |= MDL_SYSTEM_FPU;
+
+ /* Enable the FPU. */
+ fpu_state_load(l, 0);
+ }
+
+ return system_fpu;
+}
+
+void
+fpu_kthread_leave(int system_fpu)
+{
+ struct lwp *l = curlwp;
+
+ KASSERTMSG(l->l_flag & LW_SYSTEM,
+ "fpu_kthread_leave is allowed only in kthreads");
+ KASSERTMSG(l->l_md.md_flags & MDL_SYSTEM_FPU,
+ "fpu_kthread_leave without fpu_kthread_enter");
+
+ if (!system_fpu) {
+ /*
+ * Zero the fpu registers; otherwise we might leak
+ * secrets through Spectre-class attacks to userland,
+ * even if there are no bugs in fpu state management.
+ */
+ load_fpregs(&zero_fpreg);
+
+ /* Disable the FPU. */
+ fpu_state_release(l);
+
+ /* Stop asking to save our FPU state. */
+ l->l_md.md_flags &= ~MDL_SYSTEM_FPU;
+ }
+}
+
void
fpu_kern_enter(void)
{
@@ -182,6 +235,10 @@ fpu_kern_enter(void)
struct cpu_info *ci;
int s;
+ /* Nothing needed if we're in a kthread with FPU enabled. */
+ if (l->l_md.md_flags & MDL_SYSTEM_FPU)
+ return;
+
/*
* Block all interrupts. We must block preemption since -- if
* this is a user thread -- there is nowhere to save the kernel
@@ -214,10 +271,14 @@ fpu_kern_enter(void)
void
fpu_kern_leave(void)
{
- static const struct fpreg zero_fpreg;
+ struct lwp *l = curlwp;
struct cpu_info *ci = curcpu();
int s;
+ /* Nothing needed if we're in a kthread with FPU enabled. */
+ if (l->l_md.md_flags & MDL_SYSTEM_FPU)
+ return;
+
KASSERT(ci->ci_cpl == IPL_HIGH);
KASSERT(ci->ci_kfpu_spl != -1);
diff -r e7941432a3cd -r 5f0c9efc2bac sys/arch/aarch64/include/fpu.h
--- a/sys/arch/aarch64/include/fpu.h Thu Jun 04 03:23:00 2020 +0000
+++ b/sys/arch/aarch64/include/fpu.h Wed Jun 17 18:29:42 2020 +0000
@@ -29,6 +29,9 @@
#ifndef _AARCH64_FPU_H_
#define _AARCH64_FPU_H_
+int fpu_kthread_enter(void);
+void fpu_kthread_leave(int);
+
void fpu_kern_enter(void);
void fpu_kern_leave(void);
diff -r e7941432a3cd -r 5f0c9efc2bac sys/arch/aarch64/include/proc.h
--- a/sys/arch/aarch64/include/proc.h Thu Jun 04 03:23:00 2020 +0000
+++ b/sys/arch/aarch64/include/proc.h Wed Jun 17 18:29:42 2020 +0000
@@ -43,6 +43,7 @@ struct mdlwp {
struct trapframe *md_utf;
uint64_t md_cpacr;
uint32_t md_flags;
+#define MDL_SYSTEM_FPU __BIT(0)
uint64_t md_ia_kern[2]; /* APIAKey{Lo,Hi}_EL1 used in the kernel */
uint64_t md_ia_user[2]; /* APIAKey{Lo,Hi}_EL1 used in user-process */
# HG changeset patch
# User Taylor R Campbell <riastradh%NetBSD.org@localhost>
# Date 1592150319 0
# Sun Jun 14 15:58:39 2020 +0000
# Branch trunk
# Node ID 81a487955535865a6bb603c585be109c3dd1adf5
# Parent 5f0c9efc2bac72063928aca09b8529df4e63e77a
# EXP-Topic riastradh-kernelcrypto
Draft aarch64 zero_fpregs.
Just a series of sad donkeys, with no memory references.
diff -r 5f0c9efc2bac -r 81a487955535 sys/arch/aarch64/aarch64/cpuswitch.S
--- a/sys/arch/aarch64/aarch64/cpuswitch.S Wed Jun 17 18:29:42 2020 +0000
+++ b/sys/arch/aarch64/aarch64/cpuswitch.S Sun Jun 14 15:58:39 2020 +0000
@@ -538,3 +538,43 @@ ENTRY_NP(save_fpregs)
str w9, [x0, #FPREG_FPSR]
ret
END(save_fpregs)
+
+ENTRY_NP(zero_fpregs)
+ eor v0.16b, v0.16b, v0.16b
+ eor v1.16b, v1.16b, v1.16b
+ eor v2.16b, v2.16b, v2.16b
+ eor v3.16b, v3.16b, v3.16b
+ eor v4.16b, v4.16b, v4.16b
+ eor v5.16b, v5.16b, v5.16b
+ eor v6.16b, v6.16b, v6.16b
+ eor v7.16b, v7.16b, v7.16b
+ eor v8.16b, v8.16b, v8.16b
+ eor v9.16b, v9.16b, v9.16b
+ eor v10.16b, v10.16b, v10.16b
+ eor v11.16b, v11.16b, v11.16b
+ eor v12.16b, v12.16b, v12.16b
+ eor v13.16b, v13.16b, v13.16b
+ eor v14.16b, v14.16b, v14.16b
+ eor v15.16b, v15.16b, v15.16b
+ eor v16.16b, v16.16b, v16.16b
+ eor v17.16b, v17.16b, v17.16b
+ eor v18.16b, v18.16b, v18.16b
+ eor v19.16b, v19.16b, v19.16b
+ eor v20.16b, v20.16b, v20.16b
+ eor v21.16b, v21.16b, v21.16b
+ eor v22.16b, v22.16b, v22.16b
+ eor v23.16b, v23.16b, v23.16b
+ eor v24.16b, v24.16b, v24.16b
+ eor v25.16b, v25.16b, v25.16b
+ eor v26.16b, v26.16b, v26.16b
+ eor v27.16b, v27.16b, v27.16b
+ eor v28.16b, v28.16b, v28.16b
+ eor v29.16b, v29.16b, v29.16b
+ eor v30.16b, v30.16b, v30.16b
+ eor v31.16b, v31.16b, v31.16b
+ eor x8, x8, x8
+ eor x9, x9, x9
+ msr fpcr, x8
+ msr fpsr, x9
+ ret
+END(zero_fpregs)
diff -r 5f0c9efc2bac -r 81a487955535 sys/arch/aarch64/include/machdep.h
--- a/sys/arch/aarch64/include/machdep.h Wed Jun 17 18:29:42 2020 +0000
+++ b/sys/arch/aarch64/include/machdep.h Sun Jun 14 15:58:39 2020 +0000
@@ -144,6 +144,7 @@ void fpu_attach(struct cpu_info *);
struct fpreg;
void load_fpregs(const struct fpreg *);
void save_fpregs(struct fpreg *);
+void zero_fpregs(void);
void fpu_kern_enter(void);
void fpu_kern_leave(void);
# HG changeset patch
# User Taylor R Campbell <riastradh%NetBSD.org@localhost>
# Date 1591939006 0
# Fri Jun 12 05:16:46 2020 +0000
# Branch trunk
# Node ID 9d6b84c40f6517bb55848159faa9478ef1a23d02
# Parent 81a487955535865a6bb603c585be109c3dd1adf5
# EXP-Topic riastradh-kernelcrypto
Rework AES in kernel to finally address CVE-2005-1797.
1. Rip out old variable-time reference implementation.
2. Replace it by BearSSL's constant-time 32-bit logic.
=> Obtained from commit dda1f8a0c46e15b4a235163470ff700b2f13dcc5.
=> We could conditionally adopt the 64-bit logic too, which would
likely give a modest performance boost on 64-bit platforms
without AES-NI, but that's a bit more trouble.
3. Select the AES implementation at boot-time; allow an MD override.
=> Use self-tests to verify basic correctness at boot.
=> The implementation selection policy is rather rudimentary at
the moment but it is isolated to one place so it's easy to
change later on.
This (a) plugs a host of timing attacks on, e.g., cgd, and (b) paves
the way to take advantage of CPU support for AES -- both things we
should've done a decade ago. Downside: Computing AES takes 2-3x the
CPU time. But that's what hardware support will be coming for.
Rudimentary measurement of performance impact done by:
mount -t tmpfs tmpfs /tmp
dd if=/dev/zero of=/tmp/disk bs=1m count=512
vnconfig -cv vnd0 /tmp/disk
cgdconfig -s cgd0 /dev/vnd0 aes-cbc 256 < /dev/zero
dd if=/dev/rcgd0d of=/dev/null bs=64k
dd if=/dev/zero of=/dev/rcgd0d bs=64k
The AES-CBC encryption performance impact is closer to 3x because it
is inherently sequential; the AES-CBC decryption impact is closer to
2x because the bitsliced AES logic can process two blocks at once.
diff -r 81a487955535 -r 9d6b84c40f65 sys/conf/files
--- a/sys/conf/files Sun Jun 14 15:58:39 2020 +0000
+++ b/sys/conf/files Fri Jun 12 05:16:46 2020 +0000
@@ -200,10 +200,10 @@ defflag opt_machdep.h MACHDEP
# use it.
# Individual crypto transforms
+include "crypto/aes/files.aes"
include "crypto/des/files.des"
include "crypto/blowfish/files.blowfish"
include "crypto/cast128/files.cast128"
-include "crypto/rijndael/files.rijndael"
include "crypto/skipjack/files.skipjack"
include "crypto/camellia/files.camellia"
# General-purpose crypto processing framework.
diff -r 81a487955535 -r 9d6b84c40f65 sys/crypto/aes/aes.h
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/sys/crypto/aes/aes.h Fri Jun 12 05:16:46 2020 +0000
@@ -0,0 +1,101 @@
+/* $NetBSD$ */
+
+/*-
+ * Copyright (c) 2020 The NetBSD Foundation, Inc.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND CONTRIBUTORS
+ * ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
+ * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+ * POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _CRYPTO_AES_AES_H
+#define _CRYPTO_AES_AES_H
+
+#include <sys/types.h>
+#include <sys/cdefs.h>
+
+/*
+ * struct aes
+ *
+ * Expanded round keys.
+ */
+struct aes {
+ uint32_t aes_rk[60];
+} __aligned(16);
+
+#define AES_128_NROUNDS 10
+#define AES_192_NROUNDS 12
+#define AES_256_NROUNDS 14
+
+struct aesenc {
+ struct aes aese_aes;
+};
+
+struct aesdec {
+ struct aes aesd_aes;
+};
+
+struct aes_impl {
+ const char *ai_name;
+ int (*ai_probe)(void);
+ void (*ai_setenckey)(struct aesenc *, const uint8_t *, uint32_t);
+ void (*ai_setdeckey)(struct aesdec *, const uint8_t *, uint32_t);
+ void (*ai_enc)(const struct aesenc *, const uint8_t[static 16],
+ uint8_t[static 16], uint32_t);
+ void (*ai_dec)(const struct aesdec *, const uint8_t[static 16],
+ uint8_t[static 16], uint32_t);
+ void (*ai_cbc_enc)(const struct aesenc *, const uint8_t[static 16],
+ uint8_t[static 16], size_t, uint8_t[static 16], uint32_t);
+ void (*ai_cbc_dec)(const struct aesdec *, const uint8_t[static 16],
+ uint8_t[static 16], size_t, uint8_t[static 16], uint32_t);
+ void (*ai_xts_enc)(const struct aesenc *, const uint8_t[static 16],
+ uint8_t[static 16], size_t, uint8_t[static 16], uint32_t);
+ void (*ai_xts_dec)(const struct aesdec *, const uint8_t[static 16],
+ uint8_t[static 16], size_t, uint8_t[static 16], uint32_t);
+};
+
+int aes_selftest(const struct aes_impl *);
+
+uint32_t aes_setenckey128(struct aesenc *, const uint8_t[static 16]);
+uint32_t aes_setenckey192(struct aesenc *, const uint8_t[static 24]);
+uint32_t aes_setenckey256(struct aesenc *, const uint8_t[static 32]);
+uint32_t aes_setdeckey128(struct aesdec *, const uint8_t[static 16]);
+uint32_t aes_setdeckey192(struct aesdec *, const uint8_t[static 24]);
+uint32_t aes_setdeckey256(struct aesdec *, const uint8_t[static 32]);
+
+void aes_enc(const struct aesenc *, const uint8_t[static 16],
+ uint8_t[static 16], uint32_t);
+void aes_dec(const struct aesdec *, const uint8_t[static 16],
+ uint8_t[static 16], uint32_t);
+
+void aes_cbc_enc(struct aesenc *, const uint8_t[static 16],
+ uint8_t[static 16], size_t, uint8_t[static 16], uint32_t);
+void aes_cbc_dec(struct aesdec *, const uint8_t[static 16],
+ uint8_t[static 16], size_t, uint8_t[static 16], uint32_t);
+
+void aes_xts_enc(struct aesenc *, const uint8_t[static 16],
+ uint8_t[static 16], size_t, uint8_t[static 16], uint32_t);
+void aes_xts_dec(struct aesdec *, const uint8_t[static 16],
+ uint8_t[static 16], size_t, uint8_t[static 16], uint32_t);
+
+void aes_md_init(const struct aes_impl *);
+
+#endif /* _CRYPTO_AES_AES_H */
diff -r 81a487955535 -r 9d6b84c40f65 sys/crypto/aes/aes_bear.c
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/sys/crypto/aes/aes_bear.c Fri Jun 12 05:16:46 2020 +0000
@@ -0,0 +1,617 @@
+/* $NetBSD$ */
+
+/*-
+ * Copyright (c) 2020 The NetBSD Foundation, Inc.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND CONTRIBUTORS
+ * ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
+ * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+ * POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <sys/cdefs.h>
+__KERNEL_RCSID(1, "$NetBSD$");
+
+#include <sys/types.h>
+#include <sys/endian.h>
+#include <sys/systm.h>
+
+#include <crypto/aes/aes.h>
+#include <crypto/aes/aes_bear.h>
+
+static void
+aesbear_setkey(uint32_t rk[static 60], const void *key, uint32_t nrounds)
+{
+ size_t key_len;
+
+ switch (nrounds) {
+ case 10:
+ key_len = 16;
+ break;
+ case 12:
+ key_len = 24;
+ break;
+ case 14:
+ key_len = 32;
+ break;
+ default:
+ panic("invalid AES nrounds: %u", nrounds);
+ }
+
+ br_aes_ct_keysched(rk, key, key_len);
+}
+
+static void
+aesbear_setenckey(struct aesenc *enc, const uint8_t *key, uint32_t nrounds)
+{
+
+ aesbear_setkey(enc->aese_aes.aes_rk, key, nrounds);
+}
+
+static void
+aesbear_setdeckey(struct aesdec *dec, const uint8_t *key, uint32_t nrounds)
+{
+
+ /*
+ * BearSSL computes InvMixColumns on the fly -- no need for
+ * distinct decryption round keys.
+ */
+ aesbear_setkey(dec->aesd_aes.aes_rk, key, nrounds);
+}
+
+static void
+aesbear_enc(const struct aesenc *enc, const uint8_t in[static 16],
+ uint8_t out[static 16], uint32_t nrounds)
+{
+ uint32_t sk_exp[120];
+ uint32_t q[8];
+
+ /* Expand round keys for bitslicing. */
+ br_aes_ct_skey_expand(sk_exp, nrounds, enc->aese_aes.aes_rk);
+
+ /* Load input block interleaved with garbage block. */
+ q[2*0] = le32dec(in + 4*0);
+ q[2*1] = le32dec(in + 4*1);
+ q[2*2] = le32dec(in + 4*2);
+ q[2*3] = le32dec(in + 4*3);
+ q[1] = q[3] = q[5] = q[7] = 0;
+
+ /* Transform to bitslice, decrypt, transform from bitslice. */
+ br_aes_ct_ortho(q);
+ br_aes_ct_bitslice_encrypt(nrounds, sk_exp, q);
+ br_aes_ct_ortho(q);
+
+ /* Store output block. */
+ le32enc(out + 4*0, q[2*0]);
+ le32enc(out + 4*1, q[2*1]);
+ le32enc(out + 4*2, q[2*2]);
+ le32enc(out + 4*3, q[2*3]);
+
+ /* Paranoia: Zero temporary buffers. */
+ explicit_memset(sk_exp, 0, sizeof sk_exp);
+ explicit_memset(q, 0, sizeof q);
+}
+
+static void
+aesbear_dec(const struct aesdec *dec, const uint8_t in[static 16],
+ uint8_t out[static 16], uint32_t nrounds)
+{
+ uint32_t sk_exp[120];
+ uint32_t q[8];
+
+ /* Expand round keys for bitslicing. */
+ br_aes_ct_skey_expand(sk_exp, nrounds, dec->aesd_aes.aes_rk);
+
+ /* Load input block interleaved with garbage. */
+ q[2*0] = le32dec(in + 4*0);
+ q[2*1] = le32dec(in + 4*1);
+ q[2*2] = le32dec(in + 4*2);
+ q[2*3] = le32dec(in + 4*3);
+ q[1] = q[3] = q[5] = q[7] = 0;
+
+ /* Transform to bitslice, decrypt, transform from bitslice. */
+ br_aes_ct_ortho(q);
+ br_aes_ct_bitslice_decrypt(nrounds, sk_exp, q);
+ br_aes_ct_ortho(q);
+
+ /* Store output block. */
+ le32enc(out + 4*0, q[2*0]);
+ le32enc(out + 4*1, q[2*1]);
+ le32enc(out + 4*2, q[2*2]);
+ le32enc(out + 4*3, q[2*3]);
+
+ /* Paranoia: Zero temporary buffers. */
+ explicit_memset(sk_exp, 0, sizeof sk_exp);
+ explicit_memset(q, 0, sizeof q);
+}
+
+static void
+aesbear_cbc_enc(const struct aesenc *enc, const uint8_t in[static 16],
+ uint8_t out[static 16], size_t nbytes, uint8_t iv[static 16],
+ uint32_t nrounds)
+{
+ uint32_t sk_exp[120];
+ uint32_t q[8];
+ uint32_t cv0, cv1, cv2, cv3;
+
+ KASSERT(nbytes % 16 == 0);
+
+ /* Skip if there's nothing to do. */
+ if (nbytes == 0)
+ return;
+
+ /* Expand round keys for bitslicing. */
+ br_aes_ct_skey_expand(sk_exp, nrounds, enc->aese_aes.aes_rk);
+
+ /* Initialize garbage block. */
+ q[1] = q[3] = q[5] = q[7] = 0;
+
+ /* Load IV. */
+ cv0 = le32dec(iv + 4*0);
+ cv1 = le32dec(iv + 4*1);
+ cv2 = le32dec(iv + 4*2);
+ cv3 = le32dec(iv + 4*3);
+
+ for (; nbytes; nbytes -= 16, in += 16, out += 16) {
+ /* Load input block and apply CV. */
+ q[2*0] = cv0 ^ le32dec(in + 4*0);
+ q[2*1] = cv1 ^ le32dec(in + 4*1);
+ q[2*2] = cv2 ^ le32dec(in + 4*2);
+ q[2*3] = cv3 ^ le32dec(in + 4*3);
+
+ /* Transform to bitslice, encrypt, transform from bitslice. */
+ br_aes_ct_ortho(q);
+ br_aes_ct_bitslice_encrypt(nrounds, sk_exp, q);
+ br_aes_ct_ortho(q);
+
+ /* Remember ciphertext as CV and store output block. */
+ cv0 = q[2*0];
+ cv1 = q[2*1];
+ cv2 = q[2*2];
+ cv3 = q[2*3];
+ le32enc(out + 4*0, cv0);
+ le32enc(out + 4*1, cv1);
+ le32enc(out + 4*2, cv2);
+ le32enc(out + 4*3, cv3);
+ }
+
+ /* Store updated IV. */
+ le32enc(iv + 4*0, cv0);
+ le32enc(iv + 4*1, cv1);
+ le32enc(iv + 4*2, cv2);
+ le32enc(iv + 4*3, cv3);
+
+ /* Paranoia: Zero temporary buffers. */
+ explicit_memset(sk_exp, 0, sizeof sk_exp);
+ explicit_memset(q, 0, sizeof q);
+}
+
+static void
+aesbear_cbc_dec(const struct aesdec *dec, const uint8_t in[static 16],
+ uint8_t out[static 16], size_t nbytes, uint8_t iv[static 16],
+ uint32_t nrounds)
+{
+ uint32_t sk_exp[120];
+ uint32_t q[8];
+ uint32_t cv0, cv1, cv2, cv3, iv0, iv1, iv2, iv3;
+
+ KASSERT(nbytes % 16 == 0);
+
+ /* Skip if there's nothing to do. */
+ if (nbytes == 0)
+ return;
+
+ /* Expand round keys for bitslicing. */
+ br_aes_ct_skey_expand(sk_exp, nrounds, dec->aesd_aes.aes_rk);
+
+ /* Load the IV. */
+ iv0 = le32dec(iv + 4*0);
+ iv1 = le32dec(iv + 4*1);
+ iv2 = le32dec(iv + 4*2);
+ iv3 = le32dec(iv + 4*3);
+
+ /* Load the last cipher block. */
+ cv0 = le32dec(in + nbytes - 16 + 4*0);
+ cv1 = le32dec(in + nbytes - 16 + 4*1);
+ cv2 = le32dec(in + nbytes - 16 + 4*2);
+ cv3 = le32dec(in + nbytes - 16 + 4*3);
+
+ /* Store the updated IV. */
+ le32enc(iv + 4*0, cv0);
+ le32enc(iv + 4*1, cv1);
+ le32enc(iv + 4*2, cv2);
+ le32enc(iv + 4*3, cv3);
+
+ /* Handle the last cipher block separately if odd number. */
+ if (nbytes % 32) {
+ KASSERT(nbytes % 32 == 16);
+
+ /* Set up the last cipher block and a garbage block. */
+ q[2*0] = cv0;
+ q[2*1] = cv1;
+ q[2*2] = cv2;
+ q[2*3] = cv3;
+ q[1] = q[3] = q[5] = q[7] = 0;
+
+ /* Encrypt. */
+ br_aes_ct_ortho(q);
+ br_aes_ct_bitslice_decrypt(nrounds, sk_exp, q);
+ br_aes_ct_ortho(q);
+
+ /* If this was the only cipher block, we're done. */
+ nbytes -= 16;
+ if (nbytes == 0)
+ goto out;
+
+ /*
+ * Otherwise, load up the penultimate cipher block, and
+ * store the output block.
+ */
+ cv0 = le32dec(in + nbytes - 16 + 4*0);
+ cv1 = le32dec(in + nbytes - 16 + 4*1);
+ cv2 = le32dec(in + nbytes - 16 + 4*2);
+ cv3 = le32dec(in + nbytes - 16 + 4*3);
+ le32enc(out + nbytes + 4*0, cv0 ^ q[2*0]);
+ le32enc(out + nbytes + 4*1, cv1 ^ q[2*1]);
+ le32enc(out + nbytes + 4*2, cv2 ^ q[2*2]);
+ le32enc(out + nbytes + 4*3, cv3 ^ q[2*3]);
+ }
+
+ for (;;) {
+ KASSERT(nbytes >= 32);
+
+ /*
+ * 1. Set up upper cipher block from cvN.
+ * 2. Load lower cipher block into cvN and set it up.
+ * 3. Decrypt.
+ */
+ q[2*0 + 1] = cv0;
+ q[2*1 + 1] = cv1;
+ q[2*2 + 1] = cv2;
+ q[2*3 + 1] = cv3;
+ cv0 = q[2*0] = le32dec(in + nbytes - 32 + 4*0);
+ cv1 = q[2*1] = le32dec(in + nbytes - 32 + 4*1);
+ cv2 = q[2*2] = le32dec(in + nbytes - 32 + 4*2);
+ cv3 = q[2*3] = le32dec(in + nbytes - 32 + 4*3);
+
+ br_aes_ct_ortho(q);
+ br_aes_ct_bitslice_decrypt(nrounds, sk_exp, q);
+ br_aes_ct_ortho(q);
+
+ /* Store the upper output block. */
+ le32enc(out + nbytes - 16 + 4*0, q[2*0 + 1] ^ cv0);
+ le32enc(out + nbytes - 16 + 4*1, q[2*1 + 1] ^ cv1);
+ le32enc(out + nbytes - 16 + 4*2, q[2*2 + 1] ^ cv2);
+ le32enc(out + nbytes - 16 + 4*3, q[2*3 + 1] ^ cv3);
+
+ /* Stop if we've reached the first output block. */
+ nbytes -= 32;
+ if (nbytes == 0)
+ goto out;
+
+ /*
+ * Load the preceding cipher block, and apply it as the
+ * chaining value to this one.
+ */
+ cv0 = le32dec(in + nbytes - 16 + 4*0);
+ cv1 = le32dec(in + nbytes - 16 + 4*1);
+ cv2 = le32dec(in + nbytes - 16 + 4*2);
+ cv3 = le32dec(in + nbytes - 16 + 4*3);
+ le32enc(out + nbytes + 4*0, q[2*0] ^ cv0);
+ le32enc(out + nbytes + 4*1, q[2*1] ^ cv1);
+ le32enc(out + nbytes + 4*2, q[2*2] ^ cv2);
+ le32enc(out + nbytes + 4*3, q[2*3] ^ cv3);
+ }
+
+out: /* Store the first output block. */
+ le32enc(out + 4*0, q[2*0] ^ iv0);
+ le32enc(out + 4*1, q[2*1] ^ iv1);
+ le32enc(out + 4*2, q[2*2] ^ iv2);
+ le32enc(out + 4*3, q[2*3] ^ iv3);
+
+ /* Paranoia: Zero temporary buffers. */
+ explicit_memset(sk_exp, 0, sizeof sk_exp);
+ explicit_memset(q, 0, sizeof q);
+}
+
+static inline void
+aesbear_xts_update(uint32_t *t0, uint32_t *t1, uint32_t *t2, uint32_t *t3)
+{
+ uint32_t s0, s1, s2, s3;
+
+ s0 = *t0 >> 31;
+ s1 = *t1 >> 31;
+ s2 = *t2 >> 31;
+ s3 = *t3 >> 31;
+ *t0 = (*t0 << 1) ^ (-s3 & 0x87);
+ *t1 = (*t1 << 1) ^ s0;
+ *t2 = (*t2 << 1) ^ s1;
+ *t3 = (*t3 << 1) ^ s2;
+}
+
+static int
+aesbear_xts_update_selftest(void)
+{
+ static const struct {
+ uint32_t in[4], out[4];
+ } cases[] = {
+ { {1}, {2} },
+ { {0x80000000U,0,0,0}, {0,1,0,0} },
+ { {0,0x80000000U,0,0}, {0,0,1,0} },
+ { {0,0,0x80000000U,0}, {0,0,0,1} },
+ { {0,0,0,0x80000000U}, {0x87,0,0,0} },
+ { {0,0x80000000U,0,0x80000000U}, {0x87,0,1,0} },
+ };
+ unsigned i;
+ uint32_t t0, t1, t2, t3;
+
+ for (i = 0; i < sizeof(cases)/sizeof(cases[0]); i++) {
+ t0 = cases[i].in[0];
+ t1 = cases[i].in[1];
+ t2 = cases[i].in[2];
+ t3 = cases[i].in[3];
+ aesbear_xts_update(&t0, &t1, &t2, &t3);
+ if (t0 != cases[i].out[0] ||
+ t1 != cases[i].out[1] ||
+ t2 != cases[i].out[2] ||
+ t3 != cases[i].out[3])
+ return -1;
+ }
+
+ /* Success! */
+ return 0;
+}
+
+static void
+aesbear_xts_enc(const struct aesenc *enc, const uint8_t in[static 16],
+ uint8_t out[static 16], size_t nbytes, uint8_t tweak[static 16],
+ uint32_t nrounds)
+{
+ uint32_t sk_exp[120];
+ uint32_t q[8];
+ uint32_t t0, t1, t2, t3, u0, u1, u2, u3;
+
+ KASSERT(nbytes % 16 == 0);
+
+ /* Skip if there's nothing to do. */
+ if (nbytes == 0)
+ return;
+
+ /* Expand round keys for bitslicing. */
+ br_aes_ct_skey_expand(sk_exp, nrounds, enc->aese_aes.aes_rk);
+
+ /* Load tweak. */
+ t0 = le32dec(tweak + 4*0);
+ t1 = le32dec(tweak + 4*1);
+ t2 = le32dec(tweak + 4*2);
+ t3 = le32dec(tweak + 4*3);
+
+ /* Handle the first block separately if odd number. */
+ if (nbytes % 32) {
+ KASSERT(nbytes % 32 == 16);
+
+ /* Load up the first block and a garbage block. */
+ q[2*0] = le32dec(in + 4*0) ^ t0;
+ q[2*1] = le32dec(in + 4*1) ^ t1;
+ q[2*2] = le32dec(in + 4*2) ^ t2;
+ q[2*3] = le32dec(in + 4*3) ^ t3;
+ q[1] = q[3] = q[5] = q[7] = 0;
+
+ /* Encrypt two blocks. */
+ br_aes_ct_ortho(q);
+ br_aes_ct_bitslice_encrypt(nrounds, sk_exp, q);
+ br_aes_ct_ortho(q);
+
+ /* Store the first cipher block. */
+ le32enc(out + 4*0, q[2*0] ^ t0);
+ le32enc(out + 4*1, q[2*1] ^ t1);
+ le32enc(out + 4*2, q[2*2] ^ t2);
+ le32enc(out + 4*3, q[2*3] ^ t3);
+
+ /* Advance to the next block. */
+ aesbear_xts_update(&t0, &t1, &t2, &t3);
+ if ((nbytes -= 16) == 0)
+ goto out;
+ in += 16;
+ out += 16;
+ }
+
+ do {
+ KASSERT(nbytes >= 32);
+
+ /* Compute the upper tweak. */
+ u0 = t0; u1 = t1; u2 = t2; u3 = t3;
+ aesbear_xts_update(&u0, &u1, &u2, &u3);
+
+ /* Load lower and upper blocks. */
+ q[2*0] = le32dec(in + 4*0) ^ t0;
+ q[2*1] = le32dec(in + 4*1) ^ t1;
+ q[2*2] = le32dec(in + 4*2) ^ t2;
+ q[2*3] = le32dec(in + 4*3) ^ t3;
+ q[2*0 + 1] = le32dec(in + 16 + 4*0) ^ u0;
+ q[2*1 + 1] = le32dec(in + 16 + 4*1) ^ u1;
+ q[2*2 + 1] = le32dec(in + 16 + 4*2) ^ u2;
+ q[2*3 + 1] = le32dec(in + 16 + 4*3) ^ u3;
+
+ /* Encrypt two blocks. */
+ br_aes_ct_ortho(q);
+ br_aes_ct_bitslice_encrypt(nrounds, sk_exp, q);
+ br_aes_ct_ortho(q);
+
+ /* Store lower and upper blocks. */
+ le32enc(out + 4*0, q[2*0] ^ t0);
+ le32enc(out + 4*1, q[2*1] ^ t1);
+ le32enc(out + 4*2, q[2*2] ^ t2);
+ le32enc(out + 4*3, q[2*3] ^ t3);
+ le32enc(out + 16 + 4*0, q[2*0 + 1] ^ u0);
+ le32enc(out + 16 + 4*1, q[2*1 + 1] ^ u1);
+ le32enc(out + 16 + 4*2, q[2*2 + 1] ^ u2);
+ le32enc(out + 16 + 4*3, q[2*3 + 1] ^ u3);
+
+ /* Advance to the next pair of blocks. */
+ t0 = u0; t1 = u1; t2 = u2; t3 = u3;
+ aesbear_xts_update(&t0, &t1, &t2, &t3);
+ in += 32;
+ out += 32;
+ } while (nbytes -= 32, nbytes);
+
+out: /* Store the updated tweak. */
+ le32enc(tweak + 4*0, t0);
+ le32enc(tweak + 4*1, t1);
+ le32enc(tweak + 4*2, t2);
+ le32enc(tweak + 4*3, t3);
+
+ /* Paranoia: Zero temporary buffers. */
+ explicit_memset(sk_exp, 0, sizeof sk_exp);
+ explicit_memset(q, 0, sizeof q);
+}
+
+static void
+aesbear_xts_dec(const struct aesdec *dec, const uint8_t in[static 16],
+ uint8_t out[static 16], size_t nbytes, uint8_t tweak[static 16],
+ uint32_t nrounds)
+{
+ uint32_t sk_exp[120];
+ uint32_t q[8];
+ uint32_t t0, t1, t2, t3, u0, u1, u2, u3;
+
+ KASSERT(nbytes % 16 == 0);
+
+ /* Skip if there's nothing to do. */
+ if (nbytes == 0)
+ return;
+
+ /* Expand round keys for bitslicing. */
+ br_aes_ct_skey_expand(sk_exp, nrounds, dec->aesd_aes.aes_rk);
+
+ /* Load tweak. */
+ t0 = le32dec(tweak + 4*0);
+ t1 = le32dec(tweak + 4*1);
+ t2 = le32dec(tweak + 4*2);
+ t3 = le32dec(tweak + 4*3);
+
+ /* Handle the first block separately if odd number. */
+ if (nbytes % 32) {
+ KASSERT(nbytes % 32 == 16);
+
+ /* Load up the first block and a garbage block. */
+ q[2*0] = le32dec(in + 4*0) ^ t0;
+ q[2*1] = le32dec(in + 4*1) ^ t1;
+ q[2*2] = le32dec(in + 4*2) ^ t2;
+ q[2*3] = le32dec(in + 4*3) ^ t3;
+ q[1] = q[3] = q[5] = q[7] = 0;
+
+ /* Decrypt two blocks. */
+ br_aes_ct_ortho(q);
+ br_aes_ct_bitslice_decrypt(nrounds, sk_exp, q);
+ br_aes_ct_ortho(q);
+
+ /* Store the first cipher block. */
+ le32enc(out + 4*0, q[2*0] ^ t0);
+ le32enc(out + 4*1, q[2*1] ^ t1);
+ le32enc(out + 4*2, q[2*2] ^ t2);
+ le32enc(out + 4*3, q[2*3] ^ t3);
+
+ /* Advance to the next block. */
+ aesbear_xts_update(&t0, &t1, &t2, &t3);
+ if ((nbytes -= 16) == 0)
+ goto out;
+ in += 16;
+ out += 16;
+ }
+
+ do {
+ KASSERT(nbytes >= 32);
+
+ /* Compute the upper tweak. */
+ u0 = t0; u1 = t1; u2 = t2; u3 = t3;
+ aesbear_xts_update(&u0, &u1, &u2, &u3);
+
+ /* Load lower and upper blocks. */
+ q[2*0] = le32dec(in + 4*0) ^ t0;
+ q[2*1] = le32dec(in + 4*1) ^ t1;
+ q[2*2] = le32dec(in + 4*2) ^ t2;
+ q[2*3] = le32dec(in + 4*3) ^ t3;
+ q[2*0 + 1] = le32dec(in + 16 + 4*0) ^ u0;
+ q[2*1 + 1] = le32dec(in + 16 + 4*1) ^ u1;
+ q[2*2 + 1] = le32dec(in + 16 + 4*2) ^ u2;
+ q[2*3 + 1] = le32dec(in + 16 + 4*3) ^ u3;
+
+ /* Encrypt two blocks. */
+ br_aes_ct_ortho(q);
+ br_aes_ct_bitslice_decrypt(nrounds, sk_exp, q);
+ br_aes_ct_ortho(q);
+
+ /* Store lower and upper blocks. */
+ le32enc(out + 4*0, q[2*0] ^ t0);
+ le32enc(out + 4*1, q[2*1] ^ t1);
+ le32enc(out + 4*2, q[2*2] ^ t2);
+ le32enc(out + 4*3, q[2*3] ^ t3);
+ le32enc(out + 16 + 4*0, q[2*0 + 1] ^ u0);
+ le32enc(out + 16 + 4*1, q[2*1 + 1] ^ u1);
+ le32enc(out + 16 + 4*2, q[2*2 + 1] ^ u2);
+ le32enc(out + 16 + 4*3, q[2*3 + 1] ^ u3);
+
+ /* Advance to the next pair of blocks. */
+ t0 = u0; t1 = u1; t2 = u2; t3 = u3;
+ aesbear_xts_update(&t0, &t1, &t2, &t3);
+ in += 32;
+ out += 32;
+ } while (nbytes -= 32, nbytes);
+
+out: /* Store the updated tweak. */
+ le32enc(tweak + 4*0, t0);
+ le32enc(tweak + 4*1, t1);
+ le32enc(tweak + 4*2, t2);
+ le32enc(tweak + 4*3, t3);
+
+ /* Paranoia: Zero temporary buffers. */
+ explicit_memset(sk_exp, 0, sizeof sk_exp);
+ explicit_memset(q, 0, sizeof q);
+}
+
+static int
+aesbear_probe(void)
+{
+
+ if (aesbear_xts_update_selftest())
+ return -1;
+
+ /* XXX test br_aes_ct_bitslice_decrypt */
+ /* XXX test br_aes_ct_bitslice_encrypt */
+ /* XXX test br_aes_ct_keysched */
+ /* XXX test br_aes_ct_ortho */
+ /* XXX test br_aes_ct_skey_expand */
+
+ return 0;
+}
+
+struct aes_impl aes_bear_impl = {
+ .ai_name = "BearSSL aes_ct",
+ .ai_probe = aesbear_probe,
+ .ai_setenckey = aesbear_setenckey,
+ .ai_setdeckey = aesbear_setdeckey,
+ .ai_enc = aesbear_enc,
+ .ai_dec = aesbear_dec,
+ .ai_cbc_enc = aesbear_cbc_enc,
+ .ai_cbc_dec = aesbear_cbc_dec,
+ .ai_xts_enc = aesbear_xts_enc,
+ .ai_xts_dec = aesbear_xts_dec,
+};
diff -r 81a487955535 -r 9d6b84c40f65 sys/crypto/aes/aes_bear.h
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/sys/crypto/aes/aes_bear.h Fri Jun 12 05:16:46 2020 +0000
@@ -0,0 +1,50 @@
+/* $NetBSD$ */
+
+/*-
+ * Copyright (c) 2020 The NetBSD Foundation, Inc.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND CONTRIBUTORS
+ * ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
+ * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+ * POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _CRYPTO_AES_AES_BEAR_H
+#define _CRYPTO_AES_AES_BEAR_H
+
+#include <sys/types.h>
+#include <sys/endian.h>
+
+#include <crypto/aes/aes.h>
+
+#define br_dec32le le32dec
+#define br_enc32le le32enc
+
+void br_aes_ct_bitslice_Sbox(uint32_t *);
+void br_aes_ct_bitslice_invSbox(uint32_t *);
+void br_aes_ct_ortho(uint32_t *);
+u_int br_aes_ct_keysched(uint32_t *, const void *, size_t);
+void br_aes_ct_skey_expand(uint32_t *, unsigned, const uint32_t *);
+void br_aes_ct_bitslice_encrypt(unsigned, const uint32_t *, uint32_t *);
+void br_aes_ct_bitslice_decrypt(unsigned, const uint32_t *, uint32_t *);
+
+extern struct aes_impl aes_bear_impl;
+
+#endif /* _CRYPTO_AES_AES_BEAR_H */
diff -r 81a487955535 -r 9d6b84c40f65 sys/crypto/aes/aes_ct.c
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/sys/crypto/aes/aes_ct.c Fri Jun 12 05:16:46 2020 +0000
@@ -0,0 +1,335 @@
+/* $NetBSD$ */
+
+/*
+ * Copyright (c) 2016 Thomas Pornin <pornin%bolet.org@localhost>
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining
+ * a copy of this software and associated documentation files (the
+ * "Software"), to deal in the Software without restriction, including
+ * without limitation the rights to use, copy, modify, merge, publish,
+ * distribute, sublicense, and/or sell copies of the Software, and to
+ * permit persons to whom the Software is furnished to do so, subject to
+ * the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be
+ * included in all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#include <sys/cdefs.h>
+__KERNEL_RCSID(1, "$NetBSD$");
+
+#include <sys/types.h>
+
+#include <crypto/aes/aes_bear.h>
+
+/* see inner.h */
+void
+br_aes_ct_bitslice_Sbox(uint32_t *q)
+{
+ /*
+ * This S-box implementation is a straightforward translation of
+ * the circuit described by Boyar and Peralta in "A new
+ * combinational logic minimization technique with applications
+ * to cryptology" (https://eprint.iacr.org/2009/191.pdf).
+ *
+ * Note that variables x* (input) and s* (output) are numbered
+ * in "reverse" order (x0 is the high bit, x7 is the low bit).
+ */
+
+ uint32_t x0, x1, x2, x3, x4, x5, x6, x7;
+ uint32_t y1, y2, y3, y4, y5, y6, y7, y8, y9;
+ uint32_t y10, y11, y12, y13, y14, y15, y16, y17, y18, y19;
+ uint32_t y20, y21;
+ uint32_t z0, z1, z2, z3, z4, z5, z6, z7, z8, z9;
+ uint32_t z10, z11, z12, z13, z14, z15, z16, z17;
+ uint32_t t0, t1, t2, t3, t4, t5, t6, t7, t8, t9;
+ uint32_t t10, t11, t12, t13, t14, t15, t16, t17, t18, t19;
+ uint32_t t20, t21, t22, t23, t24, t25, t26, t27, t28, t29;
+ uint32_t t30, t31, t32, t33, t34, t35, t36, t37, t38, t39;
+ uint32_t t40, t41, t42, t43, t44, t45, t46, t47, t48, t49;
+ uint32_t t50, t51, t52, t53, t54, t55, t56, t57, t58, t59;
+ uint32_t t60, t61, t62, t63, t64, t65, t66, t67;
+ uint32_t s0, s1, s2, s3, s4, s5, s6, s7;
+
+ x0 = q[7];
+ x1 = q[6];
+ x2 = q[5];
+ x3 = q[4];
+ x4 = q[3];
+ x5 = q[2];
+ x6 = q[1];
+ x7 = q[0];
+
+ /*
+ * Top linear transformation.
+ */
+ y14 = x3 ^ x5;
+ y13 = x0 ^ x6;
+ y9 = x0 ^ x3;
+ y8 = x0 ^ x5;
+ t0 = x1 ^ x2;
+ y1 = t0 ^ x7;
+ y4 = y1 ^ x3;
+ y12 = y13 ^ y14;
+ y2 = y1 ^ x0;
+ y5 = y1 ^ x6;
+ y3 = y5 ^ y8;
+ t1 = x4 ^ y12;
+ y15 = t1 ^ x5;
+ y20 = t1 ^ x1;
+ y6 = y15 ^ x7;
+ y10 = y15 ^ t0;
+ y11 = y20 ^ y9;
+ y7 = x7 ^ y11;
+ y17 = y10 ^ y11;
+ y19 = y10 ^ y8;
+ y16 = t0 ^ y11;
+ y21 = y13 ^ y16;
+ y18 = x0 ^ y16;
+
+ /*
+ * Non-linear section.
+ */
+ t2 = y12 & y15;
+ t3 = y3 & y6;
+ t4 = t3 ^ t2;
+ t5 = y4 & x7;
+ t6 = t5 ^ t2;
+ t7 = y13 & y16;
+ t8 = y5 & y1;
+ t9 = t8 ^ t7;
+ t10 = y2 & y7;
+ t11 = t10 ^ t7;
+ t12 = y9 & y11;
+ t13 = y14 & y17;
+ t14 = t13 ^ t12;
+ t15 = y8 & y10;
+ t16 = t15 ^ t12;
+ t17 = t4 ^ t14;
+ t18 = t6 ^ t16;
+ t19 = t9 ^ t14;
+ t20 = t11 ^ t16;
+ t21 = t17 ^ y20;
+ t22 = t18 ^ y19;
+ t23 = t19 ^ y21;
+ t24 = t20 ^ y18;
+
+ t25 = t21 ^ t22;
+ t26 = t21 & t23;
+ t27 = t24 ^ t26;
+ t28 = t25 & t27;
+ t29 = t28 ^ t22;
+ t30 = t23 ^ t24;
+ t31 = t22 ^ t26;
+ t32 = t31 & t30;
+ t33 = t32 ^ t24;
+ t34 = t23 ^ t33;
+ t35 = t27 ^ t33;
+ t36 = t24 & t35;
+ t37 = t36 ^ t34;
+ t38 = t27 ^ t36;
+ t39 = t29 & t38;
+ t40 = t25 ^ t39;
+
+ t41 = t40 ^ t37;
+ t42 = t29 ^ t33;
+ t43 = t29 ^ t40;
+ t44 = t33 ^ t37;
+ t45 = t42 ^ t41;
+ z0 = t44 & y15;
+ z1 = t37 & y6;
+ z2 = t33 & x7;
+ z3 = t43 & y16;
+ z4 = t40 & y1;
+ z5 = t29 & y7;
+ z6 = t42 & y11;
+ z7 = t45 & y17;
+ z8 = t41 & y10;
+ z9 = t44 & y12;
+ z10 = t37 & y3;
+ z11 = t33 & y4;
+ z12 = t43 & y13;
+ z13 = t40 & y5;
+ z14 = t29 & y2;
+ z15 = t42 & y9;
+ z16 = t45 & y14;
+ z17 = t41 & y8;
+
+ /*
+ * Bottom linear transformation.
+ */
+ t46 = z15 ^ z16;
+ t47 = z10 ^ z11;
+ t48 = z5 ^ z13;
+ t49 = z9 ^ z10;
+ t50 = z2 ^ z12;
+ t51 = z2 ^ z5;
+ t52 = z7 ^ z8;
+ t53 = z0 ^ z3;
+ t54 = z6 ^ z7;
+ t55 = z16 ^ z17;
+ t56 = z12 ^ t48;
+ t57 = t50 ^ t53;
+ t58 = z4 ^ t46;
+ t59 = z3 ^ t54;
+ t60 = t46 ^ t57;
+ t61 = z14 ^ t57;
+ t62 = t52 ^ t58;
+ t63 = t49 ^ t58;
+ t64 = z4 ^ t59;
+ t65 = t61 ^ t62;
+ t66 = z1 ^ t63;
+ s0 = t59 ^ t63;
+ s6 = t56 ^ ~t62;
+ s7 = t48 ^ ~t60;
+ t67 = t64 ^ t65;
+ s3 = t53 ^ t66;
+ s4 = t51 ^ t66;
+ s5 = t47 ^ t65;
+ s1 = t64 ^ ~s3;
+ s2 = t55 ^ ~t67;
+
+ q[7] = s0;
+ q[6] = s1;
+ q[5] = s2;
+ q[4] = s3;
+ q[3] = s4;
+ q[2] = s5;
+ q[1] = s6;
+ q[0] = s7;
+}
+
+/* see inner.h */
+void
+br_aes_ct_ortho(uint32_t *q)
+{
+#define SWAPN(cl, ch, s, x, y) do { \
+ uint32_t a, b; \
+ a = (x); \
+ b = (y); \
+ (x) = (a & (uint32_t)cl) | ((b & (uint32_t)cl) << (s)); \
+ (y) = ((a & (uint32_t)ch) >> (s)) | (b & (uint32_t)ch); \
+ } while (0)
+
+#define SWAP2(x, y) SWAPN(0x55555555, 0xAAAAAAAA, 1, x, y)
+#define SWAP4(x, y) SWAPN(0x33333333, 0xCCCCCCCC, 2, x, y)
+#define SWAP8(x, y) SWAPN(0x0F0F0F0F, 0xF0F0F0F0, 4, x, y)
+
+ SWAP2(q[0], q[1]);
+ SWAP2(q[2], q[3]);
+ SWAP2(q[4], q[5]);
+ SWAP2(q[6], q[7]);
+
+ SWAP4(q[0], q[2]);
+ SWAP4(q[1], q[3]);
+ SWAP4(q[4], q[6]);
+ SWAP4(q[5], q[7]);
+
+ SWAP8(q[0], q[4]);
+ SWAP8(q[1], q[5]);
+ SWAP8(q[2], q[6]);
+ SWAP8(q[3], q[7]);
+}
+
+static const unsigned char Rcon[] = {
+ 0x01, 0x02, 0x04, 0x08, 0x10, 0x20, 0x40, 0x80, 0x1B, 0x36
+};
+
+static uint32_t
+sub_word(uint32_t x)
+{
+ uint32_t q[8];
+ int i;
+
+ for (i = 0; i < 8; i ++) {
+ q[i] = x;
+ }
+ br_aes_ct_ortho(q);
+ br_aes_ct_bitslice_Sbox(q);
+ br_aes_ct_ortho(q);
+ return q[0];
+}
+
+/* see inner.h */
+unsigned
+br_aes_ct_keysched(uint32_t *comp_skey, const void *key, size_t key_len)
+{
+ unsigned num_rounds;
+ int i, j, k, nk, nkf;
+ uint32_t tmp;
+ uint32_t skey[120];
+
+ switch (key_len) {
+ case 16:
+ num_rounds = 10;
+ break;
+ case 24:
+ num_rounds = 12;
+ break;
+ case 32:
+ num_rounds = 14;
+ break;
+ default:
+ /* abort(); */
+ return 0;
+ }
+ nk = (int)(key_len >> 2);
+ nkf = (int)((num_rounds + 1) << 2);
+ tmp = 0;
+ for (i = 0; i < nk; i ++) {
+ tmp = br_dec32le((const unsigned char *)key + (i << 2));
+ skey[(i << 1) + 0] = tmp;
+ skey[(i << 1) + 1] = tmp;
+ }
+ for (i = nk, j = 0, k = 0; i < nkf; i ++) {
+ if (j == 0) {
+ tmp = (tmp << 24) | (tmp >> 8);
+ tmp = sub_word(tmp) ^ Rcon[k];
+ } else if (nk > 6 && j == 4) {
+ tmp = sub_word(tmp);
+ }
+ tmp ^= skey[(i - nk) << 1];
+ skey[(i << 1) + 0] = tmp;
+ skey[(i << 1) + 1] = tmp;
+ if (++ j == nk) {
+ j = 0;
+ k ++;
+ }
+ }
+ for (i = 0; i < nkf; i += 4) {
+ br_aes_ct_ortho(skey + (i << 1));
+ }
+ for (i = 0, j = 0; i < nkf; i ++, j += 2) {
+ comp_skey[i] = (skey[j + 0] & 0x55555555)
+ | (skey[j + 1] & 0xAAAAAAAA);
+ }
+ return num_rounds;
+}
+
+/* see inner.h */
+void
+br_aes_ct_skey_expand(uint32_t *skey,
+ unsigned num_rounds, const uint32_t *comp_skey)
+{
+ unsigned u, v, n;
+
+ n = (num_rounds + 1) << 2;
+ for (u = 0, v = 0; u < n; u ++, v += 2) {
+ uint32_t x, y;
+
+ x = y = comp_skey[u];
+ x &= 0x55555555;
+ skey[v + 0] = x | (x << 1);
+ y &= 0xAAAAAAAA;
+ skey[v + 1] = y | (y >> 1);
+ }
+}
diff -r 81a487955535 -r 9d6b84c40f65 sys/crypto/aes/aes_ct_dec.c
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/sys/crypto/aes/aes_ct_dec.c Fri Jun 12 05:16:46 2020 +0000
@@ -0,0 +1,177 @@
+/* $NetBSD$ */
+
+/*
+ * Copyright (c) 2016 Thomas Pornin <pornin%bolet.org@localhost>
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining
+ * a copy of this software and associated documentation files (the
+ * "Software"), to deal in the Software without restriction, including
+ * without limitation the rights to use, copy, modify, merge, publish,
+ * distribute, sublicense, and/or sell copies of the Software, and to
+ * permit persons to whom the Software is furnished to do so, subject to
+ * the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be
+ * included in all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#include <sys/cdefs.h>
+__KERNEL_RCSID(1, "$NetBSD$");
+
+#include <sys/types.h>
+
+#include <crypto/aes/aes_bear.h>
+
+/* see inner.h */
+void
+br_aes_ct_bitslice_invSbox(uint32_t *q)
+{
+ /*
+ * AES S-box is:
+ * S(x) = A(I(x)) ^ 0x63
+ * where I() is inversion in GF(256), and A() is a linear
+ * transform (0 is formally defined to be its own inverse).
+ * Since inversion is an involution, the inverse S-box can be
+ * computed from the S-box as:
+ * iS(x) = B(S(B(x ^ 0x63)) ^ 0x63)
+ * where B() is the inverse of A(). Indeed, for any y in GF(256):
+ * iS(S(y)) = B(A(I(B(A(I(y)) ^ 0x63 ^ 0x63))) ^ 0x63 ^ 0x63) = y
+ *
+ * Note: we reuse the implementation of the forward S-box,
+ * instead of duplicating it here, so that total code size is
+ * lower. By merging the B() transforms into the S-box circuit
+ * we could make faster CBC decryption, but CBC decryption is
+ * already quite faster than CBC encryption because we can
+ * process two blocks in parallel.
+ */
+ uint32_t q0, q1, q2, q3, q4, q5, q6, q7;
+
+ q0 = ~q[0];
+ q1 = ~q[1];
+ q2 = q[2];
+ q3 = q[3];
+ q4 = q[4];
+ q5 = ~q[5];
+ q6 = ~q[6];
+ q7 = q[7];
+ q[7] = q1 ^ q4 ^ q6;
+ q[6] = q0 ^ q3 ^ q5;
+ q[5] = q7 ^ q2 ^ q4;
+ q[4] = q6 ^ q1 ^ q3;
+ q[3] = q5 ^ q0 ^ q2;
+ q[2] = q4 ^ q7 ^ q1;
+ q[1] = q3 ^ q6 ^ q0;
+ q[0] = q2 ^ q5 ^ q7;
+
+ br_aes_ct_bitslice_Sbox(q);
+
+ q0 = ~q[0];
+ q1 = ~q[1];
+ q2 = q[2];
+ q3 = q[3];
+ q4 = q[4];
+ q5 = ~q[5];
+ q6 = ~q[6];
+ q7 = q[7];
+ q[7] = q1 ^ q4 ^ q6;
+ q[6] = q0 ^ q3 ^ q5;
+ q[5] = q7 ^ q2 ^ q4;
+ q[4] = q6 ^ q1 ^ q3;
+ q[3] = q5 ^ q0 ^ q2;
+ q[2] = q4 ^ q7 ^ q1;
+ q[1] = q3 ^ q6 ^ q0;
+ q[0] = q2 ^ q5 ^ q7;
+}
+
+static void
+add_round_key(uint32_t *q, const uint32_t *sk)
+{
+ int i;
+
+ for (i = 0; i < 8; i ++) {
+ q[i] ^= sk[i];
+ }
+}
+
+static void
+inv_shift_rows(uint32_t *q)
+{
+ int i;
+
+ for (i = 0; i < 8; i ++) {
+ uint32_t x;
+
+ x = q[i];
+ q[i] = (x & 0x000000FF)
+ | ((x & 0x00003F00) << 2) | ((x & 0x0000C000) >> 6)
+ | ((x & 0x000F0000) << 4) | ((x & 0x00F00000) >> 4)
+ | ((x & 0x03000000) << 6) | ((x & 0xFC000000) >> 2);
+ }
+}
+
+static inline uint32_t
+rotr16(uint32_t x)
+{
+ return (x << 16) | (x >> 16);
+}
+
+static void
+inv_mix_columns(uint32_t *q)
+{
+ uint32_t q0, q1, q2, q3, q4, q5, q6, q7;
+ uint32_t r0, r1, r2, r3, r4, r5, r6, r7;
+
+ q0 = q[0];
+ q1 = q[1];
+ q2 = q[2];
+ q3 = q[3];
+ q4 = q[4];
+ q5 = q[5];
+ q6 = q[6];
+ q7 = q[7];
+ r0 = (q0 >> 8) | (q0 << 24);
+ r1 = (q1 >> 8) | (q1 << 24);
+ r2 = (q2 >> 8) | (q2 << 24);
+ r3 = (q3 >> 8) | (q3 << 24);
+ r4 = (q4 >> 8) | (q4 << 24);
+ r5 = (q5 >> 8) | (q5 << 24);
+ r6 = (q6 >> 8) | (q6 << 24);
+ r7 = (q7 >> 8) | (q7 << 24);
+
+ q[0] = q5 ^ q6 ^ q7 ^ r0 ^ r5 ^ r7 ^ rotr16(q0 ^ q5 ^ q6 ^ r0 ^ r5);
+ q[1] = q0 ^ q5 ^ r0 ^ r1 ^ r5 ^ r6 ^ r7 ^ rotr16(q1 ^ q5 ^ q7 ^ r1 ^ r5 ^ r6);
+ q[2] = q0 ^ q1 ^ q6 ^ r1 ^ r2 ^ r6 ^ r7 ^ rotr16(q0 ^ q2 ^ q6 ^ r2 ^ r6 ^ r7);
+ q[3] = q0 ^ q1 ^ q2 ^ q5 ^ q6 ^ r0 ^ r2 ^ r3 ^ r5 ^ rotr16(q0 ^ q1 ^ q3 ^ q5 ^ q6 ^ q7 ^ r0 ^ r3 ^ r5 ^ r7);
+ q[4] = q1 ^ q2 ^ q3 ^ q5 ^ r1 ^ r3 ^ r4 ^ r5 ^ r6 ^ r7 ^ rotr16(q1 ^ q2 ^ q4 ^ q5 ^ q7 ^ r1 ^ r4 ^ r5 ^ r6);
+ q[5] = q2 ^ q3 ^ q4 ^ q6 ^ r2 ^ r4 ^ r5 ^ r6 ^ r7 ^ rotr16(q2 ^ q3 ^ q5 ^ q6 ^ r2 ^ r5 ^ r6 ^ r7);
+ q[6] = q3 ^ q4 ^ q5 ^ q7 ^ r3 ^ r5 ^ r6 ^ r7 ^ rotr16(q3 ^ q4 ^ q6 ^ q7 ^ r3 ^ r6 ^ r7);
+ q[7] = q4 ^ q5 ^ q6 ^ r4 ^ r6 ^ r7 ^ rotr16(q4 ^ q5 ^ q7 ^ r4 ^ r7);
+}
+
+/* see inner.h */
+void
+br_aes_ct_bitslice_decrypt(unsigned num_rounds,
+ const uint32_t *skey, uint32_t *q)
+{
+ unsigned u;
+
+ add_round_key(q, skey + (num_rounds << 3));
+ for (u = num_rounds - 1; u > 0; u --) {
+ inv_shift_rows(q);
+ br_aes_ct_bitslice_invSbox(q);
+ add_round_key(q, skey + (u << 3));
+ inv_mix_columns(q);
+ }
+ inv_shift_rows(q);
+ br_aes_ct_bitslice_invSbox(q);
+ add_round_key(q, skey);
+}
diff -r 81a487955535 -r 9d6b84c40f65 sys/crypto/aes/aes_ct_enc.c
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/sys/crypto/aes/aes_ct_enc.c Fri Jun 12 05:16:46 2020 +0000
@@ -0,0 +1,119 @@
+/* $NetBSD$ */
+
+/*
+ * Copyright (c) 2016 Thomas Pornin <pornin%bolet.org@localhost>
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining
+ * a copy of this software and associated documentation files (the
+ * "Software"), to deal in the Software without restriction, including
+ * without limitation the rights to use, copy, modify, merge, publish,
+ * distribute, sublicense, and/or sell copies of the Software, and to
+ * permit persons to whom the Software is furnished to do so, subject to
+ * the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be
+ * included in all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#include <sys/cdefs.h>
+__KERNEL_RCSID(1, "$NetBSD$");
+
+#include <sys/types.h>
+
+#include <crypto/aes/aes_bear.h>
+
+static inline void
+add_round_key(uint32_t *q, const uint32_t *sk)
+{
+ q[0] ^= sk[0];
+ q[1] ^= sk[1];
+ q[2] ^= sk[2];
+ q[3] ^= sk[3];
+ q[4] ^= sk[4];
+ q[5] ^= sk[5];
+ q[6] ^= sk[6];
+ q[7] ^= sk[7];
+}
+
+static inline void
+shift_rows(uint32_t *q)
+{
+ int i;
+
+ for (i = 0; i < 8; i ++) {
+ uint32_t x;
+
+ x = q[i];
+ q[i] = (x & 0x000000FF)
+ | ((x & 0x0000FC00) >> 2) | ((x & 0x00000300) << 6)
+ | ((x & 0x00F00000) >> 4) | ((x & 0x000F0000) << 4)
+ | ((x & 0xC0000000) >> 6) | ((x & 0x3F000000) << 2);
+ }
+}
+
+static inline uint32_t
+rotr16(uint32_t x)
+{
+ return (x << 16) | (x >> 16);
+}
+
+static inline void
+mix_columns(uint32_t *q)
+{
+ uint32_t q0, q1, q2, q3, q4, q5, q6, q7;
+ uint32_t r0, r1, r2, r3, r4, r5, r6, r7;
+
+ q0 = q[0];
+ q1 = q[1];
+ q2 = q[2];
+ q3 = q[3];
+ q4 = q[4];
+ q5 = q[5];
+ q6 = q[6];
+ q7 = q[7];
+ r0 = (q0 >> 8) | (q0 << 24);
+ r1 = (q1 >> 8) | (q1 << 24);
+ r2 = (q2 >> 8) | (q2 << 24);
+ r3 = (q3 >> 8) | (q3 << 24);
+ r4 = (q4 >> 8) | (q4 << 24);
+ r5 = (q5 >> 8) | (q5 << 24);
+ r6 = (q6 >> 8) | (q6 << 24);
+ r7 = (q7 >> 8) | (q7 << 24);
+
+ q[0] = q7 ^ r7 ^ r0 ^ rotr16(q0 ^ r0);
+ q[1] = q0 ^ r0 ^ q7 ^ r7 ^ r1 ^ rotr16(q1 ^ r1);
+ q[2] = q1 ^ r1 ^ r2 ^ rotr16(q2 ^ r2);
+ q[3] = q2 ^ r2 ^ q7 ^ r7 ^ r3 ^ rotr16(q3 ^ r3);
+ q[4] = q3 ^ r3 ^ q7 ^ r7 ^ r4 ^ rotr16(q4 ^ r4);
+ q[5] = q4 ^ r4 ^ r5 ^ rotr16(q5 ^ r5);
+ q[6] = q5 ^ r5 ^ r6 ^ rotr16(q6 ^ r6);
+ q[7] = q6 ^ r6 ^ r7 ^ rotr16(q7 ^ r7);
+}
+
+/* see inner.h */
+void
+br_aes_ct_bitslice_encrypt(unsigned num_rounds,
+ const uint32_t *skey, uint32_t *q)
+{
+ unsigned u;
+
+ add_round_key(q, skey);
+ for (u = 1; u < num_rounds; u ++) {
+ br_aes_ct_bitslice_Sbox(q);
+ shift_rows(q);
+ mix_columns(q);
+ add_round_key(q, skey + (u << 3));
+ }
+ br_aes_ct_bitslice_Sbox(q);
+ shift_rows(q);
+ add_round_key(q, skey + (num_rounds << 3));
+}
diff -r 81a487955535 -r 9d6b84c40f65 sys/crypto/aes/aes_impl.c
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/sys/crypto/aes/aes_impl.c Fri Jun 12 05:16:46 2020 +0000
@@ -0,0 +1,256 @@
+/* $NetBSD$ */
+
+/*-
+ * Copyright (c) 2020 The NetBSD Foundation, Inc.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND CONTRIBUTORS
+ * ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
+ * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+ * POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <sys/cdefs.h>
+__KERNEL_RCSID(1, "$NetBSD$");
+
+#include <sys/types.h>
+#include <sys/kernel.h>
+#include <sys/module.h>
+#include <sys/once.h>
+#include <sys/systm.h>
+
+#include <crypto/aes/aes.h>
+#include <crypto/aes/aes_bear.h> /* default implementation */
+
+static const struct aes_impl *aes_md_impl __read_mostly;
+static const struct aes_impl *aes_impl __read_mostly;
+
+/*
+ * The timing of AES implementation selection is finicky:
+ *
+ * 1. It has to be done _after_ cpu_attach for implementations,
+ * such as AES-NI, that rely on fpu initialization done by
+ * fpu_attach.
+ *
+ * 2. It has to be done _before_ the cgd self-tests or anything
+ * else that might call AES.
+ *
+ * For the moment, doing it in module init works. However, if a
+ * driver-class module depended on the aes module, that would break.
+ */
+
+static int
+aes_select(void)
+{
+
+ KASSERT(aes_impl == NULL);
+
+ if (aes_md_impl) {
+ if (aes_selftest(aes_md_impl))
+ aprint_error("aes: self-test failed: %s\n",
+ aes_md_impl->ai_name);
+ else
+ aes_impl = aes_md_impl;
+ }
+ if (aes_impl == NULL) {
+ if (aes_selftest(&aes_bear_impl))
+ aprint_error("aes: self-test failed: %s\n",
+ aes_bear_impl.ai_name);
+ else
+ aes_impl = &aes_bear_impl;
+ }
+ if (aes_impl == NULL)
+ panic("AES self-tests failed");
+
+ aprint_verbose("aes: %s\n", aes_impl->ai_name);
+ return 0;
+}
+
+MODULE(MODULE_CLASS_MISC, aes, NULL);
+
+static int
+aes_modcmd(modcmd_t cmd, void *opaque)
+{
+
+ switch (cmd) {
+ case MODULE_CMD_INIT:
+ return aes_select();
+ case MODULE_CMD_FINI:
+ return 0;
+ default:
+ return ENOTTY;
+ }
+}
+
+static void
+aes_guarantee_selected(void)
+{
+#if 0
+ static once_t once;
+ int error;
+
+ error = RUN_ONCE(&once, aes_select);
+ KASSERT(error == 0);
+#endif
+}
+
+void
+aes_md_init(const struct aes_impl *impl)
+{
+
+ KASSERT(cold);
+ KASSERTMSG(aes_impl == NULL,
+ "AES implementation `%s' already chosen, can't offer `%s'",
+ aes_impl->ai_name, impl->ai_name);
+ KASSERTMSG(aes_md_impl == NULL,
+ "AES implementation `%s' already offered, can't offer `%s'",
+ aes_md_impl->ai_name, impl->ai_name);
+
+ aes_md_impl = impl;
+}
+
+static void
+aes_setenckey(struct aesenc *enc, const uint8_t key[static 16],
+ uint32_t nrounds)
+{
+
+ aes_guarantee_selected();
+ aes_impl->ai_setenckey(enc, key, nrounds);
+}
+
+uint32_t
+aes_setenckey128(struct aesenc *enc, const uint8_t key[static 16])
+{
+ uint32_t nrounds = AES_128_NROUNDS;
+
+ aes_setenckey(enc, key, nrounds);
+ return nrounds;
+}
+
+uint32_t
+aes_setenckey192(struct aesenc *enc, const uint8_t key[static 24])
+{
+ uint32_t nrounds = AES_192_NROUNDS;
+
+ aes_setenckey(enc, key, nrounds);
+ return nrounds;
+}
+
+uint32_t
+aes_setenckey256(struct aesenc *enc, const uint8_t key[static 32])
+{
+ uint32_t nrounds = AES_256_NROUNDS;
+
+ aes_setenckey(enc, key, nrounds);
+ return nrounds;
+}
+
+static void
+aes_setdeckey(struct aesdec *dec, const uint8_t key[static 16],
+ uint32_t nrounds)
+{
+
+ aes_guarantee_selected();
+ aes_impl->ai_setdeckey(dec, key, nrounds);
+}
+
+uint32_t
+aes_setdeckey128(struct aesdec *dec, const uint8_t key[static 16])
+{
+ uint32_t nrounds = AES_128_NROUNDS;
+
+ aes_setdeckey(dec, key, nrounds);
+ return nrounds;
+}
+
+uint32_t
+aes_setdeckey192(struct aesdec *dec, const uint8_t key[static 24])
+{
+ uint32_t nrounds = AES_192_NROUNDS;
+
+ aes_setdeckey(dec, key, nrounds);
+ return nrounds;
+}
+
+uint32_t
+aes_setdeckey256(struct aesdec *dec, const uint8_t key[static 32])
+{
+ uint32_t nrounds = AES_256_NROUNDS;
+
+ aes_setdeckey(dec, key, nrounds);
+ return nrounds;
+}
+
+void
+aes_enc(const struct aesenc *enc, const uint8_t in[static 16],
+ uint8_t out[static 16], uint32_t nrounds)
+{
+
+ aes_guarantee_selected();
+ aes_impl->ai_enc(enc, in, out, nrounds);
+}
+
+void
+aes_dec(const struct aesdec *dec, const uint8_t in[static 16],
+ uint8_t out[static 16], uint32_t nrounds)
+{
+
+ aes_guarantee_selected();
+ aes_impl->ai_dec(dec, in, out, nrounds);
+}
+
+void
+aes_cbc_enc(struct aesenc *enc, const uint8_t in[static 16],
+ uint8_t out[static 16], size_t nbytes, uint8_t iv[static 16],
+ uint32_t nrounds)
+{
+
+ aes_guarantee_selected();
+ aes_impl->ai_cbc_enc(enc, in, out, nbytes, iv, nrounds);
+}
+
+void
+aes_cbc_dec(struct aesdec *dec, const uint8_t in[static 16],
+ uint8_t out[static 16], size_t nbytes, uint8_t iv[static 16],
+ uint32_t nrounds)
+{
+
+ aes_guarantee_selected();
+ aes_impl->ai_cbc_dec(dec, in, out, nbytes, iv, nrounds);
+}
+
+void
+aes_xts_enc(struct aesenc *enc, const uint8_t in[static 16],
+ uint8_t out[static 16], size_t nbytes, uint8_t tweak[static 16],
+ uint32_t nrounds)
+{
+
+ aes_guarantee_selected();
+ aes_impl->ai_xts_enc(enc, in, out, nbytes, tweak, nrounds);
+}
+
+void
+aes_xts_dec(struct aesdec *dec, const uint8_t in[static 16],
+ uint8_t out[static 16], size_t nbytes, uint8_t tweak[static 16],
+ uint32_t nrounds)
+{
+
+ aes_guarantee_selected();
+ aes_impl->ai_xts_dec(dec, in, out, nbytes, tweak, nrounds);
+}
diff -r 81a487955535 -r 9d6b84c40f65 sys/crypto/aes/aes_rijndael.c
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/sys/crypto/aes/aes_rijndael.c Fri Jun 12 05:16:46 2020 +0000
@@ -0,0 +1,306 @@
+/* $NetBSD$ */
+
+/*-
+ * Copyright (c) 2020 The NetBSD Foundation, Inc.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND CONTRIBUTORS
+ * ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
+ * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+ * POSSIBILITY OF SUCH DAMAGE.
+ */
+
+/*
+ * Legacy `Rijndael' API
+ *
+ * rijndael_set_key
+ * rijndael_encrypt
+ * rijndael_decrypt
+ *
+ * rijndaelKeySetupEnc
+ * rijndaelKeySetupDec
+ * rijndaelEncrypt
+ * rijndaelDecrypt
+ * rijndael_makeKey
+ * rijndael_cipherInit
+ * rijndael_blockEncrypt
+ * rijndael_blockDecrypt
+ */
+
+#include <sys/cdefs.h>
+__KERNEL_RCSID(1, "$NetBSD$");
+
+#include <sys/types.h>
+#include <sys/systm.h>
+
+#include <crypto/aes/aes.h>
+#include <crypto/rijndael/rijndael.h>
+#include <crypto/rijndael/rijndael-alg-fst.h>
+#include <crypto/rijndael/rijndael-api-fst.h>
+
+void
+rijndael_set_key(rijndael_ctx *ctx, const uint8_t *key, int keybits)
+{
+
+ ctx->Nr = rijndaelKeySetupEnc(ctx->ek, key, keybits);
+ rijndaelKeySetupDec(ctx->dk, key, keybits);
+}
+
+void
+rijndael_encrypt(const rijndael_ctx *ctx, const uint8_t *in, uint8_t *out)
+{
+
+ rijndaelEncrypt(ctx->ek, ctx->Nr, in, out);
+}
+
+void
+rijndael_decrypt(const rijndael_ctx *ctx, const u_char *in, uint8_t *out)
+{
+
+ rijndaelDecrypt(ctx->dk, ctx->Nr, in, out);
+}
+
+int
+rijndaelKeySetupEnc(uint32_t *rk, const uint8_t *key, int keybits)
+{
+ struct aesenc enc;
+ unsigned nrounds;
+
+ switch (keybits) {
+ case 128:
+ nrounds = aes_setenckey128(&enc, key);
+ break;
+ case 192:
+ nrounds = aes_setenckey192(&enc, key);
+ break;
+ case 256:
+ nrounds = aes_setenckey256(&enc, key);
+ break;
+ default:
+ panic("invalid AES key bits: %d", keybits);
+ }
+
+ memcpy(rk, enc.aese_aes.aes_rk, 4*(nrounds + 1)*sizeof(rk[0]));
+ explicit_memset(&enc, 0, sizeof enc);
+
+ return nrounds;
+}
+
+int
+rijndaelKeySetupDec(uint32_t *rk, const uint8_t *key, int keybits)
+{
+ struct aesdec dec;
+ unsigned nrounds;
+
+ switch (keybits) {
+ case 128:
+ nrounds = aes_setdeckey128(&dec, key);
+ break;
+ case 192:
+ nrounds = aes_setdeckey192(&dec, key);
+ break;
+ case 256:
+ nrounds = aes_setdeckey256(&dec, key);
+ break;
+ default:
+ panic("invalid AES key bits: %d", keybits);
+ }
+
+ memcpy(rk, dec.aesd_aes.aes_rk, 4*(nrounds + 1)*sizeof(rk[0]));
+ explicit_memset(&dec, 0, sizeof dec);
+
+ return nrounds;
+}
+
+void
+rijndaelEncrypt(const uint32_t *rk, int nrounds, const uint8_t in[16],
+ uint8_t out[16])
+{
+ struct aesenc enc;
+
+ memcpy(enc.aese_aes.aes_rk, rk, 4*(nrounds + 1)*sizeof(rk[0]));
+ aes_enc(&enc, in, out, nrounds);
+ explicit_memset(&enc, 0, sizeof enc);
+}
+
+void
+rijndaelDecrypt(const uint32_t *rk, int nrounds, const uint8_t in[16],
+ uint8_t out[16])
+{
+ struct aesdec dec;
+
+ memcpy(dec.aesd_aes.aes_rk, rk, 4*(nrounds + 1)*sizeof(rk[0]));
+ aes_dec(&dec, in, out, nrounds);
+ explicit_memset(&dec, 0, sizeof dec);
+}
+
+int
+rijndael_makeKey(keyInstance *key, BYTE direction, int keybits,
+ const char *keyp)
+{
+
+ if (key == NULL)
+ return BAD_KEY_INSTANCE;
+
+ memset(key, 0x1a, sizeof(*key));
+
+ switch (direction) {
+ case DIR_ENCRYPT:
+ case DIR_DECRYPT:
+ key->direction = direction;
+ break;
+ default:
+ return BAD_KEY_DIR;
+ }
+
+ switch (keybits) {
+ case 128:
+ case 192:
+ case 256:
+ key->keyLen = keybits;
+ break;
+ default:
+ return BAD_KEY_MAT;
+ }
+
+ if (keyp)
+ memcpy(key->keyMaterial, keyp, keybits/8);
+
+ switch (direction) {
+ case DIR_ENCRYPT:
+ key->Nr = rijndaelKeySetupEnc(key->rk,
+ (const uint8_t *)key->keyMaterial, keybits);
+ break;
+ case DIR_DECRYPT:
+ key->Nr = rijndaelKeySetupDec(key->rk,
+ (const uint8_t *)key->keyMaterial, keybits);
+ break;
+ default:
+ panic("unknown encryption direction %d", direction);
+ }
+ rijndaelKeySetupEnc(key->ek, (const uint8_t *)key->keyMaterial,
+ keybits);
+
+ return 1;
+}
+
+int
+rijndael_cipherInit(cipherInstance *cipher, BYTE mode, const char *iv)
+{
+
+ switch (mode) {
+ case MODE_ECB: /* used only for encrypting one block */
+ case MODE_CBC:
+ case MODE_XTS:
+ cipher->mode = mode;
+ break;
+ case MODE_CFB1: /* unused */
+ default:
+ return BAD_CIPHER_MODE;
+ }
+
+ if (iv)
+ memcpy(cipher->IV, iv, RIJNDAEL_MAX_IV_SIZE);
+ else
+ memset(cipher->IV, 0, RIJNDAEL_MAX_IV_SIZE);
+
+ return 1;
+}
+
+int
+rijndael_blockEncrypt(cipherInstance *cipher, keyInstance *key,
+ const BYTE *in, int nbits, BYTE *out)
+{
+ struct aesenc enc;
+
+ if (cipher == NULL)
+ return BAD_CIPHER_STATE;
+ if (key == NULL)
+ return BAD_CIPHER_STATE;
+ if (key->direction != DIR_ENCRYPT)
+ return BAD_CIPHER_STATE;
+
+ if (in == NULL || nbits <= 0)
+ return 0;
+
+ memcpy(enc.aese_aes.aes_rk, key->rk,
+ 4*(key->Nr + 1)*sizeof(key->rk[0]));
+ switch (cipher->mode) {
+ case MODE_ECB:
+ KASSERT(nbits == 128);
+ aes_enc(&enc, in, out, key->Nr);
+ break;
+ case MODE_CBC:
+ KASSERT(nbits % 128 == 0);
+ aes_cbc_enc(&enc, in, out, nbits/8, (uint8_t *)cipher->IV,
+ key->Nr);
+ break;
+ case MODE_XTS:
+ KASSERT(nbits % 128 == 0);
+ aes_xts_enc(&enc, in, out, nbits/8, (uint8_t *)cipher->IV,
+ key->Nr);
+ break;
+ default:
+ panic("invalid AES mode: %d", cipher->mode);
+ }
+ explicit_memset(&enc, 0, sizeof enc);
+
+ return nbits;
+}
+
+int
+rijndael_blockDecrypt(cipherInstance *cipher, keyInstance *key,
+ const BYTE *in, int nbits, BYTE *out)
+{
+ struct aesdec dec;
+
+ if (cipher == NULL)
+ return BAD_CIPHER_STATE;
+ if (key == NULL)
+ return BAD_CIPHER_STATE;
+ if (key->direction != DIR_DECRYPT)
+ return BAD_CIPHER_STATE;
+
+ if (in == NULL || nbits <= 0)
+ return 0;
+
+ memcpy(dec.aesd_aes.aes_rk, key->rk,
+ 4*(key->Nr + 1)*sizeof(key->rk[0]));
+ switch (cipher->mode) {
+ case MODE_ECB:
+ KASSERT(nbits == 128);
+ aes_dec(&dec, in, out, key->Nr);
+ break;
+ case MODE_CBC:
+ KASSERT(nbits % 128 == 0);
+ aes_cbc_dec(&dec, in, out, nbits/8, (uint8_t *)cipher->IV,
+ key->Nr);
+ break;
+ case MODE_XTS:
+ KASSERT(nbits % 128 == 0);
+ aes_xts_dec(&dec, in, out, nbits/8, (uint8_t *)cipher->IV,
+ key->Nr);
+ break;
+ default:
+ panic("invalid AES mode: %d", cipher->mode);
+ }
+ explicit_memset(&dec, 0, sizeof dec);
+
+ return nbits;
+}
diff -r 81a487955535 -r 9d6b84c40f65 sys/crypto/aes/aes_selftest.c
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/sys/crypto/aes/aes_selftest.c Fri Jun 12 05:16:46 2020 +0000
@@ -0,0 +1,387 @@
+/* $NetBSD$ */
+
+/*-
+ * Copyright (c) 2020 The NetBSD Foundation, Inc.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND CONTRIBUTORS
+ * ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
+ * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+ * POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <sys/cdefs.h>
+__KERNEL_RCSID(1, "$NetBSD$");
+
+#include <sys/types.h>
+#include <sys/systm.h>
+
+#include <lib/libkern/libkern.h>
+
+#include <crypto/aes/aes.h>
+
+static const unsigned aes_keybytes[] __unused = { 16, 24, 32 };
+static const unsigned aes_keybits[] __unused = { 128, 192, 256 };
+static const unsigned aes_nrounds[] = { 10, 12, 14 };
+
+#define aes_selftest_fail(impl, actual, expected, nbytes, fmt, args...) \
+({ \
+ printf("%s "fmt": self-test failed\n", (impl)->ai_name, ##args); \
+ hexdump(printf, "was", (actual), (nbytes)); \
+ hexdump(printf, "expected", (expected), (nbytes)); \
+ -1; \
+})
+
+static int
+aes_selftest_encdec(const struct aes_impl *impl)
+{
+ /*
+ * head -c 16 < /dev/zero | openssl enc -aes-{128,192,256}-ecb
+ * -nopad -K 000102030405060708090a0b0c0d... | hexdump -C
+ */
+ static const uint8_t expected[3][16] = {
+ [0] = {
+ 0xc6,0xa1,0x3b,0x37,0x87,0x8f,0x5b,0x82,
+ 0x6f,0x4f,0x81,0x62,0xa1,0xc8,0xd8,0x79,
+ },
+ [1] = {
+ 0x91,0x62,0x51,0x82,0x1c,0x73,0xa5,0x22,
+ 0xc3,0x96,0xd6,0x27,0x38,0x01,0x96,0x07,
+ },
+ [2] = {
+ 0xf2,0x90,0x00,0xb6,0x2a,0x49,0x9f,0xd0,
+ 0xa9,0xf3,0x9a,0x6a,0xdd,0x2e,0x77,0x80,
+ },
+ };
+ struct aesenc enc;
+ struct aesdec dec;
+ uint8_t key[32];
+ uint8_t in[16];
+ uint8_t outbuf[18] = { [0] = 0x1a, [17] = 0x1a }, *out = outbuf + 1;
+ unsigned i;
+
+ for (i = 0; i < 32; i++)
+ key[i] = i;
+ for (i = 0; i < 16; i++)
+ in[i] = 0;
+
+ for (i = 0; i < 3; i++) {
+ impl->ai_setenckey(&enc, key, aes_nrounds[i]);
+ impl->ai_setdeckey(&dec, key, aes_nrounds[i]);
+ impl->ai_enc(&enc, in, out, aes_nrounds[i]);
+ if (memcmp(out, expected[i], 16))
+ return aes_selftest_fail(impl, out, expected[i], 16,
+ "AES-%u enc", aes_keybits[i]);
+ impl->ai_dec(&dec, out, out, aes_nrounds[i]);
+ if (memcmp(out, in, 16))
+ return aes_selftest_fail(impl, out, in, 16,
+ "AES-%u dec", aes_keybits[i]);
+ }
+
+ if (outbuf[0] != 0x1a)
+ return aes_selftest_fail(impl, outbuf,
+ (const uint8_t[1]){0x1a}, 1,
+ "AES overrun preceding");
+ if (outbuf[17] != 0x1a)
+ return aes_selftest_fail(impl, outbuf + 17,
+ (const uint8_t[1]){0x1a}, 1,
+ "AES overrun folllowing");
+
+ /* Success! */
+ return 0;
+}
+
+static int
+aes_selftest_encdec_cbc(const struct aes_impl *impl)
+{
+ static const uint8_t expected[3][144] = {
+ [0] = {
+ 0xfe,0xf1,0xa8,0xb6,0x25,0xf0,0xc4,0x3a,
+ 0x71,0x08,0xb6,0x23,0xa6,0xfb,0x90,0xca,
+ 0x9e,0x64,0x6d,0x95,0xb5,0xf5,0x41,0x24,
+ 0xd2,0xe6,0x60,0xda,0x6c,0x69,0xc4,0xa0,
+ 0x4d,0xaa,0x94,0xf6,0x66,0x1e,0xaa,0x85,
+ 0x68,0xc5,0x6b,0x2e,0x77,0x7a,0x68,0xff,
+ 0x45,0x15,0x45,0xc5,0x9c,0xbb,0x3a,0x23,
+ 0x08,0x3a,0x06,0xdd,0xc0,0x52,0xd2,0xb7,
+ 0x47,0xaa,0x1c,0xc7,0xb5,0xa9,0x7d,0x04,
+ 0x60,0x67,0x78,0xf6,0xb9,0xba,0x26,0x84,
+ 0x45,0x72,0x44,0xed,0xa3,0xd3,0xa0,0x3f,
+ 0x19,0xee,0x3f,0x94,0x59,0x52,0x4b,0x13,
+ 0xfd,0x81,0xcc,0xf9,0xf2,0x29,0xd7,0xec,
+ 0xde,0x03,0x56,0x01,0x4a,0x19,0x86,0xc0,
+ 0x87,0xce,0xe1,0xcc,0x13,0xf1,0x2e,0xda,
+ 0x3f,0xfe,0xa4,0x64,0xe7,0x48,0xb4,0x7b,
+ 0x73,0x62,0x5a,0x80,0x5e,0x01,0x20,0xa5,
+ 0x0a,0xd7,0x98,0xa7,0xd9,0x8b,0xff,0xc2,
+ },
+ [1] = {
+ 0xa6,0x87,0xf0,0x92,0x68,0xc8,0xd6,0x42,
+ 0xa8,0x83,0x1c,0x92,0x65,0x8c,0xd9,0xfe,
+ 0x0b,0x1a,0xc6,0x96,0x27,0x44,0xd4,0x14,
+ 0xfc,0xe7,0x85,0xb2,0x71,0xc7,0x11,0x39,
+ 0xed,0x36,0xd3,0x5c,0xa7,0xf7,0x3d,0xc9,
+ 0xa2,0x54,0x8b,0xb4,0xfa,0xe8,0x21,0xf9,
+ 0xfd,0x6a,0x42,0x85,0xde,0x66,0xd4,0xc0,
+ 0xa7,0xd3,0x5b,0xe1,0xe6,0xac,0xea,0xf9,
+ 0xa3,0x15,0x68,0xf4,0x66,0x4c,0x23,0x75,
+ 0x58,0xba,0x7f,0xca,0xbf,0x40,0x56,0x79,
+ 0x2f,0xbf,0xdf,0x5f,0x56,0xcb,0xa0,0xe4,
+ 0x22,0x65,0x6a,0x8f,0x4f,0xff,0x11,0x6b,
+ 0x57,0xeb,0x45,0xeb,0x9d,0x7f,0xfe,0x9c,
+ 0x8b,0x30,0xa8,0xb0,0x7e,0x27,0xf8,0xbc,
+ 0x1f,0xf8,0x15,0x34,0x36,0x4f,0x46,0x73,
+ 0x81,0x90,0x4b,0x4b,0x46,0x4d,0x01,0x45,
+ 0xa1,0xc3,0x0b,0xa8,0x5a,0xab,0xc1,0x88,
+ 0x66,0xc8,0x1a,0x94,0x17,0x64,0x6f,0xf4,
+ },
+ [2] = {
+ 0x22,0x4c,0x27,0xf4,0xba,0x37,0x8b,0x27,
+ 0xd3,0xd6,0x88,0x8a,0xdc,0xed,0x64,0x42,
+ 0x19,0x60,0x31,0x09,0xf3,0x72,0xd2,0xc2,
+ 0xd3,0xe3,0xff,0xce,0xc5,0x03,0x9f,0xce,
+ 0x99,0x49,0x8a,0xf2,0xe1,0xba,0xe2,0xa8,
+ 0xd7,0x32,0x07,0x2d,0xb0,0xb3,0xbc,0x67,
+ 0x32,0x9a,0x3e,0x7d,0x16,0x23,0xe7,0x24,
+ 0x84,0xe1,0x15,0x03,0x9c,0xa2,0x7a,0x95,
+ 0x34,0xa8,0x04,0x4e,0x79,0x31,0x50,0x26,
+ 0x76,0xd1,0x10,0xce,0xec,0x13,0xf7,0xfb,
+ 0x94,0x6b,0x76,0x50,0x5f,0xb2,0x3e,0x7c,
+ 0xbe,0x97,0xe7,0x13,0x06,0x9e,0x2d,0xc4,
+ 0x46,0x65,0xa7,0x69,0x37,0x07,0x25,0x37,
+ 0xe5,0x48,0x51,0xa8,0x58,0xe8,0x4d,0x7c,
+ 0xb5,0xbe,0x25,0x13,0xbc,0x11,0xc2,0xde,
+ 0xdb,0x00,0xef,0x1c,0x1d,0xeb,0xe3,0x49,
+ 0x1c,0xc0,0x78,0x29,0x76,0xc0,0xde,0x3a,
+ 0x0e,0x96,0x8f,0xea,0xd7,0x42,0x4e,0xb4,
+ },
+ };
+ struct aesenc enc;
+ struct aesdec dec;
+ uint8_t key[32];
+ uint8_t in[144];
+ uint8_t outbuf[146] = { [0] = 0x1a, [145] = 0x1a }, *out = outbuf + 1;
+ uint8_t iv0[16], iv[16];
+ unsigned i;
+
+ for (i = 0; i < 32; i++)
+ key[i] = i;
+ for (i = 0; i < 16; i++)
+ iv0[i] = 0x20 ^ i;
+ for (i = 0; i < 144; i++)
+ in[i] = 0x80 ^ i;
+
+ for (i = 0; i < 3; i++) {
+ impl->ai_setenckey(&enc, key, aes_nrounds[i]);
+ impl->ai_setdeckey(&dec, key, aes_nrounds[i]);
+
+ /* Try one swell foop. */
+ memcpy(iv, iv0, 16);
+ impl->ai_cbc_enc(&enc, in, out, 144, iv, aes_nrounds[i]);
+ if (memcmp(out, expected[i], 144))
+ return aes_selftest_fail(impl, out, expected[i], 144,
+ "AES-%u-CBC enc", aes_keybits[i]);
+
+ memcpy(iv, iv0, 16);
+ impl->ai_cbc_dec(&dec, out, out, 144, iv, aes_nrounds[i]);
+ if (memcmp(out, in, 144))
+ return aes_selftest_fail(impl, out, in, 144,
+ "AES-%u-CBC dec", aes_keybits[i]);
+
+ /* Try incrementally, with IV update. */
+ memcpy(iv, iv0, 16);
+ impl->ai_cbc_enc(&enc, in, out, 16, iv, aes_nrounds[i]);
+ impl->ai_cbc_enc(&enc, in + 16, out + 16, 128, iv,
+ aes_nrounds[i]);
+ if (memcmp(out, expected[i], 144))
+ return aes_selftest_fail(impl, out, expected[i], 144,
+ "AES-%u-CBC enc incremental", aes_keybits[i]);
+
+ memcpy(iv, iv0, 16);
+ impl->ai_cbc_dec(&dec, out, out, 128, iv, aes_nrounds[i]);
+ impl->ai_cbc_dec(&dec, out + 128, out + 128, 16, iv,
+ aes_nrounds[i]);
+ if (memcmp(out, in, 144))
+ return aes_selftest_fail(impl, out, in, 144,
+ "AES-%u-CBC dec incremental", aes_keybits[i]);
+ }
+
+ if (outbuf[0] != 0x1a)
+ return aes_selftest_fail(impl, outbuf,
+ (const uint8_t[1]){0x1a}, 1,
+ "AES-CBC overrun preceding");
+ if (outbuf[145] != 0x1a)
+ return aes_selftest_fail(impl, outbuf + 145,
+ (const uint8_t[1]){0x1a}, 1,
+ "AES-CBC overrun following");
+
+ /* Success! */
+ return 0;
+}
+
+static int
+aes_selftest_encdec_xts(const struct aes_impl *impl)
+{
+ uint64_t blkno[3] = { 0, 1, 0xff };
+ static const uint8_t expected[3][144] = {
+ [0] = {
+ /* IEEE P1619-D16, XTS-AES-128, Vector 4, truncated */
+ 0x27,0xa7,0x47,0x9b,0xef,0xa1,0xd4,0x76,
+ 0x48,0x9f,0x30,0x8c,0xd4,0xcf,0xa6,0xe2,
+ 0xa9,0x6e,0x4b,0xbe,0x32,0x08,0xff,0x25,
+ 0x28,0x7d,0xd3,0x81,0x96,0x16,0xe8,0x9c,
+ 0xc7,0x8c,0xf7,0xf5,0xe5,0x43,0x44,0x5f,
+ 0x83,0x33,0xd8,0xfa,0x7f,0x56,0x00,0x00,
+ 0x05,0x27,0x9f,0xa5,0xd8,0xb5,0xe4,0xad,
+ 0x40,0xe7,0x36,0xdd,0xb4,0xd3,0x54,0x12,
+ 0x32,0x80,0x63,0xfd,0x2a,0xab,0x53,0xe5,
+ 0xea,0x1e,0x0a,0x9f,0x33,0x25,0x00,0xa5,
+ 0xdf,0x94,0x87,0xd0,0x7a,0x5c,0x92,0xcc,
+ 0x51,0x2c,0x88,0x66,0xc7,0xe8,0x60,0xce,
+ 0x93,0xfd,0xf1,0x66,0xa2,0x49,0x12,0xb4,
+ 0x22,0x97,0x61,0x46,0xae,0x20,0xce,0x84,
+ 0x6b,0xb7,0xdc,0x9b,0xa9,0x4a,0x76,0x7a,
+ 0xae,0xf2,0x0c,0x0d,0x61,0xad,0x02,0x65,
+ 0x5e,0xa9,0x2d,0xc4,0xc4,0xe4,0x1a,0x89,
+ 0x52,0xc6,0x51,0xd3,0x31,0x74,0xbe,0x51,
+ },
+ [1] = {
+ },
+ [2] = {
+ /* IEEE P1619-D16, XTS-AES-256, Vector 10, truncated */
+ 0x1c,0x3b,0x3a,0x10,0x2f,0x77,0x03,0x86,
+ 0xe4,0x83,0x6c,0x99,0xe3,0x70,0xcf,0x9b,
+ 0xea,0x00,0x80,0x3f,0x5e,0x48,0x23,0x57,
+ 0xa4,0xae,0x12,0xd4,0x14,0xa3,0xe6,0x3b,
+ 0x5d,0x31,0xe2,0x76,0xf8,0xfe,0x4a,0x8d,
+ 0x66,0xb3,0x17,0xf9,0xac,0x68,0x3f,0x44,
+ 0x68,0x0a,0x86,0xac,0x35,0xad,0xfc,0x33,
+ 0x45,0xbe,0xfe,0xcb,0x4b,0xb1,0x88,0xfd,
+ 0x57,0x76,0x92,0x6c,0x49,0xa3,0x09,0x5e,
+ 0xb1,0x08,0xfd,0x10,0x98,0xba,0xec,0x70,
+ 0xaa,0xa6,0x69,0x99,0xa7,0x2a,0x82,0xf2,
+ 0x7d,0x84,0x8b,0x21,0xd4,0xa7,0x41,0xb0,
+ 0xc5,0xcd,0x4d,0x5f,0xff,0x9d,0xac,0x89,
+ 0xae,0xba,0x12,0x29,0x61,0xd0,0x3a,0x75,
+ 0x71,0x23,0xe9,0x87,0x0f,0x8a,0xcf,0x10,
+ 0x00,0x02,0x08,0x87,0x89,0x14,0x29,0xca,
+ 0x2a,0x3e,0x7a,0x7d,0x7d,0xf7,0xb1,0x03,
+ 0x55,0x16,0x5c,0x8b,0x9a,0x6d,0x0a,0x7d,
+ },
+ };
+ static const uint8_t key1[32] = {
+ 0x27,0x18,0x28,0x18,0x28,0x45,0x90,0x45,
+ 0x23,0x53,0x60,0x28,0x74,0x71,0x35,0x26,
+ 0x62,0x49,0x77,0x57,0x24,0x70,0x93,0x69,
+ 0x99,0x59,0x57,0x49,0x66,0x96,0x76,0x27,
+ };
+ static const uint8_t key2[32] = {
+ 0x31,0x41,0x59,0x26,0x53,0x58,0x97,0x93,
+ 0x23,0x84,0x62,0x64,0x33,0x83,0x27,0x95,
+ 0x02,0x88,0x41,0x97,0x16,0x93,0x99,0x37,
+ 0x51,0x05,0x82,0x09,0x74,0x94,0x45,0x92,
+ };
+ struct aesenc enc;
+ struct aesdec dec;
+ uint8_t in[144];
+ uint8_t outbuf[146] = { [0] = 0x1a, [145] = 0x1a }, *out = outbuf + 1;
+ uint8_t blkno_buf[16];
+ uint8_t iv0[16], iv[16];
+ unsigned i;
+
+ for (i = 0; i < 144; i++)
+ in[i] = i;
+
+ for (i = 0; i < 3; i++) {
+ if (i == 1) /* XXX missing AES-192 test vector */
+ continue;
+
+ /* Format the data unit sequence number. */
+ memset(blkno_buf, 0, sizeof blkno_buf);
+ le64enc(blkno_buf, blkno[i]);
+
+ /* Generate the tweak. */
+ impl->ai_setenckey(&enc, key2, aes_nrounds[i]);
+ impl->ai_enc(&enc, blkno_buf, iv0, aes_nrounds[i]);
+
+ /* Load the data encryption key. */
+ impl->ai_setenckey(&enc, key1, aes_nrounds[i]);
+ impl->ai_setdeckey(&dec, key1, aes_nrounds[i]);
+
+ /* Try one swell foop. */
+ memcpy(iv, iv0, 16);
+ impl->ai_xts_enc(&enc, in, out, 144, iv, aes_nrounds[i]);
+ if (memcmp(out, expected[i], 144))
+ return aes_selftest_fail(impl, out, expected[i], 144,
+ "AES-%u-XTS enc", aes_keybits[i]);
+
+ memcpy(iv, iv0, 16);
+ impl->ai_xts_dec(&dec, out, out, 144, iv, aes_nrounds[i]);
+ if (memcmp(out, in, 144))
+ return aes_selftest_fail(impl, out, in, 144,
+ "AES-%u-XTS dec", aes_keybits[i]);
+
+ /* Try incrementally, with IV update. */
+ memcpy(iv, iv0, 16);
+ impl->ai_xts_enc(&enc, in, out, 16, iv, aes_nrounds[i]);
+ impl->ai_xts_enc(&enc, in + 16, out + 16, 128, iv,
+ aes_nrounds[i]);
+ if (memcmp(out, expected[i], 144))
+ return aes_selftest_fail(impl, out, expected[i], 144,
+ "AES-%u-XTS enc incremental", aes_keybits[i]);
+
+ memcpy(iv, iv0, 16);
+ impl->ai_xts_dec(&dec, out, out, 128, iv, aes_nrounds[i]);
+ impl->ai_xts_dec(&dec, out + 128, out + 128, 16, iv,
+ aes_nrounds[i]);
+ if (memcmp(out, in, 144))
+ return aes_selftest_fail(impl, out, in, 144,
+ "AES-%u-XTS dec incremental", aes_keybits[i]);
+ }
+
+ if (outbuf[0] != 0x1a)
+ return aes_selftest_fail(impl, outbuf,
+ (const uint8_t[1]){0x1a}, 1,
+ "AES-XTS overrun preceding");
+ if (outbuf[145] != 0x1a)
+ return aes_selftest_fail(impl, outbuf + 145,
+ (const uint8_t[1]){0x1a}, 1,
+ "AES-XTS overrun following");
+
+ /* Success! */
+ return 0;
+}
+
+int
+aes_selftest(const struct aes_impl *impl)
+{
+ int result = 0;
+
+ if (impl->ai_probe())
+ return -1;
+
+ if (aes_selftest_encdec(impl))
+ result = -1;
+ if (aes_selftest_encdec_cbc(impl))
+ result = -1;
+ if (aes_selftest_encdec_xts(impl))
+ result = -1;
+
+ return result;
+}
diff -r 81a487955535 -r 9d6b84c40f65 sys/crypto/aes/files.aes
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/sys/crypto/aes/files.aes Fri Jun 12 05:16:46 2020 +0000
@@ -0,0 +1,12 @@
+# $NetBSD$
+
+define aes
+define rijndael: aes # legacy Rijndael API
+
+file crypto/aes/aes_bear.c aes
+file crypto/aes/aes_ct.c aes
+file crypto/aes/aes_ct_dec.c aes
+file crypto/aes/aes_ct_enc.c aes
+file crypto/aes/aes_impl.c aes
+file crypto/aes/aes_rijndael.c rijndael
+file crypto/aes/aes_selftest.c aes
diff -r 81a487955535 -r 9d6b84c40f65 sys/crypto/rijndael/files.rijndael
--- a/sys/crypto/rijndael/files.rijndael Sun Jun 14 15:58:39 2020 +0000
+++ /dev/null Thu Jan 01 00:00:00 1970 +0000
@@ -1,7 +0,0 @@
-# $NetBSD: files.rijndael,v 1.7 2020/04/22 09:15:40 rin Exp $
-
-define rijndael
-
-file crypto/rijndael/rijndael-alg-fst.c rijndael
-file crypto/rijndael/rijndael-api-fst.c rijndael
-file crypto/rijndael/rijndael.c rijndael
diff -r 81a487955535 -r 9d6b84c40f65 sys/crypto/rijndael/rijndael-alg-fst.c
--- a/sys/crypto/rijndael/rijndael-alg-fst.c Sun Jun 14 15:58:39 2020 +0000
+++ /dev/null Thu Jan 01 00:00:00 1970 +0000
@@ -1,1225 +0,0 @@
-/* $NetBSD: rijndael-alg-fst.c,v 1.7 2005/12/11 12:20:52 christos Exp $ */
-/* $KAME: rijndael-alg-fst.c,v 1.10 2003/07/15 10:47:16 itojun Exp $ */
-/**
- * rijndael-alg-fst.c
- *
- * @version 3.0 (December 2000)
- *
- * Optimised ANSI C code for the Rijndael cipher (now AES)
- *
- * @author Vincent Rijmen <vincent.rijmen%esat.kuleuven.ac.be@localhost>
- * @author Antoon Bosselaers <antoon.bosselaers%esat.kuleuven.ac.be@localhost>
- * @author Paulo Barreto <paulo.barreto%terra.com.br@localhost>
- *
- * This code is hereby placed in the public domain.
- *
- * THIS SOFTWARE IS PROVIDED BY THE AUTHORS ''AS IS'' AND ANY EXPRESS
- * OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
- * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
- * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHORS OR CONTRIBUTORS BE
- * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
- * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
- * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
- * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
- * WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE
- * OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE,
- * EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
- */
-
-#include <sys/cdefs.h>
-__KERNEL_RCSID(0, "$NetBSD: rijndael-alg-fst.c,v 1.7 2005/12/11 12:20:52 christos Exp $");
-
-#include <sys/types.h>
-#ifdef _KERNEL
-#include <sys/systm.h>
-#else
-#include <string.h>
-#endif
-
-#include <crypto/rijndael/rijndael-alg-fst.h>
-#include <crypto/rijndael/rijndael_local.h>
-
-/*
-Te0[x] = S [x].[02, 01, 01, 03];
-Te1[x] = S [x].[03, 02, 01, 01];
-Te2[x] = S [x].[01, 03, 02, 01];
-Te3[x] = S [x].[01, 01, 03, 02];
-Te4[x] = S [x].[01, 01, 01, 01];
-
-Td0[x] = Si[x].[0e, 09, 0d, 0b];
-Td1[x] = Si[x].[0b, 0e, 09, 0d];
-Td2[x] = Si[x].[0d, 0b, 0e, 09];
-Td3[x] = Si[x].[09, 0d, 0b, 0e];
-Td4[x] = Si[x].[01, 01, 01, 01];
-*/
-
-static const u32 Te0[256] = {
- 0xc66363a5U, 0xf87c7c84U, 0xee777799U, 0xf67b7b8dU,
- 0xfff2f20dU, 0xd66b6bbdU, 0xde6f6fb1U, 0x91c5c554U,
- 0x60303050U, 0x02010103U, 0xce6767a9U, 0x562b2b7dU,
- 0xe7fefe19U, 0xb5d7d762U, 0x4dababe6U, 0xec76769aU,
- 0x8fcaca45U, 0x1f82829dU, 0x89c9c940U, 0xfa7d7d87U,
- 0xeffafa15U, 0xb25959ebU, 0x8e4747c9U, 0xfbf0f00bU,
- 0x41adadecU, 0xb3d4d467U, 0x5fa2a2fdU, 0x45afafeaU,
- 0x239c9cbfU, 0x53a4a4f7U, 0xe4727296U, 0x9bc0c05bU,
- 0x75b7b7c2U, 0xe1fdfd1cU, 0x3d9393aeU, 0x4c26266aU,
- 0x6c36365aU, 0x7e3f3f41U, 0xf5f7f702U, 0x83cccc4fU,
- 0x6834345cU, 0x51a5a5f4U, 0xd1e5e534U, 0xf9f1f108U,
- 0xe2717193U, 0xabd8d873U, 0x62313153U, 0x2a15153fU,
- 0x0804040cU, 0x95c7c752U, 0x46232365U, 0x9dc3c35eU,
- 0x30181828U, 0x379696a1U, 0x0a05050fU, 0x2f9a9ab5U,
- 0x0e070709U, 0x24121236U, 0x1b80809bU, 0xdfe2e23dU,
- 0xcdebeb26U, 0x4e272769U, 0x7fb2b2cdU, 0xea75759fU,
- 0x1209091bU, 0x1d83839eU, 0x582c2c74U, 0x341a1a2eU,
- 0x361b1b2dU, 0xdc6e6eb2U, 0xb45a5aeeU, 0x5ba0a0fbU,
- 0xa45252f6U, 0x763b3b4dU, 0xb7d6d661U, 0x7db3b3ceU,
- 0x5229297bU, 0xdde3e33eU, 0x5e2f2f71U, 0x13848497U,
- 0xa65353f5U, 0xb9d1d168U, 0x00000000U, 0xc1eded2cU,
- 0x40202060U, 0xe3fcfc1fU, 0x79b1b1c8U, 0xb65b5bedU,
- 0xd46a6abeU, 0x8dcbcb46U, 0x67bebed9U, 0x7239394bU,
- 0x944a4adeU, 0x984c4cd4U, 0xb05858e8U, 0x85cfcf4aU,
- 0xbbd0d06bU, 0xc5efef2aU, 0x4faaaae5U, 0xedfbfb16U,
- 0x864343c5U, 0x9a4d4dd7U, 0x66333355U, 0x11858594U,
- 0x8a4545cfU, 0xe9f9f910U, 0x04020206U, 0xfe7f7f81U,
- 0xa05050f0U, 0x783c3c44U, 0x259f9fbaU, 0x4ba8a8e3U,
- 0xa25151f3U, 0x5da3a3feU, 0x804040c0U, 0x058f8f8aU,
- 0x3f9292adU, 0x219d9dbcU, 0x70383848U, 0xf1f5f504U,
- 0x63bcbcdfU, 0x77b6b6c1U, 0xafdada75U, 0x42212163U,
- 0x20101030U, 0xe5ffff1aU, 0xfdf3f30eU, 0xbfd2d26dU,
- 0x81cdcd4cU, 0x180c0c14U, 0x26131335U, 0xc3ecec2fU,
- 0xbe5f5fe1U, 0x359797a2U, 0x884444ccU, 0x2e171739U,
- 0x93c4c457U, 0x55a7a7f2U, 0xfc7e7e82U, 0x7a3d3d47U,
- 0xc86464acU, 0xba5d5de7U, 0x3219192bU, 0xe6737395U,
- 0xc06060a0U, 0x19818198U, 0x9e4f4fd1U, 0xa3dcdc7fU,
- 0x44222266U, 0x542a2a7eU, 0x3b9090abU, 0x0b888883U,
- 0x8c4646caU, 0xc7eeee29U, 0x6bb8b8d3U, 0x2814143cU,
- 0xa7dede79U, 0xbc5e5ee2U, 0x160b0b1dU, 0xaddbdb76U,
- 0xdbe0e03bU, 0x64323256U, 0x743a3a4eU, 0x140a0a1eU,
- 0x924949dbU, 0x0c06060aU, 0x4824246cU, 0xb85c5ce4U,
- 0x9fc2c25dU, 0xbdd3d36eU, 0x43acacefU, 0xc46262a6U,
- 0x399191a8U, 0x319595a4U, 0xd3e4e437U, 0xf279798bU,
- 0xd5e7e732U, 0x8bc8c843U, 0x6e373759U, 0xda6d6db7U,
- 0x018d8d8cU, 0xb1d5d564U, 0x9c4e4ed2U, 0x49a9a9e0U,
- 0xd86c6cb4U, 0xac5656faU, 0xf3f4f407U, 0xcfeaea25U,
- 0xca6565afU, 0xf47a7a8eU, 0x47aeaee9U, 0x10080818U,
- 0x6fbabad5U, 0xf0787888U, 0x4a25256fU, 0x5c2e2e72U,
- 0x381c1c24U, 0x57a6a6f1U, 0x73b4b4c7U, 0x97c6c651U,
- 0xcbe8e823U, 0xa1dddd7cU, 0xe874749cU, 0x3e1f1f21U,
- 0x964b4bddU, 0x61bdbddcU, 0x0d8b8b86U, 0x0f8a8a85U,
- 0xe0707090U, 0x7c3e3e42U, 0x71b5b5c4U, 0xcc6666aaU,
- 0x904848d8U, 0x06030305U, 0xf7f6f601U, 0x1c0e0e12U,
- 0xc26161a3U, 0x6a35355fU, 0xae5757f9U, 0x69b9b9d0U,
- 0x17868691U, 0x99c1c158U, 0x3a1d1d27U, 0x279e9eb9U,
- 0xd9e1e138U, 0xebf8f813U, 0x2b9898b3U, 0x22111133U,
- 0xd26969bbU, 0xa9d9d970U, 0x078e8e89U, 0x339494a7U,
- 0x2d9b9bb6U, 0x3c1e1e22U, 0x15878792U, 0xc9e9e920U,
- 0x87cece49U, 0xaa5555ffU, 0x50282878U, 0xa5dfdf7aU,
- 0x038c8c8fU, 0x59a1a1f8U, 0x09898980U, 0x1a0d0d17U,
- 0x65bfbfdaU, 0xd7e6e631U, 0x844242c6U, 0xd06868b8U,
- 0x824141c3U, 0x299999b0U, 0x5a2d2d77U, 0x1e0f0f11U,
- 0x7bb0b0cbU, 0xa85454fcU, 0x6dbbbbd6U, 0x2c16163aU,
-};
-static const u32 Te1[256] = {
- 0xa5c66363U, 0x84f87c7cU, 0x99ee7777U, 0x8df67b7bU,
- 0x0dfff2f2U, 0xbdd66b6bU, 0xb1de6f6fU, 0x5491c5c5U,
- 0x50603030U, 0x03020101U, 0xa9ce6767U, 0x7d562b2bU,
- 0x19e7fefeU, 0x62b5d7d7U, 0xe64dababU, 0x9aec7676U,
- 0x458fcacaU, 0x9d1f8282U, 0x4089c9c9U, 0x87fa7d7dU,
- 0x15effafaU, 0xebb25959U, 0xc98e4747U, 0x0bfbf0f0U,
- 0xec41adadU, 0x67b3d4d4U, 0xfd5fa2a2U, 0xea45afafU,
- 0xbf239c9cU, 0xf753a4a4U, 0x96e47272U, 0x5b9bc0c0U,
- 0xc275b7b7U, 0x1ce1fdfdU, 0xae3d9393U, 0x6a4c2626U,
- 0x5a6c3636U, 0x417e3f3fU, 0x02f5f7f7U, 0x4f83ccccU,
- 0x5c683434U, 0xf451a5a5U, 0x34d1e5e5U, 0x08f9f1f1U,
- 0x93e27171U, 0x73abd8d8U, 0x53623131U, 0x3f2a1515U,
- 0x0c080404U, 0x5295c7c7U, 0x65462323U, 0x5e9dc3c3U,
- 0x28301818U, 0xa1379696U, 0x0f0a0505U, 0xb52f9a9aU,
- 0x090e0707U, 0x36241212U, 0x9b1b8080U, 0x3ddfe2e2U,
- 0x26cdebebU, 0x694e2727U, 0xcd7fb2b2U, 0x9fea7575U,
- 0x1b120909U, 0x9e1d8383U, 0x74582c2cU, 0x2e341a1aU,
- 0x2d361b1bU, 0xb2dc6e6eU, 0xeeb45a5aU, 0xfb5ba0a0U,
- 0xf6a45252U, 0x4d763b3bU, 0x61b7d6d6U, 0xce7db3b3U,
- 0x7b522929U, 0x3edde3e3U, 0x715e2f2fU, 0x97138484U,
- 0xf5a65353U, 0x68b9d1d1U, 0x00000000U, 0x2cc1ededU,
- 0x60402020U, 0x1fe3fcfcU, 0xc879b1b1U, 0xedb65b5bU,
- 0xbed46a6aU, 0x468dcbcbU, 0xd967bebeU, 0x4b723939U,
- 0xde944a4aU, 0xd4984c4cU, 0xe8b05858U, 0x4a85cfcfU,
- 0x6bbbd0d0U, 0x2ac5efefU, 0xe54faaaaU, 0x16edfbfbU,
- 0xc5864343U, 0xd79a4d4dU, 0x55663333U, 0x94118585U,
- 0xcf8a4545U, 0x10e9f9f9U, 0x06040202U, 0x81fe7f7fU,
- 0xf0a05050U, 0x44783c3cU, 0xba259f9fU, 0xe34ba8a8U,
- 0xf3a25151U, 0xfe5da3a3U, 0xc0804040U, 0x8a058f8fU,
- 0xad3f9292U, 0xbc219d9dU, 0x48703838U, 0x04f1f5f5U,
- 0xdf63bcbcU, 0xc177b6b6U, 0x75afdadaU, 0x63422121U,
- 0x30201010U, 0x1ae5ffffU, 0x0efdf3f3U, 0x6dbfd2d2U,
- 0x4c81cdcdU, 0x14180c0cU, 0x35261313U, 0x2fc3ececU,
- 0xe1be5f5fU, 0xa2359797U, 0xcc884444U, 0x392e1717U,
- 0x5793c4c4U, 0xf255a7a7U, 0x82fc7e7eU, 0x477a3d3dU,
- 0xacc86464U, 0xe7ba5d5dU, 0x2b321919U, 0x95e67373U,
- 0xa0c06060U, 0x98198181U, 0xd19e4f4fU, 0x7fa3dcdcU,
- 0x66442222U, 0x7e542a2aU, 0xab3b9090U, 0x830b8888U,
- 0xca8c4646U, 0x29c7eeeeU, 0xd36bb8b8U, 0x3c281414U,
- 0x79a7dedeU, 0xe2bc5e5eU, 0x1d160b0bU, 0x76addbdbU,
- 0x3bdbe0e0U, 0x56643232U, 0x4e743a3aU, 0x1e140a0aU,
- 0xdb924949U, 0x0a0c0606U, 0x6c482424U, 0xe4b85c5cU,
- 0x5d9fc2c2U, 0x6ebdd3d3U, 0xef43acacU, 0xa6c46262U,
- 0xa8399191U, 0xa4319595U, 0x37d3e4e4U, 0x8bf27979U,
- 0x32d5e7e7U, 0x438bc8c8U, 0x596e3737U, 0xb7da6d6dU,
- 0x8c018d8dU, 0x64b1d5d5U, 0xd29c4e4eU, 0xe049a9a9U,
- 0xb4d86c6cU, 0xfaac5656U, 0x07f3f4f4U, 0x25cfeaeaU,
- 0xafca6565U, 0x8ef47a7aU, 0xe947aeaeU, 0x18100808U,
- 0xd56fbabaU, 0x88f07878U, 0x6f4a2525U, 0x725c2e2eU,
- 0x24381c1cU, 0xf157a6a6U, 0xc773b4b4U, 0x5197c6c6U,
- 0x23cbe8e8U, 0x7ca1ddddU, 0x9ce87474U, 0x213e1f1fU,
- 0xdd964b4bU, 0xdc61bdbdU, 0x860d8b8bU, 0x850f8a8aU,
- 0x90e07070U, 0x427c3e3eU, 0xc471b5b5U, 0xaacc6666U,
- 0xd8904848U, 0x05060303U, 0x01f7f6f6U, 0x121c0e0eU,
- 0xa3c26161U, 0x5f6a3535U, 0xf9ae5757U, 0xd069b9b9U,
- 0x91178686U, 0x5899c1c1U, 0x273a1d1dU, 0xb9279e9eU,
- 0x38d9e1e1U, 0x13ebf8f8U, 0xb32b9898U, 0x33221111U,
- 0xbbd26969U, 0x70a9d9d9U, 0x89078e8eU, 0xa7339494U,
- 0xb62d9b9bU, 0x223c1e1eU, 0x92158787U, 0x20c9e9e9U,
- 0x4987ceceU, 0xffaa5555U, 0x78502828U, 0x7aa5dfdfU,
- 0x8f038c8cU, 0xf859a1a1U, 0x80098989U, 0x171a0d0dU,
- 0xda65bfbfU, 0x31d7e6e6U, 0xc6844242U, 0xb8d06868U,
- 0xc3824141U, 0xb0299999U, 0x775a2d2dU, 0x111e0f0fU,
- 0xcb7bb0b0U, 0xfca85454U, 0xd66dbbbbU, 0x3a2c1616U,
-};
-static const u32 Te2[256] = {
- 0x63a5c663U, 0x7c84f87cU, 0x7799ee77U, 0x7b8df67bU,
- 0xf20dfff2U, 0x6bbdd66bU, 0x6fb1de6fU, 0xc55491c5U,
- 0x30506030U, 0x01030201U, 0x67a9ce67U, 0x2b7d562bU,
- 0xfe19e7feU, 0xd762b5d7U, 0xabe64dabU, 0x769aec76U,
- 0xca458fcaU, 0x829d1f82U, 0xc94089c9U, 0x7d87fa7dU,
- 0xfa15effaU, 0x59ebb259U, 0x47c98e47U, 0xf00bfbf0U,
- 0xadec41adU, 0xd467b3d4U, 0xa2fd5fa2U, 0xafea45afU,
- 0x9cbf239cU, 0xa4f753a4U, 0x7296e472U, 0xc05b9bc0U,
- 0xb7c275b7U, 0xfd1ce1fdU, 0x93ae3d93U, 0x266a4c26U,
- 0x365a6c36U, 0x3f417e3fU, 0xf702f5f7U, 0xcc4f83ccU,
- 0x345c6834U, 0xa5f451a5U, 0xe534d1e5U, 0xf108f9f1U,
- 0x7193e271U, 0xd873abd8U, 0x31536231U, 0x153f2a15U,
- 0x040c0804U, 0xc75295c7U, 0x23654623U, 0xc35e9dc3U,
- 0x18283018U, 0x96a13796U, 0x050f0a05U, 0x9ab52f9aU,
- 0x07090e07U, 0x12362412U, 0x809b1b80U, 0xe23ddfe2U,
- 0xeb26cdebU, 0x27694e27U, 0xb2cd7fb2U, 0x759fea75U,
- 0x091b1209U, 0x839e1d83U, 0x2c74582cU, 0x1a2e341aU,
- 0x1b2d361bU, 0x6eb2dc6eU, 0x5aeeb45aU, 0xa0fb5ba0U,
- 0x52f6a452U, 0x3b4d763bU, 0xd661b7d6U, 0xb3ce7db3U,
- 0x297b5229U, 0xe33edde3U, 0x2f715e2fU, 0x84971384U,
- 0x53f5a653U, 0xd168b9d1U, 0x00000000U, 0xed2cc1edU,
- 0x20604020U, 0xfc1fe3fcU, 0xb1c879b1U, 0x5bedb65bU,
- 0x6abed46aU, 0xcb468dcbU, 0xbed967beU, 0x394b7239U,
- 0x4ade944aU, 0x4cd4984cU, 0x58e8b058U, 0xcf4a85cfU,
- 0xd06bbbd0U, 0xef2ac5efU, 0xaae54faaU, 0xfb16edfbU,
- 0x43c58643U, 0x4dd79a4dU, 0x33556633U, 0x85941185U,
- 0x45cf8a45U, 0xf910e9f9U, 0x02060402U, 0x7f81fe7fU,
- 0x50f0a050U, 0x3c44783cU, 0x9fba259fU, 0xa8e34ba8U,
- 0x51f3a251U, 0xa3fe5da3U, 0x40c08040U, 0x8f8a058fU,
- 0x92ad3f92U, 0x9dbc219dU, 0x38487038U, 0xf504f1f5U,
- 0xbcdf63bcU, 0xb6c177b6U, 0xda75afdaU, 0x21634221U,
- 0x10302010U, 0xff1ae5ffU, 0xf30efdf3U, 0xd26dbfd2U,
- 0xcd4c81cdU, 0x0c14180cU, 0x13352613U, 0xec2fc3ecU,
- 0x5fe1be5fU, 0x97a23597U, 0x44cc8844U, 0x17392e17U,
- 0xc45793c4U, 0xa7f255a7U, 0x7e82fc7eU, 0x3d477a3dU,
- 0x64acc864U, 0x5de7ba5dU, 0x192b3219U, 0x7395e673U,
- 0x60a0c060U, 0x81981981U, 0x4fd19e4fU, 0xdc7fa3dcU,
- 0x22664422U, 0x2a7e542aU, 0x90ab3b90U, 0x88830b88U,
- 0x46ca8c46U, 0xee29c7eeU, 0xb8d36bb8U, 0x143c2814U,
- 0xde79a7deU, 0x5ee2bc5eU, 0x0b1d160bU, 0xdb76addbU,
- 0xe03bdbe0U, 0x32566432U, 0x3a4e743aU, 0x0a1e140aU,
- 0x49db9249U, 0x060a0c06U, 0x246c4824U, 0x5ce4b85cU,
- 0xc25d9fc2U, 0xd36ebdd3U, 0xacef43acU, 0x62a6c462U,
- 0x91a83991U, 0x95a43195U, 0xe437d3e4U, 0x798bf279U,
- 0xe732d5e7U, 0xc8438bc8U, 0x37596e37U, 0x6db7da6dU,
- 0x8d8c018dU, 0xd564b1d5U, 0x4ed29c4eU, 0xa9e049a9U,
- 0x6cb4d86cU, 0x56faac56U, 0xf407f3f4U, 0xea25cfeaU,
- 0x65afca65U, 0x7a8ef47aU, 0xaee947aeU, 0x08181008U,
- 0xbad56fbaU, 0x7888f078U, 0x256f4a25U, 0x2e725c2eU,
- 0x1c24381cU, 0xa6f157a6U, 0xb4c773b4U, 0xc65197c6U,
- 0xe823cbe8U, 0xdd7ca1ddU, 0x749ce874U, 0x1f213e1fU,
- 0x4bdd964bU, 0xbddc61bdU, 0x8b860d8bU, 0x8a850f8aU,
- 0x7090e070U, 0x3e427c3eU, 0xb5c471b5U, 0x66aacc66U,
- 0x48d89048U, 0x03050603U, 0xf601f7f6U, 0x0e121c0eU,
- 0x61a3c261U, 0x355f6a35U, 0x57f9ae57U, 0xb9d069b9U,
- 0x86911786U, 0xc15899c1U, 0x1d273a1dU, 0x9eb9279eU,
- 0xe138d9e1U, 0xf813ebf8U, 0x98b32b98U, 0x11332211U,
- 0x69bbd269U, 0xd970a9d9U, 0x8e89078eU, 0x94a73394U,
- 0x9bb62d9bU, 0x1e223c1eU, 0x87921587U, 0xe920c9e9U,
- 0xce4987ceU, 0x55ffaa55U, 0x28785028U, 0xdf7aa5dfU,
- 0x8c8f038cU, 0xa1f859a1U, 0x89800989U, 0x0d171a0dU,
- 0xbfda65bfU, 0xe631d7e6U, 0x42c68442U, 0x68b8d068U,
- 0x41c38241U, 0x99b02999U, 0x2d775a2dU, 0x0f111e0fU,
- 0xb0cb7bb0U, 0x54fca854U, 0xbbd66dbbU, 0x163a2c16U,
-};
-static const u32 Te3[256] = {
-
- 0x6363a5c6U, 0x7c7c84f8U, 0x777799eeU, 0x7b7b8df6U,
- 0xf2f20dffU, 0x6b6bbdd6U, 0x6f6fb1deU, 0xc5c55491U,
- 0x30305060U, 0x01010302U, 0x6767a9ceU, 0x2b2b7d56U,
- 0xfefe19e7U, 0xd7d762b5U, 0xababe64dU, 0x76769aecU,
- 0xcaca458fU, 0x82829d1fU, 0xc9c94089U, 0x7d7d87faU,
- 0xfafa15efU, 0x5959ebb2U, 0x4747c98eU, 0xf0f00bfbU,
- 0xadadec41U, 0xd4d467b3U, 0xa2a2fd5fU, 0xafafea45U,
- 0x9c9cbf23U, 0xa4a4f753U, 0x727296e4U, 0xc0c05b9bU,
- 0xb7b7c275U, 0xfdfd1ce1U, 0x9393ae3dU, 0x26266a4cU,
- 0x36365a6cU, 0x3f3f417eU, 0xf7f702f5U, 0xcccc4f83U,
- 0x34345c68U, 0xa5a5f451U, 0xe5e534d1U, 0xf1f108f9U,
- 0x717193e2U, 0xd8d873abU, 0x31315362U, 0x15153f2aU,
- 0x04040c08U, 0xc7c75295U, 0x23236546U, 0xc3c35e9dU,
- 0x18182830U, 0x9696a137U, 0x05050f0aU, 0x9a9ab52fU,
- 0x0707090eU, 0x12123624U, 0x80809b1bU, 0xe2e23ddfU,
- 0xebeb26cdU, 0x2727694eU, 0xb2b2cd7fU, 0x75759feaU,
- 0x09091b12U, 0x83839e1dU, 0x2c2c7458U, 0x1a1a2e34U,
- 0x1b1b2d36U, 0x6e6eb2dcU, 0x5a5aeeb4U, 0xa0a0fb5bU,
- 0x5252f6a4U, 0x3b3b4d76U, 0xd6d661b7U, 0xb3b3ce7dU,
- 0x29297b52U, 0xe3e33eddU, 0x2f2f715eU, 0x84849713U,
- 0x5353f5a6U, 0xd1d168b9U, 0x00000000U, 0xeded2cc1U,
- 0x20206040U, 0xfcfc1fe3U, 0xb1b1c879U, 0x5b5bedb6U,
- 0x6a6abed4U, 0xcbcb468dU, 0xbebed967U, 0x39394b72U,
- 0x4a4ade94U, 0x4c4cd498U, 0x5858e8b0U, 0xcfcf4a85U,
- 0xd0d06bbbU, 0xefef2ac5U, 0xaaaae54fU, 0xfbfb16edU,
- 0x4343c586U, 0x4d4dd79aU, 0x33335566U, 0x85859411U,
- 0x4545cf8aU, 0xf9f910e9U, 0x02020604U, 0x7f7f81feU,
- 0x5050f0a0U, 0x3c3c4478U, 0x9f9fba25U, 0xa8a8e34bU,
- 0x5151f3a2U, 0xa3a3fe5dU, 0x4040c080U, 0x8f8f8a05U,
- 0x9292ad3fU, 0x9d9dbc21U, 0x38384870U, 0xf5f504f1U,
- 0xbcbcdf63U, 0xb6b6c177U, 0xdada75afU, 0x21216342U,
- 0x10103020U, 0xffff1ae5U, 0xf3f30efdU, 0xd2d26dbfU,
- 0xcdcd4c81U, 0x0c0c1418U, 0x13133526U, 0xecec2fc3U,
- 0x5f5fe1beU, 0x9797a235U, 0x4444cc88U, 0x1717392eU,
- 0xc4c45793U, 0xa7a7f255U, 0x7e7e82fcU, 0x3d3d477aU,
- 0x6464acc8U, 0x5d5de7baU, 0x19192b32U, 0x737395e6U,
- 0x6060a0c0U, 0x81819819U, 0x4f4fd19eU, 0xdcdc7fa3U,
- 0x22226644U, 0x2a2a7e54U, 0x9090ab3bU, 0x8888830bU,
- 0x4646ca8cU, 0xeeee29c7U, 0xb8b8d36bU, 0x14143c28U,
- 0xdede79a7U, 0x5e5ee2bcU, 0x0b0b1d16U, 0xdbdb76adU,
- 0xe0e03bdbU, 0x32325664U, 0x3a3a4e74U, 0x0a0a1e14U,
- 0x4949db92U, 0x06060a0cU, 0x24246c48U, 0x5c5ce4b8U,
- 0xc2c25d9fU, 0xd3d36ebdU, 0xacacef43U, 0x6262a6c4U,
- 0x9191a839U, 0x9595a431U, 0xe4e437d3U, 0x79798bf2U,
- 0xe7e732d5U, 0xc8c8438bU, 0x3737596eU, 0x6d6db7daU,
- 0x8d8d8c01U, 0xd5d564b1U, 0x4e4ed29cU, 0xa9a9e049U,
- 0x6c6cb4d8U, 0x5656faacU, 0xf4f407f3U, 0xeaea25cfU,
- 0x6565afcaU, 0x7a7a8ef4U, 0xaeaee947U, 0x08081810U,
- 0xbabad56fU, 0x787888f0U, 0x25256f4aU, 0x2e2e725cU,
- 0x1c1c2438U, 0xa6a6f157U, 0xb4b4c773U, 0xc6c65197U,
- 0xe8e823cbU, 0xdddd7ca1U, 0x74749ce8U, 0x1f1f213eU,
- 0x4b4bdd96U, 0xbdbddc61U, 0x8b8b860dU, 0x8a8a850fU,
- 0x707090e0U, 0x3e3e427cU, 0xb5b5c471U, 0x6666aaccU,
- 0x4848d890U, 0x03030506U, 0xf6f601f7U, 0x0e0e121cU,
- 0x6161a3c2U, 0x35355f6aU, 0x5757f9aeU, 0xb9b9d069U,
- 0x86869117U, 0xc1c15899U, 0x1d1d273aU, 0x9e9eb927U,
- 0xe1e138d9U, 0xf8f813ebU, 0x9898b32bU, 0x11113322U,
- 0x6969bbd2U, 0xd9d970a9U, 0x8e8e8907U, 0x9494a733U,
- 0x9b9bb62dU, 0x1e1e223cU, 0x87879215U, 0xe9e920c9U,
- 0xcece4987U, 0x5555ffaaU, 0x28287850U, 0xdfdf7aa5U,
- 0x8c8c8f03U, 0xa1a1f859U, 0x89898009U, 0x0d0d171aU,
- 0xbfbfda65U, 0xe6e631d7U, 0x4242c684U, 0x6868b8d0U,
- 0x4141c382U, 0x9999b029U, 0x2d2d775aU, 0x0f0f111eU,
- 0xb0b0cb7bU, 0x5454fca8U, 0xbbbbd66dU, 0x16163a2cU,
-};
-static const u32 Te4[256] = {
- 0x63636363U, 0x7c7c7c7cU, 0x77777777U, 0x7b7b7b7bU,
- 0xf2f2f2f2U, 0x6b6b6b6bU, 0x6f6f6f6fU, 0xc5c5c5c5U,
- 0x30303030U, 0x01010101U, 0x67676767U, 0x2b2b2b2bU,
- 0xfefefefeU, 0xd7d7d7d7U, 0xababababU, 0x76767676U,
- 0xcacacacaU, 0x82828282U, 0xc9c9c9c9U, 0x7d7d7d7dU,
- 0xfafafafaU, 0x59595959U, 0x47474747U, 0xf0f0f0f0U,
- 0xadadadadU, 0xd4d4d4d4U, 0xa2a2a2a2U, 0xafafafafU,
- 0x9c9c9c9cU, 0xa4a4a4a4U, 0x72727272U, 0xc0c0c0c0U,
- 0xb7b7b7b7U, 0xfdfdfdfdU, 0x93939393U, 0x26262626U,
- 0x36363636U, 0x3f3f3f3fU, 0xf7f7f7f7U, 0xccccccccU,
- 0x34343434U, 0xa5a5a5a5U, 0xe5e5e5e5U, 0xf1f1f1f1U,
- 0x71717171U, 0xd8d8d8d8U, 0x31313131U, 0x15151515U,
- 0x04040404U, 0xc7c7c7c7U, 0x23232323U, 0xc3c3c3c3U,
- 0x18181818U, 0x96969696U, 0x05050505U, 0x9a9a9a9aU,
- 0x07070707U, 0x12121212U, 0x80808080U, 0xe2e2e2e2U,
- 0xebebebebU, 0x27272727U, 0xb2b2b2b2U, 0x75757575U,
- 0x09090909U, 0x83838383U, 0x2c2c2c2cU, 0x1a1a1a1aU,
- 0x1b1b1b1bU, 0x6e6e6e6eU, 0x5a5a5a5aU, 0xa0a0a0a0U,
- 0x52525252U, 0x3b3b3b3bU, 0xd6d6d6d6U, 0xb3b3b3b3U,
- 0x29292929U, 0xe3e3e3e3U, 0x2f2f2f2fU, 0x84848484U,
- 0x53535353U, 0xd1d1d1d1U, 0x00000000U, 0xededededU,
- 0x20202020U, 0xfcfcfcfcU, 0xb1b1b1b1U, 0x5b5b5b5bU,
- 0x6a6a6a6aU, 0xcbcbcbcbU, 0xbebebebeU, 0x39393939U,
- 0x4a4a4a4aU, 0x4c4c4c4cU, 0x58585858U, 0xcfcfcfcfU,
- 0xd0d0d0d0U, 0xefefefefU, 0xaaaaaaaaU, 0xfbfbfbfbU,
- 0x43434343U, 0x4d4d4d4dU, 0x33333333U, 0x85858585U,
- 0x45454545U, 0xf9f9f9f9U, 0x02020202U, 0x7f7f7f7fU,
- 0x50505050U, 0x3c3c3c3cU, 0x9f9f9f9fU, 0xa8a8a8a8U,
- 0x51515151U, 0xa3a3a3a3U, 0x40404040U, 0x8f8f8f8fU,
- 0x92929292U, 0x9d9d9d9dU, 0x38383838U, 0xf5f5f5f5U,
- 0xbcbcbcbcU, 0xb6b6b6b6U, 0xdadadadaU, 0x21212121U,
- 0x10101010U, 0xffffffffU, 0xf3f3f3f3U, 0xd2d2d2d2U,
- 0xcdcdcdcdU, 0x0c0c0c0cU, 0x13131313U, 0xececececU,
- 0x5f5f5f5fU, 0x97979797U, 0x44444444U, 0x17171717U,
- 0xc4c4c4c4U, 0xa7a7a7a7U, 0x7e7e7e7eU, 0x3d3d3d3dU,
- 0x64646464U, 0x5d5d5d5dU, 0x19191919U, 0x73737373U,
- 0x60606060U, 0x81818181U, 0x4f4f4f4fU, 0xdcdcdcdcU,
- 0x22222222U, 0x2a2a2a2aU, 0x90909090U, 0x88888888U,
- 0x46464646U, 0xeeeeeeeeU, 0xb8b8b8b8U, 0x14141414U,
- 0xdedededeU, 0x5e5e5e5eU, 0x0b0b0b0bU, 0xdbdbdbdbU,
- 0xe0e0e0e0U, 0x32323232U, 0x3a3a3a3aU, 0x0a0a0a0aU,
- 0x49494949U, 0x06060606U, 0x24242424U, 0x5c5c5c5cU,
- 0xc2c2c2c2U, 0xd3d3d3d3U, 0xacacacacU, 0x62626262U,
- 0x91919191U, 0x95959595U, 0xe4e4e4e4U, 0x79797979U,
- 0xe7e7e7e7U, 0xc8c8c8c8U, 0x37373737U, 0x6d6d6d6dU,
- 0x8d8d8d8dU, 0xd5d5d5d5U, 0x4e4e4e4eU, 0xa9a9a9a9U,
- 0x6c6c6c6cU, 0x56565656U, 0xf4f4f4f4U, 0xeaeaeaeaU,
- 0x65656565U, 0x7a7a7a7aU, 0xaeaeaeaeU, 0x08080808U,
- 0xbabababaU, 0x78787878U, 0x25252525U, 0x2e2e2e2eU,
- 0x1c1c1c1cU, 0xa6a6a6a6U, 0xb4b4b4b4U, 0xc6c6c6c6U,
- 0xe8e8e8e8U, 0xddddddddU, 0x74747474U, 0x1f1f1f1fU,
- 0x4b4b4b4bU, 0xbdbdbdbdU, 0x8b8b8b8bU, 0x8a8a8a8aU,
- 0x70707070U, 0x3e3e3e3eU, 0xb5b5b5b5U, 0x66666666U,
- 0x48484848U, 0x03030303U, 0xf6f6f6f6U, 0x0e0e0e0eU,
- 0x61616161U, 0x35353535U, 0x57575757U, 0xb9b9b9b9U,
- 0x86868686U, 0xc1c1c1c1U, 0x1d1d1d1dU, 0x9e9e9e9eU,
- 0xe1e1e1e1U, 0xf8f8f8f8U, 0x98989898U, 0x11111111U,
- 0x69696969U, 0xd9d9d9d9U, 0x8e8e8e8eU, 0x94949494U,
- 0x9b9b9b9bU, 0x1e1e1e1eU, 0x87878787U, 0xe9e9e9e9U,
- 0xcecececeU, 0x55555555U, 0x28282828U, 0xdfdfdfdfU,
- 0x8c8c8c8cU, 0xa1a1a1a1U, 0x89898989U, 0x0d0d0d0dU,
- 0xbfbfbfbfU, 0xe6e6e6e6U, 0x42424242U, 0x68686868U,
- 0x41414141U, 0x99999999U, 0x2d2d2d2dU, 0x0f0f0f0fU,
- 0xb0b0b0b0U, 0x54545454U, 0xbbbbbbbbU, 0x16161616U,
-};
-static const u32 Td0[256] = {
- 0x51f4a750U, 0x7e416553U, 0x1a17a4c3U, 0x3a275e96U,
- 0x3bab6bcbU, 0x1f9d45f1U, 0xacfa58abU, 0x4be30393U,
- 0x2030fa55U, 0xad766df6U, 0x88cc7691U, 0xf5024c25U,
- 0x4fe5d7fcU, 0xc52acbd7U, 0x26354480U, 0xb562a38fU,
- 0xdeb15a49U, 0x25ba1b67U, 0x45ea0e98U, 0x5dfec0e1U,
- 0xc32f7502U, 0x814cf012U, 0x8d4697a3U, 0x6bd3f9c6U,
- 0x038f5fe7U, 0x15929c95U, 0xbf6d7aebU, 0x955259daU,
- 0xd4be832dU, 0x587421d3U, 0x49e06929U, 0x8ec9c844U,
- 0x75c2896aU, 0xf48e7978U, 0x99583e6bU, 0x27b971ddU,
- 0xbee14fb6U, 0xf088ad17U, 0xc920ac66U, 0x7dce3ab4U,
- 0x63df4a18U, 0xe51a3182U, 0x97513360U, 0x62537f45U,
- 0xb16477e0U, 0xbb6bae84U, 0xfe81a01cU, 0xf9082b94U,
- 0x70486858U, 0x8f45fd19U, 0x94de6c87U, 0x527bf8b7U,
- 0xab73d323U, 0x724b02e2U, 0xe31f8f57U, 0x6655ab2aU,
- 0xb2eb2807U, 0x2fb5c203U, 0x86c57b9aU, 0xd33708a5U,
- 0x302887f2U, 0x23bfa5b2U, 0x02036abaU, 0xed16825cU,
- 0x8acf1c2bU, 0xa779b492U, 0xf307f2f0U, 0x4e69e2a1U,
- 0x65daf4cdU, 0x0605bed5U, 0xd134621fU, 0xc4a6fe8aU,
- 0x342e539dU, 0xa2f355a0U, 0x058ae132U, 0xa4f6eb75U,
- 0x0b83ec39U, 0x4060efaaU, 0x5e719f06U, 0xbd6e1051U,
- 0x3e218af9U, 0x96dd063dU, 0xdd3e05aeU, 0x4de6bd46U,
- 0x91548db5U, 0x71c45d05U, 0x0406d46fU, 0x605015ffU,
- 0x1998fb24U, 0xd6bde997U, 0x894043ccU, 0x67d99e77U,
- 0xb0e842bdU, 0x07898b88U, 0xe7195b38U, 0x79c8eedbU,
- 0xa17c0a47U, 0x7c420fe9U, 0xf8841ec9U, 0x00000000U,
- 0x09808683U, 0x322bed48U, 0x1e1170acU, 0x6c5a724eU,
- 0xfd0efffbU, 0x0f853856U, 0x3daed51eU, 0x362d3927U,
- 0x0a0fd964U, 0x685ca621U, 0x9b5b54d1U, 0x24362e3aU,
- 0x0c0a67b1U, 0x9357e70fU, 0xb4ee96d2U, 0x1b9b919eU,
- 0x80c0c54fU, 0x61dc20a2U, 0x5a774b69U, 0x1c121a16U,
- 0xe293ba0aU, 0xc0a02ae5U, 0x3c22e043U, 0x121b171dU,
- 0x0e090d0bU, 0xf28bc7adU, 0x2db6a8b9U, 0x141ea9c8U,
- 0x57f11985U, 0xaf75074cU, 0xee99ddbbU, 0xa37f60fdU,
- 0xf701269fU, 0x5c72f5bcU, 0x44663bc5U, 0x5bfb7e34U,
- 0x8b432976U, 0xcb23c6dcU, 0xb6edfc68U, 0xb8e4f163U,
- 0xd731dccaU, 0x42638510U, 0x13972240U, 0x84c61120U,
- 0x854a247dU, 0xd2bb3df8U, 0xaef93211U, 0xc729a16dU,
- 0x1d9e2f4bU, 0xdcb230f3U, 0x0d8652ecU, 0x77c1e3d0U,
- 0x2bb3166cU, 0xa970b999U, 0x119448faU, 0x47e96422U,
- 0xa8fc8cc4U, 0xa0f03f1aU, 0x567d2cd8U, 0x223390efU,
- 0x87494ec7U, 0xd938d1c1U, 0x8ccaa2feU, 0x98d40b36U,
- 0xa6f581cfU, 0xa57ade28U, 0xdab78e26U, 0x3fadbfa4U,
- 0x2c3a9de4U, 0x5078920dU, 0x6a5fcc9bU, 0x547e4662U,
- 0xf68d13c2U, 0x90d8b8e8U, 0x2e39f75eU, 0x82c3aff5U,
- 0x9f5d80beU, 0x69d0937cU, 0x6fd52da9U, 0xcf2512b3U,
- 0xc8ac993bU, 0x10187da7U, 0xe89c636eU, 0xdb3bbb7bU,
- 0xcd267809U, 0x6e5918f4U, 0xec9ab701U, 0x834f9aa8U,
- 0xe6956e65U, 0xaaffe67eU, 0x21bccf08U, 0xef15e8e6U,
- 0xbae79bd9U, 0x4a6f36ceU, 0xea9f09d4U, 0x29b07cd6U,
- 0x31a4b2afU, 0x2a3f2331U, 0xc6a59430U, 0x35a266c0U,
- 0x744ebc37U, 0xfc82caa6U, 0xe090d0b0U, 0x33a7d815U,
- 0xf104984aU, 0x41ecdaf7U, 0x7fcd500eU, 0x1791f62fU,
- 0x764dd68dU, 0x43efb04dU, 0xccaa4d54U, 0xe49604dfU,
- 0x9ed1b5e3U, 0x4c6a881bU, 0xc12c1fb8U, 0x4665517fU,
- 0x9d5eea04U, 0x018c355dU, 0xfa877473U, 0xfb0b412eU,
- 0xb3671d5aU, 0x92dbd252U, 0xe9105633U, 0x6dd64713U,
- 0x9ad7618cU, 0x37a10c7aU, 0x59f8148eU, 0xeb133c89U,
- 0xcea927eeU, 0xb761c935U, 0xe11ce5edU, 0x7a47b13cU,
- 0x9cd2df59U, 0x55f2733fU, 0x1814ce79U, 0x73c737bfU,
- 0x53f7cdeaU, 0x5ffdaa5bU, 0xdf3d6f14U, 0x7844db86U,
- 0xcaaff381U, 0xb968c43eU, 0x3824342cU, 0xc2a3405fU,
- 0x161dc372U, 0xbce2250cU, 0x283c498bU, 0xff0d9541U,
- 0x39a80171U, 0x080cb3deU, 0xd8b4e49cU, 0x6456c190U,
- 0x7bcb8461U, 0xd532b670U, 0x486c5c74U, 0xd0b85742U,
-};
-static const u32 Td1[256] = {
- 0x5051f4a7U, 0x537e4165U, 0xc31a17a4U, 0x963a275eU,
- 0xcb3bab6bU, 0xf11f9d45U, 0xabacfa58U, 0x934be303U,
- 0x552030faU, 0xf6ad766dU, 0x9188cc76U, 0x25f5024cU,
- 0xfc4fe5d7U, 0xd7c52acbU, 0x80263544U, 0x8fb562a3U,
- 0x49deb15aU, 0x6725ba1bU, 0x9845ea0eU, 0xe15dfec0U,
- 0x02c32f75U, 0x12814cf0U, 0xa38d4697U, 0xc66bd3f9U,
- 0xe7038f5fU, 0x9515929cU, 0xebbf6d7aU, 0xda955259U,
- 0x2dd4be83U, 0xd3587421U, 0x2949e069U, 0x448ec9c8U,
- 0x6a75c289U, 0x78f48e79U, 0x6b99583eU, 0xdd27b971U,
- 0xb6bee14fU, 0x17f088adU, 0x66c920acU, 0xb47dce3aU,
- 0x1863df4aU, 0x82e51a31U, 0x60975133U, 0x4562537fU,
- 0xe0b16477U, 0x84bb6baeU, 0x1cfe81a0U, 0x94f9082bU,
- 0x58704868U, 0x198f45fdU, 0x8794de6cU, 0xb7527bf8U,
- 0x23ab73d3U, 0xe2724b02U, 0x57e31f8fU, 0x2a6655abU,
- 0x07b2eb28U, 0x032fb5c2U, 0x9a86c57bU, 0xa5d33708U,
- 0xf2302887U, 0xb223bfa5U, 0xba02036aU, 0x5ced1682U,
- 0x2b8acf1cU, 0x92a779b4U, 0xf0f307f2U, 0xa14e69e2U,
- 0xcd65daf4U, 0xd50605beU, 0x1fd13462U, 0x8ac4a6feU,
- 0x9d342e53U, 0xa0a2f355U, 0x32058ae1U, 0x75a4f6ebU,
- 0x390b83ecU, 0xaa4060efU, 0x065e719fU, 0x51bd6e10U,
- 0xf93e218aU, 0x3d96dd06U, 0xaedd3e05U, 0x464de6bdU,
- 0xb591548dU, 0x0571c45dU, 0x6f0406d4U, 0xff605015U,
- 0x241998fbU, 0x97d6bde9U, 0xcc894043U, 0x7767d99eU,
- 0xbdb0e842U, 0x8807898bU, 0x38e7195bU, 0xdb79c8eeU,
- 0x47a17c0aU, 0xe97c420fU, 0xc9f8841eU, 0x00000000U,
- 0x83098086U, 0x48322bedU, 0xac1e1170U, 0x4e6c5a72U,
- 0xfbfd0effU, 0x560f8538U, 0x1e3daed5U, 0x27362d39U,
- 0x640a0fd9U, 0x21685ca6U, 0xd19b5b54U, 0x3a24362eU,
- 0xb10c0a67U, 0x0f9357e7U, 0xd2b4ee96U, 0x9e1b9b91U,
- 0x4f80c0c5U, 0xa261dc20U, 0x695a774bU, 0x161c121aU,
- 0x0ae293baU, 0xe5c0a02aU, 0x433c22e0U, 0x1d121b17U,
- 0x0b0e090dU, 0xadf28bc7U, 0xb92db6a8U, 0xc8141ea9U,
- 0x8557f119U, 0x4caf7507U, 0xbbee99ddU, 0xfda37f60U,
- 0x9ff70126U, 0xbc5c72f5U, 0xc544663bU, 0x345bfb7eU,
- 0x768b4329U, 0xdccb23c6U, 0x68b6edfcU, 0x63b8e4f1U,
- 0xcad731dcU, 0x10426385U, 0x40139722U, 0x2084c611U,
- 0x7d854a24U, 0xf8d2bb3dU, 0x11aef932U, 0x6dc729a1U,
- 0x4b1d9e2fU, 0xf3dcb230U, 0xec0d8652U, 0xd077c1e3U,
- 0x6c2bb316U, 0x99a970b9U, 0xfa119448U, 0x2247e964U,
- 0xc4a8fc8cU, 0x1aa0f03fU, 0xd8567d2cU, 0xef223390U,
- 0xc787494eU, 0xc1d938d1U, 0xfe8ccaa2U, 0x3698d40bU,
- 0xcfa6f581U, 0x28a57adeU, 0x26dab78eU, 0xa43fadbfU,
- 0xe42c3a9dU, 0x0d507892U, 0x9b6a5fccU, 0x62547e46U,
- 0xc2f68d13U, 0xe890d8b8U, 0x5e2e39f7U, 0xf582c3afU,
- 0xbe9f5d80U, 0x7c69d093U, 0xa96fd52dU, 0xb3cf2512U,
- 0x3bc8ac99U, 0xa710187dU, 0x6ee89c63U, 0x7bdb3bbbU,
- 0x09cd2678U, 0xf46e5918U, 0x01ec9ab7U, 0xa8834f9aU,
- 0x65e6956eU, 0x7eaaffe6U, 0x0821bccfU, 0xe6ef15e8U,
- 0xd9bae79bU, 0xce4a6f36U, 0xd4ea9f09U, 0xd629b07cU,
- 0xaf31a4b2U, 0x312a3f23U, 0x30c6a594U, 0xc035a266U,
- 0x37744ebcU, 0xa6fc82caU, 0xb0e090d0U, 0x1533a7d8U,
- 0x4af10498U, 0xf741ecdaU, 0x0e7fcd50U, 0x2f1791f6U,
- 0x8d764dd6U, 0x4d43efb0U, 0x54ccaa4dU, 0xdfe49604U,
- 0xe39ed1b5U, 0x1b4c6a88U, 0xb8c12c1fU, 0x7f466551U,
- 0x049d5eeaU, 0x5d018c35U, 0x73fa8774U, 0x2efb0b41U,
- 0x5ab3671dU, 0x5292dbd2U, 0x33e91056U, 0x136dd647U,
- 0x8c9ad761U, 0x7a37a10cU, 0x8e59f814U, 0x89eb133cU,
- 0xeecea927U, 0x35b761c9U, 0xede11ce5U, 0x3c7a47b1U,
- 0x599cd2dfU, 0x3f55f273U, 0x791814ceU, 0xbf73c737U,
- 0xea53f7cdU, 0x5b5ffdaaU, 0x14df3d6fU, 0x867844dbU,
- 0x81caaff3U, 0x3eb968c4U, 0x2c382434U, 0x5fc2a340U,
- 0x72161dc3U, 0x0cbce225U, 0x8b283c49U, 0x41ff0d95U,
- 0x7139a801U, 0xde080cb3U, 0x9cd8b4e4U, 0x906456c1U,
- 0x617bcb84U, 0x70d532b6U, 0x74486c5cU, 0x42d0b857U,
-};
-static const u32 Td2[256] = {
- 0xa75051f4U, 0x65537e41U, 0xa4c31a17U, 0x5e963a27U,
- 0x6bcb3babU, 0x45f11f9dU, 0x58abacfaU, 0x03934be3U,
- 0xfa552030U, 0x6df6ad76U, 0x769188ccU, 0x4c25f502U,
- 0xd7fc4fe5U, 0xcbd7c52aU, 0x44802635U, 0xa38fb562U,
- 0x5a49deb1U, 0x1b6725baU, 0x0e9845eaU, 0xc0e15dfeU,
- 0x7502c32fU, 0xf012814cU, 0x97a38d46U, 0xf9c66bd3U,
- 0x5fe7038fU, 0x9c951592U, 0x7aebbf6dU, 0x59da9552U,
- 0x832dd4beU, 0x21d35874U, 0x692949e0U, 0xc8448ec9U,
- 0x896a75c2U, 0x7978f48eU, 0x3e6b9958U, 0x71dd27b9U,
- 0x4fb6bee1U, 0xad17f088U, 0xac66c920U, 0x3ab47dceU,
- 0x4a1863dfU, 0x3182e51aU, 0x33609751U, 0x7f456253U,
- 0x77e0b164U, 0xae84bb6bU, 0xa01cfe81U, 0x2b94f908U,
- 0x68587048U, 0xfd198f45U, 0x6c8794deU, 0xf8b7527bU,
- 0xd323ab73U, 0x02e2724bU, 0x8f57e31fU, 0xab2a6655U,
- 0x2807b2ebU, 0xc2032fb5U, 0x7b9a86c5U, 0x08a5d337U,
- 0x87f23028U, 0xa5b223bfU, 0x6aba0203U, 0x825ced16U,
- 0x1c2b8acfU, 0xb492a779U, 0xf2f0f307U, 0xe2a14e69U,
- 0xf4cd65daU, 0xbed50605U, 0x621fd134U, 0xfe8ac4a6U,
- 0x539d342eU, 0x55a0a2f3U, 0xe132058aU, 0xeb75a4f6U,
- 0xec390b83U, 0xefaa4060U, 0x9f065e71U, 0x1051bd6eU,
-
- 0x8af93e21U, 0x063d96ddU, 0x05aedd3eU, 0xbd464de6U,
- 0x8db59154U, 0x5d0571c4U, 0xd46f0406U, 0x15ff6050U,
- 0xfb241998U, 0xe997d6bdU, 0x43cc8940U, 0x9e7767d9U,
- 0x42bdb0e8U, 0x8b880789U, 0x5b38e719U, 0xeedb79c8U,
- 0x0a47a17cU, 0x0fe97c42U, 0x1ec9f884U, 0x00000000U,
- 0x86830980U, 0xed48322bU, 0x70ac1e11U, 0x724e6c5aU,
- 0xfffbfd0eU, 0x38560f85U, 0xd51e3daeU, 0x3927362dU,
- 0xd9640a0fU, 0xa621685cU, 0x54d19b5bU, 0x2e3a2436U,
- 0x67b10c0aU, 0xe70f9357U, 0x96d2b4eeU, 0x919e1b9bU,
- 0xc54f80c0U, 0x20a261dcU, 0x4b695a77U, 0x1a161c12U,
- 0xba0ae293U, 0x2ae5c0a0U, 0xe0433c22U, 0x171d121bU,
- 0x0d0b0e09U, 0xc7adf28bU, 0xa8b92db6U, 0xa9c8141eU,
- 0x198557f1U, 0x074caf75U, 0xddbbee99U, 0x60fda37fU,
- 0x269ff701U, 0xf5bc5c72U, 0x3bc54466U, 0x7e345bfbU,
- 0x29768b43U, 0xc6dccb23U, 0xfc68b6edU, 0xf163b8e4U,
- 0xdccad731U, 0x85104263U, 0x22401397U, 0x112084c6U,
- 0x247d854aU, 0x3df8d2bbU, 0x3211aef9U, 0xa16dc729U,
- 0x2f4b1d9eU, 0x30f3dcb2U, 0x52ec0d86U, 0xe3d077c1U,
- 0x166c2bb3U, 0xb999a970U, 0x48fa1194U, 0x642247e9U,
- 0x8cc4a8fcU, 0x3f1aa0f0U, 0x2cd8567dU, 0x90ef2233U,
- 0x4ec78749U, 0xd1c1d938U, 0xa2fe8ccaU, 0x0b3698d4U,
- 0x81cfa6f5U, 0xde28a57aU, 0x8e26dab7U, 0xbfa43fadU,
- 0x9de42c3aU, 0x920d5078U, 0xcc9b6a5fU, 0x4662547eU,
- 0x13c2f68dU, 0xb8e890d8U, 0xf75e2e39U, 0xaff582c3U,
- 0x80be9f5dU, 0x937c69d0U, 0x2da96fd5U, 0x12b3cf25U,
- 0x993bc8acU, 0x7da71018U, 0x636ee89cU, 0xbb7bdb3bU,
- 0x7809cd26U, 0x18f46e59U, 0xb701ec9aU, 0x9aa8834fU,
- 0x6e65e695U, 0xe67eaaffU, 0xcf0821bcU, 0xe8e6ef15U,
- 0x9bd9bae7U, 0x36ce4a6fU, 0x09d4ea9fU, 0x7cd629b0U,
- 0xb2af31a4U, 0x23312a3fU, 0x9430c6a5U, 0x66c035a2U,
- 0xbc37744eU, 0xcaa6fc82U, 0xd0b0e090U, 0xd81533a7U,
- 0x984af104U, 0xdaf741ecU, 0x500e7fcdU, 0xf62f1791U,
- 0xd68d764dU, 0xb04d43efU, 0x4d54ccaaU, 0x04dfe496U,
- 0xb5e39ed1U, 0x881b4c6aU, 0x1fb8c12cU, 0x517f4665U,
- 0xea049d5eU, 0x355d018cU, 0x7473fa87U, 0x412efb0bU,
- 0x1d5ab367U, 0xd25292dbU, 0x5633e910U, 0x47136dd6U,
- 0x618c9ad7U, 0x0c7a37a1U, 0x148e59f8U, 0x3c89eb13U,
- 0x27eecea9U, 0xc935b761U, 0xe5ede11cU, 0xb13c7a47U,
- 0xdf599cd2U, 0x733f55f2U, 0xce791814U, 0x37bf73c7U,
- 0xcdea53f7U, 0xaa5b5ffdU, 0x6f14df3dU, 0xdb867844U,
- 0xf381caafU, 0xc43eb968U, 0x342c3824U, 0x405fc2a3U,
- 0xc372161dU, 0x250cbce2U, 0x498b283cU, 0x9541ff0dU,
- 0x017139a8U, 0xb3de080cU, 0xe49cd8b4U, 0xc1906456U,
- 0x84617bcbU, 0xb670d532U, 0x5c74486cU, 0x5742d0b8U,
-};
-static const u32 Td3[256] = {
- 0xf4a75051U, 0x4165537eU, 0x17a4c31aU, 0x275e963aU,
- 0xab6bcb3bU, 0x9d45f11fU, 0xfa58abacU, 0xe303934bU,
- 0x30fa5520U, 0x766df6adU, 0xcc769188U, 0x024c25f5U,
- 0xe5d7fc4fU, 0x2acbd7c5U, 0x35448026U, 0x62a38fb5U,
- 0xb15a49deU, 0xba1b6725U, 0xea0e9845U, 0xfec0e15dU,
- 0x2f7502c3U, 0x4cf01281U, 0x4697a38dU, 0xd3f9c66bU,
- 0x8f5fe703U, 0x929c9515U, 0x6d7aebbfU, 0x5259da95U,
- 0xbe832dd4U, 0x7421d358U, 0xe0692949U, 0xc9c8448eU,
- 0xc2896a75U, 0x8e7978f4U, 0x583e6b99U, 0xb971dd27U,
- 0xe14fb6beU, 0x88ad17f0U, 0x20ac66c9U, 0xce3ab47dU,
- 0xdf4a1863U, 0x1a3182e5U, 0x51336097U, 0x537f4562U,
- 0x6477e0b1U, 0x6bae84bbU, 0x81a01cfeU, 0x082b94f9U,
- 0x48685870U, 0x45fd198fU, 0xde6c8794U, 0x7bf8b752U,
- 0x73d323abU, 0x4b02e272U, 0x1f8f57e3U, 0x55ab2a66U,
- 0xeb2807b2U, 0xb5c2032fU, 0xc57b9a86U, 0x3708a5d3U,
- 0x2887f230U, 0xbfa5b223U, 0x036aba02U, 0x16825cedU,
- 0xcf1c2b8aU, 0x79b492a7U, 0x07f2f0f3U, 0x69e2a14eU,
- 0xdaf4cd65U, 0x05bed506U, 0x34621fd1U, 0xa6fe8ac4U,
- 0x2e539d34U, 0xf355a0a2U, 0x8ae13205U, 0xf6eb75a4U,
- 0x83ec390bU, 0x60efaa40U, 0x719f065eU, 0x6e1051bdU,
- 0x218af93eU, 0xdd063d96U, 0x3e05aeddU, 0xe6bd464dU,
- 0x548db591U, 0xc45d0571U, 0x06d46f04U, 0x5015ff60U,
- 0x98fb2419U, 0xbde997d6U, 0x4043cc89U, 0xd99e7767U,
- 0xe842bdb0U, 0x898b8807U, 0x195b38e7U, 0xc8eedb79U,
- 0x7c0a47a1U, 0x420fe97cU, 0x841ec9f8U, 0x00000000U,
- 0x80868309U, 0x2bed4832U, 0x1170ac1eU, 0x5a724e6cU,
- 0x0efffbfdU, 0x8538560fU, 0xaed51e3dU, 0x2d392736U,
- 0x0fd9640aU, 0x5ca62168U, 0x5b54d19bU, 0x362e3a24U,
- 0x0a67b10cU, 0x57e70f93U, 0xee96d2b4U, 0x9b919e1bU,
- 0xc0c54f80U, 0xdc20a261U, 0x774b695aU, 0x121a161cU,
- 0x93ba0ae2U, 0xa02ae5c0U, 0x22e0433cU, 0x1b171d12U,
- 0x090d0b0eU, 0x8bc7adf2U, 0xb6a8b92dU, 0x1ea9c814U,
- 0xf1198557U, 0x75074cafU, 0x99ddbbeeU, 0x7f60fda3U,
- 0x01269ff7U, 0x72f5bc5cU, 0x663bc544U, 0xfb7e345bU,
- 0x4329768bU, 0x23c6dccbU, 0xedfc68b6U, 0xe4f163b8U,
- 0x31dccad7U, 0x63851042U, 0x97224013U, 0xc6112084U,
- 0x4a247d85U, 0xbb3df8d2U, 0xf93211aeU, 0x29a16dc7U,
- 0x9e2f4b1dU, 0xb230f3dcU, 0x8652ec0dU, 0xc1e3d077U,
- 0xb3166c2bU, 0x70b999a9U, 0x9448fa11U, 0xe9642247U,
- 0xfc8cc4a8U, 0xf03f1aa0U, 0x7d2cd856U, 0x3390ef22U,
- 0x494ec787U, 0x38d1c1d9U, 0xcaa2fe8cU, 0xd40b3698U,
- 0xf581cfa6U, 0x7ade28a5U, 0xb78e26daU, 0xadbfa43fU,
- 0x3a9de42cU, 0x78920d50U, 0x5fcc9b6aU, 0x7e466254U,
- 0x8d13c2f6U, 0xd8b8e890U, 0x39f75e2eU, 0xc3aff582U,
- 0x5d80be9fU, 0xd0937c69U, 0xd52da96fU, 0x2512b3cfU,
- 0xac993bc8U, 0x187da710U, 0x9c636ee8U, 0x3bbb7bdbU,
- 0x267809cdU, 0x5918f46eU, 0x9ab701ecU, 0x4f9aa883U,
- 0x956e65e6U, 0xffe67eaaU, 0xbccf0821U, 0x15e8e6efU,
- 0xe79bd9baU, 0x6f36ce4aU, 0x9f09d4eaU, 0xb07cd629U,
- 0xa4b2af31U, 0x3f23312aU, 0xa59430c6U, 0xa266c035U,
- 0x4ebc3774U, 0x82caa6fcU, 0x90d0b0e0U, 0xa7d81533U,
- 0x04984af1U, 0xecdaf741U, 0xcd500e7fU, 0x91f62f17U,
- 0x4dd68d76U, 0xefb04d43U, 0xaa4d54ccU, 0x9604dfe4U,
- 0xd1b5e39eU, 0x6a881b4cU, 0x2c1fb8c1U, 0x65517f46U,
- 0x5eea049dU, 0x8c355d01U, 0x877473faU, 0x0b412efbU,
- 0x671d5ab3U, 0xdbd25292U, 0x105633e9U, 0xd647136dU,
- 0xd7618c9aU, 0xa10c7a37U, 0xf8148e59U, 0x133c89ebU,
- 0xa927eeceU, 0x61c935b7U, 0x1ce5ede1U, 0x47b13c7aU,
- 0xd2df599cU, 0xf2733f55U, 0x14ce7918U, 0xc737bf73U,
- 0xf7cdea53U, 0xfdaa5b5fU, 0x3d6f14dfU, 0x44db8678U,
- 0xaff381caU, 0x68c43eb9U, 0x24342c38U, 0xa3405fc2U,
- 0x1dc37216U, 0xe2250cbcU, 0x3c498b28U, 0x0d9541ffU,
- 0xa8017139U, 0x0cb3de08U, 0xb4e49cd8U, 0x56c19064U,
- 0xcb84617bU, 0x32b670d5U, 0x6c5c7448U, 0xb85742d0U,
-};
-static const u32 Td4[256] = {
- 0x52525252U, 0x09090909U, 0x6a6a6a6aU, 0xd5d5d5d5U,
- 0x30303030U, 0x36363636U, 0xa5a5a5a5U, 0x38383838U,
- 0xbfbfbfbfU, 0x40404040U, 0xa3a3a3a3U, 0x9e9e9e9eU,
- 0x81818181U, 0xf3f3f3f3U, 0xd7d7d7d7U, 0xfbfbfbfbU,
- 0x7c7c7c7cU, 0xe3e3e3e3U, 0x39393939U, 0x82828282U,
- 0x9b9b9b9bU, 0x2f2f2f2fU, 0xffffffffU, 0x87878787U,
- 0x34343434U, 0x8e8e8e8eU, 0x43434343U, 0x44444444U,
- 0xc4c4c4c4U, 0xdedededeU, 0xe9e9e9e9U, 0xcbcbcbcbU,
- 0x54545454U, 0x7b7b7b7bU, 0x94949494U, 0x32323232U,
- 0xa6a6a6a6U, 0xc2c2c2c2U, 0x23232323U, 0x3d3d3d3dU,
- 0xeeeeeeeeU, 0x4c4c4c4cU, 0x95959595U, 0x0b0b0b0bU,
- 0x42424242U, 0xfafafafaU, 0xc3c3c3c3U, 0x4e4e4e4eU,
- 0x08080808U, 0x2e2e2e2eU, 0xa1a1a1a1U, 0x66666666U,
- 0x28282828U, 0xd9d9d9d9U, 0x24242424U, 0xb2b2b2b2U,
- 0x76767676U, 0x5b5b5b5bU, 0xa2a2a2a2U, 0x49494949U,
- 0x6d6d6d6dU, 0x8b8b8b8bU, 0xd1d1d1d1U, 0x25252525U,
- 0x72727272U, 0xf8f8f8f8U, 0xf6f6f6f6U, 0x64646464U,
- 0x86868686U, 0x68686868U, 0x98989898U, 0x16161616U,
- 0xd4d4d4d4U, 0xa4a4a4a4U, 0x5c5c5c5cU, 0xccccccccU,
- 0x5d5d5d5dU, 0x65656565U, 0xb6b6b6b6U, 0x92929292U,
- 0x6c6c6c6cU, 0x70707070U, 0x48484848U, 0x50505050U,
- 0xfdfdfdfdU, 0xededededU, 0xb9b9b9b9U, 0xdadadadaU,
- 0x5e5e5e5eU, 0x15151515U, 0x46464646U, 0x57575757U,
- 0xa7a7a7a7U, 0x8d8d8d8dU, 0x9d9d9d9dU, 0x84848484U,
- 0x90909090U, 0xd8d8d8d8U, 0xababababU, 0x00000000U,
- 0x8c8c8c8cU, 0xbcbcbcbcU, 0xd3d3d3d3U, 0x0a0a0a0aU,
- 0xf7f7f7f7U, 0xe4e4e4e4U, 0x58585858U, 0x05050505U,
- 0xb8b8b8b8U, 0xb3b3b3b3U, 0x45454545U, 0x06060606U,
- 0xd0d0d0d0U, 0x2c2c2c2cU, 0x1e1e1e1eU, 0x8f8f8f8fU,
- 0xcacacacaU, 0x3f3f3f3fU, 0x0f0f0f0fU, 0x02020202U,
- 0xc1c1c1c1U, 0xafafafafU, 0xbdbdbdbdU, 0x03030303U,
- 0x01010101U, 0x13131313U, 0x8a8a8a8aU, 0x6b6b6b6bU,
- 0x3a3a3a3aU, 0x91919191U, 0x11111111U, 0x41414141U,
- 0x4f4f4f4fU, 0x67676767U, 0xdcdcdcdcU, 0xeaeaeaeaU,
- 0x97979797U, 0xf2f2f2f2U, 0xcfcfcfcfU, 0xcecececeU,
- 0xf0f0f0f0U, 0xb4b4b4b4U, 0xe6e6e6e6U, 0x73737373U,
- 0x96969696U, 0xacacacacU, 0x74747474U, 0x22222222U,
- 0xe7e7e7e7U, 0xadadadadU, 0x35353535U, 0x85858585U,
- 0xe2e2e2e2U, 0xf9f9f9f9U, 0x37373737U, 0xe8e8e8e8U,
- 0x1c1c1c1cU, 0x75757575U, 0xdfdfdfdfU, 0x6e6e6e6eU,
- 0x47474747U, 0xf1f1f1f1U, 0x1a1a1a1aU, 0x71717171U,
- 0x1d1d1d1dU, 0x29292929U, 0xc5c5c5c5U, 0x89898989U,
- 0x6f6f6f6fU, 0xb7b7b7b7U, 0x62626262U, 0x0e0e0e0eU,
- 0xaaaaaaaaU, 0x18181818U, 0xbebebebeU, 0x1b1b1b1bU,
- 0xfcfcfcfcU, 0x56565656U, 0x3e3e3e3eU, 0x4b4b4b4bU,
- 0xc6c6c6c6U, 0xd2d2d2d2U, 0x79797979U, 0x20202020U,
- 0x9a9a9a9aU, 0xdbdbdbdbU, 0xc0c0c0c0U, 0xfefefefeU,
- 0x78787878U, 0xcdcdcdcdU, 0x5a5a5a5aU, 0xf4f4f4f4U,
- 0x1f1f1f1fU, 0xddddddddU, 0xa8a8a8a8U, 0x33333333U,
- 0x88888888U, 0x07070707U, 0xc7c7c7c7U, 0x31313131U,
- 0xb1b1b1b1U, 0x12121212U, 0x10101010U, 0x59595959U,
- 0x27272727U, 0x80808080U, 0xececececU, 0x5f5f5f5fU,
- 0x60606060U, 0x51515151U, 0x7f7f7f7fU, 0xa9a9a9a9U,
- 0x19191919U, 0xb5b5b5b5U, 0x4a4a4a4aU, 0x0d0d0d0dU,
- 0x2d2d2d2dU, 0xe5e5e5e5U, 0x7a7a7a7aU, 0x9f9f9f9fU,
- 0x93939393U, 0xc9c9c9c9U, 0x9c9c9c9cU, 0xefefefefU,
- 0xa0a0a0a0U, 0xe0e0e0e0U, 0x3b3b3b3bU, 0x4d4d4d4dU,
- 0xaeaeaeaeU, 0x2a2a2a2aU, 0xf5f5f5f5U, 0xb0b0b0b0U,
- 0xc8c8c8c8U, 0xebebebebU, 0xbbbbbbbbU, 0x3c3c3c3cU,
- 0x83838383U, 0x53535353U, 0x99999999U, 0x61616161U,
- 0x17171717U, 0x2b2b2b2bU, 0x04040404U, 0x7e7e7e7eU,
- 0xbabababaU, 0x77777777U, 0xd6d6d6d6U, 0x26262626U,
- 0xe1e1e1e1U, 0x69696969U, 0x14141414U, 0x63636363U,
- 0x55555555U, 0x21212121U, 0x0c0c0c0cU, 0x7d7d7d7dU,
-};
-static const u32 rcon[] = {
- 0x01000000, 0x02000000, 0x04000000, 0x08000000,
- 0x10000000, 0x20000000, 0x40000000, 0x80000000,
- 0x1B000000, 0x36000000, /* for 128-bit blocks, Rijndael never uses more than 10 rcon values */
-};
-
-#define SWAP(x) (_lrotl(x, 8) & 0x00ff00ff | _lrotr(x, 8) & 0xff00ff00)
-
-#ifdef _MSC_VER
-#define GETU32(p) SWAP(*((u32 *)(p)))
-#define PUTU32(ct, st) { *((u32 *)(ct)) = SWAP((st)); }
-#else
-#define GETU32(pt) (((u32)(pt)[0] << 24) ^ ((u32)(pt)[1] << 16) ^ ((u32)(pt)[2] << 8) ^ ((u32)(pt)[3]))
-#define PUTU32(ct, st) { (ct)[0] = (u8)((st) >> 24); (ct)[1] = (u8)((st) >> 16); (ct)[2] = (u8)((st) >> 8); (ct)[3] = (u8)(st); }
-#endif
-
-/**
- * Expand the cipher key into the encryption key schedule.
- *
- * @return the number of rounds for the given cipher key size.
- */
-int rijndaelKeySetupEnc(u32 rk[/*4*(Nr + 1)*/], const u8 cipherKey[], int keyBits) {
- int i = 0;
- u32 temp;
-
- rk[0] = GETU32(cipherKey );
- rk[1] = GETU32(cipherKey + 4);
- rk[2] = GETU32(cipherKey + 8);
- rk[3] = GETU32(cipherKey + 12);
- if (keyBits == 128) {
- for (;;) {
- temp = rk[3];
- rk[4] = rk[0] ^
- (Te4[(temp >> 16) & 0xff] & 0xff000000) ^
- (Te4[(temp >> 8) & 0xff] & 0x00ff0000) ^
- (Te4[(temp ) & 0xff] & 0x0000ff00) ^
- (Te4[(temp >> 24) ] & 0x000000ff) ^
- rcon[i];
- rk[5] = rk[1] ^ rk[4];
- rk[6] = rk[2] ^ rk[5];
- rk[7] = rk[3] ^ rk[6];
- if (++i == 10) {
- return 10;
- }
- rk += 4;
- }
- }
- rk[4] = GETU32(cipherKey + 16);
- rk[5] = GETU32(cipherKey + 20);
- if (keyBits == 192) {
- for (;;) {
- temp = rk[ 5];
- rk[ 6] = rk[ 0] ^
- (Te4[(temp >> 16) & 0xff] & 0xff000000) ^
- (Te4[(temp >> 8) & 0xff] & 0x00ff0000) ^
- (Te4[(temp ) & 0xff] & 0x0000ff00) ^
- (Te4[(temp >> 24) ] & 0x000000ff) ^
- rcon[i];
- rk[ 7] = rk[ 1] ^ rk[ 6];
- rk[ 8] = rk[ 2] ^ rk[ 7];
- rk[ 9] = rk[ 3] ^ rk[ 8];
- if (++i == 8) {
- return 12;
- }
- rk[10] = rk[ 4] ^ rk[ 9];
- rk[11] = rk[ 5] ^ rk[10];
- rk += 6;
- }
- }
- rk[6] = GETU32(cipherKey + 24);
- rk[7] = GETU32(cipherKey + 28);
- if (keyBits == 256) {
- for (;;) {
- temp = rk[ 7];
- rk[ 8] = rk[ 0] ^
- (Te4[(temp >> 16) & 0xff] & 0xff000000) ^
- (Te4[(temp >> 8) & 0xff] & 0x00ff0000) ^
- (Te4[(temp ) & 0xff] & 0x0000ff00) ^
- (Te4[(temp >> 24) ] & 0x000000ff) ^
- rcon[i];
- rk[ 9] = rk[ 1] ^ rk[ 8];
- rk[10] = rk[ 2] ^ rk[ 9];
- rk[11] = rk[ 3] ^ rk[10];
- if (++i == 7) {
- return 14;
- }
- temp = rk[11];
- rk[12] = rk[ 4] ^
- (Te4[(temp >> 24) ] & 0xff000000) ^
- (Te4[(temp >> 16) & 0xff] & 0x00ff0000) ^
- (Te4[(temp >> 8) & 0xff] & 0x0000ff00) ^
- (Te4[(temp ) & 0xff] & 0x000000ff);
- rk[13] = rk[ 5] ^ rk[12];
- rk[14] = rk[ 6] ^ rk[13];
- rk[15] = rk[ 7] ^ rk[14];
-
- rk += 8;
- }
- }
- return 0;
-}
-
-/**
- * Expand the cipher key into the decryption key schedule.
- *
- * @return the number of rounds for the given cipher key size.
- */
-int rijndaelKeySetupDec(u32 rk[/*4*(Nr + 1)*/], const u8 cipherKey[], int keyBits) {
- int Nr, i, j;
- u32 temp;
-
- /* expand the cipher key: */
- Nr = rijndaelKeySetupEnc(rk, cipherKey, keyBits);
- /* invert the order of the round keys: */
- for (i = 0, j = 4*Nr; i < j; i += 4, j -= 4) {
- temp = rk[i ]; rk[i ] = rk[j ]; rk[j ] = temp;
- temp = rk[i + 1]; rk[i + 1] = rk[j + 1]; rk[j + 1] = temp;
- temp = rk[i + 2]; rk[i + 2] = rk[j + 2]; rk[j + 2] = temp;
- temp = rk[i + 3]; rk[i + 3] = rk[j + 3]; rk[j + 3] = temp;
- }
- /* apply the inverse MixColumn transform to all round keys but the first and the last: */
- for (i = 1; i < Nr; i++) {
- rk += 4;
- rk[0] =
- Td0[Te4[(rk[0] >> 24) ] & 0xff] ^
- Td1[Te4[(rk[0] >> 16) & 0xff] & 0xff] ^
- Td2[Te4[(rk[0] >> 8) & 0xff] & 0xff] ^
- Td3[Te4[(rk[0] ) & 0xff] & 0xff];
- rk[1] =
- Td0[Te4[(rk[1] >> 24) ] & 0xff] ^
- Td1[Te4[(rk[1] >> 16) & 0xff] & 0xff] ^
- Td2[Te4[(rk[1] >> 8) & 0xff] & 0xff] ^
- Td3[Te4[(rk[1] ) & 0xff] & 0xff];
- rk[2] =
- Td0[Te4[(rk[2] >> 24) ] & 0xff] ^
- Td1[Te4[(rk[2] >> 16) & 0xff] & 0xff] ^
- Td2[Te4[(rk[2] >> 8) & 0xff] & 0xff] ^
- Td3[Te4[(rk[2] ) & 0xff] & 0xff];
- rk[3] =
- Td0[Te4[(rk[3] >> 24) ] & 0xff] ^
- Td1[Te4[(rk[3] >> 16) & 0xff] & 0xff] ^
- Td2[Te4[(rk[3] >> 8) & 0xff] & 0xff] ^
- Td3[Te4[(rk[3] ) & 0xff] & 0xff];
- }
- return Nr;
-}
-
-void rijndaelEncrypt(const u32 rk[/*4*(Nr + 1)*/], int Nr, const u8 pt[16], u8 ct[16]) {
- u32 s0, s1, s2, s3, t0, t1, t2, t3;
-#ifndef FULL_UNROLL
- int r;
-#endif /* ?FULL_UNROLL */
-
- /*
- * map byte array block to cipher state
- * and add initial round key:
- */
- s0 = GETU32(pt ) ^ rk[0];
- s1 = GETU32(pt + 4) ^ rk[1];
- s2 = GETU32(pt + 8) ^ rk[2];
- s3 = GETU32(pt + 12) ^ rk[3];
-#ifdef FULL_UNROLL
- /* round 1: */
- t0 = Te0[s0 >> 24] ^ Te1[(s1 >> 16) & 0xff] ^ Te2[(s2 >> 8) & 0xff] ^ Te3[s3 & 0xff] ^ rk[ 4];
- t1 = Te0[s1 >> 24] ^ Te1[(s2 >> 16) & 0xff] ^ Te2[(s3 >> 8) & 0xff] ^ Te3[s0 & 0xff] ^ rk[ 5];
- t2 = Te0[s2 >> 24] ^ Te1[(s3 >> 16) & 0xff] ^ Te2[(s0 >> 8) & 0xff] ^ Te3[s1 & 0xff] ^ rk[ 6];
- t3 = Te0[s3 >> 24] ^ Te1[(s0 >> 16) & 0xff] ^ Te2[(s1 >> 8) & 0xff] ^ Te3[s2 & 0xff] ^ rk[ 7];
- /* round 2: */
- s0 = Te0[t0 >> 24] ^ Te1[(t1 >> 16) & 0xff] ^ Te2[(t2 >> 8) & 0xff] ^ Te3[t3 & 0xff] ^ rk[ 8];
- s1 = Te0[t1 >> 24] ^ Te1[(t2 >> 16) & 0xff] ^ Te2[(t3 >> 8) & 0xff] ^ Te3[t0 & 0xff] ^ rk[ 9];
- s2 = Te0[t2 >> 24] ^ Te1[(t3 >> 16) & 0xff] ^ Te2[(t0 >> 8) & 0xff] ^ Te3[t1 & 0xff] ^ rk[10];
- s3 = Te0[t3 >> 24] ^ Te1[(t0 >> 16) & 0xff] ^ Te2[(t1 >> 8) & 0xff] ^ Te3[t2 & 0xff] ^ rk[11];
- /* round 3: */
- t0 = Te0[s0 >> 24] ^ Te1[(s1 >> 16) & 0xff] ^ Te2[(s2 >> 8) & 0xff] ^ Te3[s3 & 0xff] ^ rk[12];
- t1 = Te0[s1 >> 24] ^ Te1[(s2 >> 16) & 0xff] ^ Te2[(s3 >> 8) & 0xff] ^ Te3[s0 & 0xff] ^ rk[13];
- t2 = Te0[s2 >> 24] ^ Te1[(s3 >> 16) & 0xff] ^ Te2[(s0 >> 8) & 0xff] ^ Te3[s1 & 0xff] ^ rk[14];
- t3 = Te0[s3 >> 24] ^ Te1[(s0 >> 16) & 0xff] ^ Te2[(s1 >> 8) & 0xff] ^ Te3[s2 & 0xff] ^ rk[15];
- /* round 4: */
- s0 = Te0[t0 >> 24] ^ Te1[(t1 >> 16) & 0xff] ^ Te2[(t2 >> 8) & 0xff] ^ Te3[t3 & 0xff] ^ rk[16];
- s1 = Te0[t1 >> 24] ^ Te1[(t2 >> 16) & 0xff] ^ Te2[(t3 >> 8) & 0xff] ^ Te3[t0 & 0xff] ^ rk[17];
- s2 = Te0[t2 >> 24] ^ Te1[(t3 >> 16) & 0xff] ^ Te2[(t0 >> 8) & 0xff] ^ Te3[t1 & 0xff] ^ rk[18];
- s3 = Te0[t3 >> 24] ^ Te1[(t0 >> 16) & 0xff] ^ Te2[(t1 >> 8) & 0xff] ^ Te3[t2 & 0xff] ^ rk[19];
- /* round 5: */
- t0 = Te0[s0 >> 24] ^ Te1[(s1 >> 16) & 0xff] ^ Te2[(s2 >> 8) & 0xff] ^ Te3[s3 & 0xff] ^ rk[20];
- t1 = Te0[s1 >> 24] ^ Te1[(s2 >> 16) & 0xff] ^ Te2[(s3 >> 8) & 0xff] ^ Te3[s0 & 0xff] ^ rk[21];
- t2 = Te0[s2 >> 24] ^ Te1[(s3 >> 16) & 0xff] ^ Te2[(s0 >> 8) & 0xff] ^ Te3[s1 & 0xff] ^ rk[22];
- t3 = Te0[s3 >> 24] ^ Te1[(s0 >> 16) & 0xff] ^ Te2[(s1 >> 8) & 0xff] ^ Te3[s2 & 0xff] ^ rk[23];
- /* round 6: */
- s0 = Te0[t0 >> 24] ^ Te1[(t1 >> 16) & 0xff] ^ Te2[(t2 >> 8) & 0xff] ^ Te3[t3 & 0xff] ^ rk[24];
- s1 = Te0[t1 >> 24] ^ Te1[(t2 >> 16) & 0xff] ^ Te2[(t3 >> 8) & 0xff] ^ Te3[t0 & 0xff] ^ rk[25];
- s2 = Te0[t2 >> 24] ^ Te1[(t3 >> 16) & 0xff] ^ Te2[(t0 >> 8) & 0xff] ^ Te3[t1 & 0xff] ^ rk[26];
- s3 = Te0[t3 >> 24] ^ Te1[(t0 >> 16) & 0xff] ^ Te2[(t1 >> 8) & 0xff] ^ Te3[t2 & 0xff] ^ rk[27];
- /* round 7: */
- t0 = Te0[s0 >> 24] ^ Te1[(s1 >> 16) & 0xff] ^ Te2[(s2 >> 8) & 0xff] ^ Te3[s3 & 0xff] ^ rk[28];
- t1 = Te0[s1 >> 24] ^ Te1[(s2 >> 16) & 0xff] ^ Te2[(s3 >> 8) & 0xff] ^ Te3[s0 & 0xff] ^ rk[29];
- t2 = Te0[s2 >> 24] ^ Te1[(s3 >> 16) & 0xff] ^ Te2[(s0 >> 8) & 0xff] ^ Te3[s1 & 0xff] ^ rk[30];
- t3 = Te0[s3 >> 24] ^ Te1[(s0 >> 16) & 0xff] ^ Te2[(s1 >> 8) & 0xff] ^ Te3[s2 & 0xff] ^ rk[31];
- /* round 8: */
- s0 = Te0[t0 >> 24] ^ Te1[(t1 >> 16) & 0xff] ^ Te2[(t2 >> 8) & 0xff] ^ Te3[t3 & 0xff] ^ rk[32];
- s1 = Te0[t1 >> 24] ^ Te1[(t2 >> 16) & 0xff] ^ Te2[(t3 >> 8) & 0xff] ^ Te3[t0 & 0xff] ^ rk[33];
- s2 = Te0[t2 >> 24] ^ Te1[(t3 >> 16) & 0xff] ^ Te2[(t0 >> 8) & 0xff] ^ Te3[t1 & 0xff] ^ rk[34];
- s3 = Te0[t3 >> 24] ^ Te1[(t0 >> 16) & 0xff] ^ Te2[(t1 >> 8) & 0xff] ^ Te3[t2 & 0xff] ^ rk[35];
- /* round 9: */
- t0 = Te0[s0 >> 24] ^ Te1[(s1 >> 16) & 0xff] ^ Te2[(s2 >> 8) & 0xff] ^ Te3[s3 & 0xff] ^ rk[36];
- t1 = Te0[s1 >> 24] ^ Te1[(s2 >> 16) & 0xff] ^ Te2[(s3 >> 8) & 0xff] ^ Te3[s0 & 0xff] ^ rk[37];
- t2 = Te0[s2 >> 24] ^ Te1[(s3 >> 16) & 0xff] ^ Te2[(s0 >> 8) & 0xff] ^ Te3[s1 & 0xff] ^ rk[38];
- t3 = Te0[s3 >> 24] ^ Te1[(s0 >> 16) & 0xff] ^ Te2[(s1 >> 8) & 0xff] ^ Te3[s2 & 0xff] ^ rk[39];
- if (Nr > 10) {
- /* round 10: */
- s0 = Te0[t0 >> 24] ^ Te1[(t1 >> 16) & 0xff] ^ Te2[(t2 >> 8) & 0xff] ^ Te3[t3 & 0xff] ^ rk[40];
- s1 = Te0[t1 >> 24] ^ Te1[(t2 >> 16) & 0xff] ^ Te2[(t3 >> 8) & 0xff] ^ Te3[t0 & 0xff] ^ rk[41];
- s2 = Te0[t2 >> 24] ^ Te1[(t3 >> 16) & 0xff] ^ Te2[(t0 >> 8) & 0xff] ^ Te3[t1 & 0xff] ^ rk[42];
- s3 = Te0[t3 >> 24] ^ Te1[(t0 >> 16) & 0xff] ^ Te2[(t1 >> 8) & 0xff] ^ Te3[t2 & 0xff] ^ rk[43];
- /* round 11: */
- t0 = Te0[s0 >> 24] ^ Te1[(s1 >> 16) & 0xff] ^ Te2[(s2 >> 8) & 0xff] ^ Te3[s3 & 0xff] ^ rk[44];
- t1 = Te0[s1 >> 24] ^ Te1[(s2 >> 16) & 0xff] ^ Te2[(s3 >> 8) & 0xff] ^ Te3[s0 & 0xff] ^ rk[45];
- t2 = Te0[s2 >> 24] ^ Te1[(s3 >> 16) & 0xff] ^ Te2[(s0 >> 8) & 0xff] ^ Te3[s1 & 0xff] ^ rk[46];
- t3 = Te0[s3 >> 24] ^ Te1[(s0 >> 16) & 0xff] ^ Te2[(s1 >> 8) & 0xff] ^ Te3[s2 & 0xff] ^ rk[47];
- if (Nr > 12) {
- /* round 12: */
- s0 = Te0[t0 >> 24] ^ Te1[(t1 >> 16) & 0xff] ^ Te2[(t2 >> 8) & 0xff] ^ Te3[t3 & 0xff] ^ rk[48];
- s1 = Te0[t1 >> 24] ^ Te1[(t2 >> 16) & 0xff] ^ Te2[(t3 >> 8) & 0xff] ^ Te3[t0 & 0xff] ^ rk[49];
- s2 = Te0[t2 >> 24] ^ Te1[(t3 >> 16) & 0xff] ^ Te2[(t0 >> 8) & 0xff] ^ Te3[t1 & 0xff] ^ rk[50];
- s3 = Te0[t3 >> 24] ^ Te1[(t0 >> 16) & 0xff] ^ Te2[(t1 >> 8) & 0xff] ^ Te3[t2 & 0xff] ^ rk[51];
- /* round 13: */
- t0 = Te0[s0 >> 24] ^ Te1[(s1 >> 16) & 0xff] ^ Te2[(s2 >> 8) & 0xff] ^ Te3[s3 & 0xff] ^ rk[52];
- t1 = Te0[s1 >> 24] ^ Te1[(s2 >> 16) & 0xff] ^ Te2[(s3 >> 8) & 0xff] ^ Te3[s0 & 0xff] ^ rk[53];
- t2 = Te0[s2 >> 24] ^ Te1[(s3 >> 16) & 0xff] ^ Te2[(s0 >> 8) & 0xff] ^ Te3[s1 & 0xff] ^ rk[54];
- t3 = Te0[s3 >> 24] ^ Te1[(s0 >> 16) & 0xff] ^ Te2[(s1 >> 8) & 0xff] ^ Te3[s2 & 0xff] ^ rk[55];
- }
- }
- rk += Nr << 2;
-#else /* !FULL_UNROLL */
- /*
- * Nr - 1 full rounds:
- */
- r = Nr >> 1;
- for (;;) {
- t0 =
- Te0[(s0 >> 24) ] ^
- Te1[(s1 >> 16) & 0xff] ^
- Te2[(s2 >> 8) & 0xff] ^
- Te3[(s3 ) & 0xff] ^
- rk[4];
- t1 =
- Te0[(s1 >> 24) ] ^
- Te1[(s2 >> 16) & 0xff] ^
- Te2[(s3 >> 8) & 0xff] ^
- Te3[(s0 ) & 0xff] ^
- rk[5];
- t2 =
- Te0[(s2 >> 24) ] ^
- Te1[(s3 >> 16) & 0xff] ^
- Te2[(s0 >> 8) & 0xff] ^
- Te3[(s1 ) & 0xff] ^
- rk[6];
- t3 =
- Te0[(s3 >> 24) ] ^
- Te1[(s0 >> 16) & 0xff] ^
- Te2[(s1 >> 8) & 0xff] ^
- Te3[(s2 ) & 0xff] ^
- rk[7];
-
- rk += 8;
- if (--r == 0) {
- break;
- }
-
- s0 =
- Te0[(t0 >> 24) ] ^
- Te1[(t1 >> 16) & 0xff] ^
- Te2[(t2 >> 8) & 0xff] ^
- Te3[(t3 ) & 0xff] ^
- rk[0];
- s1 =
- Te0[(t1 >> 24) ] ^
- Te1[(t2 >> 16) & 0xff] ^
- Te2[(t3 >> 8) & 0xff] ^
- Te3[(t0 ) & 0xff] ^
- rk[1];
- s2 =
- Te0[(t2 >> 24) ] ^
- Te1[(t3 >> 16) & 0xff] ^
- Te2[(t0 >> 8) & 0xff] ^
- Te3[(t1 ) & 0xff] ^
- rk[2];
- s3 =
- Te0[(t3 >> 24) ] ^
- Te1[(t0 >> 16) & 0xff] ^
- Te2[(t1 >> 8) & 0xff] ^
- Te3[(t2 ) & 0xff] ^
- rk[3];
- }
-#endif /* ?FULL_UNROLL */
- /*
- * apply last round and
- * map cipher state to byte array block:
- */
- s0 =
- (Te4[(t0 >> 24) ] & 0xff000000) ^
- (Te4[(t1 >> 16) & 0xff] & 0x00ff0000) ^
- (Te4[(t2 >> 8) & 0xff] & 0x0000ff00) ^
- (Te4[(t3 ) & 0xff] & 0x000000ff) ^
- rk[0];
- PUTU32(ct , s0);
- s1 =
- (Te4[(t1 >> 24) ] & 0xff000000) ^
- (Te4[(t2 >> 16) & 0xff] & 0x00ff0000) ^
- (Te4[(t3 >> 8) & 0xff] & 0x0000ff00) ^
- (Te4[(t0 ) & 0xff] & 0x000000ff) ^
- rk[1];
- PUTU32(ct + 4, s1);
- s2 =
- (Te4[(t2 >> 24) ] & 0xff000000) ^
- (Te4[(t3 >> 16) & 0xff] & 0x00ff0000) ^
- (Te4[(t0 >> 8) & 0xff] & 0x0000ff00) ^
- (Te4[(t1 ) & 0xff] & 0x000000ff) ^
- rk[2];
- PUTU32(ct + 8, s2);
- s3 =
- (Te4[(t3 >> 24) ] & 0xff000000) ^
- (Te4[(t0 >> 16) & 0xff] & 0x00ff0000) ^
- (Te4[(t1 >> 8) & 0xff] & 0x0000ff00) ^
- (Te4[(t2 ) & 0xff] & 0x000000ff) ^
- rk[3];
- PUTU32(ct + 12, s3);
-}
-
-void rijndaelDecrypt(const u32 rk[/*4*(Nr + 1)*/], int Nr, const u8 ct[16], u8 pt[16]) {
- u32 s0, s1, s2, s3, t0, t1, t2, t3;
-#ifndef FULL_UNROLL
- int r;
-#endif /* ?FULL_UNROLL */
-
- /*
- * map byte array block to cipher state
- * and add initial round key:
- */
- s0 = GETU32(ct ) ^ rk[0];
- s1 = GETU32(ct + 4) ^ rk[1];
- s2 = GETU32(ct + 8) ^ rk[2];
- s3 = GETU32(ct + 12) ^ rk[3];
-#ifdef FULL_UNROLL
- /* round 1: */
- t0 = Td0[s0 >> 24] ^ Td1[(s3 >> 16) & 0xff] ^ Td2[(s2 >> 8) & 0xff] ^ Td3[s1 & 0xff] ^ rk[ 4];
- t1 = Td0[s1 >> 24] ^ Td1[(s0 >> 16) & 0xff] ^ Td2[(s3 >> 8) & 0xff] ^ Td3[s2 & 0xff] ^ rk[ 5];
- t2 = Td0[s2 >> 24] ^ Td1[(s1 >> 16) & 0xff] ^ Td2[(s0 >> 8) & 0xff] ^ Td3[s3 & 0xff] ^ rk[ 6];
- t3 = Td0[s3 >> 24] ^ Td1[(s2 >> 16) & 0xff] ^ Td2[(s1 >> 8) & 0xff] ^ Td3[s0 & 0xff] ^ rk[ 7];
- /* round 2: */
- s0 = Td0[t0 >> 24] ^ Td1[(t3 >> 16) & 0xff] ^ Td2[(t2 >> 8) & 0xff] ^ Td3[t1 & 0xff] ^ rk[ 8];
- s1 = Td0[t1 >> 24] ^ Td1[(t0 >> 16) & 0xff] ^ Td2[(t3 >> 8) & 0xff] ^ Td3[t2 & 0xff] ^ rk[ 9];
- s2 = Td0[t2 >> 24] ^ Td1[(t1 >> 16) & 0xff] ^ Td2[(t0 >> 8) & 0xff] ^ Td3[t3 & 0xff] ^ rk[10];
- s3 = Td0[t3 >> 24] ^ Td1[(t2 >> 16) & 0xff] ^ Td2[(t1 >> 8) & 0xff] ^ Td3[t0 & 0xff] ^ rk[11];
- /* round 3: */
- t0 = Td0[s0 >> 24] ^ Td1[(s3 >> 16) & 0xff] ^ Td2[(s2 >> 8) & 0xff] ^ Td3[s1 & 0xff] ^ rk[12];
- t1 = Td0[s1 >> 24] ^ Td1[(s0 >> 16) & 0xff] ^ Td2[(s3 >> 8) & 0xff] ^ Td3[s2 & 0xff] ^ rk[13];
- t2 = Td0[s2 >> 24] ^ Td1[(s1 >> 16) & 0xff] ^ Td2[(s0 >> 8) & 0xff] ^ Td3[s3 & 0xff] ^ rk[14];
- t3 = Td0[s3 >> 24] ^ Td1[(s2 >> 16) & 0xff] ^ Td2[(s1 >> 8) & 0xff] ^ Td3[s0 & 0xff] ^ rk[15];
- /* round 4: */
- s0 = Td0[t0 >> 24] ^ Td1[(t3 >> 16) & 0xff] ^ Td2[(t2 >> 8) & 0xff] ^ Td3[t1 & 0xff] ^ rk[16];
- s1 = Td0[t1 >> 24] ^ Td1[(t0 >> 16) & 0xff] ^ Td2[(t3 >> 8) & 0xff] ^ Td3[t2 & 0xff] ^ rk[17];
- s2 = Td0[t2 >> 24] ^ Td1[(t1 >> 16) & 0xff] ^ Td2[(t0 >> 8) & 0xff] ^ Td3[t3 & 0xff] ^ rk[18];
- s3 = Td0[t3 >> 24] ^ Td1[(t2 >> 16) & 0xff] ^ Td2[(t1 >> 8) & 0xff] ^ Td3[t0 & 0xff] ^ rk[19];
- /* round 5: */
- t0 = Td0[s0 >> 24] ^ Td1[(s3 >> 16) & 0xff] ^ Td2[(s2 >> 8) & 0xff] ^ Td3[s1 & 0xff] ^ rk[20];
- t1 = Td0[s1 >> 24] ^ Td1[(s0 >> 16) & 0xff] ^ Td2[(s3 >> 8) & 0xff] ^ Td3[s2 & 0xff] ^ rk[21];
- t2 = Td0[s2 >> 24] ^ Td1[(s1 >> 16) & 0xff] ^ Td2[(s0 >> 8) & 0xff] ^ Td3[s3 & 0xff] ^ rk[22];
- t3 = Td0[s3 >> 24] ^ Td1[(s2 >> 16) & 0xff] ^ Td2[(s1 >> 8) & 0xff] ^ Td3[s0 & 0xff] ^ rk[23];
- /* round 6: */
- s0 = Td0[t0 >> 24] ^ Td1[(t3 >> 16) & 0xff] ^ Td2[(t2 >> 8) & 0xff] ^ Td3[t1 & 0xff] ^ rk[24];
- s1 = Td0[t1 >> 24] ^ Td1[(t0 >> 16) & 0xff] ^ Td2[(t3 >> 8) & 0xff] ^ Td3[t2 & 0xff] ^ rk[25];
- s2 = Td0[t2 >> 24] ^ Td1[(t1 >> 16) & 0xff] ^ Td2[(t0 >> 8) & 0xff] ^ Td3[t3 & 0xff] ^ rk[26];
- s3 = Td0[t3 >> 24] ^ Td1[(t2 >> 16) & 0xff] ^ Td2[(t1 >> 8) & 0xff] ^ Td3[t0 & 0xff] ^ rk[27];
- /* round 7: */
- t0 = Td0[s0 >> 24] ^ Td1[(s3 >> 16) & 0xff] ^ Td2[(s2 >> 8) & 0xff] ^ Td3[s1 & 0xff] ^ rk[28];
- t1 = Td0[s1 >> 24] ^ Td1[(s0 >> 16) & 0xff] ^ Td2[(s3 >> 8) & 0xff] ^ Td3[s2 & 0xff] ^ rk[29];
- t2 = Td0[s2 >> 24] ^ Td1[(s1 >> 16) & 0xff] ^ Td2[(s0 >> 8) & 0xff] ^ Td3[s3 & 0xff] ^ rk[30];
- t3 = Td0[s3 >> 24] ^ Td1[(s2 >> 16) & 0xff] ^ Td2[(s1 >> 8) & 0xff] ^ Td3[s0 & 0xff] ^ rk[31];
- /* round 8: */
- s0 = Td0[t0 >> 24] ^ Td1[(t3 >> 16) & 0xff] ^ Td2[(t2 >> 8) & 0xff] ^ Td3[t1 & 0xff] ^ rk[32];
- s1 = Td0[t1 >> 24] ^ Td1[(t0 >> 16) & 0xff] ^ Td2[(t3 >> 8) & 0xff] ^ Td3[t2 & 0xff] ^ rk[33];
- s2 = Td0[t2 >> 24] ^ Td1[(t1 >> 16) & 0xff] ^ Td2[(t0 >> 8) & 0xff] ^ Td3[t3 & 0xff] ^ rk[34];
- s3 = Td0[t3 >> 24] ^ Td1[(t2 >> 16) & 0xff] ^ Td2[(t1 >> 8) & 0xff] ^ Td3[t0 & 0xff] ^ rk[35];
- /* round 9: */
- t0 = Td0[s0 >> 24] ^ Td1[(s3 >> 16) & 0xff] ^ Td2[(s2 >> 8) & 0xff] ^ Td3[s1 & 0xff] ^ rk[36];
- t1 = Td0[s1 >> 24] ^ Td1[(s0 >> 16) & 0xff] ^ Td2[(s3 >> 8) & 0xff] ^ Td3[s2 & 0xff] ^ rk[37];
- t2 = Td0[s2 >> 24] ^ Td1[(s1 >> 16) & 0xff] ^ Td2[(s0 >> 8) & 0xff] ^ Td3[s3 & 0xff] ^ rk[38];
- t3 = Td0[s3 >> 24] ^ Td1[(s2 >> 16) & 0xff] ^ Td2[(s1 >> 8) & 0xff] ^ Td3[s0 & 0xff] ^ rk[39];
- if (Nr > 10) {
- /* round 10: */
- s0 = Td0[t0 >> 24] ^ Td1[(t3 >> 16) & 0xff] ^ Td2[(t2 >> 8) & 0xff] ^ Td3[t1 & 0xff] ^ rk[40];
- s1 = Td0[t1 >> 24] ^ Td1[(t0 >> 16) & 0xff] ^ Td2[(t3 >> 8) & 0xff] ^ Td3[t2 & 0xff] ^ rk[41];
- s2 = Td0[t2 >> 24] ^ Td1[(t1 >> 16) & 0xff] ^ Td2[(t0 >> 8) & 0xff] ^ Td3[t3 & 0xff] ^ rk[42];
- s3 = Td0[t3 >> 24] ^ Td1[(t2 >> 16) & 0xff] ^ Td2[(t1 >> 8) & 0xff] ^ Td3[t0 & 0xff] ^ rk[43];
- /* round 11: */
- t0 = Td0[s0 >> 24] ^ Td1[(s3 >> 16) & 0xff] ^ Td2[(s2 >> 8) & 0xff] ^ Td3[s1 & 0xff] ^ rk[44];
- t1 = Td0[s1 >> 24] ^ Td1[(s0 >> 16) & 0xff] ^ Td2[(s3 >> 8) & 0xff] ^ Td3[s2 & 0xff] ^ rk[45];
- t2 = Td0[s2 >> 24] ^ Td1[(s1 >> 16) & 0xff] ^ Td2[(s0 >> 8) & 0xff] ^ Td3[s3 & 0xff] ^ rk[46];
- t3 = Td0[s3 >> 24] ^ Td1[(s2 >> 16) & 0xff] ^ Td2[(s1 >> 8) & 0xff] ^ Td3[s0 & 0xff] ^ rk[47];
- if (Nr > 12) {
- /* round 12: */
- s0 = Td0[t0 >> 24] ^ Td1[(t3 >> 16) & 0xff] ^ Td2[(t2 >> 8) & 0xff] ^ Td3[t1 & 0xff] ^ rk[48];
- s1 = Td0[t1 >> 24] ^ Td1[(t0 >> 16) & 0xff] ^ Td2[(t3 >> 8) & 0xff] ^ Td3[t2 & 0xff] ^ rk[49];
- s2 = Td0[t2 >> 24] ^ Td1[(t1 >> 16) & 0xff] ^ Td2[(t0 >> 8) & 0xff] ^ Td3[t3 & 0xff] ^ rk[50];
- s3 = Td0[t3 >> 24] ^ Td1[(t2 >> 16) & 0xff] ^ Td2[(t1 >> 8) & 0xff] ^ Td3[t0 & 0xff] ^ rk[51];
- /* round 13: */
- t0 = Td0[s0 >> 24] ^ Td1[(s3 >> 16) & 0xff] ^ Td2[(s2 >> 8) & 0xff] ^ Td3[s1 & 0xff] ^ rk[52];
- t1 = Td0[s1 >> 24] ^ Td1[(s0 >> 16) & 0xff] ^ Td2[(s3 >> 8) & 0xff] ^ Td3[s2 & 0xff] ^ rk[53];
- t2 = Td0[s2 >> 24] ^ Td1[(s1 >> 16) & 0xff] ^ Td2[(s0 >> 8) & 0xff] ^ Td3[s3 & 0xff] ^ rk[54];
- t3 = Td0[s3 >> 24] ^ Td1[(s2 >> 16) & 0xff] ^ Td2[(s1 >> 8) & 0xff] ^ Td3[s0 & 0xff] ^ rk[55];
- }
- }
- rk += Nr << 2;
-#else /* !FULL_UNROLL */
- /*
- * Nr - 1 full rounds:
- */
- r = Nr >> 1;
- for (;;) {
- t0 =
- Td0[(s0 >> 24) ] ^
- Td1[(s3 >> 16) & 0xff] ^
- Td2[(s2 >> 8) & 0xff] ^
- Td3[(s1 ) & 0xff] ^
- rk[4];
- t1 =
- Td0[(s1 >> 24) ] ^
- Td1[(s0 >> 16) & 0xff] ^
- Td2[(s3 >> 8) & 0xff] ^
- Td3[(s2 ) & 0xff] ^
- rk[5];
- t2 =
- Td0[(s2 >> 24) ] ^
- Td1[(s1 >> 16) & 0xff] ^
- Td2[(s0 >> 8) & 0xff] ^
- Td3[(s3 ) & 0xff] ^
- rk[6];
- t3 =
- Td0[(s3 >> 24) ] ^
- Td1[(s2 >> 16) & 0xff] ^
- Td2[(s1 >> 8) & 0xff] ^
- Td3[(s0 ) & 0xff] ^
- rk[7];
-
- rk += 8;
- if (--r == 0) {
- break;
- }
-
- s0 =
- Td0[(t0 >> 24) ] ^
- Td1[(t3 >> 16) & 0xff] ^
- Td2[(t2 >> 8) & 0xff] ^
- Td3[(t1 ) & 0xff] ^
- rk[0];
- s1 =
- Td0[(t1 >> 24) ] ^
- Td1[(t0 >> 16) & 0xff] ^
- Td2[(t3 >> 8) & 0xff] ^
- Td3[(t2 ) & 0xff] ^
- rk[1];
- s2 =
- Td0[(t2 >> 24) ] ^
- Td1[(t1 >> 16) & 0xff] ^
- Td2[(t0 >> 8) & 0xff] ^
- Td3[(t3 ) & 0xff] ^
- rk[2];
- s3 =
- Td0[(t3 >> 24) ] ^
- Td1[(t2 >> 16) & 0xff] ^
- Td2[(t1 >> 8) & 0xff] ^
- Td3[(t0 ) & 0xff] ^
- rk[3];
- }
-#endif /* ?FULL_UNROLL */
- /*
- * apply last round and
- * map cipher state to byte array block:
- */
- s0 =
- (Td4[(t0 >> 24) ] & 0xff000000) ^
- (Td4[(t3 >> 16) & 0xff] & 0x00ff0000) ^
- (Td4[(t2 >> 8) & 0xff] & 0x0000ff00) ^
- (Td4[(t1 ) & 0xff] & 0x000000ff) ^
- rk[0];
- PUTU32(pt , s0);
- s1 =
- (Td4[(t1 >> 24) ] & 0xff000000) ^
- (Td4[(t0 >> 16) & 0xff] & 0x00ff0000) ^
- (Td4[(t3 >> 8) & 0xff] & 0x0000ff00) ^
- (Td4[(t2 ) & 0xff] & 0x000000ff) ^
- rk[1];
- PUTU32(pt + 4, s1);
- s2 =
- (Td4[(t2 >> 24) ] & 0xff000000) ^
- (Td4[(t1 >> 16) & 0xff] & 0x00ff0000) ^
- (Td4[(t0 >> 8) & 0xff] & 0x0000ff00) ^
- (Td4[(t3 ) & 0xff] & 0x000000ff) ^
- rk[2];
- PUTU32(pt + 8, s2);
- s3 =
- (Td4[(t3 >> 24) ] & 0xff000000) ^
- (Td4[(t2 >> 16) & 0xff] & 0x00ff0000) ^
- (Td4[(t1 >> 8) & 0xff] & 0x0000ff00) ^
- (Td4[(t0 ) & 0xff] & 0x000000ff) ^
- rk[3];
- PUTU32(pt + 12, s3);
-}
diff -r 81a487955535 -r 9d6b84c40f65 sys/crypto/rijndael/rijndael-api-fst.c
--- a/sys/crypto/rijndael/rijndael-api-fst.c Sun Jun 14 15:58:39 2020 +0000
+++ /dev/null Thu Jan 01 00:00:00 1970 +0000
@@ -1,430 +0,0 @@
-/* $NetBSD: rijndael-api-fst.c,v 1.25 2016/12/11 00:28:44 alnsn Exp $ */
-
-/**
- * rijndael-api-fst.c
- *
- * @version 2.9 (December 2000)
- *
- * Optimised ANSI C code for the Rijndael cipher (now AES)
- *
- * @author Vincent Rijmen <vincent.rijmen%esat.kuleuven.ac.be@localhost>
- * @author Antoon Bosselaers <antoon.bosselaers%esat.kuleuven.ac.be@localhost>
- * @author Paulo Barreto <paulo.barreto%terra.com.br@localhost>
- *
- * This code is hereby placed in the public domain.
- *
- * THIS SOFTWARE IS PROVIDED BY THE AUTHORS ''AS IS'' AND ANY EXPRESS
- * OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
- * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
- * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHORS OR CONTRIBUTORS BE
- * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
- * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
- * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
- * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
- * WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE
- * OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE,
- * EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
- *
- * Acknowledgements:
- *
- * We are deeply indebted to the following people for their bug reports,
- * fixes, and improvement suggestions to this implementation. Though we
- * tried to list all contributions, we apologise in advance for any
- * missing reference.
- *
- * Andrew Bales <Andrew.Bales%Honeywell.com@localhost>
- * Markus Friedl <markus.friedl%informatik.uni-erlangen.de@localhost>
- * John Skodon <skodonj%webquill.com@localhost>
- */
-
-#include <sys/cdefs.h>
-__KERNEL_RCSID(0, "$NetBSD: rijndael-api-fst.c,v 1.25 2016/12/11 00:28:44 alnsn Exp $");
-
-#include <sys/param.h>
-#ifdef _KERNEL
-#include <sys/systm.h>
-#else
-#include <stdlib.h>
-#include <string.h>
-#endif
-
-#include <crypto/rijndael/rijndael_local.h>
-#include <crypto/rijndael/rijndael-alg-fst.h>
-#include <crypto/rijndael/rijndael-api-fst.h>
-
-#define XTS_ALPHA 0x87
-
-static void xor16(uint8_t *d, const uint8_t *a, const uint8_t* b)
-{
- for (size_t i = 0; i < 4; i++) {
- *d++ = *a++ ^ *b++;
- *d++ = *a++ ^ *b++;
- *d++ = *a++ ^ *b++;
- *d++ = *a++ ^ *b++;
- }
-}
-
-static void
-xts_exponentiate(uint8_t *iv)
-{
- unsigned int carry = 0;
-
- for (size_t i = 0; i < 16; i++) {
- unsigned int msb = iv[i] >> 7;
-
- iv[i] = (iv[i] << 1) | carry;
- carry = msb;
- }
-
- if (carry != 0)
- iv[0] ^= XTS_ALPHA;
-}
-
-int
-rijndael_makeKey(keyInstance *key, BYTE direction, int keyLen,
- const char *keyMaterial)
-{
- u_int8_t cipherKey[RIJNDAEL_MAXKB];
-
- if (key == NULL) {
- return BAD_KEY_INSTANCE;
- }
-
- if ((direction == DIR_ENCRYPT) || (direction == DIR_DECRYPT)) {
- key->direction = direction;
- } else {
- return BAD_KEY_DIR;
- }
-
- if ((keyLen == 128) || (keyLen == 192) || (keyLen == 256)) {
- key->keyLen = keyLen;
- } else {
- return BAD_KEY_MAT;
- }
-
- if (keyMaterial != NULL) {
- memcpy(key->keyMaterial, keyMaterial, keyLen/8);
- }
-
- /* initialize key schedule: */
- memcpy(cipherKey, key->keyMaterial, keyLen/8);
- if (direction == DIR_ENCRYPT) {
- key->Nr = rijndaelKeySetupEnc(key->rk, cipherKey, keyLen);
- } else {
- key->Nr = rijndaelKeySetupDec(key->rk, cipherKey, keyLen);
- }
- rijndaelKeySetupEnc(key->ek, cipherKey, keyLen);
- return TRUE;
-}
-
-int
-rijndael_cipherInit(cipherInstance *cipher, BYTE mode, const char *IV)
-{
- if ((mode == MODE_ECB) || (mode == MODE_CBC) ||
- (mode == MODE_XTS) || (mode == MODE_CFB1)) {
- cipher->mode = mode;
- } else {
- return BAD_CIPHER_MODE;
- }
- if (IV != NULL) {
- memcpy(cipher->IV, IV, RIJNDAEL_MAX_IV_SIZE);
- } else {
- memset(cipher->IV, 0, RIJNDAEL_MAX_IV_SIZE);
- }
- return TRUE;
-}
-
-int
-rijndael_blockEncrypt(cipherInstance *cipher, keyInstance *key,
- const BYTE *input, int inputLen, BYTE *outBuffer)
-{
- int i, k, t, numBlocks;
- u_int8_t block[16], *iv;
-
- if (cipher == NULL ||
- key == NULL ||
- key->direction == DIR_DECRYPT) {
- return BAD_CIPHER_STATE;
- }
- if (input == NULL || inputLen <= 0) {
- return 0; /* nothing to do */
- }
-
- numBlocks = inputLen/128;
-
- switch (cipher->mode) {
- case MODE_ECB:
- for (i = numBlocks; i > 0; i--) {
- rijndaelEncrypt(key->rk, key->Nr, input, outBuffer);
- input += 16;
- outBuffer += 16;
- }
- break;
-
- case MODE_CBC:
- iv = (u_int8_t *)cipher->IV;
- for (i = numBlocks; i > 0; i--) {
- xor16(block, input, iv);
- rijndaelEncrypt(key->rk, key->Nr, block, outBuffer);
- iv = outBuffer;
- input += 16;
- outBuffer += 16;
- }
- break;
-
- case MODE_XTS:
- iv = (u_int8_t *)cipher->IV;
- for (i = numBlocks; i > 0; i--) {
- xor16(block, input, iv);
- rijndaelEncrypt(key->rk, key->Nr, block, block);
- xor16(outBuffer, block, iv);
- xts_exponentiate(iv);
- input += 16;
- outBuffer += 16;
- }
- break;
-
- case MODE_CFB1:
- iv = (u_int8_t *)cipher->IV;
- for (i = numBlocks; i > 0; i--) {
- memcpy(outBuffer, input, 16);
- for (k = 0; k < 128; k++) {
- rijndaelEncrypt(key->ek, key->Nr, iv, block);
- outBuffer[k >> 3] ^=
- (block[0] & 0x80U) >> (k & 7);
- for (t = 0; t < 15; t++) {
- iv[t] = (iv[t] << 1) | (iv[t + 1] >> 7);
- }
- iv[15] = (iv[15] << 1) |
- ((outBuffer[k >> 3] >> (7 - (k & 7))) & 1);
- }
- outBuffer += 16;
- input += 16;
- }
- break;
-
- default:
- return BAD_CIPHER_STATE;
- }
-
- return 128 * numBlocks;
-}
-
-/**
- * Encrypt data partitioned in octets, using RFC 2040-like padding.
- *
- * @param input data to be encrypted (octet sequence)
- * @param inputOctets input length in octets (not bits)
- * @param outBuffer encrypted output data
- *
- * @return length in octets (not bits) of the encrypted output buffer.
- */
-int
-rijndael_padEncrypt(cipherInstance *cipher, keyInstance *key,
- const BYTE *input, int inputOctets, BYTE *outBuffer)
-{
- int i, numBlocks, padLen;
- u_int8_t block[16], *iv;
-
- if (cipher == NULL ||
- key == NULL ||
- key->direction == DIR_DECRYPT) {
- return BAD_CIPHER_STATE;
- }
- if (input == NULL || inputOctets <= 0) {
- return 0; /* nothing to do */
- }
-
- numBlocks = inputOctets / 16;
-
- switch (cipher->mode) {
- case MODE_ECB:
- for (i = numBlocks; i > 0; i--) {
- rijndaelEncrypt(key->rk, key->Nr, input, outBuffer);
- input += 16;
- outBuffer += 16;
- }
- padLen = 16 - (inputOctets - 16*numBlocks);
- memcpy(block, input, 16 - padLen);
- memset(block + 16 - padLen, padLen, padLen);
- rijndaelEncrypt(key->rk, key->Nr, block, outBuffer);
- break;
-
- case MODE_CBC:
- iv = (u_int8_t *)cipher->IV;
- for (i = numBlocks; i > 0; i--) {
- xor16(block, input, iv);
- rijndaelEncrypt(key->rk, key->Nr, block, outBuffer);
- iv = outBuffer;
- input += 16;
- outBuffer += 16;
- }
- padLen = 16 - (inputOctets - 16*numBlocks);
- for (i = 0; i < 16 - padLen; i++) {
- block[i] = input[i] ^ iv[i];
- }
- for (i = 16 - padLen; i < 16; i++) {
- block[i] = (BYTE)padLen ^ iv[i];
- }
- rijndaelEncrypt(key->rk, key->Nr, block, outBuffer);
- break;
-
- default:
- return BAD_CIPHER_STATE;
- }
-
- return 16 * (numBlocks + 1);
-}
-
-int
-rijndael_blockDecrypt(cipherInstance *cipher, keyInstance *key,
- const BYTE *input, int inputLen, BYTE *outBuffer)
-{
- int i, k, t, numBlocks;
- u_int8_t block[16], *iv;
-
- if (cipher == NULL ||
- key == NULL ||
- (cipher->mode != MODE_CFB1 && key->direction == DIR_ENCRYPT)) {
- return BAD_CIPHER_STATE;
- }
- if (input == NULL || inputLen <= 0) {
- return 0; /* nothing to do */
- }
-
- numBlocks = inputLen/128;
-
- switch (cipher->mode) {
- case MODE_ECB:
- for (i = numBlocks; i > 0; i--) {
- rijndaelDecrypt(key->rk, key->Nr, input, outBuffer);
- input += 16;
- outBuffer += 16;
- }
- break;
-
- case MODE_CBC:
- iv = (u_int8_t *)cipher->IV;
- for (i = numBlocks; i > 0; i--) {
- rijndaelDecrypt(key->rk, key->Nr, input, block);
- xor16(block, block, iv);
- memcpy(cipher->IV, input, 16);
- memcpy(outBuffer, block, 16);
- input += 16;
- outBuffer += 16;
- }
- break;
-
- case MODE_XTS:
- iv = (u_int8_t *)cipher->IV;
- for (i = numBlocks; i > 0; i--) {
- xor16(block, input, iv);
- rijndaelDecrypt(key->rk, key->Nr, block, block);
- xor16(outBuffer, block, iv);
- xts_exponentiate(iv);
- input += 16;
- outBuffer += 16;
- }
- break;
-
- case MODE_CFB1:
- iv = (u_int8_t *)cipher->IV;
- for (i = numBlocks; i > 0; i--) {
- memcpy(outBuffer, input, 16);
- for (k = 0; k < 128; k++) {
- rijndaelEncrypt(key->ek, key->Nr, iv, block);
- for (t = 0; t < 15; t++) {
- iv[t] = (iv[t] << 1) | (iv[t + 1] >> 7);
- }
- iv[15] = (iv[15] << 1) |
- ((input[k >> 3] >> (7 - (k & 7))) & 1);
- outBuffer[k >> 3] ^= (block[0] & 0x80U) >>
- (k & 7);
- }
- outBuffer += 16;
- input += 16;
- }
- break;
-
- default:
- return BAD_CIPHER_STATE;
- }
-
- return 128 * numBlocks;
-}
-
-int
-rijndael_padDecrypt(cipherInstance *cipher, keyInstance *key,
- const BYTE *input, int inputOctets, BYTE *outBuffer)
-{
- int i, numBlocks, padLen;
- u_int8_t block[16], *iv;
-
- if (cipher == NULL ||
- key == NULL ||
- key->direction == DIR_ENCRYPT) {
- return BAD_CIPHER_STATE;
- }
- if (input == NULL || inputOctets <= 0) {
- return 0; /* nothing to do */
- }
- if (inputOctets % 16 != 0) {
- return BAD_DATA;
- }
-
- numBlocks = inputOctets/16;
-
- switch (cipher->mode) {
- case MODE_ECB:
- /* all blocks but last */
- for (i = numBlocks - 1; i > 0; i--) {
- rijndaelDecrypt(key->rk, key->Nr, input, outBuffer);
- input += 16;
- outBuffer += 16;
- }
- /* last block */
- rijndaelDecrypt(key->rk, key->Nr, input, block);
- padLen = block[15];
- if (padLen >= 16) {
- return BAD_DATA;
- }
- for (i = 16 - padLen; i < 16; i++) {
- if (block[i] != padLen) {
- return BAD_DATA;
- }
- }
- memcpy(outBuffer, block, 16 - padLen);
- break;
-
- case MODE_CBC:
- iv = (u_int8_t *)cipher->IV;
- /* all blocks but last */
- for (i = numBlocks - 1; i > 0; i--) {
- rijndaelDecrypt(key->rk, key->Nr, input, block);
- xor16(block, block, iv);
- memcpy(cipher->IV, input, 16);
- memcpy(outBuffer, block, 16);
- input += 16;
- outBuffer += 16;
- }
- /* last block */
- rijndaelDecrypt(key->rk, key->Nr, input, block);
- xor16(block, block, iv);
- padLen = block[15];
- if (padLen <= 0 || padLen > 16) {
- return BAD_DATA;
- }
- for (i = 16 - padLen; i < 16; i++) {
- if (block[i] != padLen) {
- return BAD_DATA;
- }
- }
- memcpy(outBuffer, block, 16 - padLen);
- break;
-
- default:
- return BAD_CIPHER_STATE;
- }
-
- return 16 * numBlocks - padLen;
-}
diff -r 81a487955535 -r 9d6b84c40f65 sys/crypto/rijndael/rijndael.c
--- a/sys/crypto/rijndael/rijndael.c Sun Jun 14 15:58:39 2020 +0000
+++ /dev/null Thu Jan 01 00:00:00 1970 +0000
@@ -1,57 +0,0 @@
-/* $NetBSD: rijndael.c,v 1.8 2005/12/11 12:20:52 christos Exp $ */
-
-/**
- * rijndael-alg-fst.c
- *
- * @version 3.0 (December 2000)
- *
- * Optimised ANSI C code for the Rijndael cipher (now AES)
- *
- * @author Vincent Rijmen <vincent.rijmen%esat.kuleuven.ac.be@localhost>
- * @author Antoon Bosselaers <antoon.bosselaers%esat.kuleuven.ac.be@localhost>
- * @author Paulo Barreto <paulo.barreto%terra.com.br@localhost>
- *
- * This code is hereby placed in the public domain.
- *
- * THIS SOFTWARE IS PROVIDED BY THE AUTHORS ''AS IS'' AND ANY EXPRESS
- * OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
- * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
- * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHORS OR CONTRIBUTORS BE
- * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
- * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
- * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
- * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
- * WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE
- * OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE,
- * EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
- */
-
-#include <sys/cdefs.h>
-__KERNEL_RCSID(0, "$NetBSD: rijndael.c,v 1.8 2005/12/11 12:20:52 christos Exp $");
-
-#include <sys/types.h>
-#include <sys/systm.h>
-
-#include <crypto/rijndael/rijndael.h>
-
-void
-rijndael_set_key(rijndael_ctx *ctx, const u_char *key, int bits)
-{
-
- ctx->Nr = rijndaelKeySetupEnc(ctx->ek, key, bits);
- rijndaelKeySetupDec(ctx->dk, key, bits);
-}
-
-void
-rijndael_decrypt(const rijndael_ctx *ctx, const u_char *src, u_char *dst)
-{
-
- rijndaelDecrypt(ctx->dk, ctx->Nr, src, dst);
-}
-
-void
-rijndael_encrypt(const rijndael_ctx *ctx, const u_char *src, u_char *dst)
-{
-
- rijndaelEncrypt(ctx->ek, ctx->Nr, src, dst);
-}
diff -r 81a487955535 -r 9d6b84c40f65 sys/crypto/rijndael/rijndael_local.h
--- a/sys/crypto/rijndael/rijndael_local.h Sun Jun 14 15:58:39 2020 +0000
+++ /dev/null Thu Jan 01 00:00:00 1970 +0000
@@ -1,7 +0,0 @@
-/* $NetBSD: rijndael_local.h,v 1.6 2005/12/11 12:20:52 christos Exp $ */
-/* $KAME: rijndael_local.h,v 1.4 2003/07/15 10:47:16 itojun Exp $ */
-
-/* the file should not be used from outside */
-typedef u_int8_t u8;
-typedef u_int16_t u16;
-typedef u_int32_t u32;
diff -r 81a487955535 -r 9d6b84c40f65 sys/rump/kern/lib/libcrypto/Makefile
--- a/sys/rump/kern/lib/libcrypto/Makefile Sun Jun 14 15:58:39 2020 +0000
+++ b/sys/rump/kern/lib/libcrypto/Makefile Fri Jun 12 05:16:46 2020 +0000
@@ -1,11 +1,11 @@
# $NetBSD: Makefile,v 1.6 2019/12/05 03:57:55 riastradh Exp $
#
-.PATH: ${.CURDIR}/../../../../crypto/blowfish \
+.PATH: ${.CURDIR}/../../../../crypto/aes \
+ ${.CURDIR}/../../../../crypto/blowfish \
${.CURDIR}/../../../../crypto/camellia \
${.CURDIR}/../../../../crypto/cast128 \
${.CURDIR}/../../../../crypto/des \
- ${.CURDIR}/../../../../crypto/rijndael \
${.CURDIR}/../../../../crypto/skipjack
LIB= rumpkern_crypto
@@ -23,8 +23,14 @@ SRCS+= cast128.c
# DES
SRCS+= des_ecb.c des_setkey.c des_enc.c des_cbc.c des_module.c
-# rijndael
-SRCS+= rijndael-alg-fst.c rijndael-api-fst.c rijndael.c
+# AES
+SRCS+= aes_bear.c
+SRCS+= aes_ct.c
+SRCS+= aes_ct_dec.c
+SRCS+= aes_ct_enc.c
+SRCS+= aes_impl.c
+SRCS+= aes_rijndael.c
+SRCS+= aes_selftest.c
# skipjack
SRCS+= skipjack.c
# HG changeset patch
# User Taylor R Campbell <riastradh%NetBSD.org@localhost>
# Date 1592435129 0
# Wed Jun 17 23:05:29 2020 +0000
# Branch trunk
# Node ID fea7aeacc09cf9da68d32a15edf9550ce78a4d45
# Parent 9d6b84c40f6517bb55848159faa9478ef1a23d02
# EXP-Topic riastradh-kernelcrypto
Add x86 AES-NI support.
Limited to amd64 for now. In principle, AES-NI should work in 32-bit
mode, and there may even be some 32-bit-only CPUs that support
AES-NI, but that requires work to adapt the assembly.
diff -r 9d6b84c40f65 -r fea7aeacc09c sys/arch/x86/conf/files.x86
--- a/sys/arch/x86/conf/files.x86 Fri Jun 12 05:16:46 2020 +0000
+++ b/sys/arch/x86/conf/files.x86 Wed Jun 17 23:05:29 2020 +0000
@@ -165,3 +165,6 @@ file arch/x86/pci/pciide_machdep.c pciid
file arch/x86/pci/pci_bus_fixup.c pci_bus_fixup
file arch/x86/pci/pci_addr_fixup.c pci_addr_fixup
+
+# AES-NI
+include "crypto/aes/arch/x86/files.aesni"
diff -r 9d6b84c40f65 -r fea7aeacc09c sys/arch/x86/x86/identcpu.c
--- a/sys/arch/x86/x86/identcpu.c Fri Jun 12 05:16:46 2020 +0000
+++ b/sys/arch/x86/x86/identcpu.c Wed Jun 17 23:05:29 2020 +0000
@@ -39,6 +39,8 @@
#include <sys/device.h>
#include <sys/cpu.h>
+#include <crypto/aes/arch/x86/aes_ni.h>
+
#include <uvm/uvm_extern.h>
#include <machine/specialreg.h>
@@ -995,6 +997,10 @@ cpu_probe(struct cpu_info *ci)
/* Early patch of text segment. */
x86_patch(true);
#endif
+#ifdef __x86_64__ /* not yet implemented on i386 */
+ if (cpu_feature[1] & CPUID2_AES)
+ aes_md_init(&aes_ni_impl);
+#endif
} else {
/*
* If not first. Warn about cpu_feature mismatch for
diff -r 9d6b84c40f65 -r fea7aeacc09c sys/crypto/aes/arch/x86/aes_ni.c
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/sys/crypto/aes/arch/x86/aes_ni.c Wed Jun 17 23:05:29 2020 +0000
@@ -0,0 +1,252 @@
+/* $NetBSD$ */
+
+/*-
+ * Copyright (c) 2020 The NetBSD Foundation, Inc.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND CONTRIBUTORS
+ * ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
+ * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+ * POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <sys/cdefs.h>
+__KERNEL_RCSID(1, "$NetBSD$");
+
+#include <sys/types.h>
+#include <sys/systm.h>
+
+#include <crypto/aes/aes.h>
+#include <crypto/aes/arch/x86/aes_ni.h>
+
+#include <x86/cpuvar.h>
+#include <x86/fpu.h>
+#include <x86/specialreg.h>
+
+static void
+aesni_setenckey(struct aesenc *enc, const uint8_t key[static 16],
+ uint32_t nrounds)
+{
+
+ switch (nrounds) {
+ case 10:
+ aesni_setenckey128(enc, key);
+ break;
+ case 12:
+ aesni_setenckey192(enc, key);
+ break;
+ case 14:
+ aesni_setenckey256(enc, key);
+ break;
+ default:
+ panic("invalid AES rounds: %u", nrounds);
+ }
+}
+
+static void
+aesni_setenckey_impl(struct aesenc *enc, const uint8_t key[static 16],
+ uint32_t nrounds)
+{
+
+ fpu_kern_enter();
+ aesni_setenckey(enc, key, nrounds);
+ fpu_kern_leave();
+}
+
+static void
+aesni_setdeckey_impl(struct aesdec *dec, const uint8_t key[static 16],
+ uint32_t nrounds)
+{
+ struct aesenc enc;
+
+ fpu_kern_enter();
+ aesni_setenckey(&enc, key, nrounds);
+ aesni_enctodec(&enc, dec, nrounds);
+ fpu_kern_leave();
+
+ explicit_memset(&enc, 0, sizeof enc);
+}
+
+static void
+aesni_enc_impl(const struct aesenc *enc, const uint8_t in[static 16],
+ uint8_t out[static 16], uint32_t nrounds)
+{
+
+ fpu_kern_enter();
+ aesni_enc(enc, in, out, nrounds);
+ fpu_kern_leave();
+}
+
+static void
+aesni_dec_impl(const struct aesdec *dec, const uint8_t in[static 16],
+ uint8_t out[static 16], uint32_t nrounds)
+{
+
+ fpu_kern_enter();
+ aesni_dec(dec, in, out, nrounds);
+ fpu_kern_leave();
+}
+
+static void
+aesni_cbc_enc_impl(const struct aesenc *enc, const uint8_t in[static 16],
+ uint8_t out[static 16], size_t nbytes, uint8_t iv[static 16],
+ uint32_t nrounds)
+{
+
+ KASSERT(nbytes % 16 == 0);
+
+ fpu_kern_enter();
+ aesni_cbc_enc(enc, in, out, nbytes, iv, nrounds);
+ fpu_kern_leave();
+}
+
+static void
+aesni_cbc_dec_impl(const struct aesdec *dec, const uint8_t in[static 16],
+ uint8_t out[static 16], size_t nbytes, uint8_t iv[static 16],
+ uint32_t nrounds)
+{
+
+ KASSERT(nbytes % 16 == 0);
+
+ fpu_kern_enter();
+
+ if (nbytes % 128) {
+ aesni_cbc_dec1(dec, in, out, nbytes % 128, iv, nrounds);
+ in += nbytes % 128;
+ out += nbytes % 128;
+ nbytes -= nbytes % 128;
+ }
+
+ KASSERT(nbytes % 128 == 0);
+ if (nbytes)
+ aesni_cbc_dec8(dec, in, out, nbytes, iv, nrounds);
+
+ fpu_kern_leave();
+}
+
+static void
+aesni_xts_enc_impl(const struct aesenc *enc, const uint8_t in[static 16],
+ uint8_t out[static 16], size_t nbytes, uint8_t iv[static 16],
+ uint32_t nrounds)
+{
+
+ KASSERT(nbytes % 16 == 0);
+
+ fpu_kern_enter();
+
+ if (nbytes % 128) {
+ aesni_xts_enc1(enc, in, out, nbytes % 128, iv, nrounds);
+ in += nbytes % 128;
+ out += nbytes % 128;
+ nbytes -= nbytes % 128;
+ }
+
+ KASSERT(nbytes % 128 == 0);
+ if (nbytes)
+ aesni_xts_enc8(enc, in, out, nbytes, iv, nrounds);
+
+ fpu_kern_leave();
+}
+
+static void
+aesni_xts_dec_impl(const struct aesdec *dec, const uint8_t in[static 16],
+ uint8_t out[static 16], size_t nbytes, uint8_t iv[static 16],
+ uint32_t nrounds)
+{
+
+ KASSERT(nbytes % 16 == 0);
+
+ fpu_kern_enter();
+
+ if (nbytes % 128) {
+ aesni_xts_dec1(dec, in, out, nbytes % 128, iv, nrounds);
+ in += nbytes % 128;
+ out += nbytes % 128;
+ nbytes -= nbytes % 128;
+ }
+
+ KASSERT(nbytes % 128 == 0);
+ if (nbytes)
+ aesni_xts_dec8(dec, in, out, nbytes, iv, nrounds);
+
+ fpu_kern_leave();
+}
+
+static int
+aesni_xts_update_selftest(void)
+{
+ static const struct {
+ uint8_t in[16], out[16];
+ } cases[] = {
+ {{1}, {2}},
+ {{0,0,0,0x80}, {0,0,0,0,1}},
+ {{0,0,0,0,0,0,0,0x80}, {0,0,0,0,0,0,0,0,1}},
+ {{0,0,0,0x80,0,0,0,0x80}, {0,0,0,0,1,0,0,0,1}},
+ {{0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0x80}, {0x87}},
+ {{0,0,0,0,0,0,0,0x80,0,0,0,0,0,0,0,0x80},
+ {0x87,0,0,0,0,0,0,0,1}},
+ {{0,0,0,0x80,0,0,0,0,0,0,0,0,0,0,0,0x80}, {0x87,0,0,0,1}},
+ {{0,0,0,0x80,0,0,0,0x80,0,0,0,0,0,0,0,0x80},
+ {0x87,0,0,0,1,0,0,0,1}},
+ };
+ unsigned i;
+ uint8_t tweak[16];
+
+ for (i = 0; i < sizeof(cases)/sizeof(cases[0]); i++) {
+ aesni_xts_update(cases[i].in, tweak);
+ if (memcmp(tweak, cases[i].out, 16))
+ return -1;
+ }
+
+ /* Success! */
+ return 0;
+}
+
+static int
+aesni_probe(void)
+{
+ int result = 0;
+
+ /* Verify that the CPU supports AES-NI. */
+ if ((cpu_feature[1] & CPUID2_AES) == 0)
+ return -1;
+
+ fpu_kern_enter();
+
+ /* Verify that our XTS tweak update logic works. */
+ if (aesni_xts_update_selftest())
+ result = -1;
+
+ fpu_kern_leave();
+
+ return result;
+}
+
+struct aes_impl aes_ni_impl = {
+ .ai_name = "Intel AES-NI",
+ .ai_probe = aesni_probe,
+ .ai_setenckey = aesni_setenckey_impl,
+ .ai_setdeckey = aesni_setdeckey_impl,
+ .ai_enc = aesni_enc_impl,
+ .ai_dec = aesni_dec_impl,
+ .ai_cbc_enc = aesni_cbc_enc_impl,
+ .ai_cbc_dec = aesni_cbc_dec_impl,
+ .ai_xts_enc = aesni_xts_enc_impl,
+ .ai_xts_dec = aesni_xts_dec_impl,
+};
diff -r 9d6b84c40f65 -r fea7aeacc09c sys/crypto/aes/arch/x86/aes_ni.h
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/sys/crypto/aes/arch/x86/aes_ni.h Wed Jun 17 23:05:29 2020 +0000
@@ -0,0 +1,68 @@
+/* $NetBSD$ */
+
+/*-
+ * Copyright (c) 2020 The NetBSD Foundation, Inc.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND CONTRIBUTORS
+ * ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
+ * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+ * POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _CRYPTO_AES_ARCH_X86_AES_NI_H
+#define _CRYPTO_AES_ARCH_X86_AES_NI_H
+
+#include <sys/types.h>
+
+#include <crypto/aes/aes.h>
+
+/* Assembly routines */
+
+void aesni_setenckey128(struct aesenc *, const uint8_t[static 16]);
+void aesni_setenckey192(struct aesenc *, const uint8_t[static 24]);
+void aesni_setenckey256(struct aesenc *, const uint8_t[static 32]);
+
+void aesni_enctodec(const struct aesenc *, struct aesdec *, uint32_t);
+
+void aesni_enc(const struct aesenc *, const uint8_t[static 16],
+ uint8_t[static 16], uint32_t);
+void aesni_dec(const struct aesdec *, const uint8_t[static 16],
+ uint8_t[static 16], uint32_t);
+
+void aesni_cbc_enc(const struct aesenc *, const uint8_t[static 16],
+ uint8_t[static 16], size_t, uint8_t[static 16], uint32_t);
+void aesni_cbc_dec1(const struct aesdec *, const uint8_t[static 16],
+ uint8_t[static 16], size_t, const uint8_t[static 16], uint32_t);
+void aesni_cbc_dec8(const struct aesdec *, const uint8_t[static 128],
+ uint8_t[static 128], size_t, const uint8_t[static 16], uint32_t);
+
+void aesni_xts_enc1(const struct aesenc *, const uint8_t[static 16],
+ uint8_t[static 16], size_t, uint8_t[static 16], uint32_t);
+void aesni_xts_enc8(const struct aesenc *, const uint8_t[static 128],
+ uint8_t[static 128], size_t, uint8_t[static 16], uint32_t);
+void aesni_xts_dec1(const struct aesdec *, const uint8_t[static 16],
+ uint8_t[static 16], size_t, uint8_t[static 16], uint32_t);
+void aesni_xts_dec8(const struct aesdec *, const uint8_t[static 128],
+ uint8_t[static 128], size_t, uint8_t[static 16], uint32_t);
+void aesni_xts_update(const uint8_t[static 16], uint8_t[static 16]);
+
+extern struct aes_impl aes_ni_impl;
+
+#endif /* _CRYPTO_AES_ARCH_X86_AES_NI_H */
diff -r 9d6b84c40f65 -r fea7aeacc09c sys/crypto/aes/arch/x86/aesnifunc.S
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/sys/crypto/aes/arch/x86/aesnifunc.S Wed Jun 17 23:05:29 2020 +0000
@@ -0,0 +1,1097 @@
+/* $NetBSD$ */
+
+/*-
+ * Copyright (c) 2020 The NetBSD Foundation, Inc.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND CONTRIBUTORS
+ * ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
+ * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+ * POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <machine/asm.h>
+
+/*
+ * MOVDQA/MOVDQU are Move Double Quadword (Aligned/Unaligned), defined
+ * to operate on integers; MOVAPS/MOVUPS are Move (Aligned/Unaligned)
+ * Packed Single, defined to operate on binary32 floats. They have
+ * exactly the same architectural effects (move a 128-bit quantity from
+ * memory into an xmm register).
+ *
+ * In principle, they might have different microarchitectural effects
+ * so that MOVAPS/MOVUPS might incur a penalty when the register is
+ * later used for integer paths, but in practice they don't. So we use
+ * the one whose instruction encoding is shorter -- MOVAPS/MOVUPS.
+ */
+#define movdqa movaps
+#define movdqu movups
+
+/*
+ * aesni_setenckey128(struct aesenc *enckey@rdi, const uint8_t key[16] @rsi)
+ *
+ * Expand a 16-byte AES-128 key into 10 round keys.
+ *
+ * Standard ABI calling convention.
+ */
+ENTRY(aesni_setenckey128)
+ movdqu (%rsi),%xmm0 /* load master key into %xmm0 */
+ movdqa %xmm0,(%rdi) /* store master key as the first round key */
+ lea 0x10(%rdi),%rdi /* advance %rdi to next round key */
+ aeskeygenassist $0x1,%xmm0,%xmm2
+ call aesni_expand128
+ aeskeygenassist $0x2,%xmm0,%xmm2
+ call aesni_expand128
+ aeskeygenassist $0x4,%xmm0,%xmm2
+ call aesni_expand128
+ aeskeygenassist $0x8,%xmm0,%xmm2
+ call aesni_expand128
+ aeskeygenassist $0x10,%xmm0,%xmm2
+ call aesni_expand128
+ aeskeygenassist $0x20,%xmm0,%xmm2
+ call aesni_expand128
+ aeskeygenassist $0x40,%xmm0,%xmm2
+ call aesni_expand128
+ aeskeygenassist $0x80,%xmm0,%xmm2
+ call aesni_expand128
+ aeskeygenassist $0x1b,%xmm0,%xmm2
+ call aesni_expand128
+ aeskeygenassist $0x36,%xmm0,%xmm2
+ call aesni_expand128
+ ret
+END(aesni_setenckey128)
+
+/*
+ * aesni_setenckey192(struct aesenc *enckey@rdi, const uint8_t key[24] @rsi)
+ *
+ * Expand a 24-byte AES-192 key into 12 round keys.
+ *
+ * Standard ABI calling convention.
+ */
+ENTRY(aesni_setenckey192)
+ movdqu (%rsi),%xmm0 /* load master key [0:128) into %xmm0 */
+ movq 0x10(%rsi),%xmm1 /* load master key [128:192) into %xmm1 */
+ movdqa %xmm0,(%rdi) /* store master key [0:128) as round key */
+ lea 0x10(%rdi),%rdi /* advance %rdi to next round key */
+ aeskeygenassist $0x1,%xmm1,%xmm2
+ call aesni_expand192a
+ aeskeygenassist $0x2,%xmm0,%xmm2
+ call aesni_expand192b
+ aeskeygenassist $0x4,%xmm1,%xmm2
+ call aesni_expand192a
+ aeskeygenassist $0x8,%xmm0,%xmm2
+ call aesni_expand192b
+ aeskeygenassist $0x10,%xmm1,%xmm2
+ call aesni_expand192a
+ aeskeygenassist $0x20,%xmm0,%xmm2
+ call aesni_expand192b
+ aeskeygenassist $0x40,%xmm1,%xmm2
+ call aesni_expand192a
+ aeskeygenassist $0x80,%xmm0,%xmm2
+ call aesni_expand192b
+ ret
+END(aesni_setenckey192)
+
+/*
+ * aesni_setenckey256(struct aesenc *enckey@rdi, const uint8_t key[32] @rsi)
+ *
+ * Expand a 32-byte AES-256 key into 14 round keys.
+ *
+ * Standard ABI calling convention.
+ */
+ENTRY(aesni_setenckey256)
+ movdqu (%rsi),%xmm0 /* load master key [0:128) into %xmm0 */
+ movdqu 0x10(%rsi),%xmm1 /* load master key [128:256) into %xmm1 */
+ movdqa %xmm0,(%rdi) /* store master key [0:128) as round key */
+ movdqa %xmm1,0x10(%rdi) /* store master key [128:256) as round key */
+ lea 0x20(%rdi),%rdi /* advance %rdi to next round key */
+ aeskeygenassist $0x1,%xmm1,%xmm2
+ call aesni_expand256a
+ aeskeygenassist $0x1,%xmm0,%xmm2
+ call aesni_expand256b
+ aeskeygenassist $0x2,%xmm1,%xmm2
+ call aesni_expand256a
+ aeskeygenassist $0x2,%xmm0,%xmm2
+ call aesni_expand256b
+ aeskeygenassist $0x4,%xmm1,%xmm2
+ call aesni_expand256a
+ aeskeygenassist $0x4,%xmm0,%xmm2
+ call aesni_expand256b
+ aeskeygenassist $0x8,%xmm1,%xmm2
+ call aesni_expand256a
+ aeskeygenassist $0x8,%xmm0,%xmm2
+ call aesni_expand256b
+ aeskeygenassist $0x10,%xmm1,%xmm2
+ call aesni_expand256a
+ aeskeygenassist $0x10,%xmm0,%xmm2
+ call aesni_expand256b
+ aeskeygenassist $0x20,%xmm1,%xmm2
+ call aesni_expand256a
+ aeskeygenassist $0x20,%xmm0,%xmm2
+ call aesni_expand256b
+ aeskeygenassist $0x40,%xmm1,%xmm2
+ call aesni_expand256a
+ ret
+END(aesni_setenckey256)
+
+/*
+ * aesni_expand128(uint128_t *rkp@rdi, uint128_t prk@xmm0,
+ * uint128_t keygenassist@xmm2)
+ *
+ * 1. Compute the AES-128 round key using the previous round key.
+ * 2. Store it at *rkp.
+ * 3. Set %xmm0 to it.
+ * 4. Advance %rdi to point at the next round key.
+ *
+ * Internal ABI. On entry:
+ *
+ * %rdi = rkp, pointer to round key to compute
+ * %xmm0 = (prk[0], prk[1], prk[2], prk[3])
+ * %xmm2 = (xxx, xxx, xxx, t = Rot(SubWord(prk[3])) ^ RCON)
+ *
+ * On exit:
+ *
+ * %rdi = &rkp[1], rkp advanced by one round key
+ * %xmm0 = rk, the round key we just computed
+ * %xmm2 = garbage
+ * %xmm4 = garbage
+ * %xmm5 = garbage
+ * %xmm6 = garbage
+ *
+ * Note: %xmm1 is preserved (as are %xmm3 and %xmm7 through %xmm15,
+ * and all other registers).
+ */
+ .text
+ _ALIGN_TEXT
+ .type aesni_expand128,@function
+aesni_expand128:
+ /*
+ * %xmm2 := (%xmm2[3], %xmm2[3], %xmm2[3], %xmm2[3]),
+ * i.e., set each word of %xmm2 to t := Rot(SubWord(prk[3])) ^ RCON.
+ */
+ pshufd $0b11111111,%xmm2,%xmm2
+
+ /*
+ * %xmm4 := (0, prk[0], prk[1], prk[2])
+ * %xmm5 := (0, 0, prk[0], prk[1])
+ * %xmm6 := (0, 0, 0, prk[0])
+ */
+ movdqa %xmm0,%xmm4
+ movdqa %xmm0,%xmm5
+ movdqa %xmm0,%xmm6
+ pslldq $4,%xmm4
+ pslldq $8,%xmm5
+ pslldq $12,%xmm6
+
+ /*
+ * %xmm0 := (rk[0] = t ^ prk[0],
+ * rk[1] = t ^ prk[0] ^ prk[1],
+ * rk[2] = t ^ prk[0] ^ prk[1] ^ prk[2],
+ * rk[3] = t ^ prk[0] ^ prk[1] ^ prk[2] ^ prk[3])
+ */
+ pxor %xmm2,%xmm0
+ pxor %xmm4,%xmm0
+ pxor %xmm5,%xmm0
+ pxor %xmm6,%xmm0
+
+ movdqa %xmm0,(%rdi) /* store round key */
+ lea 0x10(%rdi),%rdi /* advance to next round key address */
+ ret
+END(aesni_expand128)
+
+/*
+ * aesni_expand192a(uint128_t *rkp@rdi, uint128_t prk@xmm0,
+ * uint64_t rklo@xmm1, uint128_t keygenassist@xmm2)
+ *
+ * Set even-numbered AES-192 round key.
+ *
+ * Internal ABI. On entry:
+ *
+ * %rdi = rkp, pointer to two round keys to compute
+ * %xmm0 = (prk[0], prk[1], prk[2], prk[3])
+ * %xmm1 = (rklo[0], rklo[1], xxx, xxx)
+ * %xmm2 = (xxx, t = Rot(SubWord(rklo[1])) ^ RCON, xxx, xxx)
+ *
+ * On exit:
+ *
+ * %rdi = &rkp[2], rkp advanced by two round keys
+ * %xmm0 = nrk, second round key we just computed
+ * %xmm1 = rk, first round key we just computed
+ * %xmm2 = garbage
+ * %xmm4 = garbage
+ * %xmm5 = garbage
+ * %xmm6 = garbage
+ * %xmm7 = garbage
+ */
+ .text
+ _ALIGN_TEXT
+ .type aesni_expand192a,@function
+aesni_expand192a:
+ /*
+ * %xmm2 := (%xmm2[1], %xmm2[1], %xmm2[1], %xmm2[1]),
+ * i.e., set each word of %xmm2 to t := Rot(SubWord(rklo[1])) ^ RCON.
+ */
+ pshufd $0b01010101,%xmm2,%xmm2
+
+ /*
+ * We need to compute:
+ *
+ * rk[0] := rklo[0]
+ * rk[1] := rklo[1]
+ * rk[2] := Rot(Sub(rklo[1])) ^ RCON ^ prk[0]
+ * rk[3] := Rot(Sub(rklo[1])) ^ RCON ^ prk[0] ^ prk[1]
+ * nrk[0] := Rot(Sub(rklo[1])) ^ RCON ^ prk[0] ^ prk[1] ^ prk[2]
+ * nrk[1] := Rot(Sub(rklo[1])) ^ RCON ^ prk[0] ^ ... ^ prk[3]
+ * nrk[2] := Rot(Sub(rklo[1])) ^ RCON ^ prk[0] ^ ... ^ prk[3] ^ rklo[0]
+ * nrk[3] := Rot(Sub(rklo[1])) ^ RCON ^ prk[0] ^ ... ^ prk[3] ^ rklo[0]
+ * ^ rklo[1]
+ */
+
+ /*
+ * %xmm4 := (prk[0], prk[1], prk[2], prk[3])
+ * %xmm5 := (0, prk[0], prk[1], prk[2])
+ * %xmm6 := (0, 0, prk[0], prk[1])
+ * %xmm7 := (0, 0, 0, prk[0])
+ */
+ movdqa %xmm0,%xmm4
+ movdqa %xmm0,%xmm5
+ movdqa %xmm0,%xmm6
+ movdqa %xmm0,%xmm7
+ pslldq $4,%xmm5
+ pslldq $8,%xmm6
+ pslldq $12,%xmm7
+
+ /* %xmm4 := (rk[2], rk[3], nrk[0], nrk[1]) */
+ pxor %xmm2,%xmm4
+ pxor %xmm5,%xmm4
+ pxor %xmm6,%xmm4
+ pxor %xmm7,%xmm4
+
+ /*
+ * At this point, rk is split across %xmm1 (rk[0],rk[1],...) and
+ * %xmm4 (rk[2],rk[3],...); nrk is in %xmm4 (...,nrk[0],nrk[1]);
+ * and we have yet to compute nrk[2] or nrk[3], which requires
+ * rklo[0] and rklo[1] in %xmm1 (rklo[0], rklo[1], ...). We need
+ * nrk to end up in %xmm0 at the end, so gather rk into %xmm1 and
+ * nrk into %xmm0.
+ */
+
+ /* %xmm0 := (nrk[0], nrk[1], nrk[1], nrk[1]) */
+ pshufd $0b11111110,%xmm4,%xmm0
+
+ /*
+ * %xmm6 := (0, 0, rklo[0], rklo[1])
+ * %xmm7 := (0, 0, 0, rklo[0])
+ */
+ movdqa %xmm1,%xmm6
+ movdqa %xmm1,%xmm7
+
+ pslldq $8,%xmm6
+ pslldq $12,%xmm7
+
+ /*
+ * %xmm0 := (nrk[0],
+ * nrk[1],
+ * nrk[2] = nrk[1] ^ rklo[0],
+ * nrk[3] = nrk[1] ^ rklo[0] ^ rklo[1])
+ */
+ pxor %xmm6,%xmm0
+ pxor %xmm7,%xmm0
+
+ /* %xmm1 := (rk[0], rk[1], rk[2], rk[3]) */
+ shufps $0b01000100,%xmm4,%xmm1
+
+ movdqa %xmm1,(%rdi) /* store round key */
+ movdqa %xmm0,0x10(%rdi) /* store next round key */
+ lea 0x20(%rdi),%rdi /* advance two round keys */
+ ret
+END(aesni_expand192a)
+
+/*
+ * aesni_expand192b(uint128_t *roundkey@rdi, uint128_t prk@xmm0,
+ * uint128_t keygenassist@xmm2)
+ *
+ * Set odd-numbered AES-192 round key.
+ *
+ * Internal ABI. On entry:
+ *
+ * %rdi = rkp, pointer to round key to compute
+ * %xmm0 = (prk[0], prk[1], prk[2], prk[3])
+ * %xmm1 = (xxx, xxx, pprk[2], pprk[3])
+ * %xmm2 = (xxx, xxx, xxx, t = Rot(Sub(prk[3])) ^ RCON)
+ *
+ * On exit:
+ *
+ * %rdi = &rkp[1], rkp advanced by one round key
+ * %xmm0 = rk, the round key we just computed
+ * %xmm1 = (nrk[0], nrk[1], xxx, xxx), half of next round key
+ * %xmm2 = garbage
+ * %xmm4 = garbage
+ * %xmm5 = garbage
+ * %xmm6 = garbage
+ * %xmm7 = garbage
+ */
+ .text
+ _ALIGN_TEXT
+ .type aesni_expand192b,@function
+aesni_expand192b:
+ /*
+ * %xmm2 := (%xmm2[3], %xmm2[3], %xmm2[3], %xmm2[3]),
+ * i.e., set each word of %xmm2 to t := Rot(Sub(prk[3])) ^ RCON.
+ */
+ pshufd $0b11111111,%xmm2,%xmm2
+
+ /*
+ * We need to compute:
+ *
+ * rk[0] := Rot(Sub(prk[3])) ^ RCON ^ pprk[2]
+ * rk[1] := Rot(Sub(prk[3])) ^ RCON ^ pprk[2] ^ pprk[3]
+ * rk[2] := Rot(Sub(prk[3])) ^ RCON ^ pprk[2] ^ pprk[3] ^ prk[0]
+ * rk[3] := Rot(Sub(prk[3])) ^ RCON ^ pprk[2] ^ pprk[3] ^ prk[0]
+ * ^ prk[1]
+ * nrk[0] := Rot(Sub(prk[3])) ^ RCON ^ pprk[2] ^ pprk[3] ^ prk[0]
+ * ^ prk[1] ^ prk[2]
+ * nrk[1] := Rot(Sub(prk[3])) ^ RCON ^ pprk[2] ^ pprk[3] ^ prk[0]
+ * ^ prk[1] ^ prk[2] ^ prk[3]
+ */
+
+ /* %xmm1 := (pprk[2], pprk[3], prk[0], prk[1]) */
+ shufps $0b01001110,%xmm0,%xmm1
+
+ /*
+ * %xmm5 := (0, pprk[2], pprk[3], prk[0])
+ * %xmm6 := (0, 0, pprk[2], pprk[3])
+ * %xmm7 := (0, 0, 0, pprk[2])
+ */
+ movdqa %xmm1,%xmm5
+ movdqa %xmm1,%xmm6
+ movdqa %xmm1,%xmm7
+ pslldq $4,%xmm5
+ pslldq $8,%xmm6
+ pslldq $12,%xmm7
+
+ /* %xmm1 := (rk[0], rk[1], rk[2], rk[3) */
+ pxor %xmm2,%xmm1
+ pxor %xmm5,%xmm1
+ pxor %xmm6,%xmm1
+ pxor %xmm7,%xmm1
+
+ /* %xmm4 := (prk[2], prk[3], xxx, xxx) */
+ pshufd $0b00001110,%xmm0,%xmm4
+
+ /* %xmm5 := (0, prk[2], xxx, xxx) */
+ movdqa %xmm4,%xmm5
+ pslldq $4,%xmm5
+
+ /* %xmm0 := (rk[0], rk[1], rk[2], rk[3]) */
+ movdqa %xmm1,%xmm0
+
+ /* %xmm1 := (rk[3], rk[3], xxx, xxx) */
+ shufps $0b00001111,%xmm1,%xmm1
+
+ /*
+ * %xmm1 := (nrk[0] = rk[3] ^ prk[2],
+ * nrk[1] = rk[3] ^ prk[2] ^ prk[3],
+ * xxx,
+ * xxx)
+ */
+ pxor %xmm4,%xmm1
+ pxor %xmm5,%xmm1
+
+ movdqa %xmm0,(%rdi) /* store round key */
+ lea 0x10(%rdi),%rdi /* advance to next round key address */
+ ret
+END(aesni_expand192b)
+
+/*
+ * aesni_expand256a(uint128_t *rkp@rdi, uint128_t pprk@xmm0,
+ * uint128_t prk@xmm1, uint128_t keygenassist@xmm2)
+ *
+ * Set even-numbered AES-256 round key.
+ *
+ * Internal ABI. On entry:
+ *
+ * %rdi = rkp, pointer to round key to compute
+ * %xmm0 = (pprk[0], pprk[1], pprk[2], pprk[3])
+ * %xmm1 = (prk[0], prk[1], prk[2], prk[3])
+ * %xmm2 = (xxx, xxx, xxx, t = Rot(SubWord(prk[3])))
+ *
+ * On exit:
+ *
+ * %rdi = &rkp[1], rkp advanced by one round key
+ * %xmm0 = rk, the round key we just computed
+ * %xmm1 = prk, previous round key, preserved from entry
+ * %xmm2 = garbage
+ * %xmm4 = garbage
+ * %xmm5 = garbage
+ * %xmm6 = garbage
+ *
+ * The computation turns out to be the same as for AES-128; the
+ * previous round key does not figure into it, only the
+ * previous-previous round key.
+ */
+ aesni_expand256a = aesni_expand128
+
+/*
+ * aesni_expand256b(uint128_t *rkp@rdi, uint128_t prk@xmm0,
+ * uint128_t pprk@xmm1, uint128_t keygenassist@xmm2)
+ *
+ * Set odd-numbered AES-256 round key.
+ *
+ * Internal ABI. On entry:
+ *
+ * %rdi = rkp, pointer to round key to compute
+ * %xmm0 = (prk[0], prk[1], prk[2], prk[3])
+ * %xmm1 = (pprk[0], pprk[1], pprk[2], pprk[3])
+ * %xmm2 = (xxx, xxx, t = Sub(prk[3]), xxx)
+ *
+ * On exit:
+ *
+ * %rdi = &rkp[1], rkp advanced by one round key
+ * %xmm0 = prk, previous round key, preserved from entry
+ * %xmm1 = rk, the round key we just computed
+ * %xmm2 = garbage
+ * %xmm4 = garbage
+ * %xmm5 = garbage
+ * %xmm6 = garbage
+ */
+ .text
+ _ALIGN_TEXT
+ .type aesni_expand256b,@function
+aesni_expand256b:
+ /*
+ * %xmm2 := (%xmm2[3], %xmm2[3], %xmm2[3], %xmm2[3]),
+ * i.e., set each word of %xmm2 to t := Sub(prk[3]).
+ */
+ pshufd $0b10101010,%xmm2,%xmm2
+
+ /*
+ * %xmm4 := (0, pprk[0], pprk[1], pprk[2])
+ * %xmm5 := (0, 0, pprk[0], pprk[1])
+ * %xmm6 := (0, 0, 0, pprk[0])
+ */
+ movdqa %xmm1,%xmm4
+ movdqa %xmm1,%xmm5
+ movdqa %xmm1,%xmm6
+ pslldq $4,%xmm4
+ pslldq $8,%xmm5
+ pslldq $12,%xmm6
+
+ /*
+ * %xmm0 := (rk[0] = t ^ pprk[0],
+ * rk[1] = t ^ pprk[0] ^ pprk[1],
+ * rk[2] = t ^ pprk[0] ^ pprk[1] ^ pprk[2],
+ * rk[3] = t ^ pprk[0] ^ pprk[1] ^ pprk[2] ^ pprk[3])
+ */
+ pxor %xmm2,%xmm1
+ pxor %xmm4,%xmm1
+ pxor %xmm5,%xmm1
+ pxor %xmm6,%xmm1
+
+ movdqa %xmm1,(%rdi) /* store round key */
+ lea 0x10(%rdi),%rdi /* advance to next round key address */
+ ret
+END(aesni_expand256b)
+
+/*
+ * aesni_enctodec(const struct aesenc *enckey@rdi, struct aesdec *deckey@rsi,
+ * uint32_t nrounds@rdx)
+ *
+ * Convert AES encryption round keys to AES decryption round keys.
+ * `rounds' must be between 10 and 14.
+ *
+ * Standard ABI calling convention.
+ */
+ENTRY(aesni_enctodec)
+ shl $4,%edx /* rdx := byte offset of last round key */
+ movdqa (%rdi,%rdx),%xmm0 /* load last round key */
+ movdqa %xmm0,(%rsi) /* store last round key verbatim */
+1: sub $0x10,%rdx /* advance to next round key */
+ lea 0x10(%rsi),%rsi
+ jz 2f /* stop if this is the last one */
+ movdqa (%rdi,%rdx),%xmm0 /* load round key */
+ aesimc %xmm0,%xmm0 /* convert encryption to decryption */
+ movdqa %xmm0,(%rsi) /* store round key */
+ jmp 1b
+2: movdqa (%rdi),%xmm0 /* load first round key */
+ movdqa %xmm0,(%rsi) /* store first round key verbatim */
+ ret
+END(aesni_enctodec)
+
+/*
+ * aesni_enc(const struct aesenc *enckey@rdi, const uint8_t in[16] @rsi,
+ * uint8_t out[16] @rdx, uint32_t nrounds@ecx)
+ *
+ * Encrypt a single block.
+ *
+ * Standard ABI calling convention.
+ */
+ENTRY(aesni_enc)
+ movdqu (%rsi),%xmm0
+ call aesni_enc1
+ movdqu %xmm0,(%rdx)
+ ret
+END(aesni_enc)
+
+/*
+ * aesni_dec(const struct aesdec *deckey@rdi, const uint8_t in[16] @rsi,
+ * uint8_t out[16] @rdx, uint32_t nrounds@ecx)
+ *
+ * Decrypt a single block.
+ *
+ * Standard ABI calling convention.
+ */
+ENTRY(aesni_dec)
+ movdqu (%rsi),%xmm0
+ call aesni_dec1
+ movdqu %xmm0,(%rdx)
+ ret
+END(aesni_dec)
+
+/*
+ * aesni_cbc_enc(const struct aesenc *enckey@rdi, const uint8_t *in@rsi,
+ * uint8_t *out@rdx, size_t nbytes@rcx, uint8_t iv[16] @r8,
+ * uint32_t nrounds@r9d)
+ *
+ * Encrypt a contiguous sequence of blocks with AES-CBC.
+ *
+ * nbytes must be an integral multiple of 16.
+ *
+ * Standard ABI calling convention.
+ */
+ENTRY(aesni_cbc_enc)
+ cmp $0,%rcx
+ jz 2f
+ mov %rcx,%r10 /* r10 := nbytes */
+ movdqu (%r8),%xmm0 /* xmm0 := chaining value */
+1: movdqu (%rsi),%xmm1 /* xmm1 := plaintext block */
+ lea 0x10(%rsi),%rsi
+ pxor %xmm1,%xmm0 /* xmm0 := cv ^ ptxt */
+ mov %r9d,%ecx /* ecx := nrounds */
+ call aesni_enc1 /* xmm0 := ciphertext block */
+ movdqu %xmm0,(%rdx)
+ lea 0x10(%rdx),%rdx
+ sub $0x10,%r10
+ jnz 1b /* repeat if r10 is nonzero */
+ movdqu %xmm0,(%r8) /* store chaining value */
+2: ret
+END(aesni_cbc_enc)
+
+/*
+ * aesni_cbc_dec1(const struct aesdec *deckey@rdi, const uint8_t *in@rsi,
+ * uint8_t *out@rdx, size_t nbytes@rcx, const uint8_t iv[16] @r8,
+ * uint32_t nrounds@r9)
+ *
+ * Decrypt a contiguous sequence of blocks with AES-CBC.
+ *
+ * nbytes must be a positive integral multiple of 16. This routine
+ * is not vectorized; use aesni_cbc_dec8 for >=8 blocks at once.
+ *
+ * Standard ABI calling convention.
+ */
+ENTRY(aesni_cbc_dec1)
+ push %rbp /* create stack frame uint128[1] */
+ mov %rsp,%rbp
+ sub $0x10,%rsp
+ movdqu (%r8),%xmm8 /* xmm8 := iv */
+ movdqa %xmm8,(%rsp) /* save iv */
+ mov %rcx,%r10 /* r10 := nbytes */
+ movdqu -0x10(%rsi,%r10),%xmm0 /* xmm0 := last ciphertext block */
+ movdqu %xmm0,(%r8) /* update iv */
+1: mov %r9d,%ecx /* ecx := nrounds */
+ call aesni_dec1 /* xmm0 := cv ^ ptxt */
+ sub $0x10,%r10
+ jz 2f /* first block if r10 is now zero */
+ movdqu -0x10(%rsi,%r10),%xmm8 /* xmm8 := chaining value */
+ pxor %xmm8,%xmm0 /* xmm0 := ptxt */
+ movdqu %xmm0,(%rdx,%r10) /* store plaintext block */
+ movdqa %xmm8,%xmm0 /* move cv = ciphertext block */
+ jmp 1b
+2: pxor (%rsp),%xmm0 /* xmm0 := ptxt */
+ movdqu %xmm0,(%rdx) /* store first plaintext block */
+ leave
+ ret
+END(aesni_cbc_dec1)
+
+/*
+ * aesni_cbc_dec8(const struct aesdec *deckey@rdi, const uint8_t *in@rsi,
+ * uint8_t *out@rdx, size_t nbytes@rcx, const uint8_t iv[16] @r8,
+ * uint32_t nrounds@r9)
+ *
+ * Decrypt a contiguous sequence of 8-block units with AES-CBC.
+ *
+ * nbytes must be a positive integral multiple of 128.
+ *
+ * Standard ABI calling convention.
+ */
+ENTRY(aesni_cbc_dec8)
+ push %rbp /* create stack frame uint128[1] */
+ mov %rsp,%rbp
+ sub $0x10,%rsp
+ movdqu (%r8),%xmm8 /* xmm8 := iv */
+ movdqa %xmm8,(%rsp) /* save iv */
+ mov %rcx,%r10 /* r10 := nbytes */
+ movdqu -0x10(%rsi,%r10),%xmm7 /* xmm7 := ciphertext block[n-1] */
+ movdqu %xmm7,(%r8) /* update iv */
+1: movdqu -0x20(%rsi,%r10),%xmm6 /* xmm6 := ciphertext block[n-2] */
+ movdqu -0x30(%rsi,%r10),%xmm5 /* xmm5 := ciphertext block[n-3] */
+ movdqu -0x40(%rsi,%r10),%xmm4 /* xmm4 := ciphertext block[n-4] */
+ movdqu -0x50(%rsi,%r10),%xmm3 /* xmm3 := ciphertext block[n-5] */
+ movdqu -0x60(%rsi,%r10),%xmm2 /* xmm2 := ciphertext block[n-6] */
+ movdqu -0x70(%rsi,%r10),%xmm1 /* xmm1 := ciphertext block[n-7] */
+ movdqu -0x80(%rsi,%r10),%xmm0 /* xmm0 := ciphertext block[n-8] */
+ movdqa %xmm6,%xmm15 /* xmm[8+i] := cv[i], 0<i<8 */
+ movdqa %xmm5,%xmm14
+ movdqa %xmm4,%xmm13
+ movdqa %xmm3,%xmm12
+ movdqa %xmm2,%xmm11
+ movdqa %xmm1,%xmm10
+ movdqa %xmm0,%xmm9
+ mov %r9d,%ecx /* ecx := nrounds */
+ call aesni_dec8 /* xmm[i] := cv[i] ^ ptxt[i], 0<=i<8 */
+ pxor %xmm15,%xmm7 /* xmm[i] := ptxt[i], 0<i<8 */
+ pxor %xmm14,%xmm6
+ pxor %xmm13,%xmm5
+ pxor %xmm12,%xmm4
+ pxor %xmm11,%xmm3
+ pxor %xmm10,%xmm2
+ pxor %xmm9,%xmm1
+ movdqu %xmm7,-0x10(%rdx,%r10) /* store plaintext blocks */
+ movdqu %xmm6,-0x20(%rdx,%r10)
+ movdqu %xmm5,-0x30(%rdx,%r10)
+ movdqu %xmm4,-0x40(%rdx,%r10)
+ movdqu %xmm3,-0x50(%rdx,%r10)
+ movdqu %xmm2,-0x60(%rdx,%r10)
+ movdqu %xmm1,-0x70(%rdx,%r10)
+ sub $0x80,%r10
+ jz 2f /* first block if r10 is now zero */
+ movdqu -0x10(%rsi,%r10),%xmm7 /* xmm7 := cv[0] */
+ pxor %xmm7,%xmm0 /* xmm0 := ptxt[0] */
+ movdqu %xmm0,(%rdx,%r10) /* store plaintext block */
+ jmp 1b
+2: pxor (%rsp),%xmm0 /* xmm0 := ptxt[0] */
+ movdqu %xmm0,(%rdx) /* store first plaintext block */
+ leave
+ ret
+END(aesni_cbc_dec8)
+
+/*
+ * aesni_xts_enc1(const struct aesenc *enckey@rdi, const uint8_t *in@rsi,
+ * uint8_t *out@rdx, size_t nbytes@rcx, uint8_t tweak[16] @r8,
+ * uint32_t nrounds@r9d)
+ *
+ * Encrypt a contiguous sequence of blocks with AES-XTS.
+ *
+ * nbytes must be a positive integral multiple of 16. This routine
+ * is not vectorized; use aesni_xts_enc8 for >=8 blocks at once.
+ *
+ * Standard ABI calling convention.
+ */
+ENTRY(aesni_xts_enc1)
+ mov %rcx,%r10 /* r10 := nbytes */
+ movdqu (%r8),%xmm9 /* xmm9 := tweak */
+1: movdqu (%rsi),%xmm0 /* xmm0 := ptxt */
+ lea 0x10(%rsi),%rsi /* advance rdi to next block */
+ pxor %xmm9,%xmm0 /* xmm0 := ptxt ^ tweak */
+ mov %r9d,%ecx /* ecx := nrounds */
+ call aesni_enc1 /* xmm0 := AES(ptxt ^ tweak) */
+ pxor %xmm9,%xmm0 /* xmm0 := AES(ptxt ^ tweak) ^ tweak */
+ movdqu %xmm0,(%rdx) /* store ciphertext block */
+ lea 0x10(%rdx),%rdx /* advance rsi to next block */
+ call aesni_xts_mulx /* xmm9 *= x; trash xmm0 */
+ sub $0x10,%r10
+ jnz 1b /* repeat if more blocks */
+ movdqu %xmm9,(%r8) /* update tweak */
+ ret
+END(aesni_xts_enc1)
+
+/*
+ * aesni_xts_enc8(const struct aesenc *enckey@rdi, const uint8_t *in@rsi,
+ * uint8_t *out@rdx, size_t nbytes@rcx, uint8_t tweak[16] @r8,
+ * uint32_t nrounds@r9d)
+ *
+ * Encrypt a contiguous sequence of blocks with AES-XTS.
+ *
+ * nbytes must be a positive integral multiple of 128.
+ *
+ * Standard ABI calling convention.
+ */
+ENTRY(aesni_xts_enc8)
+ push %rbp /* create stack frame uint128[2] */
+ mov %rsp,%rbp
+ sub $0x20,%rsp
+ mov %rcx,%r10 /* r10 := nbytes */
+ movdqu (%r8),%xmm9 /* xmm9 := tweak[0] */
+1: movdqa %xmm9,(%rsp) /* save tweak[0] */
+ call aesni_xts_mulx /* xmm9 := tweak[1] */
+ movdqa %xmm9,0x10(%rsp) /* save tweak[1] */
+ call aesni_xts_mulx /* xmm9 := tweak[2] */
+ movdqa %xmm9,%xmm10 /* xmm10 := tweak[2] */
+ call aesni_xts_mulx /* xmm9 := tweak[3] */
+ movdqa %xmm9,%xmm11 /* xmm11 := tweak[3] */
+ call aesni_xts_mulx /* xmm9 := tweak[4] */
+ movdqa %xmm9,%xmm12 /* xmm12 := tweak[4] */
+ call aesni_xts_mulx /* xmm9 := tweak[5] */
+ movdqa %xmm9,%xmm13 /* xmm13 := tweak[5] */
+ call aesni_xts_mulx /* xmm9 := tweak[6] */
+ movdqa %xmm9,%xmm14 /* xmm14 := tweak[6] */
+ call aesni_xts_mulx /* xmm9 := tweak[7] */
+ movdqa %xmm9,%xmm15 /* xmm15 := tweak[7] */
+ movdqu (%rsi),%xmm0 /* xmm[i] := ptxt[i] */
+ movdqu 0x10(%rsi),%xmm1
+ movdqu 0x20(%rsi),%xmm2
+ movdqu 0x30(%rsi),%xmm3
+ movdqu 0x40(%rsi),%xmm4
+ movdqu 0x50(%rsi),%xmm5
+ movdqu 0x60(%rsi),%xmm6
+ movdqu 0x70(%rsi),%xmm7
+ lea 0x80(%rsi),%rsi /* advance rsi to next block group */
+ pxor (%rsp),%xmm0 /* xmm[i] := ptxt[i] ^ tweak[i] */
+ pxor 0x10(%rsp),%xmm1
+ pxor %xmm10,%xmm2
+ pxor %xmm11,%xmm3
+ pxor %xmm12,%xmm4
+ pxor %xmm13,%xmm5
+ pxor %xmm14,%xmm6
+ pxor %xmm15,%xmm7
+ mov %r9d,%ecx /* ecx := nrounds */
+ call aesni_enc8 /* xmm[i] := AES(ptxt[i] ^ tweak[i]) */
+ pxor (%rsp),%xmm0 /* xmm[i] := AES(...) ^ tweak[i] */
+ pxor 0x10(%rsp),%xmm1
+ pxor %xmm10,%xmm2
+ pxor %xmm11,%xmm3
+ pxor %xmm12,%xmm4
+ pxor %xmm13,%xmm5
+ pxor %xmm14,%xmm6
+ pxor %xmm15,%xmm7
+ movdqu %xmm0,(%rdx) /* store ciphertext blocks */
+ movdqu %xmm1,0x10(%rdx)
+ movdqu %xmm2,0x20(%rdx)
+ movdqu %xmm3,0x30(%rdx)
+ movdqu %xmm4,0x40(%rdx)
+ movdqu %xmm5,0x50(%rdx)
+ movdqu %xmm6,0x60(%rdx)
+ movdqu %xmm7,0x70(%rdx)
+ lea 0x80(%rdx),%rdx /* advance rdx to next block group */
+ movdqa %xmm15,%xmm9 /* xmm9 := tweak[7] */
+ call aesni_xts_mulx /* xmm9 := tweak[8] */
+ sub $0x80,%r10
+ jnz 1b /* repeat if more block groups */
+ movdqu %xmm9,(%r8) /* update tweak */
+ leave
+ ret
+END(aesni_xts_enc8)
+
+/*
+ * aesni_xts_dec1(const struct aesdec *deckey@rdi, const uint8_t *in@rsi,
+ * uint8_t *out@rdx, size_t nbytes@rcx, uint8_t tweak[16] @r8,
+ * uint32_t nrounds@r9d)
+ *
+ * Decrypt a contiguous sequence of blocks with AES-XTS.
+ *
+ * nbytes must be a positive integral multiple of 16. This routine
+ * is not vectorized; use aesni_xts_dec8 for >=8 blocks at once.
+ *
+ * Standard ABI calling convention.
+ */
+ENTRY(aesni_xts_dec1)
+ mov %rcx,%r10 /* r10 := nbytes */
+ movdqu (%r8),%xmm9 /* xmm9 := tweak */
+1: movdqu (%rsi),%xmm0 /* xmm0 := ctxt */
+ lea 0x10(%rsi),%rsi /* advance rdi to next block */
+ pxor %xmm9,%xmm0 /* xmm0 := ctxt ^ tweak */
+ mov %r9d,%ecx /* ecx := nrounds */
+ call aesni_dec1 /* xmm0 := AES(ctxt ^ tweak) */
+ pxor %xmm9,%xmm0 /* xmm0 := AES(ctxt ^ tweak) ^ tweak */
+ movdqu %xmm0,(%rdx) /* store plaintext block */
+ lea 0x10(%rdx),%rdx /* advance rsi to next block */
+ call aesni_xts_mulx /* xmm9 *= x; trash xmm0 */
+ sub $0x10,%r10
+ jnz 1b /* repeat if more blocks */
+ movdqu %xmm9,(%r8) /* update tweak */
+ ret
+END(aesni_xts_dec1)
+
+/*
+ * aesni_xts_dec8(const struct aesdec *deckey@rdi, const uint8_t *in@rsi,
+ * uint8_t *out@rdx, size_t nbytes@rcx, uint8_t tweak[16] @r8,
+ * uint32_t nrounds@r9d)
+ *
+ * Decrypt a contiguous sequence of blocks with AES-XTS.
+ *
+ * nbytes must be a positive integral multiple of 128.
+ *
+ * Standard ABI calling convention.
+ */
+ENTRY(aesni_xts_dec8)
+ push %rbp /* create stack frame uint128[2] */
+ mov %rsp,%rbp
+ sub $0x20,%rsp
+ mov %rcx,%r10 /* r10 := nbytes */
+ movdqu (%r8),%xmm9 /* xmm9 := tweak[0] */
+1: movdqa %xmm9,(%rsp) /* save tweak[0] */
+ call aesni_xts_mulx /* xmm9 := tweak[1] */
+ movdqa %xmm9,0x10(%rsp) /* save tweak[1] */
+ call aesni_xts_mulx /* xmm9 := tweak[2] */
+ movdqa %xmm9,%xmm10 /* xmm10 := tweak[2] */
+ call aesni_xts_mulx /* xmm9 := tweak[3] */
+ movdqa %xmm9,%xmm11 /* xmm11 := tweak[3] */
+ call aesni_xts_mulx /* xmm9 := tweak[4] */
+ movdqa %xmm9,%xmm12 /* xmm12 := tweak[4] */
+ call aesni_xts_mulx /* xmm9 := tweak[5] */
+ movdqa %xmm9,%xmm13 /* xmm13 := tweak[5] */
+ call aesni_xts_mulx /* xmm9 := tweak[6] */
+ movdqa %xmm9,%xmm14 /* xmm14 := tweak[6] */
+ call aesni_xts_mulx /* xmm9 := tweak[7] */
+ movdqa %xmm9,%xmm15 /* xmm15 := tweak[7] */
+ movdqu (%rsi),%xmm0 /* xmm[i] := ptxt[i] */
+ movdqu 0x10(%rsi),%xmm1
+ movdqu 0x20(%rsi),%xmm2
+ movdqu 0x30(%rsi),%xmm3
+ movdqu 0x40(%rsi),%xmm4
+ movdqu 0x50(%rsi),%xmm5
+ movdqu 0x60(%rsi),%xmm6
+ movdqu 0x70(%rsi),%xmm7
+ lea 0x80(%rsi),%rsi /* advance rsi to next block group */
+ pxor (%rsp),%xmm0 /* xmm[i] := ptxt[i] ^ tweak[i] */
+ pxor 0x10(%rsp),%xmm1
+ pxor %xmm10,%xmm2
+ pxor %xmm11,%xmm3
+ pxor %xmm12,%xmm4
+ pxor %xmm13,%xmm5
+ pxor %xmm14,%xmm6
+ pxor %xmm15,%xmm7
+ mov %r9d,%ecx /* ecx := nrounds */
+ call aesni_dec8 /* xmm[i] := AES(ptxt[i] ^ tweak[i]) */
+ pxor (%rsp),%xmm0 /* xmm[i] := AES(...) ^ tweak[i] */
+ pxor 0x10(%rsp),%xmm1
+ pxor %xmm10,%xmm2
+ pxor %xmm11,%xmm3
+ pxor %xmm12,%xmm4
+ pxor %xmm13,%xmm5
+ pxor %xmm14,%xmm6
+ pxor %xmm15,%xmm7
+ movdqu %xmm0,(%rdx) /* store ciphertext blocks */
+ movdqu %xmm1,0x10(%rdx)
+ movdqu %xmm2,0x20(%rdx)
+ movdqu %xmm3,0x30(%rdx)
+ movdqu %xmm4,0x40(%rdx)
+ movdqu %xmm5,0x50(%rdx)
+ movdqu %xmm6,0x60(%rdx)
+ movdqu %xmm7,0x70(%rdx)
+ lea 0x80(%rdx),%rdx /* advance rdx to next block group */
+ movdqa %xmm15,%xmm9 /* xmm9 := tweak[7] */
+ call aesni_xts_mulx /* xmm9 := tweak[8] */
+ sub $0x80,%r10
+ jnz 1b /* repeat if more block groups */
+ movdqu %xmm9,(%r8) /* update tweak */
+ leave
+ ret
+END(aesni_xts_dec8)
+
+/*
+ * aesni_xts_mulx(tweak@xmm9)
+ *
+ * Multiply xmm9 by x, modulo x^128 + x^7 + x^2 + x + 1, in place.
+ * Uses %xmm0 as temporary.
+ */
+ .text
+ _ALIGN_TEXT
+ .type aesni_xts_mulx,@function
+aesni_xts_mulx:
+ /*
+ * Simultaneously determine
+ * (a) whether the high bit of the low quadword must be
+ * shifted into the low bit of the high quadword, and
+ * (b) whether the high bit of the high quadword must be
+ * carried into x^128 = x^7 + x^2 + x + 1.
+ */
+ pxor %xmm0,%xmm0 /* xmm0 := 0 */
+ pcmpgtq %xmm9,%xmm0 /* xmm0[i] := -1 if 0 > xmm9[i], 0 otherwise */
+ pshufd $0b01001110,%xmm0,%xmm0 /* swap halves of xmm0 */
+ pand xtscarry,%xmm0 /* copy xtscarry according to mask */
+ psllq $1,%xmm9 /* shift */
+ pxor %xmm0,%xmm9 /* incorporate (a) and (b) */
+ ret
+END(aesni_xts_mulx)
+
+ .section .rodata
+ .align 16
+ .type xtscarry,@object
+xtscarry:
+ .byte 0x87,0,0,0, 0,0,0,0, 1,0,0,0, 0,0,0,0
+END(xtscarry)
+
+/*
+ * aesni_xts_update(const uint8_t in[16] @rdi, uint8_t out[16] @rsi)
+ *
+ * Update an AES-XTS tweak.
+ *
+ * Standard ABI calling convention.
+ */
+ENTRY(aesni_xts_update)
+ movdqu (%rdi),%xmm9
+ call aesni_xts_mulx
+ movdqu %xmm9,(%rsi)
+ ret
+END(aesni_xts_update)
+
+/*
+ * aesni_enc1(const struct aesenc *enckey@rdi, uint128_t block@xmm0,
+ * uint32_t nrounds@ecx)
+ *
+ * Encrypt a single AES block in %xmm0.
+ *
+ * Internal ABI. Uses %rax and %xmm8 as temporaries. Destroys %ecx.
+ */
+ .text
+ _ALIGN_TEXT
+ .type aesni_enc1,@function
+aesni_enc1:
+ pxor (%rdi),%xmm0 /* xor in first round key */
+ shl $4,%ecx /* ecx := total byte size of round keys */
+ lea 0x10(%rdi,%rcx),%rax /* rax := end of round key array */
+ neg %rcx /* rcx := byte offset of round key from end */
+1: movdqa (%rax,%rcx),%xmm8 /* load round key */
+ add $0x10,%rcx
+ jz 2f /* stop if this is the last one */
+ aesenc %xmm8,%xmm0
+ jmp 1b
+2: aesenclast %xmm8,%xmm0
+ ret
+END(aesni_enc1)
+
+/*
+ * aesni_enc8(const struct aesenc *enckey@rdi, uint128_t block0@xmm0, ...,
+ * block7@xmm7, uint32_t nrounds@ecx)
+ *
+ * Encrypt eight AES blocks in %xmm0 through %xmm7 in parallel.
+ *
+ * Internal ABI. Uses %rax and %xmm8 as temporaries. Destroys %ecx.
+ */
+ .text
+ _ALIGN_TEXT
+ .type aesni_enc8,@function
+aesni_enc8:
+ movdqa (%rdi),%xmm8 /* xor in first round key */
+ pxor %xmm8,%xmm0
+ pxor %xmm8,%xmm1
+ pxor %xmm8,%xmm2
+ pxor %xmm8,%xmm3
+ pxor %xmm8,%xmm4
+ pxor %xmm8,%xmm5
+ pxor %xmm8,%xmm6
+ pxor %xmm8,%xmm7
+ shl $4,%ecx /* ecx := total byte size of round keys */
+ lea 0x10(%rdi,%rcx),%rax /* rax := end of round key array */
+ neg %rcx /* rcx := byte offset of round key from end */
+1: movdqa (%rax,%rcx),%xmm8 /* load round key */
+ add $0x10,%rcx
+ jz 2f /* stop if this is the last one */
+ aesenc %xmm8,%xmm0
+ aesenc %xmm8,%xmm1
+ aesenc %xmm8,%xmm2
+ aesenc %xmm8,%xmm3
+ aesenc %xmm8,%xmm4
+ aesenc %xmm8,%xmm5
+ aesenc %xmm8,%xmm6
+ aesenc %xmm8,%xmm7
+ jmp 1b
+2: aesenclast %xmm8,%xmm0
+ aesenclast %xmm8,%xmm1
+ aesenclast %xmm8,%xmm2
+ aesenclast %xmm8,%xmm3
+ aesenclast %xmm8,%xmm4
+ aesenclast %xmm8,%xmm5
+ aesenclast %xmm8,%xmm6
+ aesenclast %xmm8,%xmm7
+ ret
+END(aesni_enc8)
+
+/*
+ * aesni_dec1(const struct aesdec *deckey@rdi, uint128_t block@xmm0,
+ * uint32_t nrounds@ecx)
+ *
+ * Decrypt a single AES block in %xmm0.
+ *
+ * Internal ABI. Uses %rax and %xmm8 as temporaries. Destroys %ecx.
+ */
+ .text
+ _ALIGN_TEXT
+ .type aesni_dec1,@function
+aesni_dec1:
+ pxor (%rdi),%xmm0 /* xor in first round key */
+ shl $4,%ecx /* ecx := byte offset of round key */
+ lea 0x10(%rdi,%rcx),%rax /* rax := pointer to round key */
+ neg %rcx /* rcx := byte offset of round key from end */
+1: movdqa (%rax,%rcx),%xmm8 /* load round key */
+ add $0x10,%rcx
+ jz 2f /* stop if this is the last one */
+ aesdec %xmm8,%xmm0
+ jmp 1b
+2: aesdeclast %xmm8,%xmm0
+ ret
+END(aesni_dec1)
+
+/*
+ * aesni_dec8(const struct aesdec *deckey@rdi, uint128_t block0@xmm0, ...,
+ * block7@xmm7, uint32_t nrounds@ecx)
+ *
+ * Decrypt eight AES blocks in %xmm0 through %xmm7 in parallel.
+ *
+ * Internal ABI. Uses %xmm8 as temporary. Destroys %rcx.
+ */
+ .text
+ _ALIGN_TEXT
+ .type aesni_dec8,@function
+aesni_dec8:
+ movdqa (%rdi),%xmm8 /* xor in first round key */
+ pxor %xmm8,%xmm0
+ pxor %xmm8,%xmm1
+ pxor %xmm8,%xmm2
+ pxor %xmm8,%xmm3
+ pxor %xmm8,%xmm4
+ pxor %xmm8,%xmm5
+ pxor %xmm8,%xmm6
+ pxor %xmm8,%xmm7
+ shl $4,%ecx /* ecx := byte offset of round key */
+ lea 0x10(%rdi,%rcx),%rax /* rax := pointer to round key */
+ neg %rcx /* rcx := byte offset of round key from end */
+1: movdqa (%rax,%rcx),%xmm8 /* load round key */
+ add $0x10,%rcx
+ jz 2f /* stop if this is the last one */
+ aesdec %xmm8,%xmm0
+ aesdec %xmm8,%xmm1
+ aesdec %xmm8,%xmm2
+ aesdec %xmm8,%xmm3
+ aesdec %xmm8,%xmm4
+ aesdec %xmm8,%xmm5
+ aesdec %xmm8,%xmm6
+ aesdec %xmm8,%xmm7
+ jmp 1b
+2: aesdeclast %xmm8,%xmm0
+ aesdeclast %xmm8,%xmm1
+ aesdeclast %xmm8,%xmm2
+ aesdeclast %xmm8,%xmm3
+ aesdeclast %xmm8,%xmm4
+ aesdeclast %xmm8,%xmm5
+ aesdeclast %xmm8,%xmm6
+ aesdeclast %xmm8,%xmm7
+ ret
+END(aesni_dec8)
diff -r 9d6b84c40f65 -r fea7aeacc09c sys/crypto/aes/arch/x86/files.aesni
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/sys/crypto/aes/arch/x86/files.aesni Wed Jun 17 23:05:29 2020 +0000
@@ -0,0 +1,6 @@
+# $NetBSD$
+
+ifdef amd64 # amd64-only for now; i386 left as exercise for reader
+file crypto/aes/arch/x86/aes_ni.c aes
+file crypto/aes/arch/x86/aesnifunc.S aes
+endif
# HG changeset patch
# User Taylor R Campbell <riastradh%NetBSD.org@localhost>
# Date 1592066612 0
# Sat Jun 13 16:43:32 2020 +0000
# Branch trunk
# Node ID 87d9e1c86afcd441a167bf5f6d485e98d8094594
# Parent fea7aeacc09cf9da68d32a15edf9550ce78a4d45
# EXP-Topic riastradh-kernelcrypto
Implement AES in kernel using ARMv8.0-AES on aarch64.
diff -r fea7aeacc09c -r 87d9e1c86afc sys/arch/aarch64/aarch64/cpu.c
--- a/sys/arch/aarch64/aarch64/cpu.c Wed Jun 17 23:05:29 2020 +0000
+++ b/sys/arch/aarch64/aarch64/cpu.c Sat Jun 13 16:43:32 2020 +0000
@@ -44,6 +44,8 @@
#include <sys/sysctl.h>
#include <sys/systm.h>
+#include <crypto/aes/arch/aarch64/aes_arm.h>
+
#include <aarch64/armreg.h>
#include <aarch64/cpu.h>
#include <aarch64/cpufunc.h>
@@ -70,6 +72,7 @@ static void cpu_init_counter(struct cpu_
static void cpu_setup_id(struct cpu_info *);
static void cpu_setup_sysctl(device_t, struct cpu_info *);
static void cpu_setup_rng(device_t, struct cpu_info *);
+static void cpu_setup_aes(device_t, struct cpu_info *);
#ifdef MULTIPROCESSOR
#define NCPUINFO MAXCPUS
@@ -158,6 +161,7 @@ cpu_attach(device_t dv, cpuid_t id)
cpu_setup_sysctl(dv, ci);
cpu_setup_rng(dv, ci);
+ cpu_setup_aes(dv, ci);
}
struct cpuidtab {
@@ -589,6 +593,26 @@ cpu_setup_rng(device_t dv, struct cpu_in
RND_FLAG_DEFAULT|RND_FLAG_HASCB);
}
+/*
+ * setup the AES implementation
+ */
+static void
+cpu_setup_aes(device_t dv, struct cpu_info *ci)
+{
+ struct aarch64_sysctl_cpu_id *id = &ci->ci_id;
+
+ /* Verify that it is supported. */
+ switch (__SHIFTOUT(id->ac_aa64isar0, ID_AA64ISAR0_EL1_AES)) {
+ case ID_AA64ISAR0_EL1_AES_AES:
+ case ID_AA64ISAR0_EL1_AES_PMUL:
+ break;
+ default:
+ return;
+ }
+
+ aes_md_init(&aes_arm_impl);
+}
+
#ifdef MULTIPROCESSOR
void
cpu_hatch(struct cpu_info *ci)
diff -r fea7aeacc09c -r 87d9e1c86afc sys/arch/aarch64/conf/files.aarch64
--- a/sys/arch/aarch64/conf/files.aarch64 Wed Jun 17 23:05:29 2020 +0000
+++ b/sys/arch/aarch64/conf/files.aarch64 Sat Jun 13 16:43:32 2020 +0000
@@ -138,3 +138,6 @@ file arch/aarch64/aarch64/netbsd32_sysca
# profiling support
file dev/tprof/tprof_armv8.c tprof needs-flag
+
+# AES
+include "crypto/aes/arch/aarch64/files.aesarm"
diff -r fea7aeacc09c -r 87d9e1c86afc sys/crypto/aes/arch/aarch64/aes_arm.c
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/sys/crypto/aes/arch/aarch64/aes_arm.c Sat Jun 13 16:43:32 2020 +0000
@@ -0,0 +1,257 @@
+/* $NetBSD$ */
+
+/*-
+ * Copyright (c) 2020 The NetBSD Foundation, Inc.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND CONTRIBUTORS
+ * ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
+ * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+ * POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <sys/cdefs.h>
+__KERNEL_RCSID(1, "$NetBSD$");
+
+#include <sys/types.h>
+#include <sys/proc.h>
+#include <sys/systm.h>
+
+#include <crypto/aes/aes.h>
+#include <crypto/aes/arch/aarch64/aes_arm.h>
+
+#include <aarch64/machdep.h>
+
+static void
+aesarm_setenckey(struct aesenc *enc, const uint8_t key[static 16],
+ uint32_t nrounds)
+{
+
+ switch (nrounds) {
+ case 10:
+ aesarm_setenckey128(enc, key);
+ break;
+ case 12:
+ aesarm_setenckey192(enc, key);
+ break;
+ case 14:
+ aesarm_setenckey256(enc, key);
+ break;
+ default:
+ panic("invalid AES rounds: %u", nrounds);
+ }
+}
+
+static void
+aesarm_setenckey_impl(struct aesenc *enc, const uint8_t key[static 16],
+ uint32_t nrounds)
+{
+
+ fpu_kern_enter();
+ aesarm_setenckey(enc, key, nrounds);
+ fpu_kern_leave();
+}
+
+static void
+aesarm_setdeckey_impl(struct aesdec *dec, const uint8_t key[static 16],
+ uint32_t nrounds)
+{
+ struct aesenc enc;
+
+ fpu_kern_enter();
+ aesarm_setenckey(&enc, key, nrounds);
+ aesarm_enctodec(&enc, dec, nrounds);
+ fpu_kern_leave();
+
+ explicit_memset(&enc, 0, sizeof enc);
+}
+
+static void
+aesarm_enc_impl(const struct aesenc *enc, const uint8_t in[static 16],
+ uint8_t out[static 16], uint32_t nrounds)
+{
+
+ fpu_kern_enter();
+ aesarm_enc(enc, in, out, nrounds);
+ fpu_kern_leave();
+}
+
+static void
+aesarm_dec_impl(const struct aesdec *dec, const uint8_t in[static 16],
+ uint8_t out[static 16], uint32_t nrounds)
+{
+
+ fpu_kern_enter();
+ aesarm_dec(dec, in, out, nrounds);
+ fpu_kern_leave();
+}
+
+static void
+aesarm_cbc_enc_impl(const struct aesenc *enc, const uint8_t in[static 16],
+ uint8_t out[static 16], size_t nbytes, uint8_t iv[static 16],
+ uint32_t nrounds)
+{
+
+ KASSERT(nbytes % 16 == 0);
+
+ fpu_kern_enter();
+ aesarm_cbc_enc(enc, in, out, nbytes, iv, nrounds);
+ fpu_kern_leave();
+}
+
+static void
+aesarm_cbc_dec_impl(const struct aesdec *dec, const uint8_t in[static 16],
+ uint8_t out[static 16], size_t nbytes, uint8_t iv[static 16],
+ uint32_t nrounds)
+{
+
+ KASSERT(nbytes % 16 == 0);
+
+ fpu_kern_enter();
+
+ if (nbytes % 128) {
+ aesarm_cbc_dec1(dec, in, out, nbytes % 128, iv, nrounds);
+ in += nbytes % 128;
+ out += nbytes % 128;
+ nbytes -= nbytes % 128;
+ }
+
+ KASSERT(nbytes % 128 == 0);
+ if (nbytes)
+ aesarm_cbc_dec8(dec, in, out, nbytes, iv, nrounds);
+
+ fpu_kern_leave();
+}
+
+static void
+aesarm_xts_enc_impl(const struct aesenc *enc, const uint8_t in[static 16],
+ uint8_t out[static 16], size_t nbytes, uint8_t tweak[static 16],
+ uint32_t nrounds)
+{
+
+ KASSERT(nbytes % 16 == 0);
+
+ fpu_kern_enter();
+
+ if (nbytes % 128) {
+ aesarm_xts_enc1(enc, in, out, nbytes % 128, tweak, nrounds);
+ in += nbytes % 128;
+ out += nbytes % 128;
+ nbytes -= nbytes % 128;
+ }
+
+ KASSERT(nbytes % 128 == 0);
+ if (nbytes)
+ aesarm_xts_enc8(enc, in, out, nbytes, tweak, nrounds);
+
+ fpu_kern_leave();
+}
+
+static void
+aesarm_xts_dec_impl(const struct aesdec *dec, const uint8_t in[static 16],
+ uint8_t out[static 16], size_t nbytes, uint8_t tweak[static 16],
+ uint32_t nrounds)
+{
+
+ KASSERT(nbytes % 16 == 0);
+
+ fpu_kern_enter();
+
+ if (nbytes % 128) {
+ aesarm_xts_dec1(dec, in, out, nbytes % 128, tweak, nrounds);
+ in += nbytes % 128;
+ out += nbytes % 128;
+ nbytes -= nbytes % 128;
+ }
+
+ KASSERT(nbytes % 128 == 0);
+ if (nbytes)
+ aesarm_xts_dec8(dec, in, out, nbytes, tweak, nrounds);
+
+ fpu_kern_leave();
+}
+
+static int
+aesarm_xts_update_selftest(void)
+{
+ static const struct {
+ uint8_t in[16], out[16];
+ } cases[] = {
+ {{1}, {2}},
+ {{0,0,0,0x80}, {0,0,0,0,1}},
+ {{0,0,0,0,0,0,0,0x80}, {0,0,0,0,0,0,0,0,1}},
+ {{0,0,0,0x80,0,0,0,0x80}, {0,0,0,0,1,0,0,0,1}},
+ {{0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0x80}, {0x87}},
+ {{0,0,0,0,0,0,0,0x80,0,0,0,0,0,0,0,0x80},
+ {0x87,0,0,0,0,0,0,0,1}},
+ {{0,0,0,0x80,0,0,0,0,0,0,0,0,0,0,0,0x80}, {0x87,0,0,0,1}},
+ {{0,0,0,0x80,0,0,0,0x80,0,0,0,0,0,0,0,0x80},
+ {0x87,0,0,0,1,0,0,0,1}},
+ };
+ unsigned i;
+ uint8_t tweak[16];
+
+ for (i = 0; i < sizeof(cases)/sizeof(cases[0]); i++) {
+ aesarm_xts_update(cases[i].in, tweak);
+ if (memcmp(tweak, cases[i].out, 16))
+ return -1;
+ }
+
+ /* Success! */
+ return 0;
+}
+
+static int
+aesarm_probe(void)
+{
+ struct aarch64_sysctl_cpu_id *id = &curcpu()->ci_id;
+ int result = 0;
+
+ /* Verify that the CPU supports AES. */
+ switch (__SHIFTOUT(id->ac_aa64isar0, ID_AA64ISAR0_EL1_AES)) {
+ case ID_AA64ISAR0_EL1_AES_AES:
+ case ID_AA64ISAR0_EL1_AES_PMUL:
+ break;
+ default:
+ return -1;
+ }
+
+ fpu_kern_enter();
+
+ /* Verify that our XTS tweak update logic works. */
+ if (aesarm_xts_update_selftest())
+ result = -1;
+
+ fpu_kern_leave();
+
+ return result;
+}
+
+struct aes_impl aes_arm_impl = {
+ .ai_name = "AArch64 ARMv8.0-AES",
+ .ai_probe = aesarm_probe,
+ .ai_setenckey = aesarm_setenckey_impl,
+ .ai_setdeckey = aesarm_setdeckey_impl,
+ .ai_enc = aesarm_enc_impl,
+ .ai_dec = aesarm_dec_impl,
+ .ai_cbc_enc = aesarm_cbc_enc_impl,
+ .ai_cbc_dec = aesarm_cbc_dec_impl,
+ .ai_xts_enc = aesarm_xts_enc_impl,
+ .ai_xts_dec = aesarm_xts_dec_impl,
+};
diff -r fea7aeacc09c -r 87d9e1c86afc sys/crypto/aes/arch/aarch64/aes_arm.h
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/sys/crypto/aes/arch/aarch64/aes_arm.h Sat Jun 13 16:43:32 2020 +0000
@@ -0,0 +1,68 @@
+/* $NetBSD$ */
+
+/*-
+ * Copyright (c) 2020 The NetBSD Foundation, Inc.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND CONTRIBUTORS
+ * ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
+ * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+ * POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _CRYPTO_AES_AES_ARCH_AARCH64_AES_ARM_H
+#define _CRYPTO_AES_AES_ARCH_AARCH64_AES_ARM_H
+
+#include <sys/types.h>
+
+#include <crypto/aes/aes.h>
+
+/* Assembly routines */
+
+void aesarm_setenckey128(struct aesenc *, const uint8_t[static 16]);
+void aesarm_setenckey192(struct aesenc *, const uint8_t[static 24]);
+void aesarm_setenckey256(struct aesenc *, const uint8_t[static 32]);
+
+void aesarm_enctodec(const struct aesenc *, struct aesdec *, uint32_t);
+
+void aesarm_enc(const struct aesenc *, const uint8_t[static 16],
+ uint8_t[static 16], uint32_t);
+void aesarm_dec(const struct aesdec *, const uint8_t[static 16],
+ uint8_t[static 16], uint32_t);
+
+void aesarm_cbc_enc(const struct aesenc *, const uint8_t[static 16],
+ uint8_t[static 16], size_t, uint8_t[static 16], uint32_t);
+void aesarm_cbc_dec1(const struct aesdec *, const uint8_t[static 16],
+ uint8_t[static 16], size_t, const uint8_t[static 16], uint32_t);
+void aesarm_cbc_dec8(const struct aesdec *, const uint8_t[static 128],
+ uint8_t[static 128], size_t, const uint8_t[static 16], uint32_t);
+
+void aesarm_xts_enc1(const struct aesenc *, const uint8_t[static 16],
+ uint8_t[static 16], size_t, uint8_t[static 16], uint32_t);
+void aesarm_xts_enc8(const struct aesenc *, const uint8_t[static 128],
+ uint8_t[static 128], size_t, const uint8_t[static 16], uint32_t);
+void aesarm_xts_dec1(const struct aesdec *, const uint8_t[static 16],
+ uint8_t[static 16], size_t, uint8_t[static 16], uint32_t);
+void aesarm_xts_dec8(const struct aesdec *, const uint8_t[static 128],
+ uint8_t[static 128], size_t, const uint8_t[static 16], uint32_t);
+void aesarm_xts_update(const uint8_t[static 16], uint8_t[static 16]);
+
+extern struct aes_impl aes_arm_impl;
+
+#endif /* _CRYPTO_AES_AES_ARCH_AARCH64_AES_ARM_H */
diff -r fea7aeacc09c -r 87d9e1c86afc sys/crypto/aes/arch/aarch64/aesarmfunc.S
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/sys/crypto/aes/arch/aarch64/aesarmfunc.S Sat Jun 13 16:43:32 2020 +0000
@@ -0,0 +1,1014 @@
+/* $NetBSD$ */
+
+/*-
+ * Copyright (c) 2020 The NetBSD Foundation, Inc.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND CONTRIBUTORS
+ * ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
+ * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+ * POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <aarch64/asm.h>
+
+ .arch_extension crypto
+
+/*
+ * uint32_t rcon[10]
+ *
+ * Table mapping n ---> x^n mod (x^8 + x^4 + x^3 + x + 1) in GF(2).
+ * Such elements of GF(8) need only eight bits to be represented,
+ * but we store them in 4-byte units so we can copy one into all
+ * four 4-byte lanes of a vector register with a single LD1R. The
+ * access pattern is fixed, so indices into this table are never
+ * secret.
+ */
+ .section .rodata
+ .align 4
+ .type rcon,@object
+rcon:
+ .long 0x01
+ .long 0x02
+ .long 0x04
+ .long 0x08
+ .long 0x10
+ .long 0x20
+ .long 0x40
+ .long 0x80
+ .long 0x1b
+ .long 0x36
+END(rcon)
+
+/*
+ * uint128_t unshiftrows_rotword_1
+ *
+ * Table for TBL instruction to undo ShiftRows, and then do
+ * RotWord on word 1, and then copy it into all the other words.
+ */
+ .section .rodata
+ .align 16
+ .type unshiftrows_rotword_1,@object
+unshiftrows_rotword_1:
+ .byte 0x01,0x0e,0x0b,0x04
+ .byte 0x01,0x0e,0x0b,0x04
+ .byte 0x01,0x0e,0x0b,0x04
+ .byte 0x01,0x0e,0x0b,0x04
+END(unshiftrows_rotword_1)
+
+/*
+ * uint128_t unshiftrows_3
+ *
+ * Table for TBL instruction to undo ShiftRows, and then copy word
+ * 3 into all the other words.
+ */
+ .section .rodata
+ .align 16
+ .type unshiftrows_3,@object
+unshiftrows_3:
+ .byte 0x0c,0x09,0x06,0x03
+ .byte 0x0c,0x09,0x06,0x03
+ .byte 0x0c,0x09,0x06,0x03
+ .byte 0x0c,0x09,0x06,0x03
+END(unshiftrows_3)
+
+/*
+ * uint128_t unshiftrows_rotword_3
+ *
+ * Table for TBL instruction to undo ShiftRows, and then do
+ * RotWord on word 3, and then copy it into all the other words.
+ */
+ .section .rodata
+ .align 16
+ .type unshiftrows_rotword_3,@object
+unshiftrows_rotword_3:
+ .byte 0x09,0x06,0x03,0x0c
+ .byte 0x09,0x06,0x03,0x0c
+ .byte 0x09,0x06,0x03,0x0c
+ .byte 0x09,0x06,0x03,0x0c
+END(unshiftrows_rotword_3)
+
+/*
+ * aesarm_setenckey128(struct aesenc *enckey@x0, const uint8_t key[16] @x1)
+ *
+ * Expand a 16-byte AES-128 key into 10 round keys.
+ *
+ * Standard ABI calling convention.
+ */
+ENTRY(aesarm_setenckey128)
+ ldr q1, [x1] /* q1 := master key */
+
+ adrl x4, unshiftrows_rotword_3
+ eor v0.16b, v0.16b, v0.16b /* q0 := 0 */
+ ldr q8, [x4] /* q8 := unshiftrows_rotword_3 table */
+
+ str q1, [x0], #0x10 /* store master key as first round key */
+ mov x2, #10 /* round count */
+ adrl x3, rcon /* round constant */
+
+1: /*
+ * q0 = 0
+ * v1.4s = (prk[0], prk[1], prk[2], prk[3])
+ * x0 = pointer to round key to compute
+ * x2 = round count
+ * x3 = rcon pointer
+ */
+
+ /* q3 := ShiftRows(SubBytes(q1)) */
+ mov v3.16b, v1.16b
+ aese v3.16b, v0.16b
+
+ /* v3.4s[i] := RotWords(SubBytes(prk[3])) ^ RCON */
+ ld1r {v4.4s}, [x3], #4
+ tbl v3.16b, {v3.16b}, v8.16b
+ eor v3.16b, v3.16b, v4.16b
+
+ /*
+ * v5.4s := (0,prk[0],prk[1],prk[2])
+ * v6.4s := (0,0,prk[0],prk[1])
+ * v7.4s := (0,0,0,prk[0])
+ */
+ ext v5.16b, v0.16b, v1.16b, #12
+ ext v6.16b, v0.16b, v1.16b, #8
+ ext v7.16b, v0.16b, v1.16b, #4
+
+ /* v1.4s := (rk[0], rk[1], rk[2], rk[3]) */
+ eor v1.16b, v1.16b, v3.16b
+ eor v1.16b, v1.16b, v5.16b
+ eor v1.16b, v1.16b, v6.16b
+ eor v1.16b, v1.16b, v7.16b
+
+ subs x2, x2, #1 /* count down rounds */
+ str q1, [x0], #0x10 /* store round key */
+ b.ne 1b
+
+ ret
+END(aesarm_setenckey128)
+
+/*
+ * aesarm_setenckey192(struct aesenc *enckey@x0, const uint8_t key[24] @x1)
+ *
+ * Expand a 24-byte AES-192 key into 12 round keys.
+ *
+ * Standard ABI calling convention.
+ */
+ENTRY(aesarm_setenckey192)
+ ldr q1, [x1], #0x10 /* q1 := master key[0:128) */
+ ldr d2, [x1] /* d2 := master key[128:192) */
+
+ adrl x4, unshiftrows_rotword_1
+ adrl x5, unshiftrows_rotword_3
+ eor v0.16b, v0.16b, v0.16b /* q0 := 0 */
+ ldr q8, [x4] /* q8 := unshiftrows_rotword_1 */
+ ldr q9, [x5] /* q9 := unshiftrows_rotword_3 */
+
+ str q1, [x0], #0x10 /* store master key[0:128) as round key */
+ mov x2, #12 /* round count */
+ adrl x3, rcon /* round constant */
+
+1: /*
+ * q0 = 0
+ * v1.4s = (prk[0], prk[1], prk[2], prk[3])
+ * v2.4s = (rklo[0], rklo[1], xxx, xxx)
+ * x0 = pointer to three round keys to compute
+ * x2 = round count
+ * x3 = rcon pointer
+ */
+
+ /* q3 := ShiftRows(SubBytes(q2)) */
+ mov v3.16b, v2.16b
+ aese v3.16b, v0.16b
+
+ /* v3.4s[i] := RotWords(SubBytes(rklo[1])) ^ RCON */
+ ld1r {v4.4s}, [x3], #4
+ tbl v3.16b, {v3.16b}, v8.16b
+ eor v3.16b, v3.16b, v4.16b
+
+ /*
+ * We need to compute:
+ *
+ * rk[0] := rklo[0]
+ * rk[1] := rklo[1]
+ * rk[2] := Rot(Sub(rklo[1])) ^ RCON ^ prk[0]
+ * rk[3] := Rot(Sub(rklo[1])) ^ RCON ^ prk[0] ^ prk[1]
+ * nrk[0] := Rot(Sub(rklo[1])) ^ RCON ^ prk[0] ^ prk[1] ^ prk[2]
+ * nrk[1] := Rot(Sub(rklo[1])) ^ RCON ^ prk[0] ^ ... ^ prk[3]
+ * nrk[2] := Rot(Sub(rklo[1])) ^ RCON ^ prk[0] ^ ... ^ prk[3] ^ rklo[0]
+ * nrk[3] := Rot(Sub(rklo[1])) ^ RCON ^ prk[0] ^ ... ^ prk[3] ^ rklo[0]
+ * ^ rklo[1]
+ */
+
+ /*
+ * v5.4s := (0,prk[0],prk[1],prk[2])
+ * v6.4s := (0,0,prk[0],prk[1])
+ * v7.4s := (0,0,0,prk[0])
+ */
+ ext v5.16b, v0.16b, v1.16b, #12
+ ext v6.16b, v0.16b, v1.16b, #8
+ ext v7.16b, v0.16b, v1.16b, #4
+
+ /* v5.4s := (rk[2], rk[3], nrk[0], nrk[1]) */
+ eor v5.16b, v5.16b, v1.16b
+ eor v5.16b, v5.16b, v3.16b
+ eor v5.16b, v5.16b, v6.16b
+ eor v5.16b, v5.16b, v7.16b
+
+ /*
+ * At this point, rk is split across v2.4s = (rk[0],rk[1],...)
+ * and v5.4s = (rk[2],rk[3],...); nrk is in v5.4s =
+ * (...,nrk[0],nrk[1]); and we have yet to compute nrk[2] or
+ * nrk[3], which requires rklo[0] and rklo[1] in v2.4s =
+ * (rklo[0],rklo[1],...).
+ */
+
+ /* v1.4s := (nrk[0], nrk[1], nrk[1], nrk[1]) */
+ dup v1.4s, v5.4s[3]
+ mov v1.4s[0], v5.4s[2]
+
+ /*
+ * v6.4s := (0, 0, rklo[0], rklo[1])
+ * v7.4s := (0, 0, 0, rklo[0])
+ */
+ ext v6.16b, v0.16b, v2.16b, #8
+ ext v7.16b, v0.16b, v2.16b, #4
+
+ /* v3.4s := (nrk[0], nrk[1], nrk[2], nrk[3]) */
+ eor v3.16b, v1.16b, v6.16b
+ eor v3.16b, v3.16b, v7.16b
+
+ /*
+ * Recall v2.4s = (rk[0], rk[1], xxx, xxx)
+ * and v5.4s = (rk[2], rk[3], xxx, xxx). Set
+ * v2.4s := (rk[0], rk[1], rk[2], rk[3])
+ */
+ mov v2.2d[1], v5.2d[0]
+
+ /* store two round keys */
+ stp q2, q3, [x0], #0x20
+
+ /*
+ * Live vector registers at this point:
+ *
+ * q0 = zero
+ * q2 = rk
+ * q3 = nrk
+ * v5.4s = (rk[2], rk[3], nrk[0], nrk[1])
+ * q8 = unshiftrows_rotword_1
+ * q9 = unshiftrows_rotword_3
+ *
+ * We have to compute, in q1:
+ *
+ * nnrk[0] := Rot(Sub(nrk[3])) ^ RCON' ^ rk[2]
+ * nnrk[1] := Rot(Sub(nrk[3])) ^ RCON' ^ rk[2] ^ rk[3]
+ * nnrk[2] := Rot(Sub(nrk[3])) ^ RCON' ^ rk[2] ^ rk[3] ^ nrk[0]
+ * nnrk[3] := Rot(Sub(nrk[3])) ^ RCON' ^ rk[2] ^ rk[3] ^ nrk[0]
+ * ^ nrk[1]
+ *
+ * And, if there's any more afterward, in q2:
+ *
+ * nnnrklo[0] := Rot(Sub(nrk[3])) ^ RCON' ^ rk[2] ^ rk[3] ^ nrk[0]
+ * ^ nrk[1] ^ nrk[2]
+ * nnnrklo[1] := Rot(Sub(nrk[3])) ^ RCON' ^ rk[2] ^ rk[3] ^ nrk[0]
+ * ^ nrk[1] ^ nrk[2] ^ nrk[3]
+ */
+
+ /* q1 := RotWords(SubBytes(q3)) */
+ mov v1.16b, v3.16b
+ aese v1.16b, v0.16b
+
+ /* v1.4s[i] := RotWords(SubBytes(nrk[3])) ^ RCON' */
+ ld1r {v4.4s}, [x3], #4
+ tbl v1.16b, {v1.16b}, v9.16b
+ eor v1.16b, v1.16b, v4.16b
+
+ /*
+ * v5.4s := (rk[2], rk[3], nrk[0], nrk[1]) [already]
+ * v4.4s := (0, rk[2], rk[3], nrk[0])
+ * v6.4s := (0, 0, rk[2], rk[3])
+ * v7.4s := (0, 0, 0, rk[2])
+ */
+ ext v4.16b, v0.16b, v5.16b, #12
+ ext v6.16b, v0.16b, v5.16b, #8
+ ext v7.16b, v0.16b, v5.16b, #4
+
+ /* v1.4s := (nnrk[0], nnrk[1], nnrk[2], nnrk[3]) */
+ eor v1.16b, v1.16b, v5.16b
+ eor v1.16b, v1.16b, v4.16b
+ eor v1.16b, v1.16b, v6.16b
+ eor v1.16b, v1.16b, v7.16b
+
+ subs x2, x2, #3 /* count down three rounds */
+ str q1, [x0], #0x10 /* store third round key */
+ b.eq 2f
+
+ /*
+ * v4.4s := (nrk[2], nrk[3], xxx, xxx)
+ * v5.4s := (0, nrk[2], xxx, xxx)
+ */
+ ext v4.16b, v3.16b, v0.16b, #8
+ ext v5.16b, v0.16b, v4.16b, #12
+
+ /* v2.4s := (nnrk[3], nnrk[3], xxx, xxx) */
+ dup v2.4s, v1.4s[3]
+
+ /*
+ * v2.4s := (nnnrklo[0] = nnrk[3] ^ nrk[2],
+ * nnnrklo[1] = nnrk[3] ^ nrk[2] ^ nrk[3],
+ * xxx, xxx)
+ */
+ eor v2.16b, v2.16b, v4.16b
+ eor v2.16b, v2.16b, v5.16b
+
+ b 1b
+
+2: ret
+END(aesarm_setenckey192)
+
+/*
+ * aesarm_setenckey256(struct aesenc *enckey@x0, const uint8_t key[32] @x1)
+ *
+ * Expand a 32-byte AES-256 key into 14 round keys.
+ *
+ * Standard ABI calling convention.
+ */
+ENTRY(aesarm_setenckey256)
+ /* q1 := key[0:128), q2 := key[128:256) */
+ ldp q1, q2, [x1], #0x20
+
+ adrl x4, unshiftrows_rotword_3
+ adrl x5, unshiftrows_3
+ eor v0.16b, v0.16b, v0.16b /* q0 := 0 */
+ ldr q8, [x4] /* q8 := unshiftrows_rotword_3 */
+ ldr q9, [x5] /* q9 := unshiftrows_3 */
+
+ /* store master key as first two round keys */
+ stp q1, q2, [x0], #0x20
+ mov x2, #14 /* round count */
+ adrl x3, rcon /* round constant */
+
+1: /*
+ * q0 = 0
+ * v1.4s = (pprk[0], pprk[1], pprk[2], pprk[3])
+ * v2.4s = (prk[0], prk[1], prk[2], prk[3])
+ * x2 = round count
+ * x3 = rcon pointer
+ */
+
+ /* q3 := ShiftRows(SubBytes(q2)) */
+ mov v3.16b, v2.16b
+ aese v3.16b, v0.16b
+
+ /* v3.4s[i] := RotWords(SubBytes(prk[3])) ^ RCON */
+ ld1r {v4.4s}, [x3], #4
+ tbl v3.16b, {v3.16b}, v8.16b
+ eor v3.16b, v3.16b, v4.16b
+
+ /*
+ * v5.4s := (0,pprk[0],pprk[1],pprk[2])
+ * v6.4s := (0,0,pprk[0],pprk[1])
+ * v7.4s := (0,0,0,pprk[0])
+ */
+ ext v5.16b, v0.16b, v1.16b, #12
+ ext v6.16b, v0.16b, v1.16b, #8
+ ext v7.16b, v0.16b, v1.16b, #4
+
+ /* v1.4s := (rk[0], rk[1], rk[2], rk[3]) */
+ eor v1.16b, v1.16b, v3.16b
+ eor v1.16b, v1.16b, v5.16b
+ eor v1.16b, v1.16b, v6.16b
+ eor v1.16b, v1.16b, v7.16b
+
+ subs x2, x2, #2 /* count down two rounds */
+ b.eq 2f /* stop if this is the last one */
+
+ /* q3 := ShiftRows(SubBytes(q1)) */
+ mov v3.16b, v1.16b
+ aese v3.16b, v0.16b
+
+ /* v3.4s[i] := SubBytes(rk[3]) */
+ tbl v3.16b, {v3.16b}, v9.16b
+
+ /*
+ * v5.4s := (0,prk[0],prk[1],prk[2])
+ * v6.4s := (0,0,prk[0],prk[1])
+ * v7.4s := (0,0,0,prk[0])
+ */
+ ext v5.16b, v0.16b, v2.16b, #12
+ ext v6.16b, v0.16b, v2.16b, #8
+ ext v7.16b, v0.16b, v2.16b, #4
+
+ /* v2.4s := (nrk[0], nrk[1], nrk[2], nrk[3]) */
+ eor v2.16b, v2.16b, v3.16b
+ eor v2.16b, v2.16b, v5.16b
+ eor v2.16b, v2.16b, v6.16b
+ eor v2.16b, v2.16b, v7.16b
+
+ stp q1, q2, [x0], #0x20 /* store two round keys */
+ b 1b
+
+2: str q1, [x0] /* store last round key */
+ ret
+END(aesarm_setenckey256)
+
+/*
+ * aesarm_enctodec(const struct aesenc *enckey@x0, struct aesdec *deckey@x1,
+ * uint32_t nrounds@x2)
+ *
+ * Convert AES encryption round keys to AES decryption round keys.
+ * `rounds' must be between 10 and 14.
+ *
+ * Standard ABI calling convention.
+ */
+ENTRY(aesarm_enctodec)
+ ldr q0, [x0, x2, lsl #4] /* load last round key */
+1: str q0, [x1], #0x10 /* store round key */
+ subs x2, x2, #1 /* count down round */
+ ldr q0, [x0, x2, lsl #4] /* load previous round key */
+ b.eq 2f /* stop if this is the last one */
+ aesimc v0.16b, v0.16b /* convert encryption to decryption */
+ b 1b
+2: str q0, [x1] /* store first round key verbatim */
+ ret
+END(aesarm_enctodec)
+
+/*
+ * aesarm_enc(const struct aesenc *enckey@x0, const uint8_t in[16] @x1,
+ * uint8_t out[16] @x2, uint32_t nrounds@x3)
+ *
+ * Encrypt a single block.
+ *
+ * Standard ABI calling convention.
+ */
+ENTRY(aesarm_enc)
+ stp fp, lr, [sp, #-16]! /* push stack frame */
+ mov fp, sp
+ ldr q0, [x1] /* q0 := block */
+ bl aesarm_enc1
+ str q0, [x2] /* store block */
+ ldp fp, lr, [sp], #16 /* pop stack frame */
+ ret
+END(aesarm_enc)
+
+/*
+ * aesarm_dec(const struct aesdec *deckey@x0, const uint8_t in[16] @x1,
+ * uint8_t out[16] @x2, uint32_t nrounds@x3)
+ *
+ * Decrypt a single block.
+ *
+ * Standard ABI calling convention.
+ */
+ENTRY(aesarm_dec)
+ stp fp, lr, [sp, #-16]! /* push stack frame */
+ mov fp, sp
+ ldr q0, [x1] /* q0 := block */
+ bl aesarm_dec1
+ str q0, [x2] /* store block */
+ ldp fp, lr, [sp], #16 /* pop stack frame */
+ ret
+END(aesarm_dec)
+
+/*
+ * aesarm_cbc_enc(const struct aesenc *enckey@x0, const uint8_t *in@x1,
+ * uint8_t *out@x2, size_t nbytes@x3, uint8_t iv[16] @x4,
+ * uint32_t nrounds@x5)
+ *
+ * Encrypt a contiguous sequence of blocks with AES-CBC.
+ *
+ * nbytes must be an integral multiple of 16.
+ *
+ * Standard ABI calling convention.
+ */
+ENTRY(aesarm_cbc_enc)
+ cbz x3, 2f /* stop if nothing to do */
+ stp fp, lr, [sp, #-16]! /* push stack frame */
+ mov fp, sp
+ mov x9, x0 /* x9 := enckey */
+ mov x10, x3 /* x10 := nbytes */
+ ldr q0, [x4] /* q0 := chaining value */
+1: ldr q1, [x1], #0x10 /* q1 := plaintext block */
+ eor v0.16b, v0.16b, v1.16b /* q0 := cv ^ ptxt */
+ mov x0, x9 /* x0 := enckey */
+ mov x3, x5 /* x3 := nrounds */
+ bl aesarm_enc1 /* q0 := ciphertext block */
+ subs x10, x10, #0x10 /* count down nbytes */
+ str q0, [x2], #0x10 /* store ciphertext block */
+ b.ne 1b /* repeat if x10 is nonzero */
+ str q0, [x4] /* store chaining value */
+ ldp fp, lr, [sp], #16 /* pop stack frame */
+2: ret
+END(aesarm_cbc_enc)
+
+/*
+ * aesarm_cbc_dec1(const struct aesdec *deckey@x0, const uint8_t *in@x1,
+ * uint8_t *out@x2, size_t nbytes@x3, const uint8_t iv[16] @x4,
+ * uint32_t nrounds@x5)
+ *
+ * Decrypt a contiguous sequence of blocks with AES-CBC.
+ *
+ * nbytes must be a positive integral multiple of 16. This routine
+ * is not vectorized; use aesarm_cbc_dec8 for >=8 blocks at once.
+ *
+ * Standard ABI calling convention.
+ */
+ENTRY(aesarm_cbc_dec1)
+ stp fp, lr, [sp, #-32]! /* push stack frame with uint128 */
+ mov fp, sp
+ ldr q8, [x4] /* q8 := iv */
+ str q8, [sp, #16] /* save iv */
+ mov x9, x0 /* x9 := enckey */
+ mov x10, x3 /* x10 := nbytes */
+ add x1, x1, x3 /* x1 := pointer past end of in */
+ add x2, x2, x3 /* x2 := pointer past end of out */
+ ldr q0, [x1, #-0x10]! /* q0 := last ciphertext block */
+ str q0, [x4] /* update iv */
+1: mov x0, x9 /* x0 := enckey */
+ mov x3, x5 /* x3 := nrounds */
+ bl aesarm_dec1 /* q0 := cv ^ ptxt; trash x0/x3 */
+ subs x10, x10, #0x10 /* count down nbytes */
+ b.eq 2f /* stop if this is the first block */
+ ldr q8, [x1, #-0x10]! /* q8 := chaining value */
+ eor v0.16b, v0.16b, v8.16b /* q0 := plaintext block */
+ str q0, [x2, #-0x10]! /* store plaintext block */
+ mov v0.16b, v8.16b /* move cv = ciphertext block */
+ b 1b
+2: ldr q8, [sp, #16] /* q8 := iv */
+ eor v0.16b, v0.16b, v8.16b /* q0 := first plaintext block */
+ str q0, [x2, #-0x10]! /* store first plaintext block */
+ ldp fp, lr, [sp], #32 /* pop stack frame */
+ ret
+END(aesarm_cbc_dec1)
+
+/*
+ * aesarm_cbc_dec8(const struct aesdec *deckey@x0, const uint8_t *in@x1,
+ * uint8_t *out@x2, size_t nbytes@x3, const uint8_t iv[16] @x4,
+ * uint32_t nrounds@x5)
+ *
+ * Decrypt a contiguous sequence of 8-block units with AES-CBC.
+ *
+ * nbytes must be a positive integral multiple of 128.
+ *
+ * Standard ABI calling convention.
+ */
+ENTRY(aesarm_cbc_dec8)
+ stp fp, lr, [sp, #-32]! /* push stack frame with uint128 */
+ mov fp, sp
+ ldr q8, [x4] /* q8 := iv */
+ str q8, [sp, #16] /* save iv */
+ mov x9, x0 /* x9 := enckey */
+ mov x10, x3 /* x10 := nbytes */
+ add x1, x1, x3 /* x1 := pointer past end of in */
+ add x2, x2, x3 /* x2 := pointer past end of out */
+ ldp q6, q7, [x1, #-0x20]! /* q6, q7 := last ciphertext blocks */
+ str q7, [x4] /* update iv */
+1: ldp q4, q5, [x1, #-0x20]!
+ ldp q2, q3, [x1, #-0x20]!
+ ldp q0, q1, [x1, #-0x20]!
+ mov v15.16b, v6.16b /* q[8+i] := cv[i], 0<i<8 */
+ mov v14.16b, v5.16b
+ mov v13.16b, v4.16b
+ mov v12.16b, v3.16b
+ mov v11.16b, v2.16b
+ mov v10.16b, v1.16b
+ mov v9.16b, v0.16b
+ mov x0, x9 /* x0 := enckey */
+ mov x3, x5 /* x3 := nrounds */
+ bl aesarm_dec8 /* q[i] := cv[i] ^ pt[i] */
+ eor v7.16b, v7.16b, v15.16b /* q[i] := pt[i] */
+ eor v6.16b, v6.16b, v14.16b
+ eor v5.16b, v5.16b, v13.16b
+ eor v4.16b, v4.16b, v12.16b
+ eor v3.16b, v3.16b, v11.16b
+ eor v2.16b, v2.16b, v10.16b
+ eor v1.16b, v1.16b, v9.16b
+ subs x10, x10, #0x80 /* count down nbytes */
+ stp q6, q7, [x2, #-0x20]! /* store plaintext blocks */
+ stp q4, q5, [x2, #-0x20]!
+ stp q2, q3, [x2, #-0x20]!
+ b.eq 2f /* stop if this is the first block */
+ ldp q6, q7, [x1, #-0x20]!
+ eor v0.16b, v0.16b, v7.16b /* q0 := pt0 */
+ stp q0, q1, [x2, #-0x20]!
+ b 1b
+2: ldr q8, [sp, #16] /* q8 := iv */
+ eor v0.16b, v0.16b, v8.16b /* q0 := pt0 */
+ stp q0, q1, [x2, #-0x20]! /* store first two plaintext blocks */
+ ldp fp, lr, [sp], #32 /* pop stack frame */
+ ret
+END(aesarm_cbc_dec8)
+
+/*
+ * aesarm_xts_enc1(const struct aesenc *enckey@x0, const uint8_t *in@x1,
+ * uint8_t *out@x2, size_t nbytes@x3, uint8_t tweak[16] @x4,
+ * uint32_t nrounds@x5)
+ *
+ * Encrypt a contiguous sequence of blocks with AES-XTS.
+ *
+ * nbytes must be a positive integral multiple of 16. This routine
+ * is not vectorized; use aesarm_xts_enc8 for >=8 blocks at once.
+ *
+ * Standard ABI calling convention.
+ */
+ENTRY(aesarm_xts_enc1)
+ stp fp, lr, [sp, #-16]! /* push stack frame */
+ mov fp, sp
+ mov x9, x0 /* x9 := enckey */
+ mov x10, x3 /* x10 := nbytes */
+ ldr q9, [x4] /* q9 := tweak */
+1: ldr q0, [x1], #0x10 /* q0 := ptxt */
+ mov x0, x9 /* x0 := enckey */
+ mov x3, x5 /* x3 := nrounds */
+ eor v0.16b, v0.16b, v9.16b /* q0 := ptxt ^ tweak */
+ bl aesarm_enc1 /* q0 := AES(ptxt ^ tweak) */
+ eor v0.16b, v0.16b, v9.16b /* q0 := AES(ptxt ^ tweak) ^ tweak */
+ str q0, [x2], #0x10 /* store ciphertext block */
+ bl aesarm_xts_mulx /* q9 *= x; trash x0/q0/q1 */
+ subs x10, x10, #0x10 /* count down nbytes */
+ b.ne 1b /* repeat if more blocks */
+ str q9, [x4] /* update tweak */
+ ldp fp, lr, [sp], #16 /* pop stack frame */
+ ret
+END(aesarm_xts_enc1)
+
+/*
+ * aesarm_xts_enc8(const struct aesenc *enckey@x0, const uint8_t *in@x1,
+ * uint8_t *out@x2, size_t nbytes@x3, uint8_t tweak[16] @x4,
+ * uint32_t nrounds@x5)
+ *
+ * Encrypt a contiguous sequence of blocks with AES-XTS.
+ *
+ * nbytes must be a positive integral multiple of 128.
+ *
+ * Standard ABI calling convention.
+ */
+ENTRY(aesarm_xts_enc8)
+ stp fp, lr, [sp, #-48]! /* push stack frame uint128[2] */
+ mov fp, sp
+ mov x9, x0 /* x9 := enckey */
+ mov x10, x3 /* x10 := nbytes */
+ ldr q9, [x4] /* q9 := tweak */
+1: str q9, [sp, #16] /* save tweak[0] */
+ bl aesarm_xts_mulx /* q9 *= x; trash x0/q0/q1 */
+ str q9, [sp, #32] /* save tweak[1] */
+ bl aesarm_xts_mulx /* q9 *= x; trash x0/q0/q1 */
+ mov v10.16b, v9.16b /* q10 := tweak[2] */
+ bl aesarm_xts_mulx /* q9 *= x; trash x0/q0/q1 */
+ mov v11.16b, v9.16b /* q11 := tweak[3] */
+ bl aesarm_xts_mulx /* q9 *= x; trash x0/q0/q1 */
+ mov v12.16b, v9.16b /* q11 := tweak[4] */
+ bl aesarm_xts_mulx /* q9 *= x; trash x0/q0/q1 */
+ mov v13.16b, v9.16b /* q11 := tweak[5] */
+ bl aesarm_xts_mulx /* q9 *= x; trash x0/q0/q1 */
+ mov v14.16b, v9.16b /* q11 := tweak[6] */
+ bl aesarm_xts_mulx /* q9 *= x; trash x0/q0/q1 */
+ mov v15.16b, v9.16b /* q11 := tweak[7] */
+ ldp q8, q9, [sp, #16] /* q8 := tweak[0], q9 := tweak[1] */
+ ldp q0, q1, [x1], #0x20 /* q[i] := pt[i] */
+ ldp q2, q3, [x1], #0x20
+ ldp q4, q5, [x1], #0x20
+ ldp q6, q7, [x1], #0x20
+ eor v0.16b, v0.16b, v8.16b /* q[i] := pt[i] ^ tweak[i] */
+ eor v1.16b, v1.16b, v9.16b
+ eor v2.16b, v2.16b, v10.16b
+ eor v3.16b, v3.16b, v11.16b
+ eor v4.16b, v4.16b, v12.16b
+ eor v5.16b, v5.16b, v13.16b
+ eor v6.16b, v6.16b, v14.16b
+ eor v7.16b, v7.16b, v15.16b
+ mov x0, x9 /* x0 := enckey */
+ mov x3, x5 /* x3 := nrounds */
+ bl aesarm_enc8 /* encrypt q0,...,q7; trash x0/x3/q8 */
+ ldr q8, [sp, #16] /* reload q8 := tweak[0] */
+ eor v1.16b, v1.16b, v9.16b /* q[i] := AES(...) ^ tweak[i] */
+ eor v2.16b, v2.16b, v10.16b
+ eor v3.16b, v3.16b, v11.16b
+ eor v0.16b, v0.16b, v8.16b
+ eor v4.16b, v4.16b, v12.16b
+ eor v5.16b, v5.16b, v13.16b
+ eor v6.16b, v6.16b, v14.16b
+ eor v7.16b, v7.16b, v15.16b
+ stp q0, q1, [x2], #0x20 /* store ciphertext blocks */
+ stp q2, q3, [x2], #0x20 /* store ciphertext blocks */
+ stp q4, q5, [x2], #0x20 /* store ciphertext blocks */
+ stp q6, q7, [x2], #0x20 /* store ciphertext blocks */
+ mov v9.16b, v15.16b /* q9 := q15 = tweak[7] */
+ bl aesarm_xts_mulx /* q9 *= x; trash x0/q0/q1 */
+ subs x10, x10, #0x80 /* count down nbytes */
+ b.ne 1b /* repeat if more block groups */
+ str q9, [x4] /* update tweak */
+ ldp fp, lr, [sp], #48 /* pop stack frame */
+ ret
+END(aesarm_xts_enc8)
+
+/*
+ * aesarm_xts_dec1(const struct aesdec *deckey@x0, const uint8_t *in@x1,
+ * uint8_t *out@x2, size_t nbytes@x3, uint8_t tweak[16] @x4,
+ * uint32_t nrounds@x5)
+ *
+ * Decrypt a contiguous sequence of blocks with AES-XTS.
+ *
+ * nbytes must be a positive integral multiple of 16. This routine
+ * is not vectorized; use aesarm_xts_dec8 for >=8 blocks at once.
+ *
+ * Standard ABI calling convention.
+ */
+ENTRY(aesarm_xts_dec1)
+ stp fp, lr, [sp, #-16]! /* push stack frame */
+ mov fp, sp
+ mov x9, x0 /* x9 := deckey */
+ mov x10, x3 /* x10 := nbytes */
+ ldr q9, [x4] /* q9 := tweak */
+1: ldr q0, [x1], #0x10 /* q0 := ptxt */
+ mov x0, x9 /* x0 := deckey */
+ mov x3, x5 /* x3 := nrounds */
+ eor v0.16b, v0.16b, v9.16b /* q0 := ptxt ^ tweak */
+ bl aesarm_dec1 /* q0 := AES(ptxt ^ tweak) */
+ eor v0.16b, v0.16b, v9.16b /* q0 := AES(ptxt ^ tweak) ^ tweak */
+ str q0, [x2], #0x10 /* store ciphertext block */
+ bl aesarm_xts_mulx /* q9 *= x; trash x0/q0/q1 */
+ subs x10, x10, #0x10 /* count down nbytes */
+ b.ne 1b /* repeat if more blocks */
+ str q9, [x4] /* update tweak */
+ ldp fp, lr, [sp], #16 /* pop stack frame */
+ ret
+END(aesarm_xts_dec1)
+
+/*
+ * aesarm_xts_dec8(const struct aesdec *deckey@x0, const uint8_t *in@x1,
+ * uint8_t *out@x2, size_t nbytes@x3, uint8_t tweak[16] @x4,
+ * uint32_t nrounds@x5)
+ *
+ * Decrypt a contiguous sequence of blocks with AES-XTS.
+ *
+ * nbytes must be a positive integral multiple of 128.
+ *
+ * Standard ABI calling convention.
+ */
+ENTRY(aesarm_xts_dec8)
+ stp fp, lr, [sp, #-48]! /* push stack frame uint128[2] */
+ mov fp, sp
+ mov x9, x0 /* x9 := deckey */
+ mov x10, x3 /* x10 := nbytes */
+ ldr q9, [x4] /* q9 := tweak */
+1: str q9, [sp, #16] /* save tweak[0] */
+ bl aesarm_xts_mulx /* q9 *= x; trash x0/q0/q1 */
+ str q9, [sp, #32] /* save tweak[1] */
+ bl aesarm_xts_mulx /* q9 *= x; trash x0/q0/q1 */
+ mov v10.16b, v9.16b /* q10 := tweak[2] */
+ bl aesarm_xts_mulx /* q9 *= x; trash x0/q0/q1 */
+ mov v11.16b, v9.16b /* q11 := tweak[3] */
+ bl aesarm_xts_mulx /* q9 *= x; trash x0/q0/q1 */
+ mov v12.16b, v9.16b /* q11 := tweak[4] */
+ bl aesarm_xts_mulx /* q9 *= x; trash x0/q0/q1 */
+ mov v13.16b, v9.16b /* q11 := tweak[5] */
+ bl aesarm_xts_mulx /* q9 *= x; trash x0/q0/q1 */
+ mov v14.16b, v9.16b /* q11 := tweak[6] */
+ bl aesarm_xts_mulx /* q9 *= x; trash x0/q0/q1 */
+ mov v15.16b, v9.16b /* q11 := tweak[7] */
+ ldp q8, q9, [sp, #16] /* q8 := tweak[0], q9 := tweak[1] */
+ ldp q0, q1, [x1], #0x20 /* q[i] := pt[i] */
+ ldp q2, q3, [x1], #0x20
+ ldp q4, q5, [x1], #0x20
+ ldp q6, q7, [x1], #0x20
+ eor v0.16b, v0.16b, v8.16b /* q[i] := pt[i] ^ tweak[i] */
+ eor v1.16b, v1.16b, v9.16b
+ eor v2.16b, v2.16b, v10.16b
+ eor v3.16b, v3.16b, v11.16b
+ eor v4.16b, v4.16b, v12.16b
+ eor v5.16b, v5.16b, v13.16b
+ eor v6.16b, v6.16b, v14.16b
+ eor v7.16b, v7.16b, v15.16b
+ mov x0, x9 /* x0 := deckey */
+ mov x3, x5 /* x3 := nrounds */
+ bl aesarm_dec8 /* decrypt q0,...,q7; trash x0/x3/q8 */
+ ldr q8, [sp, #16] /* reload q8 := tweak[0] */
+ eor v1.16b, v1.16b, v9.16b /* q[i] := AES(...) ^ tweak[i] */
+ eor v2.16b, v2.16b, v10.16b
+ eor v3.16b, v3.16b, v11.16b
+ eor v0.16b, v0.16b, v8.16b
+ eor v4.16b, v4.16b, v12.16b
+ eor v5.16b, v5.16b, v13.16b
+ eor v6.16b, v6.16b, v14.16b
+ eor v7.16b, v7.16b, v15.16b
+ stp q0, q1, [x2], #0x20 /* store ciphertext blocks */
+ stp q2, q3, [x2], #0x20 /* store ciphertext blocks */
+ stp q4, q5, [x2], #0x20 /* store ciphertext blocks */
+ stp q6, q7, [x2], #0x20 /* store ciphertext blocks */
+ mov v9.16b, v15.16b /* q9 := q15 = tweak[7] */
+ bl aesarm_xts_mulx /* q9 *= x; trash x0/q0/q1 */
+ subs x10, x10, #0x80 /* count down nbytes */
+ b.ne 1b /* repeat if more block groups */
+ str q9, [x4] /* update tweak */
+ ldp fp, lr, [sp], #48 /* pop stack frame */
+ ret
+END(aesarm_xts_dec8)
+
+/*
+ * aesarm_xts_mulx(tweak@q9)
+ *
+ * Multiply q9 by x, modulo x^128 + x^7 + x^2 + x + 1, in place.
+ * Uses x0 and q0/q1 as temporaries.
+ */
+ .text
+ _ALIGN_TEXT
+ .type aesarm_xts_mulx,@function
+aesarm_xts_mulx:
+ /*
+ * Simultaneously determine
+ * (a) whether the high bit of the low half must be
+ * shifted into the low bit of the high half, and
+ * (b) whether the high bit of the high half must be
+ * carried into x^128 = x^7 + x^2 + x + 1.
+ */
+ adrl x0, xtscarry
+ cmlt v1.2d, v9.2d, #0 /* v1.2d[i] := -1 if v9.2d[i] < 0, else 0 */
+ ldr q0, [x0] /* q0 := xtscarry */
+ ext v1.16b, v1.16b, v1.16b, #8 /* swap halves of q1 */
+ shl v9.2d, v9.2d, #1 /* shift */
+ and v0.16b, v0.16b, v1.16b /* copy xtscarry according to mask */
+ eor v9.16b, v9.16b, v0.16b /* incorporate (a) and (b) */
+ ret
+END(aesarm_xts_mulx)
+
+ .section .rodata
+ .align 16
+ .type xtscarry,@object
+xtscarry:
+ .byte 0x87,0,0,0, 0,0,0,0, 1,0,0,0, 0,0,0,0
+END(xtscarry)
+
+/*
+ * aesarm_xts_update(const uint8_t in[16] @x0, uint8_t out[16] @x1)
+ *
+ * Update an AES-XTS tweak.
+ *
+ * Standard ABI calling convention.
+ */
+ENTRY(aesarm_xts_update)
+ stp fp, lr, [sp, #-16]! /* push stack frame */
+ mov fp, sp
+ ldr q9, [x0] /* load tweak */
+ bl aesarm_xts_mulx /* q9 *= x */
+ str q9, [x1] /* store tweak */
+ ldp fp, lr, [sp], #16 /* pop stack frame */
+ ret
+END(aesarm_xts_update)
+
+/*
+ * aesarm_enc1(const struct aesenc *enckey@x0,
+ * uint128_t block@q0, uint32_t nrounds@x3)
+ *
+ * Encrypt a single AES block in q0.
+ *
+ * Internal ABI. Uses q8 as temporary. Destroys x0 and x3.
+ */
+ .text
+ _ALIGN_TEXT
+ .type aesarm_enc1,@function
+aesarm_enc1:
+ ldr q8, [x0], #0x10 /* load round key */
+1: subs x3, x3, #1
+ /* q0 := ShiftRows(SubBytes(AddRoundKey_q8(q0))) */
+ aese v0.16b, v8.16b
+ ldr q8, [x0], #0x10 /* load next round key */
+ b.eq 2f
+ /* q0 := MixColumns(q0) */
+ aesmc v0.16b, v0.16b
+ b 1b
+2: eor v0.16b, v0.16b, v8.16b
+ ret
+END(aesarm_enc1)
+
+/*
+ * aesarm_enc8(const struct aesenc *enckey@x0,
+ * uint128_t block0@q0, ..., uint128_t block7@q7,
+ * uint32_t nrounds@x3)
+ *
+ * Encrypt eight AES blocks in q0 through q7 in parallel.
+ *
+ * Internal ABI. Uses q8 as temporary. Destroys x0 and x3.
+ */
+ .text
+ _ALIGN_TEXT
+ .type aesarm_enc8,@function
+aesarm_enc8:
+ ldr q8, [x0], #0x10 /* load round key */
+1: subs x3, x3, #1
+ /* q[i] := ShiftRows(SubBytes(AddRoundKey_q8(q[i]))) */
+ aese v0.16b, v8.16b
+ aese v1.16b, v8.16b
+ aese v2.16b, v8.16b
+ aese v3.16b, v8.16b
+ aese v4.16b, v8.16b
+ aese v5.16b, v8.16b
+ aese v6.16b, v8.16b
+ aese v7.16b, v8.16b
+ ldr q8, [x0], #0x10 /* load next round key */
+ b.eq 2f
+ /* q[i] := MixColumns(q[i]) */
+ aesmc v0.16b, v0.16b
+ aesmc v1.16b, v1.16b
+ aesmc v2.16b, v2.16b
+ aesmc v3.16b, v3.16b
+ aesmc v4.16b, v4.16b
+ aesmc v5.16b, v5.16b
+ aesmc v6.16b, v6.16b
+ aesmc v7.16b, v7.16b
+ b 1b
+2: eor v0.16b, v0.16b, v8.16b /* AddRoundKey */
+ eor v1.16b, v1.16b, v8.16b
+ eor v2.16b, v2.16b, v8.16b
+ eor v3.16b, v3.16b, v8.16b
+ eor v4.16b, v4.16b, v8.16b
+ eor v5.16b, v5.16b, v8.16b
+ eor v6.16b, v6.16b, v8.16b
+ eor v7.16b, v7.16b, v8.16b
+ ret
+END(aesarm_enc8)
+
+/*
+ * aesarm_dec1(const struct aesdec *deckey@x0,
+ * uint128_t block@q0, uint32_t nrounds@x3)
+ *
+ * Decrypt a single AES block in q0.
+ *
+ * Internal ABI. Uses q8 as temporary. Destroys x0 and x3.
+ */
+ .text
+ _ALIGN_TEXT
+ .type aesarm_dec1,@function
+aesarm_dec1:
+ ldr q8, [x0], #0x10 /* load round key */
+1: subs x3, x3, #1
+ /* q0 := InSubBytes(InShiftRows(AddRoundKey_q8(q0))) */
+ aesd v0.16b, v8.16b
+ ldr q8, [x0], #0x10 /* load next round key */
+ b.eq 2f
+ /* q0 := InMixColumns(q0) */
+ aesimc v0.16b, v0.16b
+ b 1b
+2: eor v0.16b, v0.16b, v8.16b
+ ret
+END(aesarm_dec1)
+
+/*
+ * aesarm_dec8(const struct aesdec *deckey@x0,
+ * uint128_t block0@q0, ..., uint128_t block7@q7,
+ * uint32_t nrounds@x3)
+ *
+ * Decrypt eight AES blocks in q0 through q7 in parallel.
+ *
+ * Internal ABI. Uses q8 as temporary. Destroys x0 and x3.
+ */
+ .text
+ _ALIGN_TEXT
+ .type aesarm_dec8,@function
+aesarm_dec8:
+ ldr q8, [x0], #0x10 /* load round key */
+1: subs x3, x3, #1
+ /* q[i] := InSubBytes(InShiftRows(AddRoundKey_q8(q[i]))) */
+ aesd v0.16b, v8.16b
+ aesd v1.16b, v8.16b
+ aesd v2.16b, v8.16b
+ aesd v3.16b, v8.16b
+ aesd v4.16b, v8.16b
+ aesd v5.16b, v8.16b
+ aesd v6.16b, v8.16b
+ aesd v7.16b, v8.16b
+ ldr q8, [x0], #0x10 /* load next round key */
+ b.eq 2f
+ /* q[i] := InMixColumns(q[i]) */
+ aesimc v0.16b, v0.16b
+ aesimc v1.16b, v1.16b
+ aesimc v2.16b, v2.16b
+ aesimc v3.16b, v3.16b
+ aesimc v4.16b, v4.16b
+ aesimc v5.16b, v5.16b
+ aesimc v6.16b, v6.16b
+ aesimc v7.16b, v7.16b
+ b 1b
+2: eor v0.16b, v0.16b, v8.16b /* AddRoundKey */
+ eor v1.16b, v1.16b, v8.16b
+ eor v2.16b, v2.16b, v8.16b
+ eor v3.16b, v3.16b, v8.16b
+ eor v4.16b, v4.16b, v8.16b
+ eor v5.16b, v5.16b, v8.16b
+ eor v6.16b, v6.16b, v8.16b
+ eor v7.16b, v7.16b, v8.16b
+ ret
+END(aesarm_dec8)
diff -r fea7aeacc09c -r 87d9e1c86afc sys/crypto/aes/arch/aarch64/files.aesarm
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/sys/crypto/aes/arch/aarch64/files.aesarm Sat Jun 13 16:43:32 2020 +0000
@@ -0,0 +1,4 @@
+# $NetBSD$
+
+file crypto/aes/arch/aarch64/aes_arm.c aes
+file crypto/aes/arch/aarch64/aesarmfunc.S aes
# HG changeset patch
# User Taylor R Campbell <riastradh%NetBSD.org@localhost>
# Date 1592164233 0
# Sun Jun 14 19:50:33 2020 +0000
# Branch trunk
# Node ID 3ded3c0a82b5fec12d521ba1d98285d446d016d9
# Parent 87d9e1c86afcd441a167bf5f6d485e98d8094594
# EXP-Topic riastradh-kernelcrypto
glxsb(4): Remove rijndael dependency.
This doesn't actually seem to depend on it in any way.
XXX Compile-tested only.
diff -r 87d9e1c86afc -r 3ded3c0a82b5 sys/arch/i386/conf/files.i386
--- a/sys/arch/i386/conf/files.i386 Sat Jun 13 16:43:32 2020 +0000
+++ b/sys/arch/i386/conf/files.i386 Sun Jun 14 19:50:33 2020 +0000
@@ -416,7 +416,7 @@ obsolete defparam opt_vesafb.h VESAFB_WI
obsolete defflag opt_vesafb.h VESAFB_PM
# AMD Geode LX Security Block
-device glxsb: opencrypto, rijndael
+device glxsb: opencrypto
attach glxsb at pci
file arch/i386/pci/glxsb.c glxsb
diff -r 87d9e1c86afc -r 3ded3c0a82b5 sys/arch/i386/pci/glxsb.c
--- a/sys/arch/i386/pci/glxsb.c Sat Jun 13 16:43:32 2020 +0000
+++ b/sys/arch/i386/pci/glxsb.c Sun Jun 14 19:50:33 2020 +0000
@@ -44,7 +44,6 @@
#include <dev/pci/pcidevs.h>
#include <opencrypto/cryptodev.h>
-#include <crypto/rijndael/rijndael.h>
#define SB_GLD_MSR_CAP 0x58002000 /* RO - Capabilities */
#define SB_GLD_MSR_CONFIG 0x58002001 /* RW - Master Config */
# HG changeset patch
# User Taylor R Campbell <riastradh%NetBSD.org@localhost>
# Date 1592164303 0
# Sun Jun 14 19:51:43 2020 +0000
# Branch trunk
# Node ID 0eb81d1b858c9205fde1d048bd1fa6640ec93928
# Parent 3ded3c0a82b5fec12d521ba1d98285d446d016d9
# EXP-Topic riastradh-kernelcrypto
padlock(4): Convert legacy rijndael API to new aes API.
XXX Compile-tested only.
XXX The byte-order business here seems highly questionable.
diff -r 3ded3c0a82b5 -r 0eb81d1b858c sys/arch/x86/conf/files.x86
--- a/sys/arch/x86/conf/files.x86 Sun Jun 14 19:50:33 2020 +0000
+++ b/sys/arch/x86/conf/files.x86 Sun Jun 14 19:51:43 2020 +0000
@@ -59,7 +59,7 @@ device odcm
attach odcm at cpufeaturebus
file arch/x86/x86/odcm.c odcm
-device padlock: opencrypto, rijndael
+device padlock: opencrypto, aes
attach padlock at cpufeaturebus
file arch/x86/x86/via_padlock.c padlock
diff -r 3ded3c0a82b5 -r 0eb81d1b858c sys/arch/x86/include/via_padlock.h
--- a/sys/arch/x86/include/via_padlock.h Sun Jun 14 19:50:33 2020 +0000
+++ b/sys/arch/x86/include/via_padlock.h Sun Jun 14 19:51:43 2020 +0000
@@ -25,7 +25,8 @@
#include <sys/rndsource.h>
#include <sys/callout.h>
-#include <crypto/rijndael/rijndael.h>
+
+#include <crypto/aes/aes.h>
/* VIA C3 xcrypt-* instruction context control options */
#define C3_CRYPT_CWLO_ROUND_M 0x0000000f
@@ -43,9 +44,8 @@
#define C3_CRYPT_CWLO_KEY256 0x0000080e /* 256bit, 15 rds */
struct via_padlock_session {
- uint32_t ses_ekey[4 * (RIJNDAEL_MAXNR + 1) + 4]; /* 128 bit aligned */
- uint32_t ses_dkey[4 * (RIJNDAEL_MAXNR + 1) + 4]; /* 128 bit aligned */
- uint8_t ses_iv[16]; /* 128 bit aligned */
+ struct aesenc ses_ekey;
+ struct aesdec ses_dkey;
uint32_t ses_cw0;
struct swcr_data *swd;
int ses_klen;
diff -r 3ded3c0a82b5 -r 0eb81d1b858c sys/arch/x86/x86/via_padlock.c
--- a/sys/arch/x86/x86/via_padlock.c Sun Jun 14 19:50:33 2020 +0000
+++ b/sys/arch/x86/x86/via_padlock.c Sun Jun 14 19:51:43 2020 +0000
@@ -37,10 +37,11 @@
#include <machine/cpufunc.h>
#include <machine/cpuvar.h>
+#include <crypto/aes/aes.h>
+
#include <opencrypto/cryptodev.h>
#include <opencrypto/cryptosoft.h>
#include <opencrypto/xform.h>
-#include <crypto/rijndael/rijndael.h>
#include <opencrypto/cryptosoft_xform.c>
@@ -176,12 +177,18 @@ via_padlock_crypto_newsession(void *arg,
case CRYPTO_AES_CBC:
switch (c->cri_klen) {
case 128:
+ aes_setenckey128(&ses->ses_ekey, c->cri_key);
+ aes_setdeckey128(&ses->ses_dkey, c->cri_key);
cw0 = C3_CRYPT_CWLO_KEY128;
break;
case 192:
+ aes_setenckey192(&ses->ses_ekey, c->cri_key);
+ aes_setdeckey192(&ses->ses_dkey, c->cri_key);
cw0 = C3_CRYPT_CWLO_KEY192;
break;
case 256:
+ aes_setenckey256(&ses->ses_ekey, c->cri_key);
+ aes_setdeckey256(&ses->ses_dkey, c->cri_key);
cw0 = C3_CRYPT_CWLO_KEY256;
break;
default:
@@ -194,14 +201,12 @@ via_padlock_crypto_newsession(void *arg,
ses->ses_klen = c->cri_klen;
ses->ses_cw0 = cw0;
- /* Build expanded keys for both directions */
- rijndaelKeySetupEnc(ses->ses_ekey, c->cri_key,
- c->cri_klen);
- rijndaelKeySetupDec(ses->ses_dkey, c->cri_key,
- c->cri_klen);
- for (i = 0; i < 4 * (RIJNDAEL_MAXNR + 1); i++) {
- ses->ses_ekey[i] = ntohl(ses->ses_ekey[i]);
- ses->ses_dkey[i] = ntohl(ses->ses_dkey[i]);
+ /* Convert words to host byte order (???) */
+ for (i = 0; i < 4 * (AES_256_NROUNDS + 1); i++) {
+ ses->ses_ekey.aese_aes.aes_rk[i] =
+ ntohl(ses->ses_ekey.aese_aes.aes_rk[i]);
+ ses->ses_dkey.aesd_aes.aes_rk[i] =
+ ntohl(ses->ses_dkey.aesd_aes.aes_rk[i]);
}
break;
@@ -379,7 +384,7 @@ via_padlock_crypto_encdec(struct cryptop
if (crd->crd_flags & CRD_F_ENCRYPT) {
sc->op_cw[0] = ses->ses_cw0 | C3_CRYPT_CWLO_ENCRYPT;
- key = ses->ses_ekey;
+ key = ses->ses_ekey.aese_aes.aes_rk;
if (crd->crd_flags & CRD_F_IV_EXPLICIT)
memcpy(sc->op_iv, crd->crd_iv, 16);
else
@@ -398,7 +403,7 @@ via_padlock_crypto_encdec(struct cryptop
}
} else {
sc->op_cw[0] = ses->ses_cw0 | C3_CRYPT_CWLO_DECRYPT;
- key = ses->ses_dkey;
+ key = ses->ses_dkey.aesd_aes.aes_rk;
if (crd->crd_flags & CRD_F_IV_EXPLICIT)
memcpy(sc->op_iv, crd->crd_iv, 16);
else {
# HG changeset patch
# User Taylor R Campbell <riastradh%NetBSD.org@localhost>
# Date 1592164567 0
# Sun Jun 14 19:56:07 2020 +0000
# Branch trunk
# Node ID f2bfdffcb27b2e0de26513dbac99e057635654bb
# Parent 0eb81d1b858c9205fde1d048bd1fa6640ec93928
# EXP-Topic riastradh-kernelcrypto
cgd(4): Switch from legacy rijndael API to new aes API.
diff -r 0eb81d1b858c -r f2bfdffcb27b sys/conf/files
--- a/sys/conf/files Sun Jun 14 19:51:43 2020 +0000
+++ b/sys/conf/files Sun Jun 14 19:56:07 2020 +0000
@@ -1395,7 +1395,7 @@ file dev/ic/amdccp.c amdccp
defpseudodev vnd: disk
defflag opt_vnd.h VND_COMPRESSION
defpseudo ccd: disk
-defpseudodev cgd: disk, des, blowfish, cast128, rijndael
+defpseudodev cgd: disk, des, blowfish, cast128, aes
defpseudodev md: disk
defpseudodev fss: disk
diff -r 0eb81d1b858c -r f2bfdffcb27b sys/dev/cgd_crypto.c
--- a/sys/dev/cgd_crypto.c Sun Jun 14 19:51:43 2020 +0000
+++ b/sys/dev/cgd_crypto.c Sun Jun 14 19:56:07 2020 +0000
@@ -45,9 +45,9 @@
#include <dev/cgd_crypto.h>
+#include <crypto/aes/aes.h>
#include <crypto/blowfish/blowfish.h>
#include <crypto/des/des.h>
-#include <crypto/rijndael/rijndael-api-fst.h>
/*
* The general framework provides only one generic function.
@@ -114,8 +114,9 @@ cryptfuncs_find(const char *alg)
*/
struct aes_privdata {
- keyInstance ap_enckey;
- keyInstance ap_deckey;
+ struct aesenc ap_enckey;
+ struct aesdec ap_deckey;
+ uint32_t ap_nrounds;
};
static void *
@@ -132,8 +133,23 @@ cgd_cipher_aes_cbc_init(size_t keylen, c
if (*blocksize != 128)
return NULL;
ap = kmem_zalloc(sizeof(*ap), KM_SLEEP);
- rijndael_makeKey(&ap->ap_enckey, DIR_ENCRYPT, keylen, key);
- rijndael_makeKey(&ap->ap_deckey, DIR_DECRYPT, keylen, key);
+ switch (keylen) {
+ case 128:
+ aes_setenckey128(&ap->ap_enckey, key);
+ aes_setdeckey128(&ap->ap_deckey, key);
+ ap->ap_nrounds = AES_128_NROUNDS;
+ break;
+ case 192:
+ aes_setenckey192(&ap->ap_enckey, key);
+ aes_setdeckey192(&ap->ap_deckey, key);
+ ap->ap_nrounds = AES_192_NROUNDS;
+ break;
+ case 256:
+ aes_setenckey256(&ap->ap_enckey, key);
+ aes_setdeckey256(&ap->ap_deckey, key);
+ ap->ap_nrounds = AES_256_NROUNDS;
+ break;
+ }
return ap;
}
@@ -152,25 +168,18 @@ cgd_cipher_aes_cbc(void *privdata, void
{
struct aes_privdata *apd = privdata;
uint8_t iv[CGD_AES_BLOCK_SIZE] = {0};
- cipherInstance cipher;
- int cipher_ok __diagused;
/* Compute the CBC IV as AES_k(blkno). */
- cipher_ok = rijndael_cipherInit(&cipher, MODE_ECB, NULL);
- KASSERT(cipher_ok > 0);
- rijndael_blockEncrypt(&cipher, &apd->ap_enckey, blkno, /*nbits*/128,
- iv);
+ aes_enc(&apd->ap_enckey, blkno, iv, apd->ap_nrounds);
- cipher_ok = rijndael_cipherInit(&cipher, MODE_CBC, iv);
- KASSERT(cipher_ok > 0);
switch (dir) {
case CGD_CIPHER_ENCRYPT:
- rijndael_blockEncrypt(&cipher, &apd->ap_enckey, src,
- /*nbits*/nbytes * 8, dst);
+ aes_cbc_enc(&apd->ap_enckey, src, dst, nbytes, iv,
+ apd->ap_nrounds);
break;
case CGD_CIPHER_DECRYPT:
- rijndael_blockDecrypt(&cipher, &apd->ap_deckey, src,
- /*nbits*/nbytes * 8, dst);
+ aes_cbc_dec(&apd->ap_deckey, src, dst, nbytes, iv,
+ apd->ap_nrounds);
break;
default:
panic("%s: unrecognised direction %d", __func__, dir);
@@ -182,9 +191,10 @@ cgd_cipher_aes_cbc(void *privdata, void
*/
struct aesxts {
- keyInstance ax_enckey;
- keyInstance ax_deckey;
- keyInstance ax_tweakkey;
+ struct aesenc ax_enckey;
+ struct aesdec ax_deckey;
+ struct aesenc ax_tweakkey;
+ uint32_t ax_nrounds;
};
static void *
@@ -207,9 +217,20 @@ cgd_cipher_aes_xts_init(size_t keylen, c
key = xtskey;
key2 = key + keylen / CHAR_BIT;
- rijndael_makeKey(&ax->ax_enckey, DIR_ENCRYPT, keylen, key);
- rijndael_makeKey(&ax->ax_deckey, DIR_DECRYPT, keylen, key);
- rijndael_makeKey(&ax->ax_tweakkey, DIR_ENCRYPT, keylen, key2);
+ switch (keylen) {
+ case 128:
+ aes_setenckey128(&ax->ax_enckey, key);
+ aes_setdeckey128(&ax->ax_deckey, key);
+ aes_setenckey128(&ax->ax_tweakkey, key2);
+ ax->ax_nrounds = AES_128_NROUNDS;
+ break;
+ case 256:
+ aes_setenckey256(&ax->ax_enckey, key);
+ aes_setdeckey256(&ax->ax_deckey, key);
+ aes_setenckey256(&ax->ax_tweakkey, key2);
+ ax->ax_nrounds = AES_256_NROUNDS;
+ break;
+ }
return ax;
}
@@ -229,25 +250,18 @@ cgd_cipher_aes_xts(void *cookie, void *d
{
struct aesxts *ax = cookie;
uint8_t tweak[CGD_AES_BLOCK_SIZE];
- cipherInstance cipher;
- int cipher_ok __diagused;
/* Compute the initial tweak as AES_k(blkno). */
- cipher_ok = rijndael_cipherInit(&cipher, MODE_ECB, NULL);
- KASSERT(cipher_ok > 0);
- rijndael_blockEncrypt(&cipher, &ax->ax_tweakkey, blkno, /*nbits*/128,
- tweak);
+ aes_enc(&ax->ax_tweakkey, blkno, tweak, ax->ax_nrounds);
- cipher_ok = rijndael_cipherInit(&cipher, MODE_XTS, tweak);
- KASSERT(cipher_ok > 0);
switch (dir) {
case CGD_CIPHER_ENCRYPT:
- rijndael_blockEncrypt(&cipher, &ax->ax_enckey, src,
- /*nbits*/nbytes * 8, dst);
+ aes_xts_enc(&ax->ax_enckey, src, dst, nbytes, tweak,
+ ax->ax_nrounds);
break;
case CGD_CIPHER_DECRYPT:
- rijndael_blockDecrypt(&cipher, &ax->ax_deckey, src,
- /*nbits*/nbytes * 8, dst);
+ aes_xts_dec(&ax->ax_deckey, src, dst, nbytes, tweak,
+ ax->ax_nrounds);
break;
default:
panic("%s: unrecognised direction %d", __func__, dir);
# HG changeset patch
# User Taylor R Campbell <riastradh%NetBSD.org@localhost>
# Date 1592164643 0
# Sun Jun 14 19:57:23 2020 +0000
# Branch trunk
# Node ID b7131a05bde780d6bbcc795e46ecfde3a45e9bfb
# Parent f2bfdffcb27b2e0de26513dbac99e057635654bb
# EXP-Topic riastradh-kernelcrypto
uvm(9): Switch from legacy rijndael API to new aes API.
diff -r f2bfdffcb27b -r b7131a05bde7 sys/uvm/files.uvm
--- a/sys/uvm/files.uvm Sun Jun 14 19:56:07 2020 +0000
+++ b/sys/uvm/files.uvm Sun Jun 14 19:57:23 2020 +0000
@@ -8,7 +8,7 @@ defflag opt_uvmhist.h UVMHIST_PRINT: KE
defparam opt_uvmhist.h UVMHIST_MAPHIST_SIZE UVMHIST_PDHIST_SIZE
defflag opt_uvm.h USE_TOPDOWN_VM UVMMAP_COUNTERS
defparam opt_uvm.h UVM_RESERVED_PAGES_PER_CPU
-defflag opt_vmswap.h VMSWAP : rijndael
+defflag opt_vmswap.h VMSWAP : aes
defflag opt_readahead.h READAHEAD_STATS
defflag opt_ubc.h UBC_STATS
defparam opt_pagermap.h PAGER_MAP_SIZE
diff -r f2bfdffcb27b -r b7131a05bde7 sys/uvm/uvm_swap.c
--- a/sys/uvm/uvm_swap.c Sun Jun 14 19:56:07 2020 +0000
+++ b/sys/uvm/uvm_swap.c Sun Jun 14 19:57:23 2020 +0000
@@ -65,7 +65,7 @@
#include <miscfs/specfs/specdev.h>
-#include <crypto/rijndael/rijndael-api-fst.h>
+#include <crypto/aes/aes.h>
/*
* uvm_swap.c: manage configuration and i/o to swap space.
@@ -148,8 +148,8 @@ struct swapdev {
int swd_active; /* number of active buffers */
volatile uint32_t *swd_encmap; /* bitmap of encrypted slots */
- keyInstance swd_enckey; /* AES key expanded for enc */
- keyInstance swd_deckey; /* AES key expanded for dec */
+ struct aesenc swd_enckey; /* AES key expanded for enc */
+ struct aesdec swd_deckey; /* AES key expanded for dec */
bool swd_encinit; /* true if keys initialized */
};
@@ -2073,8 +2073,8 @@ uvm_swap_genkey(struct swapdev *sdp)
KASSERT(!sdp->swd_encinit);
cprng_strong(kern_cprng, key, sizeof key, 0);
- rijndael_makeKey(&sdp->swd_enckey, DIR_ENCRYPT, 256, key);
- rijndael_makeKey(&sdp->swd_deckey, DIR_DECRYPT, 256, key);
+ aes_setenckey256(&sdp->swd_enckey, key);
+ aes_setdeckey256(&sdp->swd_deckey, key);
explicit_memset(key, 0, sizeof key);
sdp->swd_encinit = true;
@@ -2089,27 +2089,17 @@ uvm_swap_genkey(struct swapdev *sdp)
static void
uvm_swap_encryptpage(struct swapdev *sdp, void *kva, int slot)
{
- cipherInstance aes;
uint8_t preiv[16] = {0}, iv[16];
- int ok __diagused, nbits __diagused;
/* iv := AES_k(le32enc(slot) || 0^96) */
le32enc(preiv, slot);
- ok = rijndael_cipherInit(&aes, MODE_ECB, NULL);
- KASSERT(ok);
- nbits = rijndael_blockEncrypt(&aes, &sdp->swd_enckey, preiv,
- /*length in bits*/128, iv);
- KASSERT(nbits == 128);
+ aes_enc(&sdp->swd_enckey, (const void *)preiv, iv, AES_256_NROUNDS);
/* *kva := AES-CBC_k(iv, *kva) */
- ok = rijndael_cipherInit(&aes, MODE_CBC, iv);
- KASSERT(ok);
- nbits = rijndael_blockEncrypt(&aes, &sdp->swd_enckey, kva,
- /*length in bits*/PAGE_SIZE*NBBY, kva);
- KASSERT(nbits == PAGE_SIZE*NBBY);
+ aes_cbc_enc(&sdp->swd_enckey, kva, kva, PAGE_SIZE, iv,
+ AES_256_NROUNDS);
explicit_memset(&iv, 0, sizeof iv);
- explicit_memset(&aes, 0, sizeof aes);
}
/*
@@ -2121,28 +2111,17 @@ uvm_swap_encryptpage(struct swapdev *sdp
static void
uvm_swap_decryptpage(struct swapdev *sdp, void *kva, int slot)
{
- cipherInstance aes;
uint8_t preiv[16] = {0}, iv[16];
- int ok __diagused, nbits __diagused;
/* iv := AES_k(le32enc(slot) || 0^96) */
le32enc(preiv, slot);
- ok = rijndael_cipherInit(&aes, MODE_ECB, NULL);
- KASSERT(ok);
- nbits = rijndael_blockEncrypt(&aes, &sdp->swd_enckey, preiv,
- /*length in bits*/128, iv);
- KASSERTMSG(nbits == 128, "nbits=%d expected %d\n", nbits, 128);
+ aes_enc(&sdp->swd_enckey, (const void *)preiv, iv, AES_256_NROUNDS);
/* *kva := AES-CBC^{-1}_k(iv, *kva) */
- ok = rijndael_cipherInit(&aes, MODE_CBC, iv);
- KASSERT(ok);
- nbits = rijndael_blockDecrypt(&aes, &sdp->swd_deckey, kva,
- /*length in bits*/PAGE_SIZE*NBBY, kva);
- KASSERTMSG(nbits == PAGE_SIZE*NBBY,
- "nbits=%d expected %d\n", nbits, PAGE_SIZE*NBBY);
+ aes_cbc_dec(&sdp->swd_deckey, kva, kva, PAGE_SIZE, iv,
+ AES_256_NROUNDS);
explicit_memset(&iv, 0, sizeof iv);
- explicit_memset(&aes, 0, sizeof aes);
}
SYSCTL_SETUP(sysctl_uvmswap_setup, "sysctl uvmswap setup")
# HG changeset patch
# User Taylor R Campbell <riastradh%NetBSD.org@localhost>
# Date 1592164753 0
# Sun Jun 14 19:59:13 2020 +0000
# Branch trunk
# Node ID a97bc0abe60d9a77b10f27d63951d60b0be7b987
# Parent b7131a05bde780d6bbcc795e46ecfde3a45e9bfb
# EXP-Topic riastradh-kernelcrypto
opencrypto: Switch from legacy rijndael API to new aes API.
While here, apply various rijndael->aes renames, reduce the size
of aesxcbc_ctx by 480 bytes, and convert some malloc->kmem.
Leave in the symbol enc_xform_rijndael128 for now, though, so this
doesn't break any kernel ABI.
diff -r b7131a05bde7 -r a97bc0abe60d sys/opencrypto/aesxcbcmac.c
--- a/sys/opencrypto/aesxcbcmac.c Sun Jun 14 19:57:23 2020 +0000
+++ b/sys/opencrypto/aesxcbcmac.c Sun Jun 14 19:59:13 2020 +0000
@@ -34,7 +34,8 @@
#include <sys/param.h>
#include <sys/systm.h>
-#include <crypto/rijndael/rijndael.h>
+
+#include <crypto/aes/aes.h>
#include <opencrypto/aesxcbcmac.h>
@@ -47,24 +48,31 @@ aes_xcbc_mac_init(void *vctx, const uint
{ 2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2 };
static const uint8_t k3seed[AES_BLOCKSIZE] =
{ 3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3 };
- u_int32_t r_ks[(RIJNDAEL_MAXNR+1)*4];
+ struct aesenc r_ks;
aesxcbc_ctx *ctx;
uint8_t k1[AES_BLOCKSIZE];
ctx = vctx;
memset(ctx, 0, sizeof(*ctx));
- if ((ctx->r_nr = rijndaelKeySetupEnc(r_ks, key, keylen * 8)) == 0)
- return -1;
- rijndaelEncrypt(r_ks, ctx->r_nr, k1seed, k1);
- rijndaelEncrypt(r_ks, ctx->r_nr, k2seed, ctx->k2);
- rijndaelEncrypt(r_ks, ctx->r_nr, k3seed, ctx->k3);
- if (rijndaelKeySetupEnc(ctx->r_k1s, k1, AES_BLOCKSIZE * 8) == 0)
- return -1;
- if (rijndaelKeySetupEnc(ctx->r_k2s, ctx->k2, AES_BLOCKSIZE * 8) == 0)
- return -1;
- if (rijndaelKeySetupEnc(ctx->r_k3s, ctx->k3, AES_BLOCKSIZE * 8) == 0)
- return -1;
+ switch (keylen) {
+ case 16:
+ ctx->r_nr = aes_setenckey128(&r_ks, key);
+ break;
+ case 24:
+ ctx->r_nr = aes_setenckey192(&r_ks, key);
+ break;
+ case 32:
+ ctx->r_nr = aes_setenckey256(&r_ks, key);
+ break;
+ }
+ aes_enc(&r_ks, k1seed, k1, ctx->r_nr);
+ aes_enc(&r_ks, k2seed, ctx->k2, ctx->r_nr);
+ aes_enc(&r_ks, k3seed, ctx->k3, ctx->r_nr);
+ aes_setenckey128(&ctx->r_k1s, k1);
+
+ explicit_memset(&r_ks, 0, sizeof(r_ks));
+ explicit_memset(k1, 0, sizeof(k1));
return 0;
}
@@ -83,7 +91,7 @@ aes_xcbc_mac_loop(void *vctx, const uint
if (ctx->buflen == sizeof(ctx->buf)) {
for (i = 0; i < sizeof(ctx->e); i++)
ctx->buf[i] ^= ctx->e[i];
- rijndaelEncrypt(ctx->r_k1s, ctx->r_nr, ctx->buf, ctx->e);
+ aes_enc(&ctx->r_k1s, ctx->buf, ctx->e, ctx->r_nr);
ctx->buflen = 0;
}
if (ctx->buflen + len < sizeof(ctx->buf)) {
@@ -96,7 +104,7 @@ aes_xcbc_mac_loop(void *vctx, const uint
sizeof(ctx->buf) - ctx->buflen);
for (i = 0; i < sizeof(ctx->e); i++)
ctx->buf[i] ^= ctx->e[i];
- rijndaelEncrypt(ctx->r_k1s, ctx->r_nr, ctx->buf, ctx->e);
+ aes_enc(&ctx->r_k1s, ctx->buf, ctx->e, ctx->r_nr);
addr += sizeof(ctx->buf) - ctx->buflen;
ctx->buflen = 0;
}
@@ -105,7 +113,7 @@ aes_xcbc_mac_loop(void *vctx, const uint
memcpy(buf, addr, AES_BLOCKSIZE);
for (i = 0; i < sizeof(buf); i++)
buf[i] ^= ctx->e[i];
- rijndaelEncrypt(ctx->r_k1s, ctx->r_nr, buf, ctx->e);
+ aes_enc(&ctx->r_k1s, buf, ctx->e, ctx->r_nr);
addr += AES_BLOCKSIZE;
}
if (addr < ep) {
@@ -129,7 +137,7 @@ aes_xcbc_mac_result(uint8_t *addr, void
ctx->buf[i] ^= ctx->e[i];
ctx->buf[i] ^= ctx->k2[i];
}
- rijndaelEncrypt(ctx->r_k1s, ctx->r_nr, ctx->buf, digest);
+ aes_enc(&ctx->r_k1s, ctx->buf, digest, ctx->r_nr);
} else {
for (i = ctx->buflen; i < sizeof(ctx->buf); i++)
ctx->buf[i] = (i == ctx->buflen) ? 0x80 : 0x00;
@@ -137,7 +145,7 @@ aes_xcbc_mac_result(uint8_t *addr, void
ctx->buf[i] ^= ctx->e[i];
ctx->buf[i] ^= ctx->k3[i];
}
- rijndaelEncrypt(ctx->r_k1s, ctx->r_nr, ctx->buf, digest);
+ aes_enc(&ctx->r_k1s, ctx->buf, digest, ctx->r_nr);
}
memcpy(addr, digest, sizeof(digest));
diff -r b7131a05bde7 -r a97bc0abe60d sys/opencrypto/aesxcbcmac.h
--- a/sys/opencrypto/aesxcbcmac.h Sun Jun 14 19:57:23 2020 +0000
+++ b/sys/opencrypto/aesxcbcmac.h Sun Jun 14 19:59:13 2020 +0000
@@ -1,5 +1,8 @@
/* $NetBSD: aesxcbcmac.h,v 1.1 2011/05/24 19:10:09 drochner Exp $ */
+#ifndef _OPENCRYPTO_AESXCBCMAC_H
+#define _OPENCRYPTO_AESXCBCMAC_H
+
#include <sys/types.h>
#define AES_BLOCKSIZE 16
@@ -8,9 +11,7 @@ typedef struct {
u_int8_t e[AES_BLOCKSIZE];
u_int8_t buf[AES_BLOCKSIZE];
size_t buflen;
- u_int32_t r_k1s[(RIJNDAEL_MAXNR+1)*4];
- u_int32_t r_k2s[(RIJNDAEL_MAXNR+1)*4];
- u_int32_t r_k3s[(RIJNDAEL_MAXNR+1)*4];
+ struct aesenc r_k1s;
int r_nr; /* key-length-dependent number of rounds */
u_int8_t k2[AES_BLOCKSIZE];
u_int8_t k3[AES_BLOCKSIZE];
@@ -19,3 +20,5 @@ typedef struct {
int aes_xcbc_mac_init(void *, const u_int8_t *, u_int16_t);
int aes_xcbc_mac_loop(void *, const u_int8_t *, u_int16_t);
void aes_xcbc_mac_result(u_int8_t *, void *);
+
+#endif /* _OPENCRYPTO_AESXCBCMAC_H */
diff -r b7131a05bde7 -r a97bc0abe60d sys/opencrypto/cryptosoft.c
--- a/sys/opencrypto/cryptosoft.c Sun Jun 14 19:57:23 2020 +0000
+++ b/sys/opencrypto/cryptosoft.c Sun Jun 14 19:59:13 2020 +0000
@@ -831,8 +831,8 @@ swcr_newsession(void *arg, u_int32_t *si
case CRYPTO_SKIPJACK_CBC:
txf = &swcr_enc_xform_skipjack;
goto enccommon;
- case CRYPTO_RIJNDAEL128_CBC:
- txf = &swcr_enc_xform_rijndael128;
+ case CRYPTO_AES_CBC:
+ txf = &swcr_enc_xform_aes;
goto enccommon;
case CRYPTO_CAMELLIA_CBC:
txf = &swcr_enc_xform_camellia;
@@ -890,15 +890,13 @@ swcr_newsession(void *arg, u_int32_t *si
axf = &swcr_auth_hash_hmac_ripemd_160_96;
goto authcommon; /* leave this for safety */
authcommon:
- (*swd)->sw_ictx = malloc(axf->ctxsize,
- M_CRYPTO_DATA, M_NOWAIT);
+ (*swd)->sw_ictx = kmem_alloc(axf->ctxsize, KM_NOSLEEP);
if ((*swd)->sw_ictx == NULL) {
swcr_freesession(NULL, i);
return ENOBUFS;
}
- (*swd)->sw_octx = malloc(axf->ctxsize,
- M_CRYPTO_DATA, M_NOWAIT);
+ (*swd)->sw_octx = kmem_alloc(axf->ctxsize, KM_NOSLEEP);
if ((*swd)->sw_octx == NULL) {
swcr_freesession(NULL, i);
return ENOBUFS;
@@ -936,16 +934,15 @@ swcr_newsession(void *arg, u_int32_t *si
CTASSERT(SHA1_DIGEST_LENGTH >= MD5_DIGEST_LENGTH);
axf = &swcr_auth_hash_key_sha1;
auth2common:
- (*swd)->sw_ictx = malloc(axf->ctxsize,
- M_CRYPTO_DATA, M_NOWAIT);
+ (*swd)->sw_ictx = kmem_alloc(axf->ctxsize, KM_NOSLEEP);
if ((*swd)->sw_ictx == NULL) {
swcr_freesession(NULL, i);
return ENOBUFS;
}
/* Store the key so we can "append" it to the payload */
- (*swd)->sw_octx = malloc(cri->cri_klen / 8, M_CRYPTO_DATA,
- M_NOWAIT);
+ (*swd)->sw_octx = kmem_alloc(cri->cri_klen / 8,
+ KM_NOSLEEP);
if ((*swd)->sw_octx == NULL) {
swcr_freesession(NULL, i);
return ENOBUFS;
@@ -968,8 +965,7 @@ swcr_newsession(void *arg, u_int32_t *si
case CRYPTO_SHA1:
axf = &swcr_auth_hash_sha1;
auth3common:
- (*swd)->sw_ictx = malloc(axf->ctxsize,
- M_CRYPTO_DATA, M_NOWAIT);
+ (*swd)->sw_ictx = kmem_alloc(axf->ctxsize, KM_NOSLEEP);
if ((*swd)->sw_ictx == NULL) {
swcr_freesession(NULL, i);
return ENOBUFS;
@@ -991,8 +987,7 @@ swcr_newsession(void *arg, u_int32_t *si
case CRYPTO_AES_256_GMAC:
axf = &swcr_auth_hash_gmac_aes_256;
auth4common:
- (*swd)->sw_ictx = malloc(axf->ctxsize,
- M_CRYPTO_DATA, M_NOWAIT);
+ (*swd)->sw_ictx = kmem_alloc(axf->ctxsize, KM_NOSLEEP);
if ((*swd)->sw_ictx == NULL) {
swcr_freesession(NULL, i);
return ENOBUFS;
@@ -1057,7 +1052,7 @@ swcr_freesession(void *arg, u_int64_t ti
case CRYPTO_BLF_CBC:
case CRYPTO_CAST_CBC:
case CRYPTO_SKIPJACK_CBC:
- case CRYPTO_RIJNDAEL128_CBC:
+ case CRYPTO_AES_CBC:
case CRYPTO_CAMELLIA_CBC:
case CRYPTO_AES_CTR:
case CRYPTO_AES_GCM_16:
@@ -1083,11 +1078,11 @@ swcr_freesession(void *arg, u_int64_t ti
if (swd->sw_ictx) {
explicit_memset(swd->sw_ictx, 0, axf->ctxsize);
- free(swd->sw_ictx, M_CRYPTO_DATA);
+ kmem_free(swd->sw_ictx, axf->ctxsize);
}
if (swd->sw_octx) {
explicit_memset(swd->sw_octx, 0, axf->ctxsize);
- free(swd->sw_octx, M_CRYPTO_DATA);
+ kmem_free(swd->sw_octx, axf->ctxsize);
}
break;
@@ -1097,11 +1092,11 @@ swcr_freesession(void *arg, u_int64_t ti
if (swd->sw_ictx) {
explicit_memset(swd->sw_ictx, 0, axf->ctxsize);
- free(swd->sw_ictx, M_CRYPTO_DATA);
+ kmem_free(swd->sw_ictx, axf->ctxsize);
}
if (swd->sw_octx) {
explicit_memset(swd->sw_octx, 0, swd->sw_klen);
- free(swd->sw_octx, M_CRYPTO_DATA);
+ kmem_free(swd->sw_octx, axf->ctxsize);
}
break;
@@ -1115,7 +1110,7 @@ swcr_freesession(void *arg, u_int64_t ti
if (swd->sw_ictx) {
explicit_memset(swd->sw_ictx, 0, axf->ctxsize);
- free(swd->sw_ictx, M_CRYPTO_DATA);
+ kmem_free(swd->sw_ictx, axf->ctxsize);
}
break;
@@ -1193,7 +1188,7 @@ swcr_process(void *arg, struct cryptop *
case CRYPTO_BLF_CBC:
case CRYPTO_CAST_CBC:
case CRYPTO_SKIPJACK_CBC:
- case CRYPTO_RIJNDAEL128_CBC:
+ case CRYPTO_AES_CBC:
case CRYPTO_CAMELLIA_CBC:
case CRYPTO_AES_CTR:
if ((crp->crp_etype = swcr_encdec(crd, sw,
@@ -1294,7 +1289,7 @@ swcr_init(void)
REGISTER(CRYPTO_AES_128_GMAC);
REGISTER(CRYPTO_AES_192_GMAC);
REGISTER(CRYPTO_AES_256_GMAC);
- REGISTER(CRYPTO_RIJNDAEL128_CBC);
+ REGISTER(CRYPTO_AES_CBC);
REGISTER(CRYPTO_DEFLATE_COMP);
REGISTER(CRYPTO_DEFLATE_COMP_NOGROW);
REGISTER(CRYPTO_GZIP_COMP);
diff -r b7131a05bde7 -r a97bc0abe60d sys/opencrypto/cryptosoft_xform.c
--- a/sys/opencrypto/cryptosoft_xform.c Sun Jun 14 19:57:23 2020 +0000
+++ b/sys/opencrypto/cryptosoft_xform.c Sun Jun 14 19:59:13 2020 +0000
@@ -42,21 +42,22 @@
#include <sys/cdefs.h>
__KERNEL_RCSID(1, "$NetBSD: cryptosoft_xform.c,v 1.28 2019/10/12 00:49:30 christos Exp $");
-#include <crypto/blowfish/blowfish.h>
-#include <crypto/cast128/cast128.h>
-#include <crypto/des/des.h>
-#include <crypto/rijndael/rijndael.h>
-#include <crypto/skipjack/skipjack.h>
-#include <crypto/camellia/camellia.h>
-
-#include <opencrypto/deflate.h>
-
+#include <sys/cprng.h>
+#include <sys/kmem.h>
#include <sys/md5.h>
#include <sys/rmd160.h>
#include <sys/sha1.h>
#include <sys/sha2.h>
-#include <sys/cprng.h>
+
+#include <crypto/aes/aes.h>
+#include <crypto/blowfish/blowfish.h>
+#include <crypto/camellia/camellia.h>
+#include <crypto/cast128/cast128.h>
+#include <crypto/des/des.h>
+#include <crypto/skipjack/skipjack.h>
+
#include <opencrypto/aesxcbcmac.h>
+#include <opencrypto/deflate.h>
#include <opencrypto/gmac.h>
struct swcr_auth_hash {
@@ -94,7 +95,7 @@ static int des3_setkey(u_int8_t **, cons
static int blf_setkey(u_int8_t **, const u_int8_t *, int);
static int cast5_setkey(u_int8_t **, const u_int8_t *, int);
static int skipjack_setkey(u_int8_t **, const u_int8_t *, int);
-static int rijndael128_setkey(u_int8_t **, const u_int8_t *, int);
+static int aes_setkey(u_int8_t **, const u_int8_t *, int);
static int cml_setkey(u_int8_t **, const u_int8_t *, int);
static int aes_ctr_setkey(u_int8_t **, const u_int8_t *, int);
static int aes_gmac_setkey(u_int8_t **, const u_int8_t *, int);
@@ -103,14 +104,14 @@ static void des3_encrypt(void *, u_int8_
static void blf_encrypt(void *, u_int8_t *);
static void cast5_encrypt(void *, u_int8_t *);
static void skipjack_encrypt(void *, u_int8_t *);
-static void rijndael128_encrypt(void *, u_int8_t *);
+static void aes_encrypt(void *, u_int8_t *);
static void cml_encrypt(void *, u_int8_t *);
static void des1_decrypt(void *, u_int8_t *);
static void des3_decrypt(void *, u_int8_t *);
static void blf_decrypt(void *, u_int8_t *);
static void cast5_decrypt(void *, u_int8_t *);
static void skipjack_decrypt(void *, u_int8_t *);
-static void rijndael128_decrypt(void *, u_int8_t *);
+static void aes_decrypt(void *, u_int8_t *);
static void cml_decrypt(void *, u_int8_t *);
static void aes_ctr_crypt(void *, u_int8_t *);
static void des1_zerokey(u_int8_t **);
@@ -118,7 +119,7 @@ static void des3_zerokey(u_int8_t **);
static void blf_zerokey(u_int8_t **);
static void cast5_zerokey(u_int8_t **);
static void skipjack_zerokey(u_int8_t **);
-static void rijndael128_zerokey(u_int8_t **);
+static void aes_zerokey(u_int8_t **);
static void cml_zerokey(u_int8_t **);
static void aes_ctr_zerokey(u_int8_t **);
static void aes_gmac_zerokey(u_int8_t **);
@@ -204,12 +205,12 @@ static const struct swcr_enc_xform swcr_
NULL
};
-static const struct swcr_enc_xform swcr_enc_xform_rijndael128 = {
+static const struct swcr_enc_xform swcr_enc_xform_aes = {
&enc_xform_rijndael128,
- rijndael128_encrypt,
- rijndael128_decrypt,
- rijndael128_setkey,
- rijndael128_zerokey,
+ aes_encrypt,
+ aes_decrypt,
+ aes_setkey,
+ aes_zerokey,
NULL
};
@@ -599,38 +600,68 @@ skipjack_zerokey(u_int8_t **sched)
*sched = NULL;
}
+struct aes_ctx {
+ struct aesenc enc;
+ struct aesdec dec;
+ uint32_t nr;
+};
+
static void
-rijndael128_encrypt(void *key, u_int8_t *blk)
+aes_encrypt(void *key, u_int8_t *blk)
{
- rijndael_encrypt((rijndael_ctx *) key, (u_char *) blk, (u_char *) blk);
+ struct aes_ctx *ctx = key;
+
+ aes_enc(&ctx->enc, blk, blk, ctx->nr);
}
static void
-rijndael128_decrypt(void *key, u_int8_t *blk)
+aes_decrypt(void *key, u_int8_t *blk)
{
- rijndael_decrypt((rijndael_ctx *) key, (u_char *) blk,
- (u_char *) blk);
+ struct aes_ctx *ctx = key;
+
+ aes_dec(&ctx->dec, blk, blk, ctx->nr);
}
static int
-rijndael128_setkey(u_int8_t **sched, const u_int8_t *key, int len)
+aes_setkey(u_int8_t **sched, const u_int8_t *key, int len)
{
+ struct aes_ctx *ctx;
if (len != 16 && len != 24 && len != 32)
return EINVAL;
- *sched = malloc(sizeof(rijndael_ctx), M_CRYPTO_DATA,
- M_NOWAIT|M_ZERO);
- if (*sched == NULL)
+ ctx = kmem_zalloc(sizeof(*ctx), KM_NOSLEEP);
+ if (ctx == NULL)
return ENOMEM;
- rijndael_set_key((rijndael_ctx *) *sched, key, len * 8);
+
+ switch (len) {
+ case 16:
+ aes_setenckey128(&ctx->enc, key);
+ aes_setdeckey128(&ctx->dec, key);
+ ctx->nr = AES_128_NROUNDS;
+ break;
+ case 24:
+ aes_setenckey192(&ctx->enc, key);
+ aes_setdeckey192(&ctx->dec, key);
+ ctx->nr = AES_192_NROUNDS;
+ break;
+ case 32:
+ aes_setenckey256(&ctx->enc, key);
+ aes_setdeckey256(&ctx->dec, key);
+ ctx->nr = AES_256_NROUNDS;
+ break;
+ }
+
+ *sched = (void *)ctx;
return 0;
}
static void
-rijndael128_zerokey(u_int8_t **sched)
+aes_zerokey(u_int8_t **sched)
{
- memset(*sched, 0, sizeof(rijndael_ctx));
- free(*sched, M_CRYPTO_DATA);
+ struct aes_ctx *ctx = (void *)*sched;
+
+ explicit_memset(ctx, 0, sizeof(*ctx));
+ kmem_free(ctx, sizeof(*ctx));
*sched = NULL;
}
@@ -678,7 +709,7 @@ cml_zerokey(u_int8_t **sched)
struct aes_ctr_ctx {
/* need only encryption half */
- u_int32_t ac_ek[4*(RIJNDAEL_MAXNR + 1)];
+ struct aesenc ac_ek;
u_int8_t ac_block[AESCTR_BLOCKSIZE];
int ac_nr;
struct {
@@ -699,10 +730,10 @@ aes_ctr_crypt(void *key, u_int8_t *blk)
i >= AESCTR_NONCESIZE + AESCTR_IVSIZE; i--)
if (++ctx->ac_block[i]) /* continue on overflow */
break;
- rijndaelEncrypt(ctx->ac_ek, ctx->ac_nr, ctx->ac_block, keystream);
+ aes_enc(&ctx->ac_ek, ctx->ac_block, keystream, ctx->ac_nr);
for (i = 0; i < AESCTR_BLOCKSIZE; i++)
blk[i] ^= keystream[i];
- memset(keystream, 0, sizeof(keystream));
+ explicit_memset(keystream, 0, sizeof(keystream));
}
int
@@ -713,13 +744,20 @@ aes_ctr_setkey(u_int8_t **sched, const u
if (len < AESCTR_NONCESIZE)
return EINVAL;
- ctx = malloc(sizeof(struct aes_ctr_ctx), M_CRYPTO_DATA,
- M_NOWAIT|M_ZERO);
+ ctx = kmem_zalloc(sizeof(*ctx), KM_NOSLEEP);
if (!ctx)
return ENOMEM;
- ctx->ac_nr = rijndaelKeySetupEnc(ctx->ac_ek, (const u_char *)key,
- (len - AESCTR_NONCESIZE) * 8);
- if (!ctx->ac_nr) { /* wrong key len */
+ switch (len) {
+ case 16 + AESCTR_NONCESIZE:
+ ctx->ac_nr = aes_setenckey128(&ctx->ac_ek, key);
+ break;
+ case 24 + AESCTR_NONCESIZE:
+ ctx->ac_nr = aes_setenckey192(&ctx->ac_ek, key);
+ break;
+ case 32 + AESCTR_NONCESIZE:
+ ctx->ac_nr = aes_setenckey256(&ctx->ac_ek, key);
+ break;
+ default:
aes_ctr_zerokey((u_int8_t **)&ctx);
return EINVAL;
}
@@ -733,9 +771,10 @@ aes_ctr_setkey(u_int8_t **sched, const u
void
aes_ctr_zerokey(u_int8_t **sched)
{
+ struct aes_ctr_ctx *ctx = (void *)*sched;
- memset(*sched, 0, sizeof(struct aes_ctr_ctx));
- free(*sched, M_CRYPTO_DATA);
+ explicit_memset(ctx, 0, sizeof(*ctx));
+ kmem_free(ctx, sizeof(*ctx));
*sched = NULL;
}
@@ -783,8 +822,7 @@ aes_gmac_setkey(u_int8_t **sched, const
{
struct aes_gmac_ctx *ctx;
- ctx = malloc(sizeof(struct aes_gmac_ctx), M_CRYPTO_DATA,
- M_NOWAIT|M_ZERO);
+ ctx = kmem_zalloc(sizeof(*ctx), KM_NOSLEEP);
if (!ctx)
return ENOMEM;
@@ -797,8 +835,9 @@ aes_gmac_setkey(u_int8_t **sched, const
void
aes_gmac_zerokey(u_int8_t **sched)
{
+ struct aes_gmac_ctx *ctx = (void *)*sched;
- free(*sched, M_CRYPTO_DATA);
+ kmem_free(ctx, sizeof(*ctx));
*sched = NULL;
}
diff -r b7131a05bde7 -r a97bc0abe60d sys/opencrypto/files.opencrypto
--- a/sys/opencrypto/files.opencrypto Sun Jun 14 19:57:23 2020 +0000
+++ b/sys/opencrypto/files.opencrypto Sun Jun 14 19:59:13 2020 +0000
@@ -7,7 +7,7 @@
# that use the opencrypto framework, should list opencrypto as a dependency
# to pull in the framework.
-define opencrypto: rijndael
+define opencrypto: aes
file opencrypto/criov.c opencrypto
file opencrypto/xform.c opencrypto
file opencrypto/crypto.c opencrypto
diff -r b7131a05bde7 -r a97bc0abe60d sys/opencrypto/gmac.c
--- a/sys/opencrypto/gmac.c Sun Jun 14 19:57:23 2020 +0000
+++ b/sys/opencrypto/gmac.c Sun Jun 14 19:59:13 2020 +0000
@@ -26,7 +26,8 @@
#include <sys/param.h>
#include <sys/systm.h>
-#include <crypto/rijndael/rijndael.h>
+#include <crypto/aes/aes.h>
+
#include <opencrypto/gmac.h>
void ghash_gfmul(const GMAC_INT *, const GMAC_INT *, GMAC_INT *);
@@ -114,13 +115,25 @@ AES_GMAC_Setkey(AES_GMAC_CTX *ctx, const
{
int i;
- ctx->rounds = rijndaelKeySetupEnc(ctx->K, (const u_char *)key,
- (klen - AESCTR_NONCESIZE) * 8);
+ switch (klen) {
+ case 16 + AESCTR_NONCESIZE:
+ ctx->rounds = aes_setenckey128(&ctx->K, key);
+ break;
+ case 24 + AESCTR_NONCESIZE:
+ ctx->rounds = aes_setenckey192(&ctx->K, key);
+ break;
+ case 32 + AESCTR_NONCESIZE:
+ ctx->rounds = aes_setenckey256(&ctx->K, key);
+ break;
+ default:
+ panic("invalid AES_GMAC_Setkey length in bytes: %u",
+ (unsigned)klen);
+ }
/* copy out salt to the counter block */
memcpy(ctx->J, key + klen - AESCTR_NONCESIZE, AESCTR_NONCESIZE);
/* prepare a hash subkey */
- rijndaelEncrypt(ctx->K, ctx->rounds, (void *)ctx->ghash.H,
- (void *)ctx->ghash.H);
+ aes_enc(&ctx->K, (const void *)ctx->ghash.H, (void *)ctx->ghash.H,
+ ctx->rounds);
#if GMAC_INTLEN == 8
for (i = 0; i < 2; i++)
ctx->ghash.H[i] = be64toh(ctx->ghash.H[i]);
@@ -163,7 +176,7 @@ AES_GMAC_Final(uint8_t digest[GMAC_DIGES
/* do one round of GCTR */
ctx->J[GMAC_BLOCK_LEN - 1] = 1;
- rijndaelEncrypt(ctx->K, ctx->rounds, ctx->J, keystream);
+ aes_enc(&ctx->K, ctx->J, keystream, ctx->rounds);
k = keystream;
d = digest;
#if GMAC_INTLEN == 8
diff -r b7131a05bde7 -r a97bc0abe60d sys/opencrypto/gmac.h
--- a/sys/opencrypto/gmac.h Sun Jun 14 19:57:23 2020 +0000
+++ b/sys/opencrypto/gmac.h Sun Jun 14 19:59:13 2020 +0000
@@ -20,7 +20,7 @@
#ifndef _GMAC_H_
#define _GMAC_H_
-#include <crypto/rijndael/rijndael.h>
+#include <crypto/aes/aes.h>
#define GMAC_BLOCK_LEN 16
#define GMAC_DIGEST_LEN 16
@@ -41,7 +41,7 @@ typedef struct _GHASH_CTX {
typedef struct _AES_GMAC_CTX {
GHASH_CTX ghash;
- uint32_t K[4*(RIJNDAEL_MAXNR + 1)];
+ struct aesenc K;
uint8_t J[GMAC_BLOCK_LEN]; /* counter block */
int rounds;
} AES_GMAC_CTX;
diff -r b7131a05bde7 -r a97bc0abe60d sys/opencrypto/xform.c
--- a/sys/opencrypto/xform.c Sun Jun 14 19:57:23 2020 +0000
+++ b/sys/opencrypto/xform.c Sun Jun 14 19:59:13 2020 +0000
@@ -145,8 +145,8 @@ const struct enc_xform enc_xform_skipjac
};
const struct enc_xform enc_xform_rijndael128 = {
- .type = CRYPTO_RIJNDAEL128_CBC,
- .name = "Rijndael-128/AES",
+ .type = CRYPTO_AES_CBC,
+ .name = "AES",
.blocksize = 16,
.ivsize = 16,
.minkey = 16,
# HG changeset patch
# User Taylor R Campbell <riastradh%NetBSD.org@localhost>
# Date 1592251492 0
# Mon Jun 15 20:04:52 2020 +0000
# Branch trunk
# Node ID 401917dcba81934295869fc7e7a3c4c7755ff186
# Parent a97bc0abe60d9a77b10f27d63951d60b0be7b987
# EXP-Topic riastradh-kernelcrypto
cgd(4): Print which key size is broken when a self-test fails.
Can be gleaned from the test index but this is a little quicker.
diff -r a97bc0abe60d -r 401917dcba81 sys/dev/cgd.c
--- a/sys/dev/cgd.c Sun Jun 14 19:59:13 2020 +0000
+++ b/sys/dev/cgd.c Mon Jun 15 20:04:52 2020 +0000
@@ -1699,8 +1699,8 @@ cgd_selftest(void)
if (memcmp(buf, selftests[i].ctxt, txtlen) != 0) {
hexdump(printf, "was", buf, txtlen);
hexdump(printf, "exp", selftests[i].ctxt, txtlen);
- panic("cgd %s encryption is broken [%zu]",
- selftests[i].alg, i);
+ panic("cgd %s-%d encryption is broken [%zu]",
+ selftests[i].alg, keylen, i);
}
cgd_cipher(&sc, buf, buf, txtlen, selftests[i].blkno,
@@ -1708,8 +1708,8 @@ cgd_selftest(void)
if (memcmp(buf, selftests[i].ptxt, txtlen) != 0) {
hexdump(printf, "was", buf, txtlen);
hexdump(printf, "exp", selftests[i].ptxt, txtlen);
- panic("cgd %s decryption is broken [%zu]",
- selftests[i].alg, i);
+ panic("cgd %s-%d decryption is broken [%zu]",
+ selftests[i].alg, keylen, i);
}
kmem_free(buf, txtlen);
# HG changeset patch
# User Taylor R Campbell <riastradh%NetBSD.org@localhost>
# Date 1592251571 0
# Mon Jun 15 20:06:11 2020 +0000
# Branch trunk
# Node ID 375cb5e0f08e74a884c537b40ac52fe31c512837
# Parent 401917dcba81934295869fc7e7a3c4c7755ff186
# EXP-Topic riastradh-kernelcrypto
cgd(4): Align IVs on the stack.
This will make it easier for some hardware crypto support.
diff -r 401917dcba81 -r 375cb5e0f08e sys/dev/cgd.c
--- a/sys/dev/cgd.c Mon Jun 15 20:04:52 2020 +0000
+++ b/sys/dev/cgd.c Mon Jun 15 20:06:11 2020 +0000
@@ -1587,7 +1587,7 @@ cgd_cipher(struct cgd_softc *sc, void *d
cfunc_cipher *cipher = sc->sc_cfuncs->cf_cipher;
size_t blocksize = sc->sc_cdata.cf_blocksize;
size_t todo;
- char blkno_buf[CGD_MAXBLOCKSIZE];
+ char blkno_buf[CGD_MAXBLOCKSIZE] __aligned(CGD_BLOCKALIGN);
DPRINTF_FOLLOW(("cgd_cipher() dir=%d\n", dir));
diff -r 401917dcba81 -r 375cb5e0f08e sys/dev/cgd_crypto.c
--- a/sys/dev/cgd_crypto.c Mon Jun 15 20:04:52 2020 +0000
+++ b/sys/dev/cgd_crypto.c Mon Jun 15 20:06:11 2020 +0000
@@ -167,7 +167,7 @@ cgd_cipher_aes_cbc(void *privdata, void
const void *blkno, int dir)
{
struct aes_privdata *apd = privdata;
- uint8_t iv[CGD_AES_BLOCK_SIZE] = {0};
+ uint8_t iv[CGD_AES_BLOCK_SIZE] __aligned(CGD_AES_BLOCK_SIZE) = {0};
/* Compute the CBC IV as AES_k(blkno). */
aes_enc(&apd->ap_enckey, blkno, iv, apd->ap_nrounds);
diff -r 401917dcba81 -r 375cb5e0f08e sys/dev/cgd_crypto.h
--- a/sys/dev/cgd_crypto.h Mon Jun 15 20:04:52 2020 +0000
+++ b/sys/dev/cgd_crypto.h Mon Jun 15 20:06:11 2020 +0000
@@ -39,6 +39,8 @@
#define CGD_3DES_BLOCK_SIZE 8
#define CGD_BF_BLOCK_SIZE 8
+#define CGD_BLOCKALIGN 16
+
typedef void *(cfunc_init)(size_t, const void *, size_t *);
typedef void (cfunc_destroy)(void *);
typedef void (cfunc_cipher)(void *, void *, const void *, size_t,
# HG changeset patch
# User Taylor R Campbell <riastradh%NetBSD.org@localhost>
# Date 1592237969 0
# Mon Jun 15 16:19:29 2020 +0000
# Branch trunk
# Node ID 28973955038a44907a800f3333d8dec03c77c8b2
# Parent 375cb5e0f08e74a884c537b40ac52fe31c512837
# EXP-Topic riastradh-kernelcrypto
Provide the standard AES key schedule.
Different AES implementations prefer different variations on it, but
some of them -- notably VIA -- require the standard key schedule to
be available and don't provide hardware support for computing it
themselves. So adapt BearSSL's logic to generate the standard key
schedule (and decryption keys, with InvMixColumns), rather than the
bitsliced key schedule that BearSSL uses natively.
diff -r 375cb5e0f08e -r 28973955038a sys/crypto/aes/aes_bear.h
--- a/sys/crypto/aes/aes_bear.h Mon Jun 15 20:06:11 2020 +0000
+++ b/sys/crypto/aes/aes_bear.h Mon Jun 15 16:19:29 2020 +0000
@@ -45,6 +45,12 @@ void br_aes_ct_skey_expand(uint32_t *, u
void br_aes_ct_bitslice_encrypt(unsigned, const uint32_t *, uint32_t *);
void br_aes_ct_bitslice_decrypt(unsigned, const uint32_t *, uint32_t *);
+/* NetBSD additions */
+
+void br_aes_ct_inv_mix_columns(uint32_t *);
+u_int br_aes_ct_keysched_stdenc(uint32_t *, const void *, size_t);
+u_int br_aes_ct_keysched_stddec(uint32_t *, const void *, size_t);
+
extern struct aes_impl aes_bear_impl;
#endif /* _CRYPTO_AES_AES_BEAR_H */
diff -r 375cb5e0f08e -r 28973955038a sys/crypto/aes/aes_ct.c
--- a/sys/crypto/aes/aes_ct.c Mon Jun 15 20:06:11 2020 +0000
+++ b/sys/crypto/aes/aes_ct.c Mon Jun 15 16:19:29 2020 +0000
@@ -29,6 +29,8 @@
#include <sys/types.h>
+#include <lib/libkern/libkern.h>
+
#include <crypto/aes/aes_bear.h>
/* see inner.h */
@@ -333,3 +335,92 @@ br_aes_ct_skey_expand(uint32_t *skey,
skey[v + 1] = y | (y >> 1);
}
}
+
+/* NetBSD additions, for computing the standard AES key schedule */
+
+unsigned
+br_aes_ct_keysched_stdenc(uint32_t *skey, const void *key, size_t key_len)
+{
+ unsigned num_rounds;
+ int i, j, k, nk, nkf;
+ uint32_t tmp;
+
+ switch (key_len) {
+ case 16:
+ num_rounds = 10;
+ break;
+ case 24:
+ num_rounds = 12;
+ break;
+ case 32:
+ num_rounds = 14;
+ break;
+ default:
+ /* abort(); */
+ return 0;
+ }
+ nk = (int)(key_len >> 2);
+ nkf = (int)((num_rounds + 1) << 2);
+ tmp = 0;
+ for (i = 0; i < nk; i ++) {
+ tmp = br_dec32le((const unsigned char *)key + (i << 2));
+ skey[i] = tmp;
+ }
+ for (i = nk, j = 0, k = 0; i < nkf; i ++) {
+ if (j == 0) {
+ tmp = (tmp << 24) | (tmp >> 8);
+ tmp = sub_word(tmp) ^ Rcon[k];
+ } else if (nk > 6 && j == 4) {
+ tmp = sub_word(tmp);
+ }
+ tmp ^= skey[i - nk];
+ skey[i] = tmp;
+ if (++ j == nk) {
+ j = 0;
+ k ++;
+ }
+ }
+ return num_rounds;
+}
+
+unsigned
+br_aes_ct_keysched_stddec(uint32_t *skey, const void *key, size_t key_len)
+{
+ uint32_t tkey[60];
+ uint32_t q[8];
+ unsigned num_rounds;
+ unsigned i;
+
+ num_rounds = br_aes_ct_keysched_stdenc(skey, key, key_len);
+ if (num_rounds == 0)
+ return 0;
+
+ tkey[0] = skey[4*num_rounds + 0];
+ tkey[1] = skey[4*num_rounds + 1];
+ tkey[2] = skey[4*num_rounds + 2];
+ tkey[3] = skey[4*num_rounds + 3];
+ for (i = 1; i < num_rounds; i++) {
+ q[2*0] = skey[4*i + 0];
+ q[2*1] = skey[4*i + 1];
+ q[2*2] = skey[4*i + 2];
+ q[2*3] = skey[4*i + 3];
+ q[1] = q[3] = q[5] = q[7] = 0;
+
+ br_aes_ct_ortho(q);
+ br_aes_ct_inv_mix_columns(q);
+ br_aes_ct_ortho(q);
+
+ tkey[4*(num_rounds - i) + 0] = q[2*0];
+ tkey[4*(num_rounds - i) + 1] = q[2*1];
+ tkey[4*(num_rounds - i) + 2] = q[2*2];
+ tkey[4*(num_rounds - i) + 3] = q[2*3];
+ }
+ tkey[4*num_rounds + 0] = skey[0];
+ tkey[4*num_rounds + 1] = skey[1];
+ tkey[4*num_rounds + 2] = skey[2];
+ tkey[4*num_rounds + 3] = skey[3];
+
+ memcpy(skey, tkey, 4*(num_rounds + 1)*sizeof(uint32_t));
+ explicit_memset(tkey, 0, 4*(num_rounds + 1)*sizeof(uint32_t));
+ return num_rounds;
+}
diff -r 375cb5e0f08e -r 28973955038a sys/crypto/aes/aes_ct_dec.c
--- a/sys/crypto/aes/aes_ct_dec.c Mon Jun 15 20:06:11 2020 +0000
+++ b/sys/crypto/aes/aes_ct_dec.c Mon Jun 15 16:19:29 2020 +0000
@@ -175,3 +175,11 @@ br_aes_ct_bitslice_decrypt(unsigned num_
br_aes_ct_bitslice_invSbox(q);
add_round_key(q, skey);
}
+
+/* NetBSD addition, for generating compatible decryption keys */
+void
+br_aes_ct_inv_mix_columns(uint32_t *q)
+{
+
+ inv_mix_columns(q);
+}
diff -r 375cb5e0f08e -r 28973955038a sys/crypto/aes/aes_impl.c
--- a/sys/crypto/aes/aes_impl.c Mon Jun 15 20:06:11 2020 +0000
+++ b/sys/crypto/aes/aes_impl.c Mon Jun 15 16:19:29 2020 +0000
@@ -38,6 +38,8 @@
#include <crypto/aes/aes.h>
#include <crypto/aes/aes_bear.h> /* default implementation */
+static int aes_selftest_stdkeysched(void);
+
static const struct aes_impl *aes_md_impl __read_mostly;
static const struct aes_impl *aes_impl __read_mostly;
@@ -61,6 +63,9 @@ aes_select(void)
KASSERT(aes_impl == NULL);
+ if (aes_selftest_stdkeysched())
+ panic("AES is busted");
+
if (aes_md_impl) {
if (aes_selftest(aes_md_impl))
aprint_error("aes: self-test failed: %s\n",
@@ -254,3 +259,131 @@ aes_xts_dec(struct aesdec *dec, const ui
aes_guarantee_selected();
aes_impl->ai_xts_dec(dec, in, out, nbytes, tweak, nrounds);
}
+
+/*
+ * Known-answer self-tests for the standard key schedule.
+ */
+static int
+aes_selftest_stdkeysched(void)
+{
+ static const uint8_t key[32] = {
+ 0x00,0x01,0x02,0x03,0x04,0x05,0x06,0x07,
+ 0x08,0x09,0x0a,0x0b,0x0c,0x0d,0x0e,0x0f,
+ 0x10,0x11,0x12,0x13,0x14,0x15,0x16,0x17,
+ 0x18,0x19,0x1a,0x1b,0x1c,0x1d,0x1e,0x1f,
+ };
+ static const uint32_t rk128enc[] = {
+ 0x03020100, 0x07060504, 0x0b0a0908, 0x0f0e0d0c,
+ 0xfd74aad6, 0xfa72afd2, 0xf178a6da, 0xfe76abd6,
+ 0x0bcf92b6, 0xf1bd3d64, 0x00c59bbe, 0xfeb33068,
+ 0x4e74ffb6, 0xbfc9c2d2, 0xbf0c596c, 0x41bf6904,
+ 0xbcf7f747, 0x033e3595, 0xbc326cf9, 0xfd8d05fd,
+ 0xe8a3aa3c, 0xeb9d9fa9, 0x57aff350, 0xaa22f6ad,
+ 0x7d0f395e, 0x9692a6f7, 0xc13d55a7, 0x6b1fa30a,
+ 0x1a70f914, 0x8ce25fe3, 0x4ddf0a44, 0x26c0a94e,
+ 0x35874347, 0xb9651ca4, 0xf4ba16e0, 0xd27abfae,
+ 0xd1329954, 0x685785f0, 0x9ced9310, 0x4e972cbe,
+ 0x7f1d1113, 0x174a94e3, 0x8ba707f3, 0xc5302b4d,
+ };
+ static const uint32_t rk192enc[] = {
+ 0x03020100, 0x07060504, 0x0b0a0908, 0x0f0e0d0c,
+ 0x13121110, 0x17161514, 0xf9f24658, 0xfef4435c,
+ 0xf5fe4a54, 0xfaf04758, 0xe9e25648, 0xfef4435c,
+ 0xb349f940, 0x4dbdba1c, 0xb843f048, 0x42b3b710,
+ 0xab51e158, 0x55a5a204, 0x41b5ff7e, 0x0c084562,
+ 0xb44bb52a, 0xf6f8023a, 0x5da9e362, 0x080c4166,
+ 0x728501f5, 0x7e8d4497, 0xcac6f1bd, 0x3c3ef387,
+ 0x619710e5, 0x699b5183, 0x9e7c1534, 0xe0f151a3,
+ 0x2a37a01e, 0x16095399, 0x779e437c, 0x1e0512ff,
+ 0x880e7edd, 0x68ff2f7e, 0x42c88f60, 0x54c1dcf9,
+ 0x235f9f85, 0x3d5a8d7a, 0x5229c0c0, 0x3ad6efbe,
+ 0x781e60de, 0x2cdfbc27, 0x0f8023a2, 0x32daaed8,
+ 0x330a97a4, 0x09dc781a, 0x71c218c4, 0x5d1da4e3,
+ };
+ static const uint32_t rk256enc[] = {
+ 0x03020100, 0x07060504, 0x0b0a0908, 0x0f0e0d0c,
+ 0x13121110, 0x17161514, 0x1b1a1918, 0x1f1e1d1c,
+ 0x9fc273a5, 0x98c476a1, 0x93ce7fa9, 0x9cc072a5,
+ 0xcda85116, 0xdabe4402, 0xc1a45d1a, 0xdeba4006,
+ 0xf0df87ae, 0x681bf10f, 0xfbd58ea6, 0x6715fc03,
+ 0x48f1e16d, 0x924fa56f, 0x53ebf875, 0x8d51b873,
+ 0x7f8256c6, 0x1799a7c9, 0xec4c296f, 0x8b59d56c,
+ 0x753ae23d, 0xe7754752, 0xb49ebf27, 0x39cf0754,
+ 0x5f90dc0b, 0x48097bc2, 0xa44552ad, 0x2f1c87c1,
+ 0x60a6f545, 0x87d3b217, 0x334d0d30, 0x0a820a64,
+ 0x1cf7cf7c, 0x54feb4be, 0xf0bbe613, 0xdfa761d2,
+ 0xfefa1af0, 0x7929a8e7, 0x4a64a5d7, 0x40e6afb3,
+ 0x71fe4125, 0x2500f59b, 0xd5bb1388, 0x0a1c725a,
+ 0x99665a4e, 0xe04ff2a9, 0xaa2b577e, 0xeacdf8cd,
+ 0xcc79fc24, 0xe97909bf, 0x3cc21a37, 0x36de686d,
+ };
+ static const uint32_t rk128dec[] = {
+ 0x7f1d1113, 0x174a94e3, 0x8ba707f3, 0xc5302b4d,
+ 0xbe29aa13, 0xf6af8f9c, 0x80f570f7, 0x03bff700,
+ 0x63a46213, 0x4886258f, 0x765aff6b, 0x834a87f7,
+ 0x74fc828d, 0x2b22479c, 0x3edcdae4, 0xf510789c,
+ 0x8d09e372, 0x5fdec511, 0x15fe9d78, 0xcbcca278,
+ 0x2710c42e, 0xd2d72663, 0x4a205869, 0xde323f00,
+ 0x04f5a2a8, 0xf5c7e24d, 0x98f77e0a, 0x94126769,
+ 0x91e3c6c7, 0xf13240e5, 0x6d309c47, 0x0ce51963,
+ 0x9902dba0, 0x60d18622, 0x9c02dca2, 0x61d58524,
+ 0xf0df568c, 0xf9d35d82, 0xfcd35a80, 0xfdd75986,
+ 0x03020100, 0x07060504, 0x0b0a0908, 0x0f0e0d0c,
+ };
+ static const uint32_t rk192dec[] = {
+ 0x330a97a4, 0x09dc781a, 0x71c218c4, 0x5d1da4e3,
+ 0x0dbdbed6, 0x49ea09c2, 0x8073b04d, 0xb91b023e,
+ 0xc999b98f, 0x3968b273, 0x9dd8f9c7, 0x728cc685,
+ 0xc16e7df7, 0xef543f42, 0x7f317853, 0x4457b714,
+ 0x90654711, 0x3b66cf47, 0x8dce0e9b, 0xf0f10bfc,
+ 0xb6a8c1dc, 0x7d3f0567, 0x4a195ccc, 0x2e3a42b5,
+ 0xabb0dec6, 0x64231e79, 0xbe5f05a4, 0xab038856,
+ 0xda7c1bdd, 0x155c8df2, 0x1dab498a, 0xcb97c4bb,
+ 0x08f7c478, 0xd63c8d31, 0x01b75596, 0xcf93c0bf,
+ 0x10efdc60, 0xce249529, 0x15efdb62, 0xcf20962f,
+ 0xdbcb4e4b, 0xdacf4d4d, 0xc7d75257, 0xdecb4949,
+ 0x1d181f1a, 0x191c1b1e, 0xd7c74247, 0xdecb4949,
+ 0x03020100, 0x07060504, 0x0b0a0908, 0x0f0e0d0c,
+ };
+ static const uint32_t rk256dec[] = {
+ 0xcc79fc24, 0xe97909bf, 0x3cc21a37, 0x36de686d,
+ 0xffd1f134, 0x2faacebf, 0x5fe2e9fc, 0x6e015825,
+ 0xeb48165e, 0x0a354c38, 0x46b77175, 0x84e680dc,
+ 0x8005a3c8, 0xd07b3f8b, 0x70482743, 0x31e3b1d9,
+ 0x138e70b5, 0xe17d5a66, 0x4c823d4d, 0xc251f1a9,
+ 0xa37bda74, 0x507e9c43, 0xa03318c8, 0x41ab969a,
+ 0x1597a63c, 0xf2f32ad3, 0xadff672b, 0x8ed3cce4,
+ 0xf3c45ff8, 0xf3054637, 0xf04d848b, 0xe1988e52,
+ 0x9a4069de, 0xe7648cef, 0x5f0c4df8, 0x232cabcf,
+ 0x1658d5ae, 0x00c119cf, 0x0348c2bc, 0x11d50ad9,
+ 0xbd68c615, 0x7d24e531, 0xb868c117, 0x7c20e637,
+ 0x0f85d77f, 0x1699cc61, 0x0389db73, 0x129dc865,
+ 0xc940282a, 0xc04c2324, 0xc54c2426, 0xc4482720,
+ 0x1d181f1a, 0x191c1b1e, 0x15101712, 0x11141316,
+ 0x03020100, 0x07060504, 0x0b0a0908, 0x0f0e0d0c,
+ };
+ static const struct {
+ unsigned len;
+ unsigned nr;
+ const uint32_t *enc, *dec;
+ } C[] = {
+ { 16, AES_128_NROUNDS, rk128enc, rk128dec },
+ { 24, AES_192_NROUNDS, rk192enc, rk192dec },
+ { 32, AES_256_NROUNDS, rk256enc, rk256dec },
+ };
+ uint32_t rk[60];
+ unsigned i;
+
+ for (i = 0; i < __arraycount(C); i++) {
+ if (br_aes_ct_keysched_stdenc(rk, key, C[i].len) != C[i].nr)
+ return -1;
+ if (memcmp(rk, C[i].enc, 4*(C[i].nr + 1)))
+ return -1;
+ if (br_aes_ct_keysched_stddec(rk, key, C[i].len) != C[i].nr)
+ return -1;
+ if (memcmp(rk, C[i].dec, 4*(C[i].nr + 1)))
+ return -1;
+ }
+
+ return 0;
+}
# HG changeset patch
# User Taylor R Campbell <riastradh%NetBSD.org@localhost>
# Date 1592238453 0
# Mon Jun 15 16:27:33 2020 +0000
# Branch trunk
# Node ID 86fed1861ac3279e6d19505769e4331842fea55c
# Parent 28973955038a44907a800f3333d8dec03c77c8b2
# EXP-Topic riastradh-kernelcrypto
Add AES implementation with VIA ACE.
diff -r 28973955038a -r 86fed1861ac3 sys/arch/x86/conf/files.x86
--- a/sys/arch/x86/conf/files.x86 Mon Jun 15 16:19:29 2020 +0000
+++ b/sys/arch/x86/conf/files.x86 Mon Jun 15 16:27:33 2020 +0000
@@ -168,3 +168,6 @@ file arch/x86/pci/pci_addr_fixup.c pci_a
# AES-NI
include "crypto/aes/arch/x86/files.aesni"
+
+# VIA ACE
+include "crypto/aes/arch/x86/files.aesvia"
diff -r 28973955038a -r 86fed1861ac3 sys/arch/x86/x86/identcpu.c
--- a/sys/arch/x86/x86/identcpu.c Mon Jun 15 16:19:29 2020 +0000
+++ b/sys/arch/x86/x86/identcpu.c Mon Jun 15 16:27:33 2020 +0000
@@ -40,6 +40,7 @@
#include <sys/cpu.h>
#include <crypto/aes/arch/x86/aes_ni.h>
+#include <crypto/aes/arch/x86/aes_via.h>
#include <uvm/uvm_extern.h>
@@ -1000,7 +1001,10 @@ cpu_probe(struct cpu_info *ci)
#ifdef __x86_64__ /* not yet implemented on i386 */
if (cpu_feature[1] & CPUID2_AES)
aes_md_init(&aes_ni_impl);
+ else
#endif
+ if (cpu_feature[4] & CPUID_VIA_HAS_ACE)
+ aes_md_init(&aes_via_impl);
} else {
/*
* If not first. Warn about cpu_feature mismatch for
diff -r 28973955038a -r 86fed1861ac3 sys/crypto/aes/arch/x86/aes_via.c
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/sys/crypto/aes/arch/x86/aes_via.c Mon Jun 15 16:27:33 2020 +0000
@@ -0,0 +1,626 @@
+/* $NetBSD$ */
+
+/*-
+ * Copyright (c) 2020 The NetBSD Foundation, Inc.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND CONTRIBUTORS
+ * ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
+ * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+ * POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <sys/cdefs.h>
+__KERNEL_RCSID(1, "$NetBSD$");
+
+#include <sys/types.h>
+#include <sys/evcnt.h>
+#include <sys/systm.h>
+
+#include <crypto/aes/aes.h>
+#include <crypto/aes/aes_bear.h>
+
+#include <x86/cpufunc.h>
+#include <x86/cpuvar.h>
+#include <x86/fpu.h>
+#include <x86/specialreg.h>
+#include <x86/via_padlock.h>
+
+static void
+aesvia_reload_keys(void)
+{
+
+ asm volatile("pushf; popf");
+}
+
+static uint32_t
+aesvia_keylen_cw0(unsigned nrounds)
+{
+
+ /*
+ * Determine the control word bits for the key size / number of
+ * rounds. For AES-128, the hardware can do key expansion on
+ * the fly; for AES-192 and AES-256, software must do it.
+ */
+ switch (nrounds) {
+ case AES_128_NROUNDS:
+ return C3_CRYPT_CWLO_KEY128;
+ case AES_192_NROUNDS:
+ return C3_CRYPT_CWLO_KEY192 | C3_CRYPT_CWLO_KEYGEN_SW;
+ case AES_256_NROUNDS:
+ return C3_CRYPT_CWLO_KEY256 | C3_CRYPT_CWLO_KEYGEN_SW;
+ default:
+ panic("invalid AES nrounds: %u", nrounds);
+ }
+}
+
+static void
+aesvia_setenckey(struct aesenc *enc, const uint8_t *key, uint32_t nrounds)
+{
+ size_t key_len;
+
+ switch (nrounds) {
+ case AES_128_NROUNDS:
+ enc->aese_aes.aes_rk[0] = le32dec(key + 4*0);
+ enc->aese_aes.aes_rk[1] = le32dec(key + 4*1);
+ enc->aese_aes.aes_rk[2] = le32dec(key + 4*2);
+ enc->aese_aes.aes_rk[3] = le32dec(key + 4*3);
+ return;
+ case AES_192_NROUNDS:
+ key_len = 24;
+ break;
+ case AES_256_NROUNDS:
+ key_len = 32;
+ break;
+ default:
+ panic("invalid AES nrounds: %u", nrounds);
+ }
+ br_aes_ct_keysched_stdenc(enc->aese_aes.aes_rk, key, key_len);
+}
+
+static void
+aesvia_setdeckey(struct aesdec *dec, const uint8_t *key, uint32_t nrounds)
+{
+ size_t key_len;
+
+ switch (nrounds) {
+ case AES_128_NROUNDS:
+ dec->aesd_aes.aes_rk[0] = le32dec(key + 4*0);
+ dec->aesd_aes.aes_rk[1] = le32dec(key + 4*1);
+ dec->aesd_aes.aes_rk[2] = le32dec(key + 4*2);
+ dec->aesd_aes.aes_rk[3] = le32dec(key + 4*3);
+ return;
+ case AES_192_NROUNDS:
+ key_len = 24;
+ break;
+ case AES_256_NROUNDS:
+ key_len = 32;
+ break;
+ default:
+ panic("invalid AES nrounds: %u", nrounds);
+ }
+ br_aes_ct_keysched_stddec(dec->aesd_aes.aes_rk, key, key_len);
+}
+
+static inline void
+aesvia_enc1(const struct aesenc *enc, const uint8_t in[static 16],
+ uint8_t out[static 16], uint32_t cw0)
+{
+ const uint32_t cw[4] __aligned(16) = {
+ [0] = (cw0
+ | C3_CRYPT_CWLO_ALG_AES
+ | C3_CRYPT_CWLO_ENCRYPT
+ | C3_CRYPT_CWLO_NORMAL),
+ };
+ size_t nblocks = 1;
+
+ KASSERT(((uintptr_t)enc & 0xf) == 0);
+ KASSERT(((uintptr_t)in & 0xf) == 0);
+ KASSERT(((uintptr_t)out & 0xf) == 0);
+
+ asm volatile("rep xcrypt-ecb"
+ : "+c"(nblocks), "+S"(in), "+D"(out)
+ : "b"(enc), "d"(cw)
+ : "memory", "cc");
+}
+
+static inline void
+aesvia_dec1(const struct aesdec *dec, const uint8_t in[static 16],
+ uint8_t out[static 16], uint32_t cw0)
+{
+ const uint32_t cw[4] __aligned(16) = {
+ [0] = (cw0
+ | C3_CRYPT_CWLO_ALG_AES
+ | C3_CRYPT_CWLO_DECRYPT
+ | C3_CRYPT_CWLO_NORMAL),
+ };
+ size_t nblocks = 1;
+
+ KASSERT(((uintptr_t)dec & 0xf) == 0);
+ KASSERT(((uintptr_t)in & 0xf) == 0);
+ KASSERT(((uintptr_t)out & 0xf) == 0);
+
+ asm volatile("rep xcrypt-ecb"
+ : "+c"(nblocks), "+S"(in), "+D"(out)
+ : "b"(dec), "d"(cw)
+ : "memory", "cc");
+}
+
+static struct evcnt enc_aligned_evcnt = EVCNT_INITIALIZER(EVCNT_TYPE_MISC,
+ NULL, "aesvia", "enc aligned");
+EVCNT_ATTACH_STATIC(enc_aligned_evcnt);
+static struct evcnt enc_unaligned_evcnt = EVCNT_INITIALIZER(EVCNT_TYPE_MISC,
+ NULL, "aesvia", "dec unaligned");
+EVCNT_ATTACH_STATIC(enc_unaligned_evcnt);
+
+static void
+aesvia_enc(const struct aesenc *enc, const uint8_t in[static 16],
+ uint8_t out[static 16], uint32_t nrounds)
+{
+ const uint32_t cw0 = aesvia_keylen_cw0(nrounds);
+
+ fpu_kern_enter();
+ aesvia_reload_keys();
+ if ((((uintptr_t)in | (uintptr_t)out) & 0xf) == 0 &&
+ ((uintptr_t)in & 0xff0) != 0xff0) {
+ enc_aligned_evcnt.ev_count++;
+ aesvia_enc1(enc, in, out, cw0);
+ } else {
+ enc_unaligned_evcnt.ev_count++;
+ /*
+ * VIA requires 16-byte/128-bit alignment, and
+ * xcrypt-ecb reads one block past the one we're
+ * working on -- which may go past the end of the page
+ * into unmapped territory. Use a bounce buffer if
+ * either constraint is violated.
+ */
+ uint8_t inbuf[16] __aligned(16);
+ uint8_t outbuf[16] __aligned(16);
+
+ memcpy(inbuf, in, 16);
+ aesvia_enc1(enc, inbuf, outbuf, cw0);
+ memcpy(out, outbuf, 16);
+
+ explicit_memset(inbuf, 0, sizeof inbuf);
+ explicit_memset(outbuf, 0, sizeof outbuf);
+ }
+ fpu_kern_leave();
+}
+
+static struct evcnt dec_aligned_evcnt = EVCNT_INITIALIZER(EVCNT_TYPE_MISC,
+ NULL, "aesvia", "dec aligned");
+EVCNT_ATTACH_STATIC(dec_aligned_evcnt);
+static struct evcnt dec_unaligned_evcnt = EVCNT_INITIALIZER(EVCNT_TYPE_MISC,
+ NULL, "aesvia", "dec unaligned");
+EVCNT_ATTACH_STATIC(dec_unaligned_evcnt);
+
+static void
+aesvia_dec(const struct aesdec *dec, const uint8_t in[static 16],
+ uint8_t out[static 16], uint32_t nrounds)
+{
+ const uint32_t cw0 = aesvia_keylen_cw0(nrounds);
+
+ fpu_kern_enter();
+ aesvia_reload_keys();
+ if ((((uintptr_t)in | (uintptr_t)out) & 0xf) == 0 &&
+ ((uintptr_t)in & 0xff0) != 0xff0) {
+ dec_aligned_evcnt.ev_count++;
+ aesvia_dec1(dec, in, out, cw0);
+ } else {
+ dec_unaligned_evcnt.ev_count++;
+ /*
+ * VIA requires 16-byte/128-bit alignment, and
+ * xcrypt-ecb reads one block past the one we're
+ * working on -- which may go past the end of the page
+ * into unmapped territory. Use a bounce buffer if
+ * either constraint is violated.
+ */
+ uint8_t inbuf[16] __aligned(16);
+ uint8_t outbuf[16] __aligned(16);
+
+ memcpy(inbuf, in, 16);
+ aesvia_dec1(dec, inbuf, outbuf, cw0);
+ memcpy(out, outbuf, 16);
+
+ explicit_memset(inbuf, 0, sizeof inbuf);
+ explicit_memset(outbuf, 0, sizeof outbuf);
+ }
+ fpu_kern_leave();
+}
+
+static inline void
+aesvia_cbc_enc1(const struct aesenc *enc, const uint8_t in[static 16],
+ uint8_t out[static 16], size_t nblocks, uint8_t **ivp, uint32_t cw0)
+{
+ const uint32_t cw[4] __aligned(16) = {
+ [0] = (cw0
+ | C3_CRYPT_CWLO_ALG_AES
+ | C3_CRYPT_CWLO_ENCRYPT
+ | C3_CRYPT_CWLO_NORMAL),
+ };
+
+ KASSERT(((uintptr_t)enc & 0xf) == 0);
+ KASSERT(((uintptr_t)in & 0xf) == 0);
+ KASSERT(((uintptr_t)out & 0xf) == 0);
+ KASSERT(((uintptr_t)*ivp & 0xf) == 0);
+
+ /*
+ * Register effects:
+ * - Counts nblocks down to zero.
+ * - Advances in by nblocks (units of blocks).
+ * - Advances out by nblocks (units of blocks).
+ * - Updates *ivp to point at the last block of out.
+ */
+ asm volatile("rep xcrypt-cbc"
+ : "+c"(nblocks), "+S"(in), "+D"(out), "+a"(*ivp)
+ : "b"(enc), "d"(cw)
+ : "memory", "cc");
+}
+
+static inline void
+aesvia_cbc_dec1(const struct aesdec *dec, const uint8_t in[static 16],
+ uint8_t out[static 16], size_t nblocks, uint8_t iv[static 16],
+ uint32_t cw0)
+{
+ const uint32_t cw[4] __aligned(16) = {
+ [0] = (cw0
+ | C3_CRYPT_CWLO_ALG_AES
+ | C3_CRYPT_CWLO_DECRYPT
+ | C3_CRYPT_CWLO_NORMAL),
+ };
+
+ KASSERT(((uintptr_t)dec & 0xf) == 0);
+ KASSERT(((uintptr_t)in & 0xf) == 0);
+ KASSERT(((uintptr_t)out & 0xf) == 0);
+ KASSERT(((uintptr_t)iv & 0xf) == 0);
+
+ /*
+ * Register effects:
+ * - Counts nblocks down to zero.
+ * - Advances in by nblocks (units of blocks).
+ * - Advances out by nblocks (units of blocks).
+ * Memory side effects:
+ * - Writes what was the last block of in at the address iv.
+ */
+ asm volatile("rep xcrypt-cbc"
+ : "+c"(nblocks), "+S"(in), "+D"(out)
+ : "a"(iv), "b"(dec), "d"(cw)
+ : "memory", "cc");
+}
+
+static inline void
+xor128(void *x, const void *a, const void *b)
+{
+ uint32_t *x32 = x;
+ const uint32_t *a32 = a;
+ const uint32_t *b32 = b;
+
+ x32[0] = a32[0] ^ b32[0];
+ x32[1] = a32[1] ^ b32[1];
+ x32[2] = a32[2] ^ b32[2];
+ x32[3] = a32[3] ^ b32[3];
+}
+
+static struct evcnt cbcenc_aligned_evcnt = EVCNT_INITIALIZER(EVCNT_TYPE_MISC,
+ NULL, "aesvia", "cbcenc aligned");
+EVCNT_ATTACH_STATIC(cbcenc_aligned_evcnt);
+static struct evcnt cbcenc_unaligned_evcnt = EVCNT_INITIALIZER(EVCNT_TYPE_MISC,
+ NULL, "aesvia", "cbcenc unaligned");
+EVCNT_ATTACH_STATIC(cbcenc_unaligned_evcnt);
+
+static void
+aesvia_cbc_enc(const struct aesenc *enc, const uint8_t in[static 16],
+ uint8_t out[static 16], size_t nbytes, uint8_t iv[static 16],
+ uint32_t nrounds)
+{
+ const uint32_t cw0 = aesvia_keylen_cw0(nrounds);
+
+ KASSERT(nbytes % 16 == 0);
+ if (nbytes == 0)
+ return;
+
+ fpu_kern_enter();
+ aesvia_reload_keys();
+ if ((((uintptr_t)in | (uintptr_t)out | (uintptr_t)iv) & 0xf) == 0) {
+ cbcenc_aligned_evcnt.ev_count++;
+ uint8_t *ivp = iv;
+ aesvia_cbc_enc1(enc, in, out, nbytes/16, &ivp, cw0);
+ memcpy(iv, ivp, 16);
+ } else {
+ cbcenc_unaligned_evcnt.ev_count++;
+ uint8_t cv[16] __aligned(16);
+ uint8_t tmp[16] __aligned(16);
+
+ memcpy(cv, iv, 16);
+ for (; nbytes; nbytes -= 16, in += 16, out += 16) {
+ memcpy(tmp, in, 16);
+ xor128(tmp, tmp, cv);
+ aesvia_enc1(enc, tmp, cv, cw0);
+ memcpy(out, cv, 16);
+ }
+ memcpy(iv, cv, 16);
+ }
+ fpu_kern_leave();
+}
+
+static struct evcnt cbcdec_aligned_evcnt = EVCNT_INITIALIZER(EVCNT_TYPE_MISC,
+ NULL, "aesvia", "cbcdec aligned");
+EVCNT_ATTACH_STATIC(cbcdec_aligned_evcnt);
+static struct evcnt cbcdec_unaligned_evcnt = EVCNT_INITIALIZER(EVCNT_TYPE_MISC,
+ NULL, "aesvia", "cbcdec unaligned");
+EVCNT_ATTACH_STATIC(cbcdec_unaligned_evcnt);
+
+static void
+aesvia_cbc_dec(const struct aesdec *dec, const uint8_t in[static 16],
+ uint8_t out[static 16], size_t nbytes, uint8_t iv[static 16],
+ uint32_t nrounds)
+{
+ const uint32_t cw0 = aesvia_keylen_cw0(nrounds);
+
+ KASSERT(nbytes % 16 == 0);
+ if (nbytes == 0)
+ return;
+
+ fpu_kern_enter();
+ aesvia_reload_keys();
+ if ((((uintptr_t)in | (uintptr_t)out | (uintptr_t)iv) & 0xf) == 0) {
+ cbcdec_aligned_evcnt.ev_count++;
+ aesvia_cbc_dec1(dec, in, out, nbytes/16, iv, cw0);
+ } else {
+ cbcdec_unaligned_evcnt.ev_count++;
+ uint8_t iv0[16] __aligned(16);
+ uint8_t cv[16] __aligned(16);
+ uint8_t tmp[16] __aligned(16);
+
+ memcpy(iv0, iv, 16);
+ memcpy(cv, in + nbytes - 16, 16);
+ memcpy(iv, cv, 16);
+
+ for (;;) {
+ aesvia_dec1(dec, cv, tmp, cw0);
+ if ((nbytes -= 16) == 0)
+ break;
+ memcpy(cv, in + nbytes - 16, 16);
+ xor128(tmp, tmp, cv);
+ memcpy(out + nbytes, tmp, 16);
+ }
+
+ xor128(tmp, tmp, iv0);
+ memcpy(out, tmp, 16);
+ explicit_memset(tmp, 0, sizeof tmp);
+ }
+ fpu_kern_leave();
+}
+
+static inline void
+aesvia_xts_update(uint32_t *t0, uint32_t *t1, uint32_t *t2, uint32_t *t3)
+{
+ uint32_t s0, s1, s2, s3;
+
+ s0 = *t0 >> 31;
+ s1 = *t1 >> 31;
+ s2 = *t2 >> 31;
+ s3 = *t3 >> 31;
+ *t0 = (*t0 << 1) ^ (-s3 & 0x87);
+ *t1 = (*t1 << 1) ^ s0;
+ *t2 = (*t2 << 1) ^ s1;
+ *t3 = (*t3 << 1) ^ s2;
+}
+
+static int
+aesvia_xts_update_selftest(void)
+{
+ static const struct {
+ uint32_t in[4], out[4];
+ } cases[] = {
+ { {1}, {2} },
+ { {0x80000000U,0,0,0}, {0,1,0,0} },
+ { {0,0x80000000U,0,0}, {0,0,1,0} },
+ { {0,0,0x80000000U,0}, {0,0,0,1} },
+ { {0,0,0,0x80000000U}, {0x87,0,0,0} },
+ { {0,0x80000000U,0,0x80000000U}, {0x87,0,1,0} },
+ };
+ unsigned i;
+ uint32_t t0, t1, t2, t3;
+
+ for (i = 0; i < sizeof(cases)/sizeof(cases[0]); i++) {
+ t0 = cases[i].in[0];
+ t1 = cases[i].in[1];
+ t2 = cases[i].in[2];
+ t3 = cases[i].in[3];
+ aesvia_xts_update(&t0, &t1, &t2, &t3);
+ if (t0 != cases[i].out[0] ||
+ t1 != cases[i].out[1] ||
+ t2 != cases[i].out[2] ||
+ t3 != cases[i].out[3])
+ return -1;
+ }
+
+ /* Success! */
+ return 0;
+}
+
+static struct evcnt xtsenc_aligned_evcnt = EVCNT_INITIALIZER(EVCNT_TYPE_MISC,
+ NULL, "aesvia", "xtsenc aligned");
+EVCNT_ATTACH_STATIC(xtsenc_aligned_evcnt);
+static struct evcnt xtsenc_unaligned_evcnt = EVCNT_INITIALIZER(EVCNT_TYPE_MISC,
+ NULL, "aesvia", "xtsenc unaligned");
+EVCNT_ATTACH_STATIC(xtsenc_unaligned_evcnt);
+
+static void
+aesvia_xts_enc(const struct aesenc *enc, const uint8_t in[static 16],
+ uint8_t out[static 16], size_t nbytes, uint8_t tweak[static 16],
+ uint32_t nrounds)
+{
+ const uint32_t cw0 = aesvia_keylen_cw0(nrounds);
+ uint32_t t[4];
+
+ KASSERT(nbytes % 16 == 0);
+
+ memcpy(t, tweak, 16);
+
+ fpu_kern_enter();
+ aesvia_reload_keys();
+ if ((((uintptr_t)in | (uintptr_t)out) & 0xf) == 0) {
+ xtsenc_aligned_evcnt.ev_count++;
+ unsigned lastblock = 0;
+
+ /*
+ * Make sure the last block is not the last block of a
+ * page. (Note that we store the AES input in `out' as
+ * a temporary buffer, rather than reading it directly
+ * from `in', since we have to combine the tweak
+ * first.)
+ */
+ lastblock = 16*(((uintptr_t)(out + nbytes) & 0xfff) == 0);
+ nbytes -= lastblock;
+
+ for (; nbytes; nbytes -= 16, in += 16, out += 16) {
+ xor128(out, in, t);
+ aesvia_enc1(enc, out, out, cw0);
+ xor128(out, out, t);
+ aesvia_xts_update(&t[0], &t[1], &t[2], &t[3]);
+ }
+
+ /* Handle the last block of a page, if necessary. */
+ if (lastblock) {
+ uint8_t buf[16] __aligned(16);
+ xor128(buf, in, t);
+ aesvia_enc1(enc, buf, out, cw0);
+ explicit_memset(buf, 0, sizeof buf);
+ }
+ } else {
+ xtsenc_unaligned_evcnt.ev_count++;
+ uint8_t buf[16] __aligned(16);
+
+ for (; nbytes; nbytes -= 16, in += 16, out += 16) {
+ memcpy(buf, in, 16);
+ xor128(buf, buf, t);
+ aesvia_enc1(enc, buf, buf, cw0);
+ xor128(buf, buf, t);
+ memcpy(out, buf, 16);
+ aesvia_xts_update(&t[0], &t[1], &t[2], &t[3]);
+ }
+
+ explicit_memset(buf, 0, sizeof buf);
+ }
+ fpu_kern_leave();
+
+ memcpy(tweak, t, 16);
+ explicit_memset(t, 0, sizeof t);
+}
+
+static struct evcnt xtsdec_aligned_evcnt = EVCNT_INITIALIZER(EVCNT_TYPE_MISC,
+ NULL, "aesvia", "xtsdec aligned");
+EVCNT_ATTACH_STATIC(xtsdec_aligned_evcnt);
+static struct evcnt xtsdec_unaligned_evcnt = EVCNT_INITIALIZER(EVCNT_TYPE_MISC,
+ NULL, "aesvia", "xtsdec unaligned");
+EVCNT_ATTACH_STATIC(xtsdec_unaligned_evcnt);
+
+static void
+aesvia_xts_dec(const struct aesdec *dec, const uint8_t in[static 16],
+ uint8_t out[static 16], size_t nbytes, uint8_t tweak[static 16],
+ uint32_t nrounds)
+{
+ const uint32_t cw0 = aesvia_keylen_cw0(nrounds);
+ uint32_t t[4];
+
+ KASSERT(nbytes % 16 == 0);
+
+ memcpy(t, tweak, 16);
+
+ fpu_kern_enter();
+ aesvia_reload_keys();
+ if ((((uintptr_t)in | (uintptr_t)out) & 0xf) == 0) {
+ xtsdec_aligned_evcnt.ev_count++;
+ unsigned lastblock = 0;
+
+ /*
+ * Make sure the last block is not the last block of a
+ * page. (Note that we store the AES input in `out' as
+ * a temporary buffer, rather than reading it directly
+ * from `in', since we have to combine the tweak
+ * first.)
+ */
+ lastblock = 16*(((uintptr_t)(out + nbytes) & 0xfff) == 0);
+ nbytes -= lastblock;
+
+ for (; nbytes; nbytes -= 16, in += 16, out += 16) {
+ xor128(out, in, t);
+ aesvia_dec1(dec, out, out, cw0);
+ xor128(out, out, t);
+ aesvia_xts_update(&t[0], &t[1], &t[2], &t[3]);
+ }
+
+ /* Handle the last block of a page, if necessary. */
+ if (lastblock) {
+ uint8_t buf[16] __aligned(16);
+ xor128(buf, in, t);
+ aesvia_dec1(dec, buf, out, cw0);
+ explicit_memset(buf, 0, sizeof buf);
+ }
+ } else {
+ xtsdec_unaligned_evcnt.ev_count++;
+ uint8_t buf[16] __aligned(16);
+
+ for (; nbytes; nbytes -= 16, in += 16, out += 16) {
+ memcpy(buf, in, 16);
+ xor128(buf, buf, t);
+ aesvia_dec1(dec, buf, buf, cw0);
+ xor128(buf, buf, t);
+ memcpy(out, buf, 16);
+ aesvia_xts_update(&t[0], &t[1], &t[2], &t[3]);
+ }
+
+ explicit_memset(buf, 0, sizeof buf);
+ }
+ fpu_kern_leave();
+
+ memcpy(tweak, t, 16);
+ explicit_memset(t, 0, sizeof t);
+}
+
+static int
+aesvia_probe(void)
+{
+
+ /* Verify that the CPU advertises VIA ACE support. */
+ if ((cpu_feature[4] & CPUID_VIA_HAS_ACE) == 0)
+ return -1;
+
+ /* Verify that our XTS tweak update logic works. */
+ if (aesvia_xts_update_selftest())
+ return -1;
+
+ /* Success! */
+ return 0;
+}
+
+struct aes_impl aes_via_impl = {
+ .ai_name = "VIA ACE",
+ .ai_probe = aesvia_probe,
+ .ai_setenckey = aesvia_setenckey,
+ .ai_setdeckey = aesvia_setdeckey,
+ .ai_enc = aesvia_enc,
+ .ai_dec = aesvia_dec,
+ .ai_cbc_enc = aesvia_cbc_enc,
+ .ai_cbc_dec = aesvia_cbc_dec,
+ .ai_xts_enc = aesvia_xts_enc,
+ .ai_xts_dec = aesvia_xts_dec,
+};
diff -r 28973955038a -r 86fed1861ac3 sys/crypto/aes/arch/x86/aes_via.h
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/sys/crypto/aes/arch/x86/aes_via.h Mon Jun 15 16:27:33 2020 +0000
@@ -0,0 +1,36 @@
+/* $NetBSD$ */
+
+/*-
+ * Copyright (c) 2020 The NetBSD Foundation, Inc.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND CONTRIBUTORS
+ * ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
+ * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+ * POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _CRYPTO_AES_ARCH_X86_AES_VIA_H
+#define _CRYPTO_AES_ARCH_X86_AES_VIA_H
+
+#include <crypto/aes/aes.h>
+
+extern struct aes_impl aes_via_impl;
+
+#endif /* _CRYPTO_AES_ARCH_X86_AES_VIA_H */
diff -r 28973955038a -r 86fed1861ac3 sys/crypto/aes/arch/x86/files.aesvia
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/sys/crypto/aes/arch/x86/files.aesvia Mon Jun 15 16:27:33 2020 +0000
@@ -0,0 +1,3 @@
+# $NetBSD$
+
+file crypto/aes/arch/x86/aes_via.c aes
# HG changeset patch
# User Taylor R Campbell <riastradh%NetBSD.org@localhost>
# Date 1592258370 0
# Mon Jun 15 21:59:30 2020 +0000
# Branch trunk
# Node ID 1ff6250fd07e4ed180d6b61958c9e7ae3667d4f7
# Parent 86fed1861ac3279e6d19505769e4331842fea55c
# EXP-Topic riastradh-kernelcrypto
uvm: Make sure swap encryption IV is 128-bit-aligned on stack.
Will help hardware-assisted AES.
diff -r 86fed1861ac3 -r 1ff6250fd07e sys/uvm/uvm_swap.c
--- a/sys/uvm/uvm_swap.c Mon Jun 15 16:27:33 2020 +0000
+++ b/sys/uvm/uvm_swap.c Mon Jun 15 21:59:30 2020 +0000
@@ -2089,7 +2089,7 @@ uvm_swap_genkey(struct swapdev *sdp)
static void
uvm_swap_encryptpage(struct swapdev *sdp, void *kva, int slot)
{
- uint8_t preiv[16] = {0}, iv[16];
+ uint8_t preiv[16] __aligned(16) = {0}, iv[16] __aligned(16);
/* iv := AES_k(le32enc(slot) || 0^96) */
le32enc(preiv, slot);
@@ -2111,7 +2111,7 @@ uvm_swap_encryptpage(struct swapdev *sdp
static void
uvm_swap_decryptpage(struct swapdev *sdp, void *kva, int slot)
{
- uint8_t preiv[16] = {0}, iv[16];
+ uint8_t preiv[16] __aligned(16) = {0}, iv[16] __aligned(16);
/* iv := AES_k(le32enc(slot) || 0^96) */
le32enc(preiv, slot);
# HG changeset patch
# User Taylor R Campbell <riastradh%NetBSD.org@localhost>
# Date 1592261759 0
# Mon Jun 15 22:55:59 2020 +0000
# Branch trunk
# Node ID 36794fee0d0481ed3f3253e8d4ef6b87c96c13b7
# Parent 1ff6250fd07e4ed180d6b61958c9e7ae3667d4f7
# EXP-Topic riastradh-kernelcrypto
Batch AES-XTS computation into eight blocks at a time.
Experimental -- performance improvement is not clearly worth the
complexity.
diff -r 1ff6250fd07e -r 36794fee0d04 sys/crypto/aes/arch/x86/aes_via.c
--- a/sys/crypto/aes/arch/x86/aes_via.c Mon Jun 15 21:59:30 2020 +0000
+++ b/sys/crypto/aes/arch/x86/aes_via.c Mon Jun 15 22:55:59 2020 +0000
@@ -119,8 +119,8 @@ aesvia_setdeckey(struct aesdec *dec, con
}
static inline void
-aesvia_enc1(const struct aesenc *enc, const uint8_t in[static 16],
- uint8_t out[static 16], uint32_t cw0)
+aesvia_encN(const struct aesenc *enc, const uint8_t in[static 16],
+ uint8_t out[static 16], size_t nblocks, uint32_t cw0)
{
const uint32_t cw[4] __aligned(16) = {
[0] = (cw0
@@ -128,7 +128,6 @@ aesvia_enc1(const struct aesenc *enc, co
| C3_CRYPT_CWLO_ENCRYPT
| C3_CRYPT_CWLO_NORMAL),
};
- size_t nblocks = 1;
KASSERT(((uintptr_t)enc & 0xf) == 0);
KASSERT(((uintptr_t)in & 0xf) == 0);
@@ -141,8 +140,8 @@ aesvia_enc1(const struct aesenc *enc, co
}
static inline void
-aesvia_dec1(const struct aesdec *dec, const uint8_t in[static 16],
- uint8_t out[static 16], uint32_t cw0)
+aesvia_decN(const struct aesdec *dec, const uint8_t in[static 16],
+ uint8_t out[static 16], size_t nblocks, uint32_t cw0)
{
const uint32_t cw[4] __aligned(16) = {
[0] = (cw0
@@ -150,7 +149,6 @@ aesvia_dec1(const struct aesdec *dec, co
| C3_CRYPT_CWLO_DECRYPT
| C3_CRYPT_CWLO_NORMAL),
};
- size_t nblocks = 1;
KASSERT(((uintptr_t)dec & 0xf) == 0);
KASSERT(((uintptr_t)in & 0xf) == 0);
@@ -180,7 +178,7 @@ aesvia_enc(const struct aesenc *enc, con
if ((((uintptr_t)in | (uintptr_t)out) & 0xf) == 0 &&
((uintptr_t)in & 0xff0) != 0xff0) {
enc_aligned_evcnt.ev_count++;
- aesvia_enc1(enc, in, out, cw0);
+ aesvia_encN(enc, in, out, 1, cw0);
} else {
enc_unaligned_evcnt.ev_count++;
/*
@@ -194,7 +192,7 @@ aesvia_enc(const struct aesenc *enc, con
uint8_t outbuf[16] __aligned(16);
memcpy(inbuf, in, 16);
- aesvia_enc1(enc, inbuf, outbuf, cw0);
+ aesvia_encN(enc, inbuf, outbuf, 1, cw0);
memcpy(out, outbuf, 16);
explicit_memset(inbuf, 0, sizeof inbuf);
@@ -221,7 +219,7 @@ aesvia_dec(const struct aesdec *dec, con
if ((((uintptr_t)in | (uintptr_t)out) & 0xf) == 0 &&
((uintptr_t)in & 0xff0) != 0xff0) {
dec_aligned_evcnt.ev_count++;
- aesvia_dec1(dec, in, out, cw0);
+ aesvia_decN(dec, in, out, 1, cw0);
} else {
dec_unaligned_evcnt.ev_count++;
/*
@@ -235,7 +233,7 @@ aesvia_dec(const struct aesdec *dec, con
uint8_t outbuf[16] __aligned(16);
memcpy(inbuf, in, 16);
- aesvia_dec1(dec, inbuf, outbuf, cw0);
+ aesvia_decN(dec, inbuf, outbuf, 1, cw0);
memcpy(out, outbuf, 16);
explicit_memset(inbuf, 0, sizeof inbuf);
@@ -245,7 +243,7 @@ aesvia_dec(const struct aesdec *dec, con
}
static inline void
-aesvia_cbc_enc1(const struct aesenc *enc, const uint8_t in[static 16],
+aesvia_cbc_encN(const struct aesenc *enc, const uint8_t in[static 16],
uint8_t out[static 16], size_t nblocks, uint8_t **ivp, uint32_t cw0)
{
const uint32_t cw[4] __aligned(16) = {
@@ -274,7 +272,7 @@ aesvia_cbc_enc1(const struct aesenc *enc
}
static inline void
-aesvia_cbc_dec1(const struct aesdec *dec, const uint8_t in[static 16],
+aesvia_cbc_decN(const struct aesdec *dec, const uint8_t in[static 16],
uint8_t out[static 16], size_t nblocks, uint8_t iv[static 16],
uint32_t cw0)
{
@@ -340,7 +338,7 @@ aesvia_cbc_enc(const struct aesenc *enc,
if ((((uintptr_t)in | (uintptr_t)out | (uintptr_t)iv) & 0xf) == 0) {
cbcenc_aligned_evcnt.ev_count++;
uint8_t *ivp = iv;
- aesvia_cbc_enc1(enc, in, out, nbytes/16, &ivp, cw0);
+ aesvia_cbc_encN(enc, in, out, nbytes/16, &ivp, cw0);
memcpy(iv, ivp, 16);
} else {
cbcenc_unaligned_evcnt.ev_count++;
@@ -351,7 +349,7 @@ aesvia_cbc_enc(const struct aesenc *enc,
for (; nbytes; nbytes -= 16, in += 16, out += 16) {
memcpy(tmp, in, 16);
xor128(tmp, tmp, cv);
- aesvia_enc1(enc, tmp, cv, cw0);
+ aesvia_encN(enc, tmp, cv, 1, cw0);
memcpy(out, cv, 16);
}
memcpy(iv, cv, 16);
@@ -381,7 +379,7 @@ aesvia_cbc_dec(const struct aesdec *dec,
aesvia_reload_keys();
if ((((uintptr_t)in | (uintptr_t)out | (uintptr_t)iv) & 0xf) == 0) {
cbcdec_aligned_evcnt.ev_count++;
- aesvia_cbc_dec1(dec, in, out, nbytes/16, iv, cw0);
+ aesvia_cbc_decN(dec, in, out, nbytes/16, iv, cw0);
} else {
cbcdec_unaligned_evcnt.ev_count++;
uint8_t iv0[16] __aligned(16);
@@ -393,7 +391,7 @@ aesvia_cbc_dec(const struct aesdec *dec,
memcpy(iv, cv, 16);
for (;;) {
- aesvia_dec1(dec, cv, tmp, cw0);
+ aesvia_decN(dec, cv, tmp, 1, cw0);
if ((nbytes -= 16) == 0)
break;
memcpy(cv, in + nbytes - 16, 16);
@@ -480,6 +478,7 @@ aesvia_xts_enc(const struct aesenc *enc,
if ((((uintptr_t)in | (uintptr_t)out) & 0xf) == 0) {
xtsenc_aligned_evcnt.ev_count++;
unsigned lastblock = 0;
+ uint32_t buf[8*4] __aligned(16);
/*
* Make sure the last block is not the last block of a
@@ -491,20 +490,43 @@ aesvia_xts_enc(const struct aesenc *enc,
lastblock = 16*(((uintptr_t)(out + nbytes) & 0xfff) == 0);
nbytes -= lastblock;
- for (; nbytes; nbytes -= 16, in += 16, out += 16) {
- xor128(out, in, t);
- aesvia_enc1(enc, out, out, cw0);
- xor128(out, out, t);
- aesvia_xts_update(&t[0], &t[1], &t[2], &t[3]);
+ /*
+ * Handle an odd number of initial blocks so we can
+ * process the rest in eight-block (128-byte) chunks.
+ */
+ if (nbytes % 128) {
+ unsigned nbytes128 = nbytes % 128;
+
+ nbytes -= nbytes128;
+ for (; nbytes128; nbytes128 -= 16, in += 16, out += 16)
+ {
+ xor128(out, in, t);
+ aesvia_encN(enc, out, out, 1, cw0);
+ xor128(out, out, t);
+ aesvia_xts_update(&t[0], &t[1], &t[2], &t[3]);
+ }
+ }
+
+ /* Process eight blocks at a time. */
+ for (; nbytes; nbytes -= 128, in += 128, out += 128) {
+ unsigned i;
+ for (i = 0; i < 8; i++) {
+ memcpy(buf + 4*i, t, 16);
+ xor128(out + 4*i, in + 4*i, t);
+ aesvia_xts_update(&t[0], &t[1], &t[2], &t[3]);
+ }
+ aesvia_encN(enc, out, out, 8, cw0);
+ for (i = 0; i < 8; i++)
+ xor128(out + 4*i, in + 4*i, buf + 4*i);
}
/* Handle the last block of a page, if necessary. */
if (lastblock) {
- uint8_t buf[16] __aligned(16);
xor128(buf, in, t);
- aesvia_enc1(enc, buf, out, cw0);
- explicit_memset(buf, 0, sizeof buf);
+ aesvia_encN(enc, (const void *)buf, out, 1, cw0);
}
+
+ explicit_memset(buf, 0, sizeof buf);
} else {
xtsenc_unaligned_evcnt.ev_count++;
uint8_t buf[16] __aligned(16);
@@ -512,7 +534,7 @@ aesvia_xts_enc(const struct aesenc *enc,
for (; nbytes; nbytes -= 16, in += 16, out += 16) {
memcpy(buf, in, 16);
xor128(buf, buf, t);
- aesvia_enc1(enc, buf, buf, cw0);
+ aesvia_encN(enc, buf, buf, 1, cw0);
xor128(buf, buf, t);
memcpy(out, buf, 16);
aesvia_xts_update(&t[0], &t[1], &t[2], &t[3]);
@@ -550,6 +572,7 @@ aesvia_xts_dec(const struct aesdec *dec,
if ((((uintptr_t)in | (uintptr_t)out) & 0xf) == 0) {
xtsdec_aligned_evcnt.ev_count++;
unsigned lastblock = 0;
+ uint32_t buf[8*4] __aligned(16);
/*
* Make sure the last block is not the last block of a
@@ -561,20 +584,43 @@ aesvia_xts_dec(const struct aesdec *dec,
lastblock = 16*(((uintptr_t)(out + nbytes) & 0xfff) == 0);
nbytes -= lastblock;
- for (; nbytes; nbytes -= 16, in += 16, out += 16) {
- xor128(out, in, t);
- aesvia_dec1(dec, out, out, cw0);
- xor128(out, out, t);
- aesvia_xts_update(&t[0], &t[1], &t[2], &t[3]);
+ /*
+ * Handle an odd number of initial blocks so we can
+ * process the rest in eight-block (128-byte) chunks.
+ */
+ if (nbytes % 128) {
+ unsigned nbytes128 = nbytes % 128;
+
+ nbytes -= nbytes128;
+ for (; nbytes128; nbytes128 -= 16, in += 16, out += 16)
+ {
+ xor128(out, in, t);
+ aesvia_decN(dec, out, out, 1, cw0);
+ xor128(out, out, t);
+ aesvia_xts_update(&t[0], &t[1], &t[2], &t[3]);
+ }
+ }
+
+ /* Process eight blocks at a time. */
+ for (; nbytes; nbytes -= 128, in += 128, out += 128) {
+ unsigned i;
+ for (i = 0; i < 8; i++) {
+ memcpy(buf + 4*i, t, 16);
+ xor128(out + 4*i, in + 4*i, t);
+ aesvia_xts_update(&t[0], &t[1], &t[2], &t[3]);
+ }
+ aesvia_decN(dec, out, out, 8, cw0);
+ for (i = 0; i < 8; i++)
+ xor128(out + 4*i, in + 4*i, buf + 4*i);
}
/* Handle the last block of a page, if necessary. */
if (lastblock) {
- uint8_t buf[16] __aligned(16);
xor128(buf, in, t);
- aesvia_dec1(dec, buf, out, cw0);
- explicit_memset(buf, 0, sizeof buf);
+ aesvia_decN(dec, (const void *)buf, out, 1, cw0);
}
+
+ explicit_memset(buf, 0, sizeof buf);
} else {
xtsdec_unaligned_evcnt.ev_count++;
uint8_t buf[16] __aligned(16);
@@ -582,7 +628,7 @@ aesvia_xts_dec(const struct aesdec *dec,
for (; nbytes; nbytes -= 16, in += 16, out += 16) {
memcpy(buf, in, 16);
xor128(buf, buf, t);
- aesvia_dec1(dec, buf, buf, cw0);
+ aesvia_decN(dec, buf, buf, 1, cw0);
xor128(buf, buf, t);
memcpy(out, buf, 16);
aesvia_xts_update(&t[0], &t[1], &t[2], &t[3]);
# HG changeset patch
# User Taylor R Campbell <riastradh%NetBSD.org@localhost>
# Date 1592362063 0
# Wed Jun 17 02:47:43 2020 +0000
# Branch trunk
# Node ID 9fde04e138c10fd0fca4362c7d93fd3ef4b325ad
# Parent 36794fee0d0481ed3f3253e8d4ef6b87c96c13b7
# EXP-Topic riastradh-kernelcrypto
New cgd cipher adiantum.
Adiantum is a wide-block cipher, built out of AES, XChaCha12,
Poly1305, and NH, defined in
Paul Crowley and Eric Biggers, `Adiantum: length-preserving
encryption for entry-level processors', IACR Transactions on
Symmetric Cryptology 2018(4), pp. 39--61.
Adiantum provides better security than a narrow-block cipher with CBC
or XTS, because every bit of each sector affects every other bit,
whereas with CBC each block of plaintext only affects the following
blocks of ciphertext in the disk sector, and with XTS each block of
plaintext only affects its own block of ciphertext and nothing else.
Adiantum generally provides much better performance than
constant-time AES-CBC or AES-XTS software do without hardware
support, and performance comparable to or better than the
variable-time (i.e., leaky) AES-CBC and AES-XTS software we had
before. (Note: Adiantum also uses AES as a subroutine, but only once
per disk sector. It takes only a small fraction of the time spent by
Adiantum, so there's relatively little performance impact to using
constant-time AES software over using variable-time AES software for
it.)
Adiantum naturally scales to essentially arbitrary disk sector sizes;
sizes >=1024-bytes take the most advantage of Adiantum's design for
performance, so 4096-byte sectors would be a natural choice if we
taught cgd to change the disk sector size. (However, it's a
different cipher for each disk sector size, so it _must_ be a cgd
parameter.)
The paper presents a similar construction HPolyC. The salient
difference is that HPolyC uses Poly1305 directly, whereas Adiantum
uses Poly1395(NH(...)). NH is annoying because it requires a
1072-byte key, which means the test vectors are ginormous, and
changing keys is costly; HPolyC avoids these shortcomings by using
Poly1305 directly, but HPolyC is measurably slower, costing about
1.5x what Adiantum costs on 4096-byte sectors.
For the purposes of cgd, we will reuse each key for many messages,
and there will be very few keys in total (one per cgd volume) so --
except for the annoying verbosity of test vectors -- the tradeoff
weighs in the favour of Adiantum, especially if we teach cgd to do
>>512-byte sectors.
For now, everything that Adiantum needs beyond what's already in the
kernel is gathered into a single file, including NH, Poly1305, and
XChaCha12. We can split those out -- and reuse them, and provide MD
tuned implementations, and so on -- as needed; this is just a first
pass to get Adiantum implemented for experimentation.
diff -r 36794fee0d04 -r 9fde04e138c1 sys/conf/files
--- a/sys/conf/files Mon Jun 15 22:55:59 2020 +0000
+++ b/sys/conf/files Wed Jun 17 02:47:43 2020 +0000
@@ -200,6 +200,7 @@ defflag opt_machdep.h MACHDEP
# use it.
# Individual crypto transforms
+include "crypto/adiantum/files.adiantum"
include "crypto/aes/files.aes"
include "crypto/des/files.des"
include "crypto/blowfish/files.blowfish"
@@ -1395,7 +1396,7 @@ file dev/ic/amdccp.c amdccp
defpseudodev vnd: disk
defflag opt_vnd.h VND_COMPRESSION
defpseudo ccd: disk
-defpseudodev cgd: disk, des, blowfish, cast128, aes
+defpseudodev cgd: disk, des, blowfish, cast128, aes, adiantum
defpseudodev md: disk
defpseudodev fss: disk
diff -r 36794fee0d04 -r 9fde04e138c1 sys/crypto/adiantum/adiantum.c
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/sys/crypto/adiantum/adiantum.c Wed Jun 17 02:47:43 2020 +0000
@@ -0,0 +1,2316 @@
+/* $NetBSD$ */
+
+/*-
+ * Copyright (c) 2020 The NetBSD Foundation, Inc.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND CONTRIBUTORS
+ * ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
+ * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+ * POSSIBILITY OF SUCH DAMAGE.
+ */
+
+/*
+ * The Adiantum wide-block cipher, from
+ *
+ * Paul Crowley and Eric Biggers, `Adiantum: length-preserving
+ * encryption for entry-level processors', IACR Transactions on
+ * Symmetric Cryptology 2018(4), pp. 39--61.
+ *
+ * https://doi.org/10.13154/tosc.v2018.i4.39-61
+ */
+
+#include <sys/cdefs.h>
+__KERNEL_RCSID(1, "$NetBSD$");
+
+#include <sys/types.h>
+#include <sys/endian.h>
+
+#ifdef _KERNEL
+
+#include <sys/module.h>
+#include <sys/systm.h>
+
+#include <lib/libkern/libkern.h>
+
+#include <crypto/adiantum/adiantum.h>
+#include <crypto/aes/aes.h>
+
+#else /* !defined(_KERNEL) */
+
+#include <sys/cdefs.h>
+
+#include <assert.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <string.h>
+
+#include <openssl/aes.h>
+
+struct aesenc {
+ AES_KEY enckey;
+};
+
+struct aesdec {
+ AES_KEY deckey;
+};
+
+#define AES_256_NROUNDS 14
+#define aes_setenckey256(E, K) AES_set_encrypt_key((K), 256, &(E)->enckey)
+#define aes_setdeckey256(D, K) AES_set_decrypt_key((K), 256, &(D)->deckey)
+#define aes_enc(E, P, C, NR) AES_encrypt(P, C, &(E)->enckey)
+#define aes_dec(D, C, P, NR) AES_decrypt(C, P, &(D)->deckey)
+
+#include "adiantum.h"
+
+#define CTASSERT __CTASSERT
+#define KASSERT assert
+#define MIN(x,y) ((x) < (y) ? (x) : (y))
+
+static void
+hexdump(int (*prf)(const char *, ...) __printflike(1,2), const char *prefix,
+ const void *buf, size_t len)
+{
+ const uint8_t *p = buf;
+ size_t i;
+
+ (*prf)("%s (%zu bytes)\n", prefix, len);
+ for (i = 0; i < len; i++) {
+ if (i % 16 == 8)
+ (*prf)(" ");
+ else
+ (*prf)(" ");
+ (*prf)("%02hhx", p[i]);
+ if ((i + 1) % 16 == 0)
+ (*prf)("\n");
+ }
+ if (i % 16)
+ (*prf)("\n");
+}
+
+#endif /* _KERNEL */
+
+/* Arithmetic modulo 2^128, represented by 16-digit strings in radix 2^8. */
+
+/* s := a + b (mod 2^128) */
+static inline void
+add128(uint8_t s[restrict static 16],
+ const uint8_t a[static 16], const uint8_t b[static 16])
+{
+ unsigned i, c;
+
+ c = 0;
+ for (i = 0; i < 16; i++) {
+ c = a[i] + b[i] + c;
+ s[i] = c & 0xff;
+ c >>= 8;
+ }
+}
+
+/* s := a - b (mod 2^128) */
+static inline void
+sub128(uint8_t d[restrict static 16],
+ const uint8_t a[static 16], const uint8_t b[static 16])
+{
+ unsigned i, c;
+
+ c = 0;
+ for (i = 0; i < 16; i++) {
+ c = a[i] - b[i] - c;
+ d[i] = c & 0xff;
+ c = 1 & (c >> 8);
+ }
+}
+
+static int
+addsub128_selftest(void)
+{
+ static const uint8_t zero[16] = {
+ 0x00,0x00,0x00,0x00, 0x00,0x00,0x00,0x00,
+ 0x00,0x00,0x00,0x00, 0x00,0x00,0x00,0x00,
+ };
+ static const uint8_t one[16] = {
+ 0x01,0x00,0x00,0x00, 0x00,0x00,0x00,0x00,
+ 0x00,0x00,0x00,0x00, 0x00,0x00,0x00,0x00,
+ };
+ static const uint8_t negativeone[16] = {
+ 0xff,0xff,0xff,0xff, 0xff,0xff,0xff,0xff,
+ 0xff,0xff,0xff,0xff, 0xff,0xff,0xff,0xff,
+ };
+ static const uint8_t a[16] = {
+ 0x03,0x80,0x00,0x00, 0x00,0x00,0x00,0x00,
+ 0x00,0x00,0x00,0x00, 0x00,0x00,0x00,0x00,
+ };
+ static const uint8_t b[16] = {
+ 0x01,0x82,0x00,0x00, 0x00,0x00,0x00,0x00,
+ 0x00,0x00,0x00,0x00, 0x00,0x00,0x00,0x00,
+ };
+ static const uint8_t c[16] = {
+ 0x02,0xfe,0xff,0xff, 0xff,0xff,0xff,0xff,
+ 0xff,0xff,0xff,0xff, 0xff,0xff,0xff,0xff,
+ };
+ uint8_t r[16];
+ int result = 0;
+
+ sub128(r, zero, one);
+ if (memcmp(r, negativeone, 16)) {
+ hexdump(printf, "sub128 1", r, sizeof r);
+ result = -1;
+ }
+
+ sub128(r, a, b);
+ if (memcmp(r, c, 16)) {
+ hexdump(printf, "sub128 2", r, sizeof r);
+ result = -1;
+ }
+
+ return result;
+}
+
+/* Poly1305 */
+
+struct poly1305 {
+ uint32_t r[5]; /* evaluation point */
+ uint32_t h[5]; /* value */
+};
+
+static void
+poly1305_init(struct poly1305 *P, const uint8_t key[static 16])
+{
+
+ /* clamp */
+ P->r[0] = (le32dec(key + 0) >> 0) & 0x03ffffff;
+ P->r[1] = (le32dec(key + 3) >> 2) & 0x03ffff03;
+ P->r[2] = (le32dec(key + 6) >> 4) & 0x03ffc0ff;
+ P->r[3] = (le32dec(key + 9) >> 6) & 0x03f03fff;
+ P->r[4] = (le32dec(key + 12) >> 8) & 0x000fffff;
+
+ /* initialize polynomial evaluation */
+ P->h[0] = P->h[1] = P->h[2] = P->h[3] = P->h[4] = 0;
+}
+
+static void
+poly1305_update_internal(struct poly1305 *P, const uint8_t m[static 16],
+ uint32_t pad)
+{
+ uint32_t r0 = P->r[0];
+ uint32_t r1 = P->r[1];
+ uint32_t r2 = P->r[2];
+ uint32_t r3 = P->r[3];
+ uint32_t r4 = P->r[4];
+ uint32_t h0 = P->h[0];
+ uint32_t h1 = P->h[1];
+ uint32_t h2 = P->h[2];
+ uint32_t h3 = P->h[3];
+ uint32_t h4 = P->h[4];
+ uint64_t k0, k1, k2, k3, k4; /* 64-bit extension of h */
+ uint64_t p0, p1, p2, p3, p4; /* columns of product */
+ uint32_t c; /* carry */
+
+ /* h' := h + m */
+ h0 += (le32dec(m + 0) >> 0) & 0x03ffffff;
+ h1 += (le32dec(m + 3) >> 2) & 0x03ffffff;
+ h2 += (le32dec(m + 6) >> 4) & 0x03ffffff;
+ h3 += (le32dec(m + 9) >> 6);
+ h4 += (le32dec(m + 12) >> 8) | (pad << 24);
+
+ /* extend to 64 bits */
+ k0 = h0;
+ k1 = h1;
+ k2 = h2;
+ k3 = h3;
+ k4 = h4;
+
+ /* p := h' * r = (h + m)*r mod 2^130 - 5 */
+ p0 = r0*k0 + 5*r4*k1 + 5*r3*k2 + 5*r2*k3 + 5*r1*k4;
+ p1 = r1*k0 + r0*k1 + 5*r4*k2 + 5*r3*k3 + 5*r2*k4;
+ p2 = r2*k0 + r1*k1 + r0*k2 + 5*r4*k3 + 5*r3*k4;
+ p3 = r3*k0 + r2*k1 + r1*k2 + r0*k3 + 5*r4*k4;
+ p4 = r4*k0 + r3*k1 + r2*k2 + r1*k3 + r0*k4;
+
+ /* propagate carries */
+ p0 += 0; c = p0 >> 26; h0 = p0 & 0x03ffffff;
+ p1 += c; c = p1 >> 26; h1 = p1 & 0x03ffffff;
+ p2 += c; c = p2 >> 26; h2 = p2 & 0x03ffffff;
+ p3 += c; c = p3 >> 26; h3 = p3 & 0x03ffffff;
+ p4 += c; c = p4 >> 26; h4 = p4 & 0x03ffffff;
+
+ /* reduce 2^130 = 5 */
+ h0 += c*5; c = h0 >> 26; h0 &= 0x03ffffff;
+ h1 += c;
+
+ /* update hash values */
+ P->h[0] = h0;
+ P->h[1] = h1;
+ P->h[2] = h2;
+ P->h[3] = h3;
+ P->h[4] = h4;
+}
+
+static void
+poly1305_update_block(struct poly1305 *P, const uint8_t m[static 16])
+{
+
+ poly1305_update_internal(P, m, 1);
+}
+
+static void
+poly1305_update_last(struct poly1305 *P, const uint8_t *m, size_t mlen)
+{
+ uint8_t buf[16];
+ unsigned i;
+
+ if (mlen == 16) {
+ poly1305_update_internal(P, m, 1);
+ return;
+ }
+
+ for (i = 0; i < mlen; i++)
+ buf[i] = m[i];
+ buf[i++] = 1;
+ for (; i < 16; i++)
+ buf[i] = 0;
+ poly1305_update_internal(P, buf, 0);
+}
+
+static void
+poly1305_final(uint8_t *h, struct poly1305 *P)
+{
+ uint32_t h0 = P->h[0];
+ uint32_t h1 = P->h[1];
+ uint32_t h2 = P->h[2];
+ uint32_t h3 = P->h[3];
+ uint32_t h4 = P->h[4];
+ uint32_t s0, s1, s2, s3, s4; /* h - (2^130 - 5) */
+ uint32_t m; /* mask */
+ uint32_t c;
+
+ /* propagate carries */
+ h1 += 0; c = h1 >> 26; h1 &= 0x03ffffff;
+ h2 += c; c = h2 >> 26; h2 &= 0x03ffffff;
+ h3 += c; c = h3 >> 26; h3 &= 0x03ffffff;
+ h4 += c; c = h4 >> 26; h4 &= 0x03ffffff;
+
+ /* reduce 2^130 = 5 */
+ h0 += c*5; c = h0 >> 26; h0 &= 0x03ffffff;
+ h1 += c;
+
+ /* s := h - (2^130 - 5) */
+ c = 5;
+ s0 = h0 + c; c = s0 >> 26; s0 &= 0x03ffffff;
+ s1 = h1 + c; c = s1 >> 26; s1 &= 0x03ffffff;
+ s2 = h2 + c; c = s2 >> 26; s2 &= 0x03ffffff;
+ s3 = h3 + c; c = s3 >> 26; s3 &= 0x03ffffff;
+ s4 = h4 + c;
+ s4 -= 0x04000000;
+
+ /* m := -1 if h < 2^130 - 5 else 0 */
+ m = -(s4 >> 31);
+
+ /* conditional subtract */
+ h0 = (m & h0) | (~m & s0);
+ h1 = (m & h1) | (~m & s1);
+ h2 = (m & h2) | (~m & s2);
+ h3 = (m & h3) | (~m & s3);
+ h4 = (m & h4) | (~m & s4);
+
+ /* reduce modulo 2^128 */
+ le32enc(h + 0, ((h1 << 26) | (h0 >> 0)) & 0xffffffff);
+ le32enc(h + 4, ((h2 << 20) | (h1 >> 6)) & 0xffffffff);
+ le32enc(h + 8, ((h3 << 14) | (h2 >> 12)) & 0xffffffff);
+ le32enc(h + 12, ((h4 << 8) | (h3 >> 18)) & 0xffffffff);
+}
+
+static void
+poly1305(uint8_t h[static 16], const uint8_t *m, size_t mlen,
+ const uint8_t k[static 16])
+{
+ struct poly1305 P;
+
+ poly1305_init(&P, k);
+ for (; mlen > 16; mlen -= 16, m += 16)
+ poly1305_update_block(&P, m);
+ poly1305_update_last(&P, m, mlen);
+ poly1305_final(h, &P);
+}
+
+static int
+poly1305_selftest(void)
+{
+ /* https://tools.ietf.org/html/rfc7539#section-2.5.2 */
+ static const uint8_t r[16] = {
+ 0x85,0xd6,0xbe,0x78, 0x57,0x55,0x6d,0x33,
+ 0x7f,0x44,0x52,0xfe, 0x42,0xd5,0x06,0xa8,
+ };
+ static const uint8_t s[16] = {
+ 0x01,0x03,0x80,0x8a, 0xfb,0x0d,0xb2,0xfd,
+ 0x4a,0xbf,0xf6,0xaf, 0x41,0x49,0xf5,0x1b,
+ };
+ static const uint8_t m[] = {
+ 0x43,0x72,0x79,0x70, 0x74,0x6f,0x67,0x72,
+ 0x61,0x70,0x68,0x69, 0x63,0x20,0x46,0x6f,
+ 0x72,0x75,0x6d,0x20, 0x52,0x65,0x73,0x65,
+ 0x61,0x72,0x63,0x68, 0x20,0x47,0x72,0x6f,
+ 0x75,0x70,
+ };
+ static const uint8_t expected[16] = {
+ 0xa8,0x06,0x1d,0xc1, 0x30,0x51,0x36,0xc6,
+ 0xc2,0x2b,0x8b,0xaf, 0x0c,0x01,0x27,0xa9,
+ };
+ uint8_t h[16], t[16];
+ int result = 0;
+
+ poly1305(h, m, sizeof m, r);
+ add128(t, h, s);
+ if (memcmp(t, expected, 16)) {
+ hexdump(printf, "poly1305 h", h, sizeof h);
+ hexdump(printf, "poly1305 t", t, sizeof t);
+ result = -1;
+ }
+
+ return result;
+}
+
+/* NHPoly1305 */
+
+static void
+nh(uint8_t h[32], const uint8_t *m, size_t mlen,
+ const uint32_t k[268 /* u/w + 2s(r - 1) */])
+{
+ const unsigned w = 32; /* word size */
+ const unsigned s = 2; /* stride */
+ const unsigned r = 4; /* rounds */
+ const unsigned u = 8192; /* unit count (bits per msg unit) */
+ uint64_t h0 = 0, h1 = 0, h2 = 0, h3 = 0;
+ unsigned i;
+
+ CTASSERT(r*w/8 == 16);
+
+ KASSERT(mlen <= u/8);
+ KASSERT(mlen % 16 == 0);
+
+ for (i = 0; i < mlen/16; i++) {
+ uint32_t m0 = le32dec(m + 16*i + 4*0);
+ uint32_t m1 = le32dec(m + 16*i + 4*1);
+ uint32_t m2 = le32dec(m + 16*i + 4*2);
+ uint32_t m3 = le32dec(m + 16*i + 4*3);
+
+ uint32_t k00 = k[4*i + 4*0 + 0];
+ uint32_t k01 = k[4*i + 4*0 + 1];
+ uint32_t k02 = k[4*i + 4*0 + 2];
+ uint32_t k03 = k[4*i + 4*0 + 3];
+ uint32_t k10 = k[4*i + 4*1 + 0];
+ uint32_t k11 = k[4*i + 4*1 + 1];
+ uint32_t k12 = k[4*i + 4*1 + 2];
+ uint32_t k13 = k[4*i + 4*1 + 3];
+ uint32_t k20 = k[4*i + 4*2 + 0];
+ uint32_t k21 = k[4*i + 4*2 + 1];
+ uint32_t k22 = k[4*i + 4*2 + 2];
+ uint32_t k23 = k[4*i + 4*2 + 3];
+ uint32_t k30 = k[4*i + 4*3 + 0];
+ uint32_t k31 = k[4*i + 4*3 + 1];
+ uint32_t k32 = k[4*i + 4*3 + 2];
+ uint32_t k33 = k[4*i + 4*3 + 3];
+
+ CTASSERT(s == 2);
+ h0 += (uint64_t)(m0 + k00) * (m2 + k02);
+ h1 += (uint64_t)(m0 + k10) * (m2 + k12);
+ h2 += (uint64_t)(m0 + k20) * (m2 + k22);
+ h3 += (uint64_t)(m0 + k30) * (m2 + k32);
+ h0 += (uint64_t)(m1 + k01) * (m3 + k03);
+ h1 += (uint64_t)(m1 + k11) * (m3 + k13);
+ h2 += (uint64_t)(m1 + k21) * (m3 + k23);
+ h3 += (uint64_t)(m1 + k31) * (m3 + k33);
+ }
+
+ le64enc(h + 8*0, h0);
+ le64enc(h + 8*1, h1);
+ le64enc(h + 8*2, h2);
+ le64enc(h + 8*3, h3);
+}
+
+static void
+nhpoly1305(uint8_t h[restrict static 16], const uint8_t *m, size_t mlen,
+ const uint8_t pk[static 16],
+ const uint32_t nhk[static 268 /* u/w + 2s(r - 1) */])
+{
+ struct poly1305 P;
+ uint8_t h0[32];
+
+ /*
+ * In principle NHPoly1305 is defined on uneven message
+ * lengths, but that's a pain in the patootie.
+ */
+ KASSERT(mlen % 16 == 0);
+
+ poly1305_init(&P, pk);
+ for (; mlen; m += MIN(mlen, 1024), mlen -= MIN(mlen, 1024)) {
+ nh(h0, m, MIN(mlen, 1024), nhk);
+ poly1305_update_block(&P, h0 + 16*0);
+ poly1305_update_block(&P, h0 + 16*1);
+ }
+ poly1305_final(h, &P);
+}
+
+/* https://github.com/google/adiantum/blob/68971e9c6684121b2203b4b05a22768b84051b58/test_vectors/ours/NH/NH.json */
+static int
+nh_selftest(void)
+{
+ static const struct {
+ uint8_t k[1072];
+ unsigned mlen;
+ uint8_t m[1024];
+ uint8_t h[32];
+ } C[] = {
+ [0] = { /* 16-byte message */
+ .k = {
+ 0x22,0x5b,0x80,0xc8, 0x18,0x05,0x37,0x09,
+ 0x76,0x14,0x4b,0x67, 0xc4,0x50,0x7f,0x2b,
+ 0x2c,0xff,0x56,0xc5, 0xd5,0x66,0x45,0x68,
+ 0x35,0xe6,0xd2,0x9a, 0xe5,0xd0,0xc1,0xfb,
+ 0xac,0x59,0x81,0x1a, 0x60,0xb0,0x3d,0x81,
+ 0x4b,0xa3,0x5b,0xa9, 0xcc,0xb3,0xfe,0x2d,
+ 0xc2,0x4d,0xd9,0x26, 0xad,0x36,0xcf,0x8c,
+ 0x05,0x11,0x3b,0x8a, 0x99,0x15,0x81,0xc8,
+ 0x23,0xf5,0x5a,0x94, 0x10,0x2f,0x92,0x80,
+ 0x38,0xc5,0xb2,0x63, 0x80,0xd5,0xdc,0xa3,
+ 0x6c,0x2f,0xaa,0x03, 0x96,0x4a,0x75,0x33,
+ 0x4c,0xa8,0x60,0x05, 0x96,0xbf,0xe5,0x7a,
+ 0xc8,0x4f,0x5c,0x22, 0xf9,0x92,0x74,0x4a,
+ 0x75,0x5f,0xa2,0x2a, 0x8d,0x3f,0xe2,0x43,
+ 0xfd,0xd9,0x04,0x8c, 0x8e,0xea,0x84,0xcc,
+ 0x4d,0x3f,0x94,0x96, 0xed,0x1a,0x51,0xbb,
+ 0x2f,0xc4,0x63,0x28, 0x31,0x0b,0xda,0x92,
+ 0x1e,0x4d,0xe2,0x1d, 0x82,0xb5,0x65,0xb4,
+ 0x75,0x69,0xd7,0x6f, 0x29,0xe4,0xbe,0x7e,
+ 0xcc,0xbd,0x95,0xbd, 0x7a,0x62,0xea,0xfa,
+ 0x33,0x34,0x80,0x58, 0xbf,0xfa,0x00,0x7e,
+ 0xa7,0xb4,0xc9,0x32, 0x7c,0xc7,0x8f,0x8a,
+ 0x28,0x27,0xdd,0xeb, 0xb9,0x1c,0x01,0xad,
+ 0xec,0xf4,0x30,0x5e, 0xce,0x3b,0xaa,0x22,
+ 0x60,0xbd,0x84,0xd9, 0x9e,0xaf,0xe8,0x4c,
+ 0x44,0xb6,0x84,0x2d, 0x5c,0xe6,0x26,0xee,
+ 0x8a,0xa2,0x0d,0xe3, 0x97,0xed,0xf5,0x47,
+ 0xdb,0x50,0x72,0x4a, 0x5e,0x9a,0x8d,0x10,
+ 0xc2,0x25,0xdd,0x5b, 0xd0,0x39,0xc4,0x5b,
+ 0x2a,0x79,0x81,0xb7, 0x5c,0xda,0xed,0x77,
+ 0x17,0x53,0xb5,0x8b, 0x1e,0x5f,0xf3,0x48,
+ 0x30,0xac,0x97,0x7d, 0x29,0xe3,0xc9,0x18,
+ 0xe1,0x2b,0x31,0xa0, 0x08,0xe9,0x15,0x59,
+ 0x29,0xdb,0x84,0x2a, 0x33,0x98,0x8a,0xd4,
+ 0xc3,0xfc,0xf7,0xca, 0x65,0x02,0x4d,0x9f,
+ 0xe2,0xb1,0x5e,0xa6, 0x6a,0x01,0xf9,0xcf,
+ 0x7e,0xa6,0x09,0xd9, 0x16,0x90,0x14,0x5f,
+ 0x3a,0xf8,0xd8,0x34, 0x38,0xd6,0x1f,0x89,
+ 0x0c,0x81,0xc2,0x68, 0xc4,0x65,0x78,0xf3,
+ 0xfe,0x27,0x48,0x70, 0x38,0x43,0x48,0x5a,
+ 0xc1,0x24,0xc5,0x6f, 0x65,0x63,0x1b,0xb0,
+ 0x5b,0xb4,0x07,0x1e, 0x69,0x08,0x8f,0xfc,
+ 0x93,0x29,0x04,0x16, 0x6a,0x8b,0xb3,0x3d,
+ 0x0f,0xba,0x5f,0x46, 0xff,0xfe,0x77,0xa1,
+ 0xb9,0xdc,0x29,0x66, 0x9a,0xd1,0x08,0xdd,
+ 0x32,0xe3,0x21,0x7b, 0xcc,0x2e,0x5c,0xf7,
+ 0x79,0x68,0xd4,0xc1, 0x8b,0x3c,0x5d,0x0e,
+ 0xd4,0x26,0xa6,0x19, 0x92,0x45,0xf7,0x19,
+ 0x0e,0xa2,0x17,0xd8, 0x1c,0x7f,0x8d,0xd6,
+ 0x68,0x37,0x6c,0xbf, 0xb1,0x8a,0x5e,0x36,
+ 0x4b,0xc0,0xca,0x21, 0x02,0x24,0x69,0x9b,
+ 0x2b,0x19,0x0a,0x1b, 0xe3,0x17,0x30,0x57,
+ 0xf6,0xfc,0xd6,0x66, 0x36,0x30,0xc2,0x11,
+ 0x08,0x8d,0xc5,0x84, 0x67,0xa0,0x89,0xc3,
+ 0x74,0x48,0x15,0xca, 0x6e,0x0c,0x6d,0x78,
+ 0x66,0x15,0x73,0x85, 0xf9,0x8b,0xba,0xb2,
+ 0x09,0xda,0x79,0xe6, 0x00,0x08,0x2a,0xda,
+ 0x6b,0xd7,0xd1,0xa7, 0x8b,0x5f,0x11,0x87,
+ 0x96,0x1b,0x23,0xb0, 0x6c,0x55,0xb6,0x86,
+ 0xfb,0xff,0xe3,0x69, 0xac,0x43,0xcd,0x8f,
+ 0x8a,0xe7,0x1c,0x3c, 0xa0,0x6a,0xd5,0x63,
+ 0x80,0x66,0xd8,0x7f, 0xb5,0xb8,0x96,0xd4,
+ 0xe2,0x20,0x40,0x53, 0x6d,0x0d,0x8b,0x6d,
+ 0xd5,0x5d,0x51,0xfb, 0x4d,0x80,0x82,0x01,
+ 0x14,0x97,0x96,0x9b, 0x13,0xb8,0x1d,0x76,
+ 0x7a,0xa1,0xca,0x19, 0x90,0xec,0x7b,0xe0,
+ 0x8e,0xa8,0xb4,0xf2, 0x33,0x67,0x0e,0x10,
+ 0xb1,0xa2,0x82,0xea, 0x81,0x82,0xa2,0xc6,
+ 0x78,0x51,0xa6,0xd3, 0x25,0xe4,0x9c,0xf2,
+ 0x6b,0xa8,0xec,0xfb, 0xd4,0x1d,0x5b,0xa4,
+ 0x79,0x66,0x62,0xb8, 0x2b,0x6f,0x9e,0x0f,
+ 0xcc,0xcb,0x9e,0x92, 0x6f,0x06,0xdb,0xf0,
+ 0x97,0xce,0x3f,0x90, 0xa2,0x1f,0xbe,0x3b,
+ 0x7b,0x10,0xf0,0x23, 0x30,0x0c,0xc5,0x0c,
+ 0x6c,0x78,0xfc,0xa8, 0x71,0x62,0xcf,0x98,
+ 0xa2,0xb1,0x44,0xb5, 0xc6,0x3b,0x5c,0x63,
+ 0x83,0x1d,0x35,0xf2, 0xc7,0x42,0x67,0x5d,
+ 0xc1,0x26,0x36,0xc8, 0x6e,0x1d,0xf6,0xd5,
+ 0x52,0x35,0xa4,0x9e, 0xce,0x4c,0x3b,0x92,
+ 0x20,0x86,0xb7,0x89, 0x63,0x73,0x1a,0x8b,
+ 0xa6,0x35,0xfe,0xb9, 0xdf,0x5e,0x0e,0x53,
+ 0x0b,0xf2,0xb3,0x4d, 0x34,0x1d,0x66,0x33,
+ 0x1f,0x08,0xf5,0xf5, 0x0a,0xab,0x76,0x19,
+ 0xde,0x82,0x2f,0xcf, 0x11,0xa6,0xcb,0xb3,
+ 0x17,0xec,0x8d,0xaf, 0xcb,0xf0,0x92,0x1e,
+ 0xb8,0xa3,0x04,0x0a, 0xac,0x2c,0xae,0xc5,
+ 0x0b,0xc4,0x4e,0xef, 0x0a,0xe2,0xda,0xe9,
+ 0xd7,0x75,0x2d,0x95, 0xc7,0x1b,0xf3,0x0b,
+ 0x43,0x19,0x16,0xd7, 0xc6,0x90,0x2d,0x6b,
+ 0xe1,0xb2,0xce,0xbe, 0xd0,0x7d,0x15,0x99,
+ 0x24,0x37,0xbc,0xb6, 0x8c,0x89,0x7a,0x8c,
+ 0xcb,0xa7,0xf7,0x0b, 0x5f,0xd4,0x96,0x8d,
+ 0xf5,0x80,0xa3,0xce, 0xf5,0x9e,0xed,0x60,
+ 0x00,0x92,0xa5,0x67, 0xc9,0x21,0x79,0x0b,
+ 0xfb,0xe2,0x57,0x0e, 0xdf,0xb6,0x16,0x90,
+ 0xd3,0x75,0xf6,0xb0, 0xa3,0x4e,0x43,0x9a,
+ 0xb7,0xf4,0x73,0xd8, 0x34,0x46,0xc6,0xbe,
+ 0x80,0xec,0x4a,0xc0, 0x7f,0x9e,0xb6,0xb0,
+ 0x58,0xc2,0xae,0xa1, 0xf3,0x60,0x04,0x62,
+ 0x11,0xea,0x0f,0x90, 0xa9,0xea,0x6f,0x0c,
+ 0x4c,0xcf,0xe8,0xd0, 0xea,0xbf,0xdb,0xf2,
+ 0x53,0x0c,0x09,0x4d, 0xd4,0xed,0xf3,0x22,
+ 0x10,0x99,0xc6,0x4f, 0xcf,0xcf,0x96,0xc9,
+ 0xd9,0x6b,0x08,0x3b, 0xf0,0x62,0x2d,0xac,
+ 0x55,0x38,0xd5,0x5c, 0x57,0xad,0x51,0xc3,
+ 0xf5,0xd2,0x37,0x45, 0xb3,0x3f,0x6d,0xaf,
+ 0x10,0x62,0x57,0xb9, 0x58,0x40,0xb3,0x3c,
+ 0x6a,0x98,0x97,0x1a, 0x9c,0xeb,0x66,0xf1,
+ 0xa5,0x93,0x0b,0xe7, 0x8b,0x29,0x0f,0xff,
+ 0x2c,0xd0,0x90,0xf2, 0x67,0xa0,0x69,0xcd,
+ 0xd3,0x59,0xad,0xad, 0xf1,0x1f,0xd7,0xad,
+ 0x24,0x74,0x29,0xcd, 0x06,0xd5,0x42,0x90,
+ 0xf9,0x96,0x4a,0xd9, 0xa0,0x37,0xe4,0x64,
+ 0x8e,0x13,0x2a,0x2a, 0xe7,0xc2,0x1e,0xf6,
+ 0xb2,0xd3,0xdc,0x9f, 0x33,0x32,0x0c,0x50,
+ 0x88,0x37,0x8b,0x9b, 0xfe,0x6f,0xfd,0x05,
+ 0x96,0x26,0x6c,0x96, 0x73,0x73,0xe1,0x09,
+ 0x28,0xf3,0x7f,0xa6, 0x59,0xc5,0x2e,0xf4,
+ 0xd3,0xd5,0xda,0x6b, 0xca,0x42,0x05,0xe5,
+ 0xed,0x13,0xe2,0x4e, 0xcd,0xd5,0xd0,0xfb,
+ 0x6e,0xf7,0x8a,0x3e, 0x91,0x9d,0x6b,0xc5,
+ 0x33,0x05,0x07,0x86, 0xb2,0x26,0x41,0x6e,
+ 0xf8,0x38,0x38,0x7a, 0xf0,0x6c,0x27,0x5a,
+ 0x01,0xd8,0x03,0xe5, 0x91,0x33,0xaa,0x20,
+ 0xcd,0xa7,0x4f,0x18, 0xa0,0x91,0x28,0x74,
+ 0xc0,0x58,0x27,0x0f, 0x9b,0xa8,0x85,0xb0,
+ 0xe0,0xfd,0x5b,0xdb, 0x5b,0xb8,0x86,0x79,
+ 0x94,0x6d,0xde,0x26, 0x64,0x2d,0x6c,0xb9,
+ 0xba,0xc7,0xf0,0xd7, 0xaa,0x68,0x68,0xd0,
+ 0x40,0x71,0xdb,0x94, 0x54,0x62,0xa5,0x7f,
+ 0x98,0xea,0xe3,0x4c, 0xe4,0x44,0x9a,0x03,
+ 0xf9,0x1c,0x20,0x36, 0xeb,0x0d,0xa4,0x41,
+ 0x24,0x06,0xcb,0x94, 0x86,0x35,0x22,0x62,
+ 0x80,0x19,0x16,0xba, 0x2c,0x10,0x38,0x96,
+ },
+ .mlen = 16,
+ .m = {
+ 0xd3,0x82,0xe7,0x04, 0x35,0xcc,0xf7,0xa4,
+ 0xf9,0xb2,0xc5,0xed, 0x5a,0xd9,0x58,0xeb,
+ },
+ .h = {
+ 0x41,0xd9,0xad,0x54, 0x5a,0x0d,0xcc,0x53,
+ 0x48,0xf6,0x4c,0x75, 0x43,0x5d,0xdd,0x77,
+ 0xda,0xca,0x7d,0xec, 0x91,0x3b,0x53,0x16,
+ 0x5c,0x4b,0x58,0xdc, 0x70,0x0a,0x7b,0x37,
+ },
+ },
+ [1] = { /* 1008-byte message */
+ .k = {
+ 0xd9,0x94,0x65,0xda, 0xc2,0x60,0xdd,0xa9,
+ 0x39,0xe5,0x37,0x11, 0xf6,0x74,0xa5,0x95,
+ 0x36,0x07,0x24,0x99, 0x64,0x6b,0xda,0xe2,
+ 0xd5,0xd1,0xd2,0xd9, 0x25,0xd5,0xcc,0x48,
+ 0xf8,0xa5,0x9e,0xff, 0x84,0x5a,0xd1,0x6f,
+ 0xb7,0x6a,0x4d,0xd2, 0xc8,0x13,0x3d,0xde,
+ 0x17,0xed,0x64,0xf1, 0x2b,0xcc,0xdd,0x65,
+ 0x11,0x16,0xf2,0xaf, 0x34,0xd2,0xc5,0x31,
+ 0xaa,0x69,0x33,0x0a, 0x0b,0xc1,0xb4,0x6d,
+ 0xaa,0xcd,0x43,0xc4, 0x0b,0xef,0xf9,0x7d,
+ 0x97,0x3c,0xa7,0x22, 0xda,0xa6,0x6a,0xf0,
+ 0xad,0xe3,0x6f,0xde, 0xfb,0x33,0xf3,0xd8,
+ 0x96,0x5f,0xca,0xda, 0x18,0x63,0x03,0xd0,
+ 0x8f,0xb6,0xc4,0x62, 0x9d,0x50,0x6c,0x8f,
+ 0x85,0xdd,0x6d,0x52, 0x2d,0x45,0x01,0x36,
+ 0x57,0x9f,0x51,0xf0, 0x70,0xe0,0xb2,0x99,
+ 0x3a,0x11,0x68,0xbd, 0xe5,0xfa,0x7c,0x59,
+ 0x12,0x5a,0xbc,0xd9, 0xd6,0x9a,0x09,0xe6,
+ 0xa2,0x80,0x1f,0xd6, 0x47,0x20,0x82,0x4e,
+ 0xac,0xb5,0x6d,0xde, 0x5b,0xff,0x9c,0xd4,
+ 0x2a,0xae,0x27,0x7c, 0x0f,0x5a,0x5d,0x35,
+ 0x2d,0xff,0x07,0xf9, 0x79,0x6a,0xf9,0x3e,
+ 0xd9,0x22,0x62,0x30, 0x40,0xce,0xe1,0xf4,
+ 0x46,0x0a,0x24,0xca, 0x7a,0x3e,0xa1,0x92,
+ 0x1a,0x29,0xa0,0xbf, 0x23,0x95,0x99,0x31,
+ 0xe3,0x51,0x25,0x3d, 0xaf,0x1e,0xfc,0xb3,
+ 0x65,0xa2,0x10,0x37, 0xe6,0xa7,0x20,0xa0,
+ 0xe3,0x6a,0xd4,0x81, 0x2c,0x8d,0xa0,0x87,
+ 0xec,0xae,0x9f,0x44, 0x10,0xda,0x2e,0x17,
+ 0xba,0xb2,0xa5,0x5c, 0x89,0xc6,0xfa,0x70,
+ 0x7e,0xc2,0xe3,0xb6, 0xa0,0x98,0x9c,0xb8,
+ 0x14,0x33,0x27,0x3a, 0x6e,0x4d,0x94,0x72,
+ 0x4b,0xc8,0xac,0x24, 0x2f,0x85,0xd9,0xa4,
+ 0xda,0x22,0x95,0xc5, 0xb3,0xfc,0xbe,0xd2,
+ 0x96,0x57,0x91,0xf9, 0xfd,0x18,0x9c,0x56,
+ 0x70,0x15,0x5f,0xe7, 0x40,0x45,0x28,0xb3,
+ 0x2b,0x56,0x44,0xca, 0x6a,0x2b,0x0e,0x25,
+ 0x66,0x3e,0x32,0x04, 0xe2,0xb7,0x91,0xc8,
+ 0xd2,0x02,0x79,0x0f, 0x7e,0xa9,0xb3,0x86,
+ 0xb2,0x76,0x74,0x18, 0x57,0x16,0x63,0x06,
+ 0x6e,0x16,0xfa,0xef, 0x52,0x3c,0x5e,0x0d,
+ 0x33,0x55,0xd2,0x8d, 0x57,0x4d,0xfe,0x54,
+ 0x65,0x7a,0x54,0x52, 0xf0,0x7b,0x2c,0xf8,
+ 0xd5,0x43,0xba,0x92, 0xa5,0x2e,0xbe,0x1a,
+ 0xce,0x25,0x4f,0x34, 0x31,0xe7,0xa3,0xff,
+ 0x90,0xf6,0xbc,0x0c, 0xbc,0x98,0xdf,0x4a,
+ 0xc3,0xeb,0xb6,0x27, 0x68,0xa9,0xb5,0x33,
+ 0xbc,0x13,0xe8,0x13, 0x7c,0x6b,0xec,0x31,
+ 0xd9,0x79,0x2a,0xa7, 0xe4,0x02,0x4f,0x02,
+ 0xd4,0x5c,0x57,0x4f, 0xa4,0xbc,0xa3,0xe1,
+ 0x7e,0x36,0x8a,0xde, 0x11,0x55,0xec,0xb3,
+ 0x8b,0x65,0x06,0x02, 0x9a,0x68,0x06,0x64,
+ 0x63,0xc7,0x9a,0x67, 0xdc,0x70,0xbf,0xb5,
+ 0xf8,0x49,0x2a,0xe1, 0x59,0x4c,0xe4,0x1e,
+ 0xb5,0x56,0xa5,0xad, 0x24,0x82,0x8c,0xd0,
+ 0x66,0xe4,0x72,0x79, 0x02,0x5d,0x0d,0xf9,
+ 0x19,0x44,0xe3,0x86, 0x1a,0xda,0xda,0xf0,
+ 0x2d,0x47,0xc0,0x07, 0x47,0x0b,0xf8,0x06,
+ 0xf6,0x45,0x8a,0x7f, 0xb9,0xf9,0x33,0x2e,
+ 0xc2,0xf1,0xf1,0x81, 0x41,0x99,0xcd,0xf6,
+ 0xb1,0x71,0x1b,0xfa, 0x21,0x53,0x7c,0xa1,
+ 0xeb,0x2a,0x38,0x5b, 0x9b,0xfe,0x96,0xa5,
+ 0xe3,0x78,0x77,0x47, 0x98,0x0f,0x7d,0xef,
+ 0xf6,0x05,0x37,0x88, 0x79,0x0c,0x21,0x8d,
+ 0x87,0x1f,0xae,0xce, 0x83,0xaf,0xa3,0xd6,
+ 0x6e,0xc5,0x3c,0x47, 0xc6,0xd6,0x4a,0xdc,
+ 0x7c,0xcc,0xdc,0x11, 0x7c,0x7d,0x0f,0x03,
+ 0xc1,0x80,0x75,0x2a, 0x64,0x76,0xf0,0x08,
+ 0x0c,0x11,0x4b,0xe4, 0x05,0x41,0x78,0x0f,
+ 0x86,0xa0,0xd6,0x61, 0xb0,0xfb,0x15,0x3d,
+ 0x3c,0xc3,0xd5,0x1b, 0x72,0x0e,0x79,0x53,
+ 0x07,0xd2,0x2c,0x6e, 0x83,0xbd,0x72,0x88,
+ 0x41,0x07,0x4b,0xd2, 0xe9,0xcc,0x2a,0x9d,
+ 0x5b,0x82,0x0d,0x02, 0x29,0x6e,0xf3,0xbc,
+ 0x34,0x31,0x62,0x8d, 0x83,0xc1,0x7e,0x94,
+ 0x21,0xd5,0xfd,0xa6, 0x6a,0x2b,0xe8,0x86,
+ 0x05,0x48,0x97,0x41, 0xad,0xca,0xef,0x79,
+ 0x5e,0xd8,0x51,0xc4, 0xae,0xf7,0xfa,0xac,
+ 0x3d,0x74,0x2e,0xf4, 0x41,0x3b,0x19,0xc2,
+ 0x04,0xf3,0x40,0xfe, 0x77,0x7c,0x6a,0x4c,
+ 0x8e,0x24,0x84,0xe0, 0x70,0xe4,0xb2,0x19,
+ 0x6c,0x0c,0x85,0x9e, 0xe1,0xad,0xa4,0x73,
+ 0x90,0xdd,0xbf,0x7d, 0x1b,0x6f,0x8b,0x4d,
+ 0x3b,0xec,0xd7,0xb0, 0xd9,0x90,0xf1,0xf5,
+ 0xb9,0x32,0xe3,0x79, 0x15,0x08,0x3e,0x71,
+ 0xed,0x91,0xc4,0x5c, 0x18,0xe8,0x16,0x52,
+ 0xae,0x9d,0xf3,0x09, 0xac,0x57,0x11,0xf8,
+ 0x16,0x55,0xd0,0x28, 0x60,0xc1,0x7e,0x6d,
+ 0x87,0xc1,0x7a,0xe8, 0x5d,0xc5,0x12,0x68,
+ 0x6d,0x63,0x39,0x27, 0x49,0xb8,0x0c,0x78,
+ 0x92,0xea,0x6f,0x52, 0xeb,0x43,0xc2,0x0b,
+ 0xd8,0x28,0x77,0xe5, 0x43,0x5f,0xb8,0xa6,
+ 0x32,0xb7,0xaa,0x01, 0x1e,0xa6,0xde,0xe4,
+ 0x9b,0x0f,0xb6,0x49, 0xcc,0x6f,0x2c,0x04,
+ 0x41,0xcb,0xd8,0x80, 0xd1,0x15,0x5e,0x57,
+ 0x1e,0x4a,0x77,0xbf, 0xc4,0xcb,0x09,0x7c,
+ 0x6e,0x81,0xb8,0x64, 0x51,0x6a,0xf2,0x71,
+ 0x06,0xf6,0x00,0xac, 0x79,0x2c,0x83,0x7a,
+ 0x6c,0xa4,0x85,0x89, 0x69,0x06,0x26,0x72,
+ 0xe1,0x00,0x66,0xc0, 0xc5,0x8e,0xc8,0x51,
+ 0x6e,0x25,0xdd,0xc9, 0x54,0x98,0x45,0x64,
+ 0xaa,0x51,0x18,0x1b, 0xe4,0xbe,0x1b,0xee,
+ 0x13,0xd6,0x34,0x50, 0x4c,0xcf,0x3c,0x31,
+ 0x9b,0xd2,0x6f,0x07, 0x79,0xf4,0x63,0x3f,
+ 0x09,0x01,0x64,0xf1, 0xc1,0xf1,0xae,0xa9,
+ 0x0c,0x60,0xc9,0x62, 0x84,0xf6,0xe8,0x15,
+ 0x55,0xdf,0xdd,0x71, 0x95,0xa9,0x0f,0x65,
+ 0x97,0x40,0x79,0x86, 0x95,0xd9,0x57,0x23,
+ 0x2f,0x61,0x51,0xb5, 0x16,0x18,0x62,0xd2,
+ 0x1a,0xd9,0x8b,0x88, 0x84,0xa9,0x9b,0x47,
+ 0xd7,0x22,0x68,0xe9, 0x9c,0x69,0x68,0x74,
+ 0x13,0x95,0xd3,0x99, 0x33,0xdb,0x30,0x96,
+ 0xbf,0x01,0xc6,0x68, 0xbd,0x19,0x32,0xc1,
+ 0xf8,0xa9,0x7f,0x2b, 0xc5,0x69,0x2f,0xa2,
+ 0xce,0x5a,0x46,0x43, 0x8d,0x36,0x9c,0xfa,
+ 0x5c,0x7f,0x03,0xe0, 0x80,0xaa,0xc7,0x9e,
+ 0x3b,0xa3,0x27,0x6b, 0x2e,0xc6,0x59,0x0a,
+ 0xf6,0x36,0x37,0xa6, 0xc0,0xd1,0xa1,0xa1,
+ 0x7e,0xc1,0xf8,0x5b, 0x0f,0x9b,0xdd,0x6d,
+ 0x9f,0x54,0x16,0x6b, 0x6e,0x53,0xfd,0xe8,
+ 0x72,0xd0,0x3e,0x46, 0xce,0xaf,0x94,0x36,
+ 0x85,0xa8,0xae,0x4c, 0x8d,0xb5,0xc2,0x1b,
+ 0x5d,0x29,0x46,0x40, 0x87,0x50,0x59,0xdd,
+ 0x04,0xbe,0xba,0x8f, 0x0b,0x9b,0xd2,0x50,
+ 0x67,0x19,0x83,0x80, 0x87,0x5c,0x58,0x86,
+ 0x20,0x39,0xbf,0xdf, 0xd2,0xc8,0xbb,0xe8,
+ 0xc8,0xd8,0xe8,0x8d, 0xcc,0x97,0xe0,0xc9,
+ 0x6c,0x2f,0x47,0xb6, 0x75,0x8f,0x0d,0x37,
+ 0x5a,0x83,0xb0,0xce, 0x59,0xc2,0x0b,0x84,
+ 0xa2,0x54,0xe5,0x38, 0x59,0x29,0x0f,0xa8,
+ 0x26,0x2d,0x11,0xa9, 0x89,0x0e,0x0b,0x75,
+ 0xe0,0xbc,0xf0,0xf8, 0x92,0x1f,0x29,0x71,
+ 0x91,0xc4,0x63,0xcc, 0xf8,0x52,0xb5,0xd4,
+ 0xb8,0x94,0x6a,0x30, 0x90,0xf7,0x44,0xbe,
+ },
+ .mlen = 1008,
+ .m = {
+ 0x05,0xe3,0x6f,0x44, 0xa4,0x40,0x35,0xf6,
+ 0xeb,0x86,0xa9,0x6d, 0xed,0x16,0xdb,0xb6,
+ 0x5b,0x59,0xda,0x30, 0x54,0x6c,0x59,0x35,
+ 0x42,0x59,0x56,0x45, 0x9a,0x85,0x20,0x73,
+ 0xcf,0x21,0xf5,0x98, 0x58,0x07,0x0e,0x7f,
+ 0x44,0x1f,0xf1,0x53, 0x92,0xc7,0x81,0x53,
+ 0x5e,0x97,0x8a,0x23, 0x1d,0xe8,0xad,0xca,
+ 0x19,0x55,0x96,0x9d, 0x9b,0xfd,0x0a,0x0a,
+ 0xad,0xa8,0x0f,0x76, 0xe2,0x6a,0x8f,0x33,
+ 0x36,0xbf,0xcb,0x7a, 0xfd,0x61,0xc6,0xfb,
+ 0x75,0xea,0xd4,0x09, 0x5e,0x70,0xfb,0x32,
+ 0x54,0xe3,0x47,0x48, 0xd4,0x8c,0xa9,0x7c,
+ 0x72,0xdb,0xdb,0xf7, 0x09,0x6d,0x58,0xa6,
+ 0x42,0xb5,0x74,0x8c, 0x98,0x66,0x83,0x7a,
+ 0x6d,0xeb,0x91,0xfb, 0x22,0x1c,0x78,0x3d,
+ 0x22,0xa6,0xf8,0xb0, 0xd1,0x9f,0xc8,0x69,
+ 0x8a,0xba,0xd3,0x78, 0x21,0xb0,0x7b,0x9f,
+ 0xb8,0xed,0xe0,0x65, 0xff,0xa0,0x8b,0x4c,
+ 0x17,0x9e,0xf7,0x3e, 0xa2,0x5f,0x82,0x77,
+ 0xce,0x2a,0xda,0x41, 0x76,0x07,0x68,0xa4,
+ 0xa1,0xbb,0xe0,0x1d, 0x7b,0xab,0x9c,0x03,
+ 0x90,0x2c,0xd2,0x93, 0x46,0x43,0x3a,0x44,
+ 0x29,0xe8,0xb5,0x7a, 0x23,0xbb,0xe9,0xaf,
+ 0x2b,0x17,0x88,0x8f, 0x7a,0x81,0x7a,0x25,
+ 0x3b,0xc7,0x1e,0x6e, 0xde,0x3e,0x54,0xbc,
+ 0xc6,0xff,0x07,0xdc, 0xe6,0x29,0x02,0x4c,
+ 0x95,0x57,0x0e,0x44, 0xc4,0x9c,0xc7,0x45,
+ 0x01,0xd7,0x17,0xfd, 0x0f,0x1a,0x83,0x74,
+ 0xa0,0xd5,0xb3,0x1a, 0xc0,0x97,0xdc,0xc3,
+ 0x0f,0x3d,0x5d,0x8c, 0x02,0x58,0xc6,0x4d,
+ 0x43,0x10,0xae,0xc9, 0x94,0xe2,0x9b,0xcd,
+ 0xf9,0xcc,0xfe,0xbd, 0x9c,0x69,0xd0,0xec,
+ 0xf8,0x67,0xde,0x98, 0xe5,0x50,0x5e,0x93,
+ 0x6a,0x5b,0x31,0x2a, 0x62,0xee,0x03,0xbe,
+ 0x76,0x9c,0x1d,0x13, 0x16,0x13,0xcf,0x63,
+ 0x30,0x18,0x7d,0x1e, 0x55,0x94,0xf5,0x29,
+ 0xb4,0x91,0xb4,0x76, 0x1c,0x31,0x9e,0xe5,
+ 0x1b,0x0a,0xee,0x89, 0xb4,0xd9,0x45,0x19,
+ 0xd7,0x47,0x2c,0x01, 0x20,0xe6,0x1d,0x7c,
+ 0xb3,0x5e,0x1b,0x2a, 0x8c,0x3d,0x4d,0x1a,
+ 0x6b,0x35,0x84,0x41, 0x6a,0xe4,0x32,0x8f,
+ 0x9a,0x0d,0xbf,0x90, 0xff,0xcf,0x4c,0xfb,
+ 0x9b,0x07,0x81,0x94, 0xcf,0x8e,0x1a,0x8a,
+ 0xfc,0xbd,0x91,0xfe, 0xc3,0xe1,0x18,0xc7,
+ 0x1f,0x0d,0x8e,0x1c, 0x2e,0xfc,0x02,0xe8,
+ 0x39,0xbf,0x05,0x90, 0x58,0x94,0xee,0xe7,
+ 0x15,0x31,0x5d,0x9f, 0x68,0x36,0x64,0x32,
+ 0x25,0x49,0xdd,0x3e, 0xc8,0xb6,0x83,0x5e,
+ 0x09,0x90,0xcd,0x48, 0xaf,0x9e,0xfe,0xd6,
+ 0x79,0x8e,0x69,0x4b, 0x94,0xd5,0xf4,0x84,
+ 0x7b,0xce,0xea,0x2f, 0x9b,0x79,0x7a,0x7c,
+ 0x22,0x28,0x4d,0xa1, 0x38,0x1a,0x66,0x24,
+ 0x79,0xa3,0xfa,0xfa, 0x8d,0x98,0x7c,0x54,
+ 0x71,0x54,0xef,0x37, 0xa6,0xf1,0x97,0x54,
+ 0xad,0xe7,0x67,0xa0, 0xf3,0x33,0xcf,0x4f,
+ 0x4e,0xa3,0x47,0xee, 0x31,0xd3,0x98,0xf9,
+ 0x7f,0x9f,0x44,0x18, 0x2f,0x13,0x1b,0x44,
+ 0x57,0xcd,0x15,0x5b, 0xde,0x8f,0x1a,0x3c,
+ 0xb5,0x1e,0xa7,0x2d, 0x4d,0xbe,0x85,0x08,
+ 0x78,0xeb,0xe2,0x35, 0x3a,0xbe,0x55,0x6b,
+ 0xc3,0xe1,0x0f,0x77, 0x43,0x41,0x11,0x5a,
+ 0x61,0xc9,0x3b,0xbc, 0xad,0x88,0x9e,0xba,
+ 0xc6,0xd2,0xdc,0x87, 0xd9,0x54,0xcc,0x86,
+ 0x46,0xe6,0xa5,0x29, 0x2c,0x08,0x49,0x53,
+ 0x2c,0xe3,0x0e,0x60, 0xc5,0x48,0xca,0x62,
+ 0x3f,0xf6,0x93,0xc1, 0xba,0x8d,0x36,0x49,
+ 0xe7,0x0f,0x9c,0x49, 0x7d,0xee,0x2a,0x22,
+ 0xc3,0xe5,0x11,0x21, 0xfa,0xc7,0xeb,0x79,
+ 0xcc,0x4d,0x75,0x4e, 0x66,0x33,0xf5,0x09,
+ 0xa3,0xb9,0x60,0xa5, 0xd6,0xbd,0x38,0x75,
+ 0x0c,0x2f,0x5f,0x1f, 0xea,0xa5,0x9d,0x45,
+ 0x3c,0xe4,0x41,0xb8, 0xf6,0x4e,0x15,0x87,
+ 0x0b,0x7f,0x42,0x4e, 0x51,0x3d,0xc4,0x9a,
+ 0xb2,0xca,0x37,0x16, 0x0f,0xed,0x9e,0x0b,
+ 0x93,0x86,0x12,0x93, 0x36,0x5e,0x39,0xc4,
+ 0xf0,0xf4,0x48,0xdb, 0xeb,0x18,0x5e,0x50,
+ 0x71,0x30,0x83,0xe5, 0x0f,0xb1,0x73,0xa7,
+ 0xc6,0xf0,0xca,0x29, 0x0e,0xc4,0x07,0x5b,
+ 0x8b,0x09,0x68,0x68, 0x10,0x32,0x92,0x62,
+ 0x6a,0x6c,0x56,0x8b, 0x01,0x46,0x9a,0x20,
+ 0x89,0xe0,0x93,0x85, 0x8c,0x53,0x87,0xf6,
+ 0x02,0xd3,0x8d,0x72, 0x31,0x35,0xa1,0x34,
+ 0x63,0x70,0x61,0x80, 0x06,0xf1,0x54,0xb3,
+ 0x5d,0xdf,0xad,0x9c, 0x7e,0x3a,0xc2,0x8f,
+ 0x76,0x8b,0x4c,0x74, 0x2c,0x8c,0x6f,0x0a,
+ 0x60,0x13,0xa8,0xce, 0x4c,0x49,0x70,0x90,
+ 0x59,0x57,0xf5,0x7b, 0x03,0x94,0x37,0x87,
+ 0xfa,0xfe,0xeb,0xe7, 0x2d,0x01,0x45,0x69,
+ 0xb4,0x10,0x80,0x6d, 0x13,0x26,0xe3,0x9b,
+ 0x49,0x2a,0x0b,0xb1, 0x36,0xf9,0x62,0x63,
+ 0x33,0x2a,0xee,0x51, 0x5e,0x35,0xa4,0x2e,
+ 0x34,0xa1,0x77,0xac, 0x27,0x99,0x03,0xc6,
+ 0xe2,0x83,0x11,0x72, 0x77,0x30,0x8b,0xb7,
+ 0xde,0x1a,0xa1,0x4b, 0xa9,0x9c,0x07,0x02,
+ 0xf2,0xdc,0x06,0x45, 0xf2,0xab,0x31,0x46,
+ 0x50,0x25,0x34,0x54, 0xa8,0x06,0x88,0x6c,
+ 0xfc,0x88,0xb5,0xae, 0x30,0xbd,0xe1,0xe7,
+ 0xfe,0x51,0x46,0x05, 0x9a,0x29,0xd9,0x93,
+ 0x99,0x60,0x69,0x4a, 0x5c,0xb2,0x29,0x6b,
+ 0xa1,0xbb,0x9d,0xe4, 0x9b,0x7d,0x4a,0xe5,
+ 0x37,0xcb,0x16,0x6f, 0x44,0x93,0xe4,0x71,
+ 0x34,0x7b,0x54,0xec, 0x5b,0x2b,0xe0,0xf7,
+ 0x32,0xed,0x77,0xa6, 0xb3,0x7c,0x8d,0x1a,
+ 0xc0,0x57,0xbe,0x2b, 0x6d,0x7f,0xd7,0x35,
+ 0xe6,0x93,0xed,0x90, 0x26,0xfe,0x41,0xf3,
+ 0x58,0x55,0x03,0xb7, 0xb2,0x94,0xe2,0x0c,
+ 0x34,0xc3,0x06,0xc6, 0x9e,0x4b,0x17,0xc7,
+ 0xb9,0x58,0x23,0x58, 0xd3,0x73,0x18,0x5e,
+ 0xcf,0x28,0xac,0x90, 0xa0,0xba,0x35,0x90,
+ 0x96,0xb3,0xc7,0x6c, 0xe1,0x07,0xdf,0x5d,
+ 0xaa,0x2c,0xa6,0x6b, 0x82,0x2d,0x71,0x66,
+ 0xb7,0x76,0x37,0xdb, 0x39,0x7f,0x22,0x8f,
+ 0x38,0x70,0xd4,0xeb, 0xf8,0xf0,0x73,0xed,
+ 0xb6,0x67,0x75,0xaf, 0xd7,0x5d,0x01,0x01,
+ 0xc4,0xd6,0x7c,0xbc, 0xc3,0xe6,0xad,0x9a,
+ 0x9c,0x6a,0x43,0x9b, 0xfb,0x34,0x55,0x47,
+ 0xcd,0xeb,0x4e,0x2c, 0x29,0x6f,0xb0,0xeb,
+ 0xb5,0x08,0xdb,0x6b, 0x40,0x26,0x51,0x54,
+ 0x5a,0x97,0x64,0x74, 0x95,0xe6,0xae,0x8a,
+ 0x4c,0xe9,0x44,0x47, 0x85,0xd6,0xcf,0xe0,
+ 0x11,0x65,0x45,0xb3, 0xe1,0xfc,0x6a,0x01,
+ 0x38,0x40,0x8a,0x71, 0xc5,0xd6,0x64,0xa8,
+ 0x36,0x95,0x44,0x9c, 0x10,0x41,0xa3,0x71,
+ 0xb4,0x70,0x02,0xdf, 0xf9,0xad,0x2b,0xec,
+ 0x75,0xf7,0x09,0x6c, 0x5d,0x2a,0xd0,0x0b,
+ 0x2e,0xb3,0xf0,0xd3, 0xce,0xdb,0x26,0x80,
+ },
+ .h = {
+ 0x2d,0xb3,0x7e,0x73, 0xde,0x6a,0x9e,0xa9,
+ 0x54,0x9a,0x0f,0xb3, 0x0b,0xcc,0xc9,0xde,
+ 0x7a,0x4e,0x4a,0x71, 0x07,0x33,0xee,0x06,
+ 0x5c,0x9a,0xa1,0x30, 0x5e,0x39,0x4e,0x10,
+ },
+ },
+ [2] = { /* 1024-byte message */
+ .k = {
+ 0x4c,0xe4,0x3c,0x6e, 0xa0,0xe3,0x0e,0x64,
+ 0x35,0x44,0x3e,0x0b, 0x4d,0x29,0xbe,0x04,
+ 0xa7,0xaa,0x88,0xe0, 0xe0,0x07,0x7d,0xa8,
+ 0x2b,0x87,0x7d,0x08, 0xa6,0x59,0xd0,0xa5,
+ 0x03,0xae,0x9b,0xee, 0xd4,0x11,0x39,0x7d,
+ 0x9e,0x1d,0x89,0xe3, 0xc6,0x92,0x36,0x07,
+ 0xa4,0x43,0xad,0x2f, 0xd5,0x71,0x84,0x2d,
+ 0xc0,0x37,0xed,0x62, 0x4e,0x2b,0x8c,0xd5,
+ 0x1d,0xf7,0x00,0xbb, 0x3d,0x5e,0xcc,0xc5,
+ 0x6d,0xdd,0x17,0xf2, 0x89,0x25,0x30,0x16,
+ 0x04,0xd7,0x1f,0x84, 0x7d,0x61,0xa0,0x7a,
+ 0x49,0x88,0x44,0x46, 0xc6,0x05,0xd1,0xc9,
+ 0xa0,0x2a,0x86,0xdd, 0xd3,0x80,0x40,0xa4,
+ 0x28,0xb3,0xa4,0x3b, 0x71,0x0a,0x7f,0x2d,
+ 0x3b,0xcd,0xe6,0xac, 0x59,0xda,0x43,0x56,
+ 0x6e,0x9a,0x3f,0x1e, 0x82,0xcf,0xb3,0xa0,
+ 0xa1,0x46,0xcf,0x2e, 0x32,0x05,0xcd,0x68,
+ 0xbb,0x51,0x71,0x8a, 0x16,0x75,0xbe,0x49,
+ 0x7e,0xb3,0x63,0x30, 0x95,0x34,0xe6,0x85,
+ 0x7e,0x9a,0xdd,0xe6, 0x43,0xd6,0x59,0xf8,
+ 0x6a,0xb8,0x8f,0x5f, 0x5d,0xd9,0x55,0x41,
+ 0x12,0xf9,0x98,0xc6, 0x93,0x7c,0x3f,0x46,
+ 0xab,0x7c,0x8b,0x28, 0xde,0x9a,0xb1,0xf0,
+ 0x6c,0x43,0x2a,0xb3, 0x70,0xc5,0x9d,0xc0,
+ 0x26,0xcf,0xad,0x9c, 0x87,0x9b,0x3f,0x7c,
+ 0x24,0xac,0xe7,0xd4, 0xe8,0x14,0xe3,0x3e,
+ 0xf6,0x8a,0x97,0x87, 0x63,0x2c,0x88,0xdc,
+ 0xc5,0x23,0x68,0x6e, 0x94,0xe1,0x09,0xc4,
+ 0x44,0xda,0x8f,0xa7, 0x9f,0xc4,0x52,0xa4,
+ 0x18,0x1d,0x3c,0x08, 0xca,0x0a,0x3e,0xb4,
+ 0xbf,0xbe,0xc6,0x47, 0xe2,0x89,0x2b,0x07,
+ 0x71,0xd9,0xc8,0x6a, 0x06,0xd5,0xd0,0x47,
+ 0x4e,0x07,0x4f,0x6b, 0xdb,0xdf,0x3d,0xf0,
+ 0x7c,0x5f,0x49,0x70, 0x17,0x4f,0x9f,0x33,
+ 0x7e,0x4b,0x72,0x3b, 0x8c,0x68,0x22,0xf9,
+ 0xd2,0xad,0xe4,0xe4, 0xb2,0x61,0x9d,0xb8,
+ 0xc2,0x5c,0xf0,0x3b, 0x08,0xb2,0x75,0x30,
+ 0x3a,0xd0,0x7d,0xf9, 0xb2,0x00,0x40,0x56,
+ 0x79,0xe2,0x0d,0x31, 0x72,0xe2,0xc2,0xd1,
+ 0x2e,0x27,0xe7,0xc8, 0x96,0x1a,0xc6,0x7e,
+ 0xb8,0xc1,0x93,0xfb, 0x1d,0xbc,0xed,0x97,
+ 0x2f,0x2f,0xea,0xa1, 0x40,0x49,0xf6,0x1d,
+ 0xab,0x54,0x46,0x2e, 0x73,0xf2,0x74,0xf1,
+ 0x6d,0x5c,0xe6,0xa0, 0xd4,0x73,0x1c,0xbc,
+ 0x07,0x81,0xf5,0x94, 0xe6,0x18,0xdc,0x42,
+ 0x68,0xb9,0xeb,0xfb, 0xa3,0x76,0x8c,0x83,
+ 0x98,0xe9,0x96,0xa6, 0xa6,0x5e,0x0e,0xd1,
+ 0xfc,0xb7,0x8e,0x8b, 0x9e,0xa4,0x00,0x76,
+ 0x0e,0x35,0x92,0x5e, 0x05,0xa1,0x92,0xc4,
+ 0x0c,0xd1,0xec,0x8c, 0x04,0x8e,0x65,0x56,
+ 0x43,0xae,0x16,0x18, 0x2e,0x3e,0xfe,0x47,
+ 0x92,0xe1,0x76,0x1b, 0xb6,0xcc,0x0b,0x82,
+ 0xe1,0x8c,0x7b,0x43, 0xe4,0x90,0xed,0x28,
+ 0x0b,0xe6,0x05,0xea, 0x4a,0xc0,0xf1,0x12,
+ 0x54,0x09,0x93,0xda, 0xfc,0xf4,0x86,0xff,
+ 0x4c,0xaa,0x7d,0xbe, 0xd0,0x4a,0xa6,0x9d,
+ 0x6b,0x27,0x8f,0xb1, 0xb5,0x3a,0x9b,0xce,
+ 0xe2,0x5c,0x29,0x35, 0xd6,0xe7,0xf3,0xa4,
+ 0x5e,0x70,0xf6,0xc6, 0xde,0x63,0x86,0xf7,
+ 0xc9,0xab,0x42,0xb9, 0xe7,0x5d,0x1c,0x68,
+ 0x73,0xa3,0xed,0xb0, 0xa0,0xb6,0x18,0x15,
+ 0xe6,0x57,0x4c,0x21, 0xf7,0xf3,0xc6,0x32,
+ 0x4d,0x07,0x4a,0x14, 0xde,0xb2,0xc7,0xca,
+ 0xf0,0x78,0xc4,0x85, 0xe3,0xdc,0xfb,0x35,
+ 0x7c,0x6b,0xc0,0xb8, 0xcd,0x7a,0x22,0xfc,
+ 0xe4,0xe8,0xe2,0x98, 0x6c,0x8e,0xdf,0x37,
+ 0x8e,0x0f,0x25,0x23, 0xdd,0xea,0x40,0x6f,
+ 0xb3,0x07,0x7e,0x7a, 0x6b,0xa1,0xa1,0xcf,
+ 0x24,0xd9,0xad,0x72, 0x7a,0x45,0x49,0xca,
+ 0xfe,0xc7,0x2e,0x6d, 0xaa,0xc1,0x08,0x2c,
+ 0xe6,0xde,0xde,0x73, 0x01,0x9c,0xdc,0x65,
+ 0x3a,0xdf,0xc6,0x15, 0x37,0x62,0x0b,0x2c,
+ 0x9a,0x36,0xed,0x37, 0xd9,0xfc,0xa9,0xb3,
+ 0x32,0xc3,0xde,0x26, 0xe7,0xf0,0x3f,0x02,
+ 0xed,0x35,0x74,0xea, 0xdd,0x32,0xe9,0x96,
+ 0x75,0x66,0xb8,0xf0, 0x75,0x98,0x8f,0x3a,
+ 0xd0,0xc2,0xa1,0x98, 0x5f,0xf9,0x32,0x31,
+ 0x00,0x18,0x7d,0xc5, 0x9d,0x15,0x5b,0xdc,
+ 0x13,0x37,0x69,0xfc, 0x95,0x7a,0x62,0x0e,
+ 0x8a,0x86,0xed,0x18, 0x78,0x3c,0x49,0xf4,
+ 0x18,0x73,0xcd,0x2e, 0x7b,0xa3,0x40,0xd7,
+ 0x01,0xf6,0xc7,0x2a, 0xc5,0xce,0x13,0x09,
+ 0xb1,0xe5,0x25,0x17, 0xdf,0x9d,0x7e,0x0b,
+ 0x50,0x46,0x62,0x78, 0xb5,0x25,0xb2,0xd9,
+ 0x65,0xfa,0x5b,0xf7, 0xfe,0xc6,0xe0,0x7b,
+ 0x7b,0x4e,0x14,0x2e, 0x0d,0x3a,0xd0,0xe0,
+ 0xa0,0xd2,0xeb,0x4d, 0x87,0x11,0x42,0x28,
+ 0x02,0x7e,0xa8,0x56, 0x5b,0x53,0xbd,0x76,
+ 0x47,0x8f,0x5f,0x8b, 0xc7,0xd9,0x72,0xf7,
+ 0x11,0xbb,0x94,0xdb, 0x0d,0x07,0xb7,0x0a,
+ 0xcc,0x41,0x00,0xcd, 0xd0,0x50,0x25,0x31,
+ 0xc9,0x47,0x6b,0xdd, 0x3f,0x70,0x24,0x3e,
+ 0xde,0x02,0x62,0x6c, 0xb4,0x44,0x92,0x8e,
+ 0x98,0x9c,0x0e,0x30, 0x2f,0x80,0xb9,0x5e,
+ 0x75,0x90,0xa6,0x02, 0xf0,0xed,0xb0,0x8b,
+ 0x44,0xa3,0x59,0x2d, 0xc3,0x08,0xe5,0xd9,
+ 0x89,0x6a,0x71,0x44, 0x04,0xc4,0xb2,0x61,
+ 0x5b,0xf5,0x46,0x44, 0xdc,0x36,0x2e,0xfd,
+ 0x41,0xf5,0xa1,0x3a, 0xb3,0x93,0x74,0x7d,
+ 0x54,0x5e,0x64,0xdc, 0xbc,0xd7,0x07,0x48,
+ 0x3e,0x73,0x81,0x22, 0x9c,0x5a,0xf6,0xde,
+ 0x94,0x42,0xe1,0x6c, 0x92,0xe7,0x6d,0xa0,
+ 0x5e,0xc3,0xd6,0xe9, 0x84,0xd9,0xba,0x57,
+ 0xef,0x85,0x6a,0x9b, 0xe6,0x9a,0x2b,0xf8,
+ 0x8d,0xfe,0x9d,0xad, 0x70,0x26,0x05,0x14,
+ 0x45,0x07,0xcb,0x72, 0xd4,0x8b,0x14,0x44,
+ 0x74,0x40,0x9c,0x29, 0x8b,0xba,0x40,0x09,
+ 0x52,0xfc,0xc5,0x40, 0xb1,0x25,0x69,0xaa,
+ 0x8f,0x12,0xc4,0xc6, 0x2b,0x3f,0x73,0x9d,
+ 0xff,0x52,0xd4,0xac, 0x77,0x43,0xdc,0xd2,
+ 0x06,0x9a,0x1b,0xfc, 0x0c,0x8f,0x6b,0x59,
+ 0xa5,0xd4,0xde,0x06, 0x16,0x34,0xef,0x75,
+ 0x22,0x54,0x9c,0x53, 0x38,0x0b,0x57,0xc7,
+ 0xaa,0x78,0x2d,0x3a, 0x9b,0xdd,0xed,0xb5,
+ 0x0b,0xb0,0x08,0x5f, 0x57,0xdb,0xfc,0xbe,
+ 0x44,0xfd,0x71,0x5f, 0x71,0x14,0xd5,0x14,
+ 0x70,0xb6,0xee,0xd0, 0xf3,0x37,0x6f,0x57,
+ 0x55,0x3c,0x7c,0x23, 0x6f,0xbe,0x83,0x5c,
+ 0xb5,0x64,0xfd,0x6d, 0x7c,0xe4,0x05,0x2b,
+ 0xdb,0xc4,0xf5,0xa0, 0xd3,0xa6,0x15,0x48,
+ 0xc2,0x50,0xf8,0xf7, 0xc2,0xab,0xb5,0x6a,
+ 0x0d,0x1a,0xb5,0x30, 0x33,0xf8,0x12,0x2d,
+ 0xfb,0xa6,0x2e,0xe5, 0xbe,0x40,0xba,0x48,
+ 0xef,0x05,0xc8,0x37, 0x3a,0x36,0xad,0x99,
+ 0x77,0x87,0x84,0xac, 0xd8,0xcb,0x7a,0x88,
+ 0x3e,0x2d,0x8b,0xbe, 0x9a,0x35,0x88,0x26,
+ 0xe9,0x20,0xd4,0x66, 0x80,0x8b,0xf8,0x54,
+ 0xba,0xcd,0xa8,0x47, 0x35,0x1b,0xc4,0x09,
+ 0x6d,0xff,0x0e,0x60, 0x7c,0xf3,0x68,0xbf,
+ 0xe3,0xe9,0x73,0x07, 0x84,0xf0,0x08,0x45,
+ 0x97,0x65,0x94,0xd1, 0x35,0x4e,0x67,0x0c,
+ 0xe3,0xb7,0x61,0x7b, 0x09,0x22,0xed,0x18,
+ 0xee,0x0b,0x54,0xc0, 0xab,0x8b,0xaa,0x71,
+ 0x4c,0x40,0xbf,0xf7, 0xe0,0x7e,0x08,0xaa,
+ },
+ .mlen = 1024,
+ .m = {
+ 0x1d,0xea,0xe5,0x2b, 0x4c,0x22,0x4d,0xf3,
+ 0x15,0x53,0xcb,0x41, 0xf5,0xcf,0x0b,0x7b,
+ 0xc9,0x80,0xc0,0x95, 0xd2,0x7b,0x08,0x4b,
+ 0x3d,0xcd,0xd8,0x3b, 0x2f,0x18,0xd4,0x70,
+ 0x38,0xb2,0xa7,0x2f, 0x7f,0xba,0xd8,0xed,
+ 0xbc,0x8f,0xac,0xe4, 0xe2,0x11,0x2d,0x6d,
+ 0xe6,0xa4,0x36,0x90, 0xc2,0x7f,0xdf,0xe3,
+ 0xdc,0x50,0xdb,0x6c, 0x56,0xcf,0x7d,0xd6,
+ 0xd0,0xcb,0xd6,0x9b, 0x01,0xbb,0xef,0x1c,
+ 0x0a,0x6c,0x92,0x23, 0xeb,0x77,0xf9,0xd1,
+ 0x25,0xdc,0x94,0x30, 0x30,0xa4,0x96,0x3e,
+ 0xdf,0x52,0x4c,0xe7, 0xdf,0x27,0x9f,0x73,
+ 0x78,0x0c,0x8c,0x7f, 0x9d,0xae,0x79,0x5d,
+ 0x91,0x5e,0x4b,0x02, 0xa9,0x31,0x9c,0xff,
+ 0x46,0x73,0xec,0x0d, 0x5a,0xb8,0xeb,0x48,
+ 0x19,0x9c,0x44,0xe0, 0xc8,0x81,0x96,0x4c,
+ 0x47,0x0c,0xe7,0x1d, 0x2a,0x9c,0xd5,0xe0,
+ 0xe7,0xd6,0xa0,0x88, 0xf0,0xf6,0xda,0xa7,
+ 0x6a,0xdd,0xfd,0x4f, 0x00,0x6e,0x25,0x7d,
+ 0xb9,0x81,0x19,0x2f, 0x4e,0xcc,0x8d,0x6e,
+ 0xa6,0x92,0xcf,0xd8, 0x6e,0x78,0x0a,0xf6,
+ 0x8a,0x43,0xeb,0x60, 0x0c,0x8b,0x93,0x50,
+ 0x88,0xd1,0x67,0x05, 0x0c,0xdc,0x43,0x85,
+ 0x50,0x91,0x63,0xa4, 0x32,0x14,0x66,0x84,
+ 0xdb,0x04,0x9f,0x77, 0x95,0x60,0x19,0xc6,
+ 0x98,0x60,0x62,0xe4, 0xc6,0xee,0x70,0x76,
+ 0xb0,0x59,0x80,0x59, 0x46,0xae,0x99,0x26,
+ 0x62,0x4a,0xf0,0x45, 0x8f,0xf0,0x70,0x5b,
+ 0x52,0xfc,0xee,0x4d, 0x30,0x47,0xc8,0xae,
+ 0xe2,0xbc,0x2c,0x73, 0x78,0x67,0xf1,0x00,
+ 0xb4,0xda,0x01,0xad, 0x3b,0xc4,0x5c,0x6c,
+ 0x65,0xca,0x84,0x22, 0x95,0x32,0x95,0x20,
+ 0x4d,0xdc,0x96,0x2e, 0x61,0xe4,0xc8,0xec,
+ 0x2d,0xbf,0xc1,0x5d, 0x70,0xf9,0x75,0xf2,
+ 0xad,0x0a,0xc9,0xd7, 0x0a,0x81,0x3c,0xa1,
+ 0x13,0xec,0x63,0xd4, 0xd0,0x67,0xf4,0xcc,
+ 0x6e,0xb8,0x52,0x08, 0x46,0xc9,0x2a,0x92,
+ 0x59,0xd9,0x14,0x17, 0xde,0x2f,0xc7,0x36,
+ 0xd5,0xd5,0xfc,0x8a, 0x63,0xd5,0x5f,0xe3,
+ 0xdd,0x55,0x00,0x8e, 0x5e,0xc9,0xed,0x04,
+ 0x1d,0xeb,0xae,0xc5, 0xd0,0xf9,0x73,0x28,
+ 0xf3,0x81,0xd5,0xb4, 0x60,0xb2,0x42,0x81,
+ 0x68,0xf3,0xb9,0x73, 0x07,0x2e,0x34,0x8e,
+ 0x47,0x12,0xae,0x7c, 0xa8,0xc2,0xce,0xad,
+ 0x0f,0x6e,0x44,0xa5, 0x35,0x5e,0x61,0x6b,
+ 0xfc,0x67,0x9c,0x82, 0xa1,0xd2,0xff,0xfe,
+ 0x60,0x7c,0x40,0x02, 0x24,0x9e,0x8b,0x90,
+ 0xa0,0x89,0xd9,0x83, 0x04,0xd8,0xef,0x9c,
+ 0x96,0x28,0x77,0x3e, 0xe3,0xb0,0xf8,0x3d,
+ 0xfb,0x91,0x8f,0x6f, 0x83,0x58,0x1e,0x4b,
+ 0x64,0xc7,0xf6,0xe0, 0x85,0x03,0xe3,0xf9,
+ 0x6b,0xc9,0x9e,0x9d, 0x57,0x25,0xe4,0x69,
+ 0x08,0x59,0x28,0x4a, 0x52,0x9c,0x49,0x19,
+ 0x24,0x49,0xba,0xb1, 0x82,0xd4,0xcf,0xd0,
+ 0x1e,0x1d,0xc2,0x02, 0x42,0x4e,0xdf,0xf7,
+ 0x2b,0x3d,0x99,0xf6, 0x99,0xa4,0x3a,0xe1,
+ 0x9d,0x68,0xc8,0x08, 0xec,0xec,0x1c,0xa8,
+ 0x41,0x4a,0x27,0x84, 0xe9,0x0d,0x95,0x54,
+ 0x1a,0xca,0x5f,0x5d, 0x5a,0x96,0xb9,0x5b,
+ 0x6e,0xbc,0x39,0x7f, 0x7a,0x20,0xc5,0xb2,
+ 0x60,0x0c,0xa3,0x78, 0xc3,0x2b,0x87,0xcc,
+ 0xea,0xb0,0x4d,0x27, 0xfb,0x6c,0x58,0x51,
+ 0xce,0x90,0xca,0xd6, 0x86,0x91,0x4d,0x2c,
+ 0x8c,0x82,0xf0,0xc9, 0x9a,0x0a,0x73,0xb3,
+ 0xcb,0xa9,0xd4,0x26, 0x4d,0x74,0xbe,0x0e,
+ 0x4a,0x6e,0x10,0xeb, 0x4e,0xba,0x4e,0xba,
+ 0x0d,0x26,0x69,0x87, 0x5e,0x08,0x2b,0x43,
+ 0xbe,0x97,0x4e,0x2a, 0x63,0xbc,0x52,0xb7,
+ 0xda,0x23,0x23,0x11, 0xfa,0xcf,0x89,0xac,
+ 0x90,0x5f,0x60,0x7a, 0x50,0xb7,0xbe,0x79,
+ 0x0b,0x2c,0xf0,0x27, 0xf0,0xfb,0xaf,0x64,
+ 0xc8,0x57,0x7c,0xeb, 0x1c,0xf7,0x36,0xec,
+ 0x09,0x97,0x66,0x31, 0x54,0xe4,0x00,0xcf,
+ 0x68,0x24,0x77,0x1a, 0xbc,0x27,0x3a,0xad,
+ 0x8a,0x01,0x7e,0x45, 0xe7,0xe4,0xa4,0xeb,
+ 0x38,0x62,0x9d,0x90, 0xea,0x00,0x9c,0x03,
+ 0x5e,0xb2,0x7d,0xd8, 0x2f,0xe9,0xc9,0x3c,
+ 0x1a,0x5c,0x21,0x1a, 0x59,0x45,0x62,0x47,
+ 0x93,0x1b,0xdc,0xd8, 0x3e,0x07,0x8b,0x75,
+ 0xd0,0x6d,0xcc,0x8d, 0xec,0x79,0xa8,0x9a,
+ 0x51,0xa5,0x50,0x18, 0xae,0x44,0x93,0x75,
+ 0xc1,0xc8,0x1e,0x10, 0x59,0x1e,0x0b,0xb3,
+ 0x06,0x30,0xa8,0x66, 0x8d,0x8e,0xd6,0x4d,
+ 0x0d,0x8a,0xb4,0x28, 0xdc,0xfb,0x5d,0x59,
+ 0xe0,0x92,0x77,0x38, 0xfa,0xad,0x46,0x46,
+ 0x25,0x15,0x4c,0xca, 0x09,0x2b,0x31,0xe9,
+ 0x36,0xe8,0xc2,0x67, 0x34,0x4d,0x5e,0xa0,
+ 0x8f,0x9a,0xe8,0x7f, 0xf2,0x2a,0x92,0x78,
+ 0xde,0x09,0x75,0xe7, 0xe5,0x50,0x0a,0x2e,
+ 0x88,0x63,0xc0,0x8f, 0xa8,0x73,0x0f,0xe5,
+ 0x1e,0x9d,0xdb,0xce, 0x53,0xe0,0x42,0x94,
+ 0x7b,0x5c,0xa1,0x5e, 0x1e,0x8f,0x0a,0x6e,
+ 0x8b,0x1a,0xad,0x93, 0x70,0x86,0xf1,0x69,
+ 0x70,0x93,0x24,0xe3, 0x83,0x2f,0xa8,0x04,
+ 0xba,0x27,0x0a,0x2e, 0x03,0xeb,0x69,0xd9,
+ 0x56,0x0e,0xc4,0x10, 0x55,0x31,0x2c,0x3f,
+ 0xd1,0xb2,0x94,0x0f, 0x28,0x15,0x3c,0x02,
+ 0x15,0x5e,0xec,0x26, 0x9c,0xc3,0xfc,0xa7,
+ 0x5c,0xb0,0xfa,0xc0, 0x02,0xf9,0x01,0x3f,
+ 0x01,0x73,0x24,0x22, 0x50,0x28,0x2a,0xca,
+ 0xb1,0xf2,0x03,0x00, 0x2f,0xc6,0x6f,0x28,
+ 0x4f,0x4b,0x4f,0x1a, 0x9a,0xb8,0x16,0x93,
+ 0x31,0x60,0x7c,0x3d, 0x35,0xc8,0xd6,0x90,
+ 0xde,0x8c,0x89,0x39, 0xbd,0x21,0x11,0x05,
+ 0xe8,0xc4,0x04,0x3b, 0x65,0xa5,0x15,0xcf,
+ 0xcf,0x15,0x14,0xf6, 0xe7,0x2e,0x3c,0x47,
+ 0x59,0x0b,0xaa,0xc0, 0xd4,0xab,0x04,0x14,
+ 0x9c,0xd7,0xe2,0x43, 0xc7,0x87,0x09,0x03,
+ 0x27,0xd2,0x0a,0xff, 0x8d,0xd5,0x80,0x34,
+ 0x93,0xa2,0x2c,0xb1, 0x4e,0x16,0x2d,0x82,
+ 0x51,0x5c,0x3c,0xe5, 0x75,0x51,0x7b,0xb4,
+ 0xd8,0x1e,0x59,0x98, 0x0f,0x75,0xed,0x02,
+ 0x1c,0x13,0xf6,0x02, 0xda,0xf9,0x47,0xf7,
+ 0x45,0x25,0x0f,0x58, 0x22,0x5d,0xef,0xf0,
+ 0x1b,0xdb,0xae,0xaf, 0xbe,0xc6,0xe1,0xcd,
+ 0x70,0x46,0x6e,0x03, 0x9a,0x20,0x77,0x00,
+ 0x3c,0x32,0xb5,0x8f, 0x04,0xb6,0x6f,0xa2,
+ 0x31,0xc9,0x7c,0xf9, 0x84,0x67,0x87,0xfb,
+ 0x7b,0x13,0xb0,0x4d, 0x35,0xfd,0x37,0x5b,
+ 0xf4,0x25,0xf0,0x02, 0x74,0xa0,0x69,0xd4,
+ 0x53,0x61,0x4b,0x54, 0x68,0x94,0x0e,0x08,
+ 0x25,0x82,0x90,0xfc, 0x25,0xb6,0x63,0xe2,
+ 0x07,0x9f,0x42,0xf1, 0xbb,0x33,0xea,0xab,
+ 0x92,0x54,0x2b,0x9f, 0x88,0xc0,0x31,0x2b,
+ 0xfd,0x36,0x50,0x80, 0xfc,0x1a,0xff,0xab,
+ 0xe8,0xc4,0x7f,0xb6, 0x98,0xb9,0x2e,0x17,
+ 0xca,0x28,0x3d,0xdf, 0x0f,0x07,0x43,0x20,
+ 0xf0,0x07,0xea,0xe5, 0xcd,0x4e,0x81,0x34,
+ },
+ .h = {
+ 0x9d,0x22,0x88,0xfd, 0x41,0x43,0x88,0x45,
+ 0x34,0xfe,0x85,0xc4, 0xb9,0xff,0xe1,0x55,
+ 0x40,0x1d,0x25,0x37, 0xd1,0xf8,0xfc,0x2b,
+ 0x3a,0xf5,0x3b,0x69, 0xbf,0xa6,0x9d,0xed,
+ },
+ },
+ };
+ static uint32_t k[268];
+ uint8_t h[32];
+ unsigned i, j;
+ int result = 0;
+
+ for (i = 0; i < __arraycount(C); i++) {
+ for (j = 0; j < 268; j++)
+ k[j] = le32dec(C[i].k + 4*j);
+ nh(h, C[i].m, C[i].mlen, k);
+ if (memcmp(h, C[i].h, 32)) {
+ char prefix[10];
+ snprintf(prefix, sizeof prefix, "nh %u", i);
+ hexdump(printf, prefix, h, 32);
+ result = -1;
+ }
+ }
+
+ return result;
+}
+
+/* https://github.com/google/adiantum/blob/a5ad5134ab11b10a3ee982c52385953fac88fedc/test_vectors/ours/NHPoly1305/NHPoly1305.json */
+static int
+nhpoly1305_selftest(void)
+{
+ static const struct {
+ uint8_t k[1088];
+ unsigned mlen;
+ uint8_t m[1024];
+ uint8_t h[16];
+ } C[] = {
+ [0] = { /* 0-byte message */
+ .k = {
+ /* Poly1305 key */
+ 0xd2,0x5d,0x4c,0xdd, 0x8d,0x2b,0x7f,0x7a,
+ 0xd9,0xbe,0x71,0xec, 0xd1,0x83,0x52,0xe3,
+
+ /* NH key */
+ 0xe1,0xad,0xd7,0x5c, 0x0a,0x75,0x9d,0xec,
+ 0x1d,0x13,0x7e,0x5d, 0x71,0x07,0xc9,0xe4,
+ 0x57,0x2d,0x44,0x68, 0xcf,0xd8,0xd6,0xc5,
+ 0x39,0x69,0x7d,0x32, 0x75,0x51,0x4f,0x7e,
+ 0xb2,0x4c,0xc6,0x90, 0x51,0x6e,0xd9,0xd6,
+ 0xa5,0x8b,0x2d,0xf1, 0x94,0xf9,0xf7,0x5e,
+ 0x2c,0x84,0x7b,0x41, 0x0f,0x88,0x50,0x89,
+ 0x30,0xd9,0xa1,0x38, 0x46,0x6c,0xc0,0x4f,
+ 0xe8,0xdf,0xdc,0x66, 0xab,0x24,0x43,0x41,
+ 0x91,0x55,0x29,0x65, 0x86,0x28,0x5e,0x45,
+ 0xd5,0x2d,0xb7,0x80, 0x08,0x9a,0xc3,0xd4,
+ 0x9a,0x77,0x0a,0xd4, 0xef,0x3e,0xe6,0x3f,
+ 0x6f,0x2f,0x9b,0x3a, 0x7d,0x12,0x1e,0x80,
+ 0x6c,0x44,0xa2,0x25, 0xe1,0xf6,0x60,0xe9,
+ 0x0d,0xaf,0xc5,0x3c, 0xa5,0x79,0xae,0x64,
+ 0xbc,0xa0,0x39,0xa3, 0x4d,0x10,0xe5,0x4d,
+ 0xd5,0xe7,0x89,0x7a, 0x13,0xee,0x06,0x78,
+ 0xdc,0xa4,0xdc,0x14, 0x27,0xe6,0x49,0x38,
+ 0xd0,0xe0,0x45,0x25, 0x36,0xc5,0xf4,0x79,
+ 0x2e,0x9a,0x98,0x04, 0xe4,0x2b,0x46,0x52,
+ 0x7c,0x33,0xca,0xe2, 0x56,0x51,0x50,0xe2,
+ 0xa5,0x9a,0xae,0x18, 0x6a,0x13,0xf8,0xd2,
+ 0x21,0x31,0x66,0x02, 0xe2,0xda,0x8d,0x7e,
+ 0x41,0x19,0xb2,0x61, 0xee,0x48,0x8f,0xf1,
+ 0x65,0x24,0x2e,0x1e, 0x68,0xce,0x05,0xd9,
+ 0x2a,0xcf,0xa5,0x3a, 0x57,0xdd,0x35,0x91,
+ 0x93,0x01,0xca,0x95, 0xfc,0x2b,0x36,0x04,
+ 0xe6,0x96,0x97,0x28, 0xf6,0x31,0xfe,0xa3,
+ 0x9d,0xf6,0x6a,0x1e, 0x80,0x8d,0xdc,0xec,
+ 0xaf,0x66,0x11,0x13, 0x02,0x88,0xd5,0x27,
+ 0x33,0xb4,0x1a,0xcd, 0xa3,0xf6,0xde,0x31,
+ 0x8e,0xc0,0x0e,0x6c, 0xd8,0x5a,0x97,0x5e,
+ 0xdd,0xfd,0x60,0x69, 0x38,0x46,0x3f,0x90,
+ 0x5e,0x97,0xd3,0x32, 0x76,0xc7,0x82,0x49,
+ 0xfe,0xba,0x06,0x5f, 0x2f,0xa2,0xfd,0xff,
+ 0x80,0x05,0x40,0xe4, 0x33,0x03,0xfb,0x10,
+ 0xc0,0xde,0x65,0x8c, 0xc9,0x8d,0x3a,0x9d,
+ 0xb5,0x7b,0x36,0x4b, 0xb5,0x0c,0xcf,0x00,
+ 0x9c,0x87,0xe4,0x49, 0xad,0x90,0xda,0x4a,
+ 0xdd,0xbd,0xff,0xe2, 0x32,0x57,0xd6,0x78,
+ 0x36,0x39,0x6c,0xd3, 0x5b,0x9b,0x88,0x59,
+ 0x2d,0xf0,0x46,0xe4, 0x13,0x0e,0x2b,0x35,
+ 0x0d,0x0f,0x73,0x8a, 0x4f,0x26,0x84,0x75,
+ 0x88,0x3c,0xc5,0x58, 0x66,0x18,0x1a,0xb4,
+ 0x64,0x51,0x34,0x27, 0x1b,0xa4,0x11,0xc9,
+ 0x6d,0x91,0x8a,0xfa, 0x32,0x60,0x9d,0xd7,
+ 0x87,0xe5,0xaa,0x43, 0x72,0xf8,0xda,0xd1,
+ 0x48,0x44,0x13,0x61, 0xdc,0x8c,0x76,0x17,
+ 0x0c,0x85,0x4e,0xf3, 0xdd,0xa2,0x42,0xd2,
+ 0x74,0xc1,0x30,0x1b, 0xeb,0x35,0x31,0x29,
+ 0x5b,0xd7,0x4c,0x94, 0x46,0x35,0xa1,0x23,
+ 0x50,0xf2,0xa2,0x8e, 0x7e,0x4f,0x23,0x4f,
+ 0x51,0xff,0xe2,0xc9, 0xa3,0x7d,0x56,0x8b,
+ 0x41,0xf2,0xd0,0xc5, 0x57,0x7e,0x59,0xac,
+ 0xbb,0x65,0xf3,0xfe, 0xf7,0x17,0xef,0x63,
+ 0x7c,0x6f,0x23,0xdd, 0x22,0x8e,0xed,0x84,
+ 0x0e,0x3b,0x09,0xb3, 0xf3,0xf4,0x8f,0xcd,
+ 0x37,0xa8,0xe1,0xa7, 0x30,0xdb,0xb1,0xa2,
+ 0x9c,0xa2,0xdf,0x34, 0x17,0x3e,0x68,0x44,
+ 0xd0,0xde,0x03,0x50, 0xd1,0x48,0x6b,0x20,
+ 0xe2,0x63,0x45,0xa5, 0xea,0x87,0xc2,0x42,
+ 0x95,0x03,0x49,0x05, 0xed,0xe0,0x90,0x29,
+ 0x1a,0xb8,0xcf,0x9b, 0x43,0xcf,0x29,0x7a,
+ 0x63,0x17,0x41,0x9f, 0xe0,0xc9,0x10,0xfd,
+ 0x2c,0x56,0x8c,0x08, 0x55,0xb4,0xa9,0x27,
+ 0x0f,0x23,0xb1,0x05, 0x6a,0x12,0x46,0xc7,
+ 0xe1,0xfe,0x28,0x93, 0x93,0xd7,0x2f,0xdc,
+ 0x98,0x30,0xdb,0x75, 0x8a,0xbe,0x97,0x7a,
+ 0x02,0xfb,0x8c,0xba, 0xbe,0x25,0x09,0xbe,
+ 0xce,0xcb,0xa2,0xef, 0x79,0x4d,0x0e,0x9d,
+ 0x1b,0x9d,0xb6,0x39, 0x34,0x38,0xfa,0x07,
+ 0xec,0xe8,0xfc,0x32, 0x85,0x1d,0xf7,0x85,
+ 0x63,0xc3,0x3c,0xc0, 0x02,0x75,0xd7,0x3f,
+ 0xb2,0x68,0x60,0x66, 0x65,0x81,0xc6,0xb1,
+ 0x42,0x65,0x4b,0x4b, 0x28,0xd7,0xc7,0xaa,
+ 0x9b,0xd2,0xdc,0x1b, 0x01,0xe0,0x26,0x39,
+ 0x01,0xc1,0x52,0x14, 0xd1,0x3f,0xb7,0xe6,
+ 0x61,0x41,0xc7,0x93, 0xd2,0xa2,0x67,0xc6,
+ 0xf7,0x11,0xb5,0xf5, 0xea,0xdd,0x19,0xfb,
+ 0x4d,0x21,0x12,0xd6, 0x7d,0xf1,0x10,0xb0,
+ 0x89,0x07,0xc7,0x5a, 0x52,0x73,0x70,0x2f,
+ 0x32,0xef,0x65,0x2b, 0x12,0xb2,0xf0,0xf5,
+ 0x20,0xe0,0x90,0x59, 0x7e,0x64,0xf1,0x4c,
+ 0x41,0xb3,0xa5,0x91, 0x08,0xe6,0x5e,0x5f,
+ 0x05,0x56,0x76,0xb4, 0xb0,0xcd,0x70,0x53,
+ 0x10,0x48,0x9c,0xff, 0xc2,0x69,0x55,0x24,
+ 0x87,0xef,0x84,0xea, 0xfb,0xa7,0xbf,0xa0,
+ 0x91,0x04,0xad,0x4f, 0x8b,0x57,0x54,0x4b,
+ 0xb6,0xe9,0xd1,0xac, 0x37,0x2f,0x1d,0x2e,
+ 0xab,0xa5,0xa4,0xe8, 0xff,0xfb,0xd9,0x39,
+ 0x2f,0xb7,0xac,0xd1, 0xfe,0x0b,0x9a,0x80,
+ 0x0f,0xb6,0xf4,0x36, 0x39,0x90,0x51,0xe3,
+ 0x0a,0x2f,0xb6,0x45, 0x76,0x89,0xcd,0x61,
+ 0xfe,0x48,0x5f,0x75, 0x1d,0x13,0x00,0x62,
+ 0x80,0x24,0x47,0xe7, 0xbc,0x37,0xd7,0xe3,
+ 0x15,0xe8,0x68,0x22, 0xaf,0x80,0x6f,0x4b,
+ 0xa8,0x9f,0x01,0x10, 0x48,0x14,0xc3,0x02,
+ 0x52,0xd2,0xc7,0x75, 0x9b,0x52,0x6d,0x30,
+ 0xac,0x13,0x85,0xc8, 0xf7,0xa3,0x58,0x4b,
+ 0x49,0xf7,0x1c,0x45, 0x55,0x8c,0x39,0x9a,
+ 0x99,0x6d,0x97,0x27, 0x27,0xe6,0xab,0xdd,
+ 0x2c,0x42,0x1b,0x35, 0xdd,0x9d,0x73,0xbb,
+ 0x6c,0xf3,0x64,0xf1, 0xfb,0xb9,0xf7,0xe6,
+ 0x4a,0x3c,0xc0,0x92, 0xc0,0x2e,0xb7,0x1a,
+ 0xbe,0xab,0xb3,0x5a, 0xe5,0xea,0xb1,0x48,
+ 0x58,0x13,0x53,0x90, 0xfd,0xc3,0x8e,0x54,
+ 0xf9,0x18,0x16,0x73, 0xe8,0xcb,0x6d,0x39,
+ 0x0e,0xd7,0xe0,0xfe, 0xb6,0x9f,0x43,0x97,
+ 0xe8,0xd0,0x85,0x56, 0x83,0x3e,0x98,0x68,
+ 0x7f,0xbd,0x95,0xa8, 0x9a,0x61,0x21,0x8f,
+ 0x06,0x98,0x34,0xa6, 0xc8,0xd6,0x1d,0xf3,
+ 0x3d,0x43,0xa4,0x9a, 0x8c,0xe5,0xd3,0x5a,
+ 0x32,0xa2,0x04,0x22, 0xa4,0x19,0x1a,0x46,
+ 0x42,0x7e,0x4d,0xe5, 0xe0,0xe6,0x0e,0xca,
+ 0xd5,0x58,0x9d,0x2c, 0xaf,0xda,0x33,0x5c,
+ 0xb0,0x79,0x9e,0xc9, 0xfc,0xca,0xf0,0x2f,
+ 0xa8,0xb2,0x77,0xeb, 0x7a,0xa2,0xdd,0x37,
+ 0x35,0x83,0x07,0xd6, 0x02,0x1a,0xb6,0x6c,
+ 0x24,0xe2,0x59,0x08, 0x0e,0xfd,0x3e,0x46,
+ 0xec,0x40,0x93,0xf4, 0x00,0x26,0x4f,0x2a,
+ 0xff,0x47,0x2f,0xeb, 0x02,0x92,0x26,0x5b,
+ 0x53,0x17,0xc2,0x8d, 0x2a,0xc7,0xa3,0x1b,
+ 0xcd,0xbc,0xa7,0xe8, 0xd1,0x76,0xe3,0x80,
+ 0x21,0xca,0x5d,0x3b, 0xe4,0x9c,0x8f,0xa9,
+ 0x5b,0x7f,0x29,0x7f, 0x7c,0xd8,0xed,0x6d,
+ 0x8c,0xb2,0x86,0x85, 0xe7,0x77,0xf2,0x85,
+ 0xab,0x38,0xa9,0x9d, 0xc1,0x4e,0xc5,0x64,
+ 0x33,0x73,0x8b,0x59, 0x03,0xad,0x05,0xdf,
+ 0x25,0x98,0x31,0xde, 0xef,0x13,0xf1,0x9b,
+ 0x3c,0x91,0x9d,0x7b, 0xb1,0xfa,0xe6,0xbf,
+ 0x5b,0xed,0xa5,0x55, 0xe6,0xea,0x6c,0x74,
+ 0xf4,0xb9,0xe4,0x45, 0x64,0x72,0x81,0xc2,
+ 0x4c,0x28,0xd4,0xcd, 0xac,0xe2,0xde,0xf9,
+ 0xeb,0x5c,0xeb,0x61, 0x60,0x5a,0xe5,0x28,
+ },
+ .mlen = 0,
+ .h = {0},
+ },
+ [1] = { /* 16-byte message */
+ .k = {
+ /* Poly1305 key */
+ 0x29,0x21,0x43,0xcb, 0xcb,0x13,0x07,0xde,
+ 0xbf,0x48,0xdf,0x8a, 0x7f,0xa2,0x84,0xde,
+
+ /* NH key */
+ 0x72,0x23,0x9d,0xf5, 0xf0,0x07,0xf2,0x4c,
+ 0x20,0x3a,0x93,0xb9, 0xcd,0x5d,0xfe,0xcb,
+ 0x99,0x2c,0x2b,0x58, 0xc6,0x50,0x5f,0x94,
+ 0x56,0xc3,0x7c,0x0d, 0x02,0x3f,0xb8,0x5e,
+ 0x7b,0xc0,0x6c,0x51, 0x34,0x76,0xc0,0x0e,
+ 0xc6,0x22,0xc8,0x9e, 0x92,0xa0,0x21,0xc9,
+ 0x85,0x5c,0x7c,0xf8, 0xe2,0x64,0x47,0xc9,
+ 0xe4,0xa2,0x57,0x93, 0xf8,0xa2,0x69,0xcd,
+ 0x62,0x98,0x99,0xf4, 0xd7,0x7b,0x14,0xb1,
+ 0xd8,0x05,0xff,0x04, 0x15,0xc9,0xe1,0x6e,
+ 0x9b,0xe6,0x50,0x6b, 0x0b,0x3f,0x22,0x1f,
+ 0x08,0xde,0x0c,0x5b, 0x08,0x7e,0xc6,0x2f,
+ 0x6c,0xed,0xd6,0xb2, 0x15,0xa4,0xb3,0xf9,
+ 0xa7,0x46,0x38,0x2a, 0xea,0x69,0xa5,0xde,
+ 0x02,0xc3,0x96,0x89, 0x4d,0x55,0x3b,0xed,
+ 0x3d,0x3a,0x85,0x77, 0xbf,0x97,0x45,0x5c,
+ 0x9e,0x02,0x69,0xe2, 0x1b,0x68,0xbe,0x96,
+ 0xfb,0x64,0x6f,0x0f, 0xf6,0x06,0x40,0x67,
+ 0xfa,0x04,0xe3,0x55, 0xfa,0xbe,0xa4,0x60,
+ 0xef,0x21,0x66,0x97, 0xe6,0x9d,0x5c,0x1f,
+ 0x62,0x37,0xaa,0x31, 0xde,0xe4,0x9c,0x28,
+ 0x95,0xe0,0x22,0x86, 0xf4,0x4d,0xf3,0x07,
+ 0xfd,0x5f,0x3a,0x54, 0x2c,0x51,0x80,0x71,
+ 0xba,0x78,0x69,0x5b, 0x65,0xab,0x1f,0x81,
+ 0xed,0x3b,0xff,0x34, 0xa3,0xfb,0xbc,0x73,
+ 0x66,0x7d,0x13,0x7f, 0xdf,0x6e,0xe2,0xe2,
+ 0xeb,0x4f,0x6c,0xda, 0x7d,0x33,0x57,0xd0,
+ 0xd3,0x7c,0x95,0x4f, 0x33,0x58,0x21,0xc7,
+ 0xc0,0xe5,0x6f,0x42, 0x26,0xc6,0x1f,0x5e,
+ 0x85,0x1b,0x98,0x9a, 0xa2,0x1e,0x55,0x77,
+ 0x23,0xdf,0x81,0x5e, 0x79,0x55,0x05,0xfc,
+ 0xfb,0xda,0xee,0xba, 0x5a,0xba,0xf7,0x77,
+ 0x7f,0x0e,0xd3,0xe1, 0x37,0xfe,0x8d,0x2b,
+ 0xd5,0x3f,0xfb,0xd0, 0xc0,0x3c,0x0b,0x3f,
+ 0xcf,0x3c,0x14,0xcf, 0xfb,0x46,0x72,0x4c,
+ 0x1f,0x39,0xe2,0xda, 0x03,0x71,0x6d,0x23,
+ 0xef,0x93,0xcd,0x39, 0xd9,0x37,0x80,0x4d,
+ 0x65,0x61,0xd1,0x2c, 0x03,0xa9,0x47,0x72,
+ 0x4d,0x1e,0x0e,0x16, 0x33,0x0f,0x21,0x17,
+ 0xec,0x92,0xea,0x6f, 0x37,0x22,0xa4,0xd8,
+ 0x03,0x33,0x9e,0xd8, 0x03,0x69,0x9a,0xe8,
+ 0xb2,0x57,0xaf,0x78, 0x99,0x05,0x12,0xab,
+ 0x48,0x90,0x80,0xf0, 0x12,0x9b,0x20,0x64,
+ 0x7a,0x1d,0x47,0x5f, 0xba,0x3c,0xf9,0xc3,
+ 0x0a,0x0d,0x8d,0xa1, 0xf9,0x1b,0x82,0x13,
+ 0x3e,0x0d,0xec,0x0a, 0x83,0xc0,0x65,0xe1,
+ 0xe9,0x95,0xff,0x97, 0xd6,0xf2,0xe4,0xd5,
+ 0x86,0xc0,0x1f,0x29, 0x27,0x63,0xd7,0xde,
+ 0xb7,0x0a,0x07,0x99, 0x04,0x2d,0xa3,0x89,
+ 0xa2,0x43,0xcf,0xf3, 0xe1,0x43,0xac,0x4a,
+ 0x06,0x97,0xd0,0x05, 0x4f,0x87,0xfa,0xf9,
+ 0x9b,0xbf,0x52,0x70, 0xbd,0xbc,0x6c,0xf3,
+ 0x03,0x13,0x60,0x41, 0x28,0x09,0xec,0xcc,
+ 0xb1,0x1a,0xec,0xd6, 0xfb,0x6f,0x2a,0x89,
+ 0x5d,0x0b,0x53,0x9c, 0x59,0xc1,0x84,0x21,
+ 0x33,0x51,0x47,0x19, 0x31,0x9c,0xd4,0x0a,
+ 0x4d,0x04,0xec,0x50, 0x90,0x61,0xbd,0xbc,
+ 0x7e,0xc8,0xd9,0x6c, 0x98,0x1d,0x45,0x41,
+ 0x17,0x5e,0x97,0x1c, 0xc5,0xa8,0xe8,0xea,
+ 0x46,0x58,0x53,0xf7, 0x17,0xd5,0xad,0x11,
+ 0xc8,0x54,0xf5,0x7a, 0x33,0x90,0xf5,0x19,
+ 0xba,0x36,0xb4,0xfc, 0x52,0xa5,0x72,0x3d,
+ 0x14,0xbb,0x55,0xa7, 0xe9,0xe3,0x12,0xf7,
+ 0x1c,0x30,0xa2,0x82, 0x03,0xbf,0x53,0x91,
+ 0x2e,0x60,0x41,0x9f, 0x5b,0x69,0x39,0xf6,
+ 0x4d,0xc8,0xf8,0x46, 0x7a,0x7f,0xa4,0x98,
+ 0x36,0xff,0x06,0xcb, 0xca,0xe7,0x33,0xf2,
+ 0xc0,0x4a,0xf4,0x3c, 0x14,0x44,0x5f,0x6b,
+ 0x75,0xef,0x02,0x36, 0x75,0x08,0x14,0xfd,
+ 0x10,0x8e,0xa5,0x58, 0xd0,0x30,0x46,0x49,
+ 0xaf,0x3a,0xf8,0x40, 0x3d,0x35,0xdb,0x84,
+ 0x11,0x2e,0x97,0x6a, 0xb7,0x87,0x7f,0xad,
+ 0xf1,0xfa,0xa5,0x63, 0x60,0xd8,0x5e,0xbf,
+ 0x41,0x78,0x49,0xcf, 0x77,0xbb,0x56,0xbb,
+ 0x7d,0x01,0x67,0x05, 0x22,0xc8,0x8f,0x41,
+ 0xba,0x81,0xd2,0xca, 0x2c,0x38,0xac,0x76,
+ 0x06,0xc1,0x1a,0xc2, 0xce,0xac,0x90,0x67,
+ 0x57,0x3e,0x20,0x12, 0x5b,0xd9,0x97,0x58,
+ 0x65,0x05,0xb7,0x04, 0x61,0x7e,0xd8,0x3a,
+ 0xbf,0x55,0x3b,0x13, 0xe9,0x34,0x5a,0x37,
+ 0x36,0xcb,0x94,0x45, 0xc5,0x32,0xb3,0xa0,
+ 0x0c,0x3e,0x49,0xc5, 0xd3,0xed,0xa7,0xf0,
+ 0x1c,0x69,0xcc,0xea, 0xcc,0x83,0xc9,0x16,
+ 0x95,0x72,0x4b,0xf4, 0x89,0xd5,0xb9,0x10,
+ 0xf6,0x2d,0x60,0x15, 0xea,0x3c,0x06,0x66,
+ 0x9f,0x82,0xad,0x17, 0xce,0xd2,0xa4,0x48,
+ 0x7c,0x65,0xd9,0xf8, 0x02,0x4d,0x9b,0x4c,
+ 0x89,0x06,0x3a,0x34, 0x85,0x48,0x89,0x86,
+ 0xf9,0x24,0xa9,0x54, 0x72,0xdb,0x44,0x95,
+ 0xc7,0x44,0x1c,0x19, 0x11,0x4c,0x04,0xdc,
+ 0x13,0xb9,0x67,0xc8, 0xc3,0x3a,0x6a,0x50,
+ 0xfa,0xd1,0xfb,0xe1, 0x88,0xb6,0xf1,0xa3,
+ 0xc5,0x3b,0xdc,0x38, 0x45,0x16,0x26,0x02,
+ 0x3b,0xb8,0x8f,0x8b, 0x58,0x7d,0x23,0x04,
+ 0x50,0x6b,0x81,0x9f, 0xae,0x66,0xac,0x6f,
+ 0xcf,0x2a,0x9d,0xf1, 0xfd,0x1d,0x57,0x07,
+ 0xbe,0x58,0xeb,0x77, 0x0c,0xe3,0xc2,0x19,
+ 0x14,0x74,0x1b,0x51, 0x1c,0x4f,0x41,0xf3,
+ 0x32,0x89,0xb3,0xe7, 0xde,0x62,0xf6,0x5f,
+ 0xc7,0x6a,0x4a,0x2a, 0x5b,0x0f,0x5f,0x87,
+ 0x9c,0x08,0xb9,0x02, 0x88,0xc8,0x29,0xb7,
+ 0x94,0x52,0xfa,0x52, 0xfe,0xaa,0x50,0x10,
+ 0xba,0x48,0x75,0x5e, 0x11,0x1b,0xe6,0x39,
+ 0xd7,0x82,0x2c,0x87, 0xf1,0x1e,0xa4,0x38,
+ 0x72,0x3e,0x51,0xe7, 0xd8,0x3e,0x5b,0x7b,
+ 0x31,0x16,0x89,0xba, 0xd6,0xad,0x18,0x5e,
+ 0xba,0xf8,0x12,0xb3, 0xf4,0x6c,0x47,0x30,
+ 0xc0,0x38,0x58,0xb3, 0x10,0x8d,0x58,0x5d,
+ 0xb4,0xfb,0x19,0x7e, 0x41,0xc3,0x66,0xb8,
+ 0xd6,0x72,0x84,0xe1, 0x1a,0xc2,0x71,0x4c,
+ 0x0d,0x4a,0x21,0x7a, 0xab,0xa2,0xc0,0x36,
+ 0x15,0xc5,0xe9,0x46, 0xd7,0x29,0x17,0x76,
+ 0x5e,0x47,0x36,0x7f, 0x72,0x05,0xa7,0xcc,
+ 0x36,0x63,0xf9,0x47, 0x7d,0xe6,0x07,0x3c,
+ 0x8b,0x79,0x1d,0x96, 0x61,0x8d,0x90,0x65,
+ 0x7c,0xf5,0xeb,0x4e, 0x6e,0x09,0x59,0x6d,
+ 0x62,0x50,0x1b,0x0f, 0xe0,0xdc,0x78,0xf2,
+ 0x5b,0x83,0x1a,0xa1, 0x11,0x75,0xfd,0x18,
+ 0xd7,0xe2,0x8d,0x65, 0x14,0x21,0xce,0xbe,
+ 0xb5,0x87,0xe3,0x0a, 0xda,0x24,0x0a,0x64,
+ 0xa9,0x9f,0x03,0x8d, 0x46,0x5d,0x24,0x1a,
+ 0x8a,0x0c,0x42,0x01, 0xca,0xb1,0x5f,0x7c,
+ 0xa5,0xac,0x32,0x4a, 0xb8,0x07,0x91,0x18,
+ 0x6f,0xb0,0x71,0x3c, 0xc9,0xb1,0xa8,0xf8,
+ 0x5f,0x69,0xa5,0xa1, 0xca,0x9e,0x7a,0xaa,
+ 0xac,0xe9,0xc7,0x47, 0x41,0x75,0x25,0xc3,
+ 0x73,0xe2,0x0b,0xdd, 0x6d,0x52,0x71,0xbe,
+ 0xc5,0xdc,0xb4,0xe7, 0x01,0x26,0x53,0x77,
+ 0x86,0x90,0x85,0x68, 0x6b,0x7b,0x03,0x53,
+ 0xda,0x52,0x52,0x51, 0x68,0xc8,0xf3,0xec,
+ 0x6c,0xd5,0x03,0x7a, 0xa3,0x0e,0xb4,0x02,
+ 0x5f,0x1a,0xab,0xee, 0xca,0x67,0x29,0x7b,
+ 0xbd,0x96,0x59,0xb3, 0x8b,0x32,0x7a,0x92,
+ 0x9f,0xd8,0x25,0x2b, 0xdf,0xc0,0x4c,0xda,
+ },
+ .mlen = 16,
+ .m = {
+ 0xbc,0xda,0x81,0xa8, 0x78,0x79,0x1c,0xbf,
+ 0x77,0x53,0xba,0x4c, 0x30,0x5b,0xb8,0x33,
+ },
+ .h = {
+ 0x04,0xbf,0x7f,0x6a, 0xce,0x72,0xea,0x6a,
+ 0x79,0xdb,0xb0,0xc9, 0x60,0xf6,0x12,0xcc,
+ },
+ },
+ [2] = { /* 1024-byte message */
+ .k = {
+ 0x65,0x4d,0xe3,0xf8, 0xd2,0x4c,0xac,0x28,
+ 0x68,0xf5,0xb3,0x81, 0x71,0x4b,0xa1,0xfa,
+ 0x04,0x0e,0xd3,0x81, 0x36,0xbe,0x0c,0x81,
+ 0x5e,0xaf,0xbc,0x3a, 0xa4,0xc0,0x8e,0x8b,
+ 0x55,0x63,0xd3,0x52, 0x97,0x88,0xd6,0x19,
+ 0xbc,0x96,0xdf,0x49, 0xff,0x04,0x63,0xf5,
+ 0x0c,0x11,0x13,0xaa, 0x9e,0x1f,0x5a,0xf7,
+ 0xdd,0xbd,0x37,0x80, 0xc3,0xd0,0xbe,0xa7,
+ 0x05,0xc8,0x3c,0x98, 0x1e,0x05,0x3c,0x84,
+ 0x39,0x61,0xc4,0xed, 0xed,0x71,0x1b,0xc4,
+ 0x74,0x45,0x2c,0xa1, 0x56,0x70,0x97,0xfd,
+ 0x44,0x18,0x07,0x7d, 0xca,0x60,0x1f,0x73,
+ 0x3b,0x6d,0x21,0xcb, 0x61,0x87,0x70,0x25,
+ 0x46,0x21,0xf1,0x1f, 0x21,0x91,0x31,0x2d,
+ 0x5d,0xcc,0xb7,0xd1, 0x84,0x3e,0x3d,0xdb,
+ 0x03,0x53,0x2a,0x82, 0xa6,0x9a,0x95,0xbc,
+ 0x1a,0x1e,0x0a,0x5e, 0x07,0x43,0xab,0x43,
+ 0xaf,0x92,0x82,0x06, 0x91,0x04,0x09,0xf4,
+ 0x17,0x0a,0x9a,0x2c, 0x54,0xdb,0xb8,0xf4,
+ 0xd0,0xf0,0x10,0x66, 0x24,0x8d,0xcd,0xda,
+ 0xfe,0x0e,0x45,0x9d, 0x6f,0xc4,0x4e,0xf4,
+ 0x96,0xaf,0x13,0xdc, 0xa9,0xd4,0x8c,0xc4,
+ 0xc8,0x57,0x39,0x3c, 0xc2,0xd3,0x0a,0x76,
+ 0x4a,0x1f,0x75,0x83, 0x44,0xc7,0xd1,0x39,
+ 0xd8,0xb5,0x41,0xba, 0x73,0x87,0xfa,0x96,
+ 0xc7,0x18,0x53,0xfb, 0x9b,0xda,0xa0,0x97,
+ 0x1d,0xee,0x60,0x85, 0x9e,0x14,0xc3,0xce,
+ 0xc4,0x05,0x29,0x3b, 0x95,0x30,0xa3,0xd1,
+ 0x9f,0x82,0x6a,0x04, 0xf5,0xa7,0x75,0x57,
+ 0x82,0x04,0xfe,0x71, 0x51,0x71,0xb1,0x49,
+ 0x50,0xf8,0xe0,0x96, 0xf1,0xfa,0xa8,0x88,
+ 0x3f,0xa0,0x86,0x20, 0xd4,0x60,0x79,0x59,
+ 0x17,0x2d,0xd1,0x09, 0xf4,0xec,0x05,0x57,
+ 0xcf,0x62,0x7e,0x0e, 0x7e,0x60,0x78,0xe6,
+ 0x08,0x60,0x29,0xd8, 0xd5,0x08,0x1a,0x24,
+ 0xc4,0x6c,0x24,0xe7, 0x92,0x08,0x3d,0x8a,
+ 0x98,0x7a,0xcf,0x99, 0x0a,0x65,0x0e,0xdc,
+ 0x8c,0x8a,0xbe,0x92, 0x82,0x91,0xcc,0x62,
+ 0x30,0xb6,0xf4,0x3f, 0xc6,0x8a,0x7f,0x12,
+ 0x4a,0x8a,0x49,0xfa, 0x3f,0x5c,0xd4,0x5a,
+ 0xa6,0x82,0xa3,0xe6, 0xaa,0x34,0x76,0xb2,
+ 0xab,0x0a,0x30,0xef, 0x6c,0x77,0x58,0x3f,
+ 0x05,0x6b,0xcc,0x5c, 0xae,0xdc,0xd7,0xb9,
+ 0x51,0x7e,0x8d,0x32, 0x5b,0x24,0x25,0xbe,
+ 0x2b,0x24,0x01,0xcf, 0x80,0xda,0x16,0xd8,
+ 0x90,0x72,0x2c,0xad, 0x34,0x8d,0x0c,0x74,
+ 0x02,0xcb,0xfd,0xcf, 0x6e,0xef,0x97,0xb5,
+ 0x4c,0xf2,0x68,0xca, 0xde,0x43,0x9e,0x8a,
+ 0xc5,0x5f,0x31,0x7f, 0x14,0x71,0x38,0xec,
+ 0xbd,0x98,0xe5,0x71, 0xc4,0xb5,0xdb,0xef,
+ 0x59,0xd2,0xca,0xc0, 0xc1,0x86,0x75,0x01,
+ 0xd4,0x15,0x0d,0x6f, 0xa4,0xf7,0x7b,0x37,
+ 0x47,0xda,0x18,0x93, 0x63,0xda,0xbe,0x9e,
+ 0x07,0xfb,0xb2,0x83, 0xd5,0xc4,0x34,0x55,
+ 0xee,0x73,0xa1,0x42, 0x96,0xf9,0x66,0x41,
+ 0xa4,0xcc,0xd2,0x93, 0x6e,0xe1,0x0a,0xbb,
+ 0xd2,0xdd,0x18,0x23, 0xe6,0x6b,0x98,0x0b,
+ 0x8a,0x83,0x59,0x2c, 0xc3,0xa6,0x59,0x5b,
+ 0x01,0x22,0x59,0xf7, 0xdc,0xb0,0x87,0x7e,
+ 0xdb,0x7d,0xf4,0x71, 0x41,0xab,0xbd,0xee,
+ 0x79,0xbe,0x3c,0x01, 0x76,0x0b,0x2d,0x0a,
+ 0x42,0xc9,0x77,0x8c, 0xbb,0x54,0x95,0x60,
+ 0x43,0x2e,0xe0,0x17, 0x52,0xbd,0x90,0xc9,
+ 0xc2,0x2c,0xdd,0x90, 0x24,0x22,0x76,0x40,
+ 0x5c,0xb9,0x41,0xc9, 0xa1,0xd5,0xbd,0xe3,
+ 0x44,0xe0,0xa4,0xab, 0xcc,0xb8,0xe2,0x32,
+ 0x02,0x15,0x04,0x1f, 0x8c,0xec,0x5d,0x14,
+ 0xac,0x18,0xaa,0xef, 0x6e,0x33,0x19,0x6e,
+ 0xde,0xfe,0x19,0xdb, 0xeb,0x61,0xca,0x18,
+ 0xad,0xd8,0x3d,0xbf, 0x09,0x11,0xc7,0xa5,
+ 0x86,0x0b,0x0f,0xe5, 0x3e,0xde,0xe8,0xd9,
+ 0x0a,0x69,0x9e,0x4c, 0x20,0xff,0xf9,0xc5,
+ 0xfa,0xf8,0xf3,0x7f, 0xa5,0x01,0x4b,0x5e,
+ 0x0f,0xf0,0x3b,0x68, 0xf0,0x46,0x8c,0x2a,
+ 0x7a,0xc1,0x8f,0xa0, 0xfe,0x6a,0x5b,0x44,
+ 0x70,0x5c,0xcc,0x92, 0x2c,0x6f,0x0f,0xbd,
+ 0x25,0x3e,0xb7,0x8e, 0x73,0x58,0xda,0xc9,
+ 0xa5,0xaa,0x9e,0xf3, 0x9b,0xfd,0x37,0x3e,
+ 0xe2,0x88,0xa4,0x7b, 0xc8,0x5c,0xa8,0x93,
+ 0x0e,0xe7,0x9a,0x9c, 0x2e,0x95,0x18,0x9f,
+ 0xc8,0x45,0x0c,0x88, 0x9e,0x53,0x4f,0x3a,
+ 0x76,0xc1,0x35,0xfa, 0x17,0xd8,0xac,0xa0,
+ 0x0c,0x2d,0x47,0x2e, 0x4f,0x69,0x9b,0xf7,
+ 0xd0,0xb6,0x96,0x0c, 0x19,0xb3,0x08,0x01,
+ 0x65,0x7a,0x1f,0xc7, 0x31,0x86,0xdb,0xc8,
+ 0xc1,0x99,0x8f,0xf8, 0x08,0x4a,0x9d,0x23,
+ 0x22,0xa8,0xcf,0x27, 0x01,0x01,0x88,0x93,
+ 0x9c,0x86,0x45,0xbd, 0xe0,0x51,0xca,0x52,
+ 0x84,0xba,0xfe,0x03, 0xf7,0xda,0xc5,0xce,
+ 0x3e,0x77,0x75,0x86, 0xaf,0x84,0xc8,0x05,
+ 0x44,0x01,0x0f,0x02, 0xf3,0x58,0xb0,0x06,
+ 0x5a,0xd7,0x12,0x30, 0x8d,0xdf,0x1f,0x1f,
+ 0x0a,0xe6,0xd2,0xea, 0xf6,0x3a,0x7a,0x99,
+ 0x63,0xe8,0xd2,0xc1, 0x4a,0x45,0x8b,0x40,
+ 0x4d,0x0a,0xa9,0x76, 0x92,0xb3,0xda,0x87,
+ 0x36,0x33,0xf0,0x78, 0xc3,0x2f,0x5f,0x02,
+ 0x1a,0x6a,0x2c,0x32, 0xcd,0x76,0xbf,0xbd,
+ 0x5a,0x26,0x20,0x28, 0x8c,0x8c,0xbc,0x52,
+ 0x3d,0x0a,0xc9,0xcb, 0xab,0xa4,0x21,0xb0,
+ 0x54,0x40,0x81,0x44, 0xc7,0xd6,0x1c,0x11,
+ 0x44,0xc6,0x02,0x92, 0x14,0x5a,0xbf,0x1a,
+ 0x09,0x8a,0x18,0xad, 0xcd,0x64,0x3d,0x53,
+ 0x4a,0xb6,0xa5,0x1b, 0x57,0x0e,0xef,0xe0,
+ 0x8c,0x44,0x5f,0x7d, 0xbd,0x6c,0xfd,0x60,
+ 0xae,0x02,0x24,0xb6, 0x99,0xdd,0x8c,0xaf,
+ 0x59,0x39,0x75,0x3c, 0xd1,0x54,0x7b,0x86,
+ 0xcc,0x99,0xd9,0x28, 0x0c,0xb0,0x94,0x62,
+ 0xf9,0x51,0xd1,0x19, 0x96,0x2d,0x66,0xf5,
+ 0x55,0xcf,0x9e,0x59, 0xe2,0x6b,0x2c,0x08,
+ 0xc0,0x54,0x48,0x24, 0x45,0xc3,0x8c,0x73,
+ 0xea,0x27,0x6e,0x66, 0x7d,0x1d,0x0e,0x6e,
+ 0x13,0xe8,0x56,0x65, 0x3a,0xb0,0x81,0x5c,
+ 0xf0,0xe8,0xd8,0x00, 0x6b,0xcd,0x8f,0xad,
+ 0xdd,0x53,0xf3,0xa4, 0x6c,0x43,0xd6,0x31,
+ 0xaf,0xd2,0x76,0x1e, 0x91,0x12,0xdb,0x3c,
+ 0x8c,0xc2,0x81,0xf0, 0x49,0xdb,0xe2,0x6b,
+ 0x76,0x62,0x0a,0x04, 0xe4,0xaa,0x8a,0x7c,
+ 0x08,0x0b,0x5d,0xd0, 0xee,0x1d,0xfb,0xc4,
+ 0x02,0x75,0x42,0xd6, 0xba,0xa7,0x22,0xa8,
+ 0x47,0x29,0xb7,0x85, 0x6d,0x93,0x3a,0xdb,
+ 0x00,0x53,0x0b,0xa2, 0xeb,0xf8,0xfe,0x01,
+ 0x6f,0x8a,0x31,0xd6, 0x17,0x05,0x6f,0x67,
+ 0x88,0x95,0x32,0xfe, 0x4f,0xa6,0x4b,0xf8,
+ 0x03,0xe4,0xcd,0x9a, 0x18,0xe8,0x4e,0x2d,
+ 0xf7,0x97,0x9a,0x0c, 0x7d,0x9f,0x7e,0x44,
+ 0x69,0x51,0xe0,0x32, 0x6b,0x62,0x86,0x8f,
+ 0xa6,0x8e,0x0b,0x21, 0x96,0xe5,0xaf,0x77,
+ 0xc0,0x83,0xdf,0xa5, 0x0e,0xd0,0xa1,0x04,
+ 0xaf,0xc1,0x10,0xcb, 0x5a,0x40,0xe4,0xe3,
+ 0x38,0x7e,0x07,0xe8, 0x4d,0xfa,0xed,0xc5,
+ 0xf0,0x37,0xdf,0xbb, 0x8a,0xcf,0x3d,0xdc,
+ 0x61,0xd2,0xc6,0x2b, 0xff,0x07,0xc9,0x2f,
+ 0x0c,0x2d,0x5c,0x07, 0xa8,0x35,0x6a,0xfc,
+ 0xae,0x09,0x03,0x45, 0x74,0x51,0x4d,0xc4,
+ 0xb8,0x23,0x87,0x4a, 0x99,0x27,0x20,0x87,
+ 0x62,0x44,0x0a,0x4a, 0xce,0x78,0x47,0x22,
+ },
+ .mlen = 1024,
+ .m = {
+ 0x8e,0xb0,0x4c,0xde, 0x9c,0x4a,0x04,0x5a,
+ 0xf6,0xa9,0x7f,0x45, 0x25,0xa5,0x7b,0x3a,
+ 0xbc,0x4d,0x73,0x39, 0x81,0xb5,0xbd,0x3d,
+ 0x21,0x6f,0xd7,0x37, 0x50,0x3c,0x7b,0x28,
+ 0xd1,0x03,0x3a,0x17, 0xed,0x7b,0x7c,0x2a,
+ 0x16,0xbc,0xdf,0x19, 0x89,0x52,0x71,0x31,
+ 0xb6,0xc0,0xfd,0xb5, 0xd3,0xba,0x96,0x99,
+ 0xb6,0x34,0x0b,0xd0, 0x99,0x93,0xfc,0x1a,
+ 0x01,0x3c,0x85,0xc6, 0x9b,0x78,0x5c,0x8b,
+ 0xfe,0xae,0xd2,0xbf, 0xb2,0x6f,0xf9,0xed,
+ 0xc8,0x25,0x17,0xfe, 0x10,0x3b,0x7d,0xda,
+ 0xf4,0x8d,0x35,0x4b, 0x7c,0x7b,0x82,0xe7,
+ 0xc2,0xb3,0xee,0x60, 0x4a,0x03,0x86,0xc9,
+ 0x4e,0xb5,0xc4,0xbe, 0xd2,0xbd,0x66,0xf1,
+ 0x13,0xf1,0x09,0xab, 0x5d,0xca,0x63,0x1f,
+ 0xfc,0xfb,0x57,0x2a, 0xfc,0xca,0x66,0xd8,
+ 0x77,0x84,0x38,0x23, 0x1d,0xac,0xd3,0xb3,
+ 0x7a,0xad,0x4c,0x70, 0xfa,0x9c,0xc9,0x61,
+ 0xa6,0x1b,0xba,0x33, 0x4b,0x4e,0x33,0xec,
+ 0xa0,0xa1,0x64,0x39, 0x40,0x05,0x1c,0xc2,
+ 0x3f,0x49,0x9d,0xae, 0xf2,0xc5,0xf2,0xc5,
+ 0xfe,0xe8,0xf4,0xc2, 0xf9,0x96,0x2d,0x28,
+ 0x92,0x30,0x44,0xbc, 0xd2,0x7f,0xe1,0x6e,
+ 0x62,0x02,0x8f,0x3d, 0x1c,0x80,0xda,0x0e,
+ 0x6a,0x90,0x7e,0x75, 0xff,0xec,0x3e,0xc4,
+ 0xcd,0x16,0x34,0x3b, 0x05,0x6d,0x4d,0x20,
+ 0x1c,0x7b,0xf5,0x57, 0x4f,0xfa,0x3d,0xac,
+ 0xd0,0x13,0x55,0xe8, 0xb3,0xe1,0x1b,0x78,
+ 0x30,0xe6,0x9f,0x84, 0xd4,0x69,0xd1,0x08,
+ 0x12,0x77,0xa7,0x4a, 0xbd,0xc0,0xf2,0xd2,
+ 0x78,0xdd,0xa3,0x81, 0x12,0xcb,0x6c,0x14,
+ 0x90,0x61,0xe2,0x84, 0xc6,0x2b,0x16,0xcc,
+ 0x40,0x99,0x50,0x88, 0x01,0x09,0x64,0x4f,
+ 0x0a,0x80,0xbe,0x61, 0xae,0x46,0xc9,0x0a,
+ 0x5d,0xe0,0xfb,0x72, 0x7a,0x1a,0xdd,0x61,
+ 0x63,0x20,0x05,0xa0, 0x4a,0xf0,0x60,0x69,
+ 0x7f,0x92,0xbc,0xbf, 0x4e,0x39,0x4d,0xdd,
+ 0x74,0xd1,0xb7,0xc0, 0x5a,0x34,0xb7,0xae,
+ 0x76,0x65,0x2e,0xbc, 0x36,0xb9,0x04,0x95,
+ 0x42,0xe9,0x6f,0xca, 0x78,0xb3,0x72,0x07,
+ 0xa3,0xba,0x02,0x94, 0x67,0x4c,0xb1,0xd7,
+ 0xe9,0x30,0x0d,0xf0, 0x3b,0xb8,0x10,0x6d,
+ 0xea,0x2b,0x21,0xbf, 0x74,0x59,0x82,0x97,
+ 0x85,0xaa,0xf1,0xd7, 0x54,0x39,0xeb,0x05,
+ 0xbd,0xf3,0x40,0xa0, 0x97,0xe6,0x74,0xfe,
+ 0xb4,0x82,0x5b,0xb1, 0x36,0xcb,0xe8,0x0d,
+ 0xce,0x14,0xd9,0xdf, 0xf1,0x94,0x22,0xcd,
+ 0xd6,0x00,0xba,0x04, 0x4c,0x05,0x0c,0xc0,
+ 0xd1,0x5a,0xeb,0x52, 0xd5,0xa8,0x8e,0xc8,
+ 0x97,0xa1,0xaa,0xc1, 0xea,0xc1,0xbe,0x7c,
+ 0x36,0xb3,0x36,0xa0, 0xc6,0x76,0x66,0xc5,
+ 0xe2,0xaf,0xd6,0x5c, 0xe2,0xdb,0x2c,0xb3,
+ 0x6c,0xb9,0x99,0x7f, 0xff,0x9f,0x03,0x24,
+ 0xe1,0x51,0x44,0x66, 0xd8,0x0c,0x5d,0x7f,
+ 0x5c,0x85,0x22,0x2a, 0xcf,0x6d,0x79,0x28,
+ 0xab,0x98,0x01,0x72, 0xfe,0x80,0x87,0x5f,
+ 0x46,0xba,0xef,0x81, 0x24,0xee,0xbf,0xb0,
+ 0x24,0x74,0xa3,0x65, 0x97,0x12,0xc4,0xaf,
+ 0x8b,0xa0,0x39,0xda, 0x8a,0x7e,0x74,0x6e,
+ 0x1b,0x42,0xb4,0x44, 0x37,0xfc,0x59,0xfd,
+ 0x86,0xed,0xfb,0x8c, 0x66,0x33,0xda,0x63,
+ 0x75,0xeb,0xe1,0xa4, 0x85,0x4f,0x50,0x8f,
+ 0x83,0x66,0x0d,0xd3, 0x37,0xfa,0xe6,0x9c,
+ 0x4f,0x30,0x87,0x35, 0x18,0xe3,0x0b,0xb7,
+ 0x6e,0x64,0x54,0xcd, 0x70,0xb3,0xde,0x54,
+ 0xb7,0x1d,0xe6,0x4c, 0x4d,0x55,0x12,0x12,
+ 0xaf,0x5f,0x7f,0x5e, 0xee,0x9d,0xe8,0x8e,
+ 0x32,0x9d,0x4e,0x75, 0xeb,0xc6,0xdd,0xaa,
+ 0x48,0x82,0xa4,0x3f, 0x3c,0xd7,0xd3,0xa8,
+ 0x63,0x9e,0x64,0xfe, 0xe3,0x97,0x00,0x62,
+ 0xe5,0x40,0x5d,0xc3, 0xad,0x72,0xe1,0x28,
+ 0x18,0x50,0xb7,0x75, 0xef,0xcd,0x23,0xbf,
+ 0x3f,0xc0,0x51,0x36, 0xf8,0x41,0xc3,0x08,
+ 0xcb,0xf1,0x8d,0x38, 0x34,0xbd,0x48,0x45,
+ 0x75,0xed,0xbc,0x65, 0x7b,0xb5,0x0c,0x9b,
+ 0xd7,0x67,0x7d,0x27, 0xb4,0xc4,0x80,0xd7,
+ 0xa9,0xb9,0xc7,0x4a, 0x97,0xaa,0xda,0xc8,
+ 0x3c,0x74,0xcf,0x36, 0x8f,0xe4,0x41,0xe3,
+ 0xd4,0xd3,0x26,0xa7, 0xf3,0x23,0x9d,0x8f,
+ 0x6c,0x20,0x05,0x32, 0x3e,0xe0,0xc3,0xc8,
+ 0x56,0x3f,0xa7,0x09, 0xb7,0xfb,0xc7,0xf7,
+ 0xbe,0x2a,0xdd,0x0f, 0x06,0x7b,0x0d,0xdd,
+ 0xb0,0xb4,0x86,0x17, 0xfd,0xb9,0x04,0xe5,
+ 0xc0,0x64,0x5d,0xad, 0x2a,0x36,0x38,0xdb,
+ 0x24,0xaf,0x5b,0xff, 0xca,0xf9,0x41,0xe8,
+ 0xf9,0x2f,0x1e,0x5e, 0xf9,0xf5,0xd5,0xf2,
+ 0xb2,0x88,0xca,0xc9, 0xa1,0x31,0xe2,0xe8,
+ 0x10,0x95,0x65,0xbf, 0xf1,0x11,0x61,0x7a,
+ 0x30,0x1a,0x54,0x90, 0xea,0xd2,0x30,0xf6,
+ 0xa5,0xad,0x60,0xf9, 0x4d,0x84,0x21,0x1b,
+ 0xe4,0x42,0x22,0xc8, 0x12,0x4b,0xb0,0x58,
+ 0x3e,0x9c,0x2d,0x32, 0x95,0x0a,0x8e,0xb0,
+ 0x0a,0x7e,0x77,0x2f, 0xe8,0x97,0x31,0x6a,
+ 0xf5,0x59,0xb4,0x26, 0xe6,0x37,0x12,0xc9,
+ 0xcb,0xa0,0x58,0x33, 0x6f,0xd5,0x55,0x55,
+ 0x3c,0xa1,0x33,0xb1, 0x0b,0x7e,0x2e,0xb4,
+ 0x43,0x2a,0x84,0x39, 0xf0,0x9c,0xf4,0x69,
+ 0x4f,0x1e,0x79,0xa6, 0x15,0x1b,0x87,0xbb,
+ 0xdb,0x9b,0xe0,0xf1, 0x0b,0xba,0xe3,0x6e,
+ 0xcc,0x2f,0x49,0x19, 0x22,0x29,0xfc,0x71,
+ 0xbb,0x77,0x38,0x18, 0x61,0xaf,0x85,0x76,
+ 0xeb,0xd1,0x09,0xcc, 0x86,0x04,0x20,0x9a,
+ 0x66,0x53,0x2f,0x44, 0x8b,0xc6,0xa3,0xd2,
+ 0x5f,0xc7,0x79,0x82, 0x66,0xa8,0x6e,0x75,
+ 0x7d,0x94,0xd1,0x86, 0x75,0x0f,0xa5,0x4f,
+ 0x3c,0x7a,0x33,0xce, 0xd1,0x6e,0x9d,0x7b,
+ 0x1f,0x91,0x37,0xb8, 0x37,0x80,0xfb,0xe0,
+ 0x52,0x26,0xd0,0x9a, 0xd4,0x48,0x02,0x41,
+ 0x05,0xe3,0x5a,0x94, 0xf1,0x65,0x61,0x19,
+ 0xb8,0x88,0x4e,0x2b, 0xea,0xba,0x8b,0x58,
+ 0x8b,0x42,0x01,0x00, 0xa8,0xfe,0x00,0x5c,
+ 0xfe,0x1c,0xee,0x31, 0x15,0x69,0xfa,0xb3,
+ 0x9b,0x5f,0x22,0x8e, 0x0d,0x2c,0xe3,0xa5,
+ 0x21,0xb9,0x99,0x8a, 0x8e,0x94,0x5a,0xef,
+ 0x13,0x3e,0x99,0x96, 0x79,0x6e,0xd5,0x42,
+ 0x36,0x03,0xa9,0xe2, 0xca,0x65,0x4e,0x8a,
+ 0x8a,0x30,0xd2,0x7d, 0x74,0xe7,0xf0,0xaa,
+ 0x23,0x26,0xdd,0xcb, 0x82,0x39,0xfc,0x9d,
+ 0x51,0x76,0x21,0x80, 0xa2,0xbe,0x93,0x03,
+ 0x47,0xb0,0xc1,0xb6, 0xdc,0x63,0xfd,0x9f,
+ 0xca,0x9d,0xa5,0xca, 0x27,0x85,0xe2,0xd8,
+ 0x15,0x5b,0x7e,0x14, 0x7a,0xc4,0x89,0xcc,
+ 0x74,0x14,0x4b,0x46, 0xd2,0xce,0xac,0x39,
+ 0x6b,0x6a,0x5a,0xa4, 0x0e,0xe3,0x7b,0x15,
+ 0x94,0x4b,0x0f,0x74, 0xcb,0x0c,0x7f,0xa9,
+ 0xbe,0x09,0x39,0xa3, 0xdd,0x56,0x5c,0xc7,
+ 0x99,0x56,0x65,0x39, 0xf4,0x0b,0x7d,0x87,
+ 0xec,0xaa,0xe3,0x4d, 0x22,0x65,0x39,0x4e,
+ },
+ .h = {
+ 0x64,0x3a,0xbc,0xc3, 0x3f,0x74,0x40,0x51,
+ 0x6e,0x56,0x01,0x1a, 0x51,0xec,0x36,0xde,
+ },
+ },
+ };
+ const uint8_t *pk;
+ const uint8_t *nhk;
+ static uint32_t nhk32[268];
+ uint8_t h[16];
+ unsigned i, j;
+ int result = 0;
+
+ for (i = 0; i < __arraycount(C); i++) {
+ pk = C[i].k;
+ nhk = C[i].k + 16;
+ for (j = 0; j < 268; j++)
+ nhk32[j] = le32dec(nhk + 4*j);
+ nhpoly1305(h, C[i].m, C[i].mlen, pk, nhk32);
+ if (memcmp(h, C[i].h, 16)) {
+ char prefix[16];
+ snprintf(prefix, sizeof prefix, "nhpoly1305 %u", i);
+ hexdump(printf, prefix, h, 32);
+ result = -1;
+ }
+ }
+
+ return result;
+}
+
+/* ChaCha core */
+
+static uint32_t
+rol32(uint32_t u, unsigned c)
+{
+
+ return (u << c) | (u >> (32 - c));
+}
+
+#define CHACHA_QUARTERROUND(a, b, c, d) do { \
+ (a) += (b); (d) ^= (a); (d) = rol32((d), 16); \
+ (c) += (d); (b) ^= (c); (b) = rol32((b), 12); \
+ (a) += (b); (d) ^= (a); (d) = rol32((d), 8); \
+ (c) += (d); (b) ^= (c); (b) = rol32((b), 7); \
+} while (/*CONSTCOND*/0)
+
+const uint8_t chacha_const32[16] = "expand 32-byte k";
+
+static void
+chacha_core(uint8_t out[restrict static 64], const uint8_t in[static 16],
+ const uint8_t k[static 32], const uint8_t c[static 16], unsigned nr)
+{
+ uint32_t x0,x1,x2,x3,x4,x5,x6,x7,x8,x9,x10,x11,x12,x13,x14,x15;
+ uint32_t y0,y1,y2,y3,y4,y5,y6,y7,y8,y9,y10,y11,y12,y13,y14,y15;
+ int i;
+
+ x0 = y0 = le32dec(c + 0);
+ x1 = y1 = le32dec(c + 4);
+ x2 = y2 = le32dec(c + 8);
+ x3 = y3 = le32dec(c + 12);
+ x4 = y4 = le32dec(k + 0);
+ x5 = y5 = le32dec(k + 4);
+ x6 = y6 = le32dec(k + 8);
+ x7 = y7 = le32dec(k + 12);
+ x8 = y8 = le32dec(k + 16);
+ x9 = y9 = le32dec(k + 20);
+ x10 = y10 = le32dec(k + 24);
+ x11 = y11 = le32dec(k + 28);
+ x12 = y12 = le32dec(in + 0);
+ x13 = y13 = le32dec(in + 4);
+ x14 = y14 = le32dec(in + 8);
+ x15 = y15 = le32dec(in + 12);
+
+ for (i = nr; i > 0; i -= 2) {
+ CHACHA_QUARTERROUND( y0, y4, y8,y12);
+ CHACHA_QUARTERROUND( y1, y5, y9,y13);
+ CHACHA_QUARTERROUND( y2, y6,y10,y14);
+ CHACHA_QUARTERROUND( y3, y7,y11,y15);
+ CHACHA_QUARTERROUND( y0, y5,y10,y15);
+ CHACHA_QUARTERROUND( y1, y6,y11,y12);
+ CHACHA_QUARTERROUND( y2, y7, y8,y13);
+ CHACHA_QUARTERROUND( y3, y4, y9,y14);
+ }
+
+ le32enc(out + 0, x0 + y0);
+ le32enc(out + 4, x1 + y1);
+ le32enc(out + 8, x2 + y2);
+ le32enc(out + 12, x3 + y3);
+ le32enc(out + 16, x4 + y4);
+ le32enc(out + 20, x5 + y5);
+ le32enc(out + 24, x6 + y6);
+ le32enc(out + 28, x7 + y7);
+ le32enc(out + 32, x8 + y8);
+ le32enc(out + 36, x9 + y9);
+ le32enc(out + 40, x10 + y10);
+ le32enc(out + 44, x11 + y11);
+ le32enc(out + 48, x12 + y12);
+ le32enc(out + 52, x13 + y13);
+ le32enc(out + 56, x14 + y14);
+ le32enc(out + 60, x15 + y15);
+}
+
+/* https://tools.ietf.org/html/draft-strombergson-chacha-test-vectors-00 */
+static int
+chacha_core_selftest(void)
+{
+ /* TC1, 32-byte key, rounds=12, keystream block 1 */
+ static const uint8_t zero[32];
+ static const uint8_t expected0[64] = {
+ 0x9b,0xf4,0x9a,0x6a, 0x07,0x55,0xf9,0x53,
+ 0x81,0x1f,0xce,0x12, 0x5f,0x26,0x83,0xd5,
+ 0x04,0x29,0xc3,0xbb, 0x49,0xe0,0x74,0x14,
+ 0x7e,0x00,0x89,0xa5, 0x2e,0xae,0x15,0x5f,
+ 0x05,0x64,0xf8,0x79, 0xd2,0x7a,0xe3,0xc0,
+ 0x2c,0xe8,0x28,0x34, 0xac,0xfa,0x8c,0x79,
+ 0x3a,0x62,0x9f,0x2c, 0xa0,0xde,0x69,0x19,
+ 0x61,0x0b,0xe8,0x2f, 0x41,0x13,0x26,0xbe,
+ };
+ /* TC7, 32-byte key, rounds=12, keystream block 2 */
+ static const uint8_t k1[32] = {
+ 0x00,0x11,0x22,0x33, 0x44,0x55,0x66,0x77,
+ 0x88,0x99,0xaa,0xbb, 0xcc,0xdd,0xee,0xff,
+ 0xff,0xee,0xdd,0xcc, 0xbb,0xaa,0x99,0x88,
+ 0x77,0x66,0x55,0x44, 0x33,0x22,0x11,0x00,
+ };
+ static const uint8_t in1[16] = {
+ 0x01,0x00,0x00,0x00, 0x00,0x00,0x00,0x00,
+ 0x0f,0x1e,0x2d,0x3c, 0x4b,0x59,0x68,0x77,
+ };
+ static const uint8_t expected1[64] = {
+ 0xcd,0x9a,0x2a,0xa9, 0xea,0x93,0xc2,0x67,
+ 0x5e,0x82,0x88,0x14, 0x08,0xde,0x85,0x2c,
+ 0x62,0xfa,0x74,0x6a, 0x30,0xe5,0x2b,0x45,
+ 0xa2,0x69,0x62,0xcf, 0x43,0x51,0xe3,0x04,
+ 0xd3,0x13,0x20,0xbb, 0xd6,0xaa,0x6c,0xc8,
+ 0xf3,0x26,0x37,0xf9, 0x59,0x34,0xe4,0xc1,
+ 0x45,0xef,0xd5,0x62, 0x31,0xef,0x31,0x61,
+ 0x03,0x28,0x36,0xf4, 0x96,0x71,0x83,0x3e,
+ };
+ uint8_t out[64];
+ int result = 0;
+
+ chacha_core(out, zero, zero, chacha_const32, 12);
+ if (memcmp(out, expected0, 64)) {
+ hexdump(printf, "chacha core 1", out, sizeof out);
+ result = -1;
+ }
+
+ chacha_core(out, in1, k1, chacha_const32, 12);
+ if (memcmp(out, expected1, 64)) {
+ hexdump(printf, "chacha core 2", out, sizeof out);
+ result = -1;
+ }
+
+ return result;
+}
+
+/* HChaCha */
+
+static void
+hchacha(uint8_t out[restrict static 32], const uint8_t in[static 16],
+ const uint8_t k[static 32], const uint8_t c[static 16], unsigned nr)
+{
+ uint8_t t[64];
+
+ chacha_core(t, in, k, c, nr);
+ le32enc(out + 0, le32dec(t + 0) - le32dec(c + 0));
+ le32enc(out + 4, le32dec(t + 4) - le32dec(c + 4));
+ le32enc(out + 8, le32dec(t + 8) - le32dec(c + 8));
+ le32enc(out + 12, le32dec(t + 12) - le32dec(c + 12));
+ le32enc(out + 16, le32dec(t + 48) - le32dec(in + 0));
+ le32enc(out + 20, le32dec(t + 52) - le32dec(in + 4));
+ le32enc(out + 24, le32dec(t + 56) - le32dec(in + 8));
+ le32enc(out + 28, le32dec(t + 60) - le32dec(in + 12));
+}
+
+static int
+hchacha_selftest(void)
+{
+ /* https://tools.ietf.org/html/draft-irtf-cfrg-xchacha-03, §2.2.1 */
+ static const uint8_t k[32] = {
+ 0x00,0x01,0x02,0x03, 0x04,0x05,0x06,0x07,
+ 0x08,0x09,0x0a,0x0b, 0x0c,0x0d,0x0e,0x0f,
+ 0x10,0x11,0x12,0x13, 0x14,0x15,0x16,0x17,
+ 0x18,0x19,0x1a,0x1b, 0x1c,0x1d,0x1e,0x1f,
+ };
+ static const uint8_t in[16] = {
+ 0x00,0x00,0x00,0x09, 0x00,0x00,0x00,0x4a,
+ 0x00,0x00,0x00,0x00, 0x31,0x41,0x59,0x27,
+ };
+ static const uint8_t expected[32] = {
+ 0x82,0x41,0x3b,0x42, 0x27,0xb2,0x7b,0xfe,
+ 0xd3,0x0e,0x42,0x50, 0x8a,0x87,0x7d,0x73,
+ 0xa0,0xf9,0xe4,0xd5, 0x8a,0x74,0xa8,0x53,
+ 0xc1,0x2e,0xc4,0x13, 0x26,0xd3,0xec,0xdc,
+ };
+ uint8_t out[32];
+ int result = 0;
+
+ hchacha(out, in, k, chacha_const32, 20);
+ if (memcmp(out, expected, 32)) {
+ hexdump(printf, "hchacha", out, sizeof out);
+ result = -1;
+ }
+
+ return result;
+}
+
+/* XChaCha */
+
+static void
+xchacha_xor(uint8_t *c, const uint8_t *p, size_t nbytes,
+ const uint8_t nonce[static 24], const uint8_t k[static 32], unsigned nr)
+{
+ uint8_t h[32];
+ uint8_t in[16];
+ uint8_t block[64];
+ unsigned i;
+
+ hchacha(h, nonce, k, chacha_const32, nr);
+ memset(in, 0, 8);
+ memcpy(in + 8, nonce + 16, 8);
+
+ for (; nbytes; nbytes -= i, c += i, p += i) {
+ chacha_core(block, in, h, chacha_const32, nr);
+ for (i = 0; i < MIN(nbytes, 64); i++)
+ c[i] = p[i] ^ block[i];
+ le32enc(in, 1 + le32dec(in));
+ }
+}
+
+static int
+xchacha_selftest(void)
+{
+ /* https://tools.ietf.org/html/draft-irtf-cfrg-xchacha-03, A.2.2 */
+ static const uint8_t k[32] = {
+ 0x80,0x81,0x82,0x83, 0x84,0x85,0x86,0x87,
+ 0x88,0x89,0x8a,0x8b, 0x8c,0x8d,0x8e,0x8f,
+ 0x90,0x91,0x92,0x93, 0x94,0x95,0x96,0x97,
+ 0x98,0x99,0x9a,0x9b, 0x9c,0x9d,0x9e,0x9f,
+ };
+ static const uint8_t nonce[24] = {
+ 0x40,0x41,0x42,0x43, 0x44,0x45,0x46,0x47,
+ 0x48,0x49,0x4a,0x4b, 0x4c,0x4d,0x4e,0x4f,
+ 0x50,0x51,0x52,0x53, 0x54,0x55,0x56,0x58,
+ };
+ static const uint8_t p[128] = {
+ 0x54,0x68,0x65,0x20, 0x64,0x68,0x6f,0x6c,
+ 0x65,0x20,0x28,0x70, 0x72,0x6f,0x6e,0x6f,
+ 0x75,0x6e,0x63,0x65, 0x64,0x20,0x22,0x64,
+ 0x6f,0x6c,0x65,0x22, 0x29,0x20,0x69,0x73,
+ 0x20,0x61,0x6c,0x73, 0x6f,0x20,0x6b,0x6e,
+ 0x6f,0x77,0x6e,0x20, 0x61,0x73,0x20,0x74,
+ 0x68,0x65,0x20,0x41, 0x73,0x69,0x61,0x74,
+ 0x69,0x63,0x20,0x77, 0x69,0x6c,0x64,0x20,
+ 0x64,0x6f,0x67,0x2c, 0x20,0x72,0x65,0x64,
+ 0x20,0x64,0x6f,0x67, 0x2c,0x20,0x61,0x6e,
+ 0x64,0x20,0x77,0x68, 0x69,0x73,0x74,0x6c,
+ 0x69,0x6e,0x67,0x20, 0x64,0x6f,0x67,0x2e,
+ 0x20,0x49,0x74,0x20, 0x69,0x73,0x20,0x61,
+ 0x62,0x6f,0x75,0x74, 0x20,0x74,0x68,0x65,
+ 0x20,0x73,0x69,0x7a, 0x65,0x20,0x6f,0x66,
+ 0x20,0x61,0x20,0x47, 0x65,0x72,0x6d,0x61,
+ };
+ static const uint8_t expected[128] = {
+ 0x45,0x59,0xab,0xba, 0x4e,0x48,0xc1,0x61,
+ 0x02,0xe8,0xbb,0x2c, 0x05,0xe6,0x94,0x7f,
+ 0x50,0xa7,0x86,0xde, 0x16,0x2f,0x9b,0x0b,
+ 0x7e,0x59,0x2a,0x9b, 0x53,0xd0,0xd4,0xe9,
+ 0x8d,0x8d,0x64,0x10, 0xd5,0x40,0xa1,0xa6,
+ 0x37,0x5b,0x26,0xd8, 0x0d,0xac,0xe4,0xfa,
+ 0xb5,0x23,0x84,0xc7, 0x31,0xac,0xbf,0x16,
+ 0xa5,0x92,0x3c,0x0c, 0x48,0xd3,0x57,0x5d,
+ 0x4d,0x0d,0x2c,0x67, 0x3b,0x66,0x6f,0xaa,
+ 0x73,0x10,0x61,0x27, 0x77,0x01,0x09,0x3a,
+ 0x6b,0xf7,0xa1,0x58, 0xa8,0x86,0x42,0x92,
+ 0xa4,0x1c,0x48,0xe3, 0xa9,0xb4,0xc0,0xda,
+ 0xec,0xe0,0xf8,0xd9, 0x8d,0x0d,0x7e,0x05,
+ 0xb3,0x7a,0x30,0x7b, 0xbb,0x66,0x33,0x31,
+ 0x64,0xec,0x9e,0x1b, 0x24,0xea,0x0d,0x6c,
+ 0x3f,0xfd,0xdc,0xec, 0x4f,0x68,0xe7,0x44,
+ };
+ uint8_t c[128];
+ int result = 0;
+
+ xchacha_xor(c, p, 128, nonce, k, 20);
+ if (memcmp(c, expected, 128)) {
+ hexdump(printf, "xchacha", c, sizeof c);
+ result = -1;
+ }
+
+ return result;
+}
+
+void
+adiantum_init(struct adiantum *A, const uint8_t key[static 32])
+{
+ uint8_t nonce[24] = {1};
+ unsigned i;
+
+ memcpy(A->ks, key, 32);
+
+ /* Relies on ordering of struct members. */
+ memset(A->kk, 0, 32 + 16 + 16 + 1072);
+ xchacha_xor(A->kk, A->kk, 32 + 16 + 16 + 1072, nonce, A->ks, 12);
+
+ /* Put the NH key words into host byte order. */
+ for (i = 0; i < __arraycount(A->kn); i++)
+ A->kn[i] = le32toh(A->kn[i]);
+
+ /* Expand the AES key. */
+ aes_setenckey256(&A->kk_enc, A->kk);
+ aes_setdeckey256(&A->kk_dec, A->kk);
+}
+
+static void
+adiantum_hash(uint8_t h[static 16], const void *l, size_t llen,
+ const void *t, size_t tlen,
+ const uint8_t kt[static 16],
+ const uint8_t kl[static 16],
+ const uint32_t kn[static 268])
+{
+ const uint8_t *t8 = t;
+ struct poly1305 P;
+ uint8_t llenbuf[16];
+ uint8_t ht[16];
+ uint8_t hl[16];
+
+ KASSERT(llen % 16 == 0);
+
+ memset(llenbuf, 0, sizeof llenbuf);
+ le64enc(llenbuf, 8*llen);
+
+ /* Compute H_T := Poly1305_{K_T}(le128(|l|) || tweak). */
+ poly1305_init(&P, kt);
+ if (tlen == 0) {
+ poly1305_update_last(&P, llenbuf, 16);
+ } else {
+ poly1305_update_block(&P, llenbuf);
+ for (; tlen > 16; t8 += 16, tlen -= 16)
+ poly1305_update_block(&P, t8);
+ poly1305_update_last(&P, t8, tlen);
+ }
+ poly1305_final(ht, &P);
+
+ /* Compute H_L := Poly1305_{K_L}(NH(pad_128(l))). */
+ nhpoly1305(hl, l, llen, kl, kn);
+
+ /* Compute H := H_T + H_L (mod 2^128). */
+ add128(h, ht, hl);
+}
+
+void
+adiantum_enc(void *c, const void *p, size_t len, const void *t, size_t tlen,
+ const struct adiantum *A)
+{
+ size_t Rlen = 16;
+ size_t Llen = len - Rlen;
+ uint8_t *c8 = c;
+ uint8_t *cL = c8;
+ uint8_t *cR = c8 + Llen;
+ const uint8_t *p8 = p;
+ const uint8_t *pL = p8;
+ const uint8_t *pR = p8 + Llen;
+ uint8_t h[16];
+ uint8_t buf[16] __aligned(16);
+ uint8_t nonce[24];
+
+ KASSERT(len % 16 == 0);
+
+ aes_enc(&A->kk_enc, p, buf, AES_256_NROUNDS);
+
+ adiantum_hash(h, pL, Llen, t, tlen, A->kt, A->kl, A->kn);
+ add128(buf, pR, h); /* buf := P_M */
+ aes_enc(&A->kk_enc, buf, buf, AES_256_NROUNDS); /* buf := C_M */
+
+ memcpy(nonce, buf, 16);
+ le64enc(nonce + 16, 1);
+ xchacha_xor(cL, pL, Llen, nonce, A->ks, 12);
+
+ adiantum_hash(h, cL, Llen, t, tlen, A->kt, A->kl, A->kn);
+ sub128(cR, buf, h);
+
+ explicit_memset(h, 0, sizeof h);
+ explicit_memset(buf, 0, sizeof buf);
+}
+
+void
+adiantum_dec(void *p, const void *c, size_t len, const void *t, size_t tlen,
+ const struct adiantum *A)
+{
+ size_t Rlen = 16;
+ size_t Llen = len - Rlen;
+ const uint8_t *c8 = c;
+ const uint8_t *cL = c8;
+ const uint8_t *cR = c8 + Llen;
+ uint8_t *p8 = p;
+ uint8_t *pL = p8;
+ uint8_t *pR = p8 + Llen;
+ uint8_t h[16];
+ uint8_t buf[16] __aligned(16);
+ uint8_t nonce[24];
+
+ KASSERT(len % 16 == 0);
+
+ adiantum_hash(h, cL, Llen, t, tlen, A->kt, A->kl, A->kn);
+ add128(buf, cR, h); /* buf := P_M */
+
+ memcpy(nonce, buf, 16);
+ le64enc(nonce + 16, 1);
+ xchacha_xor(pL, cL, Llen, nonce, A->ks, 12);
+
+ aes_dec(&A->kk_dec, buf, buf, AES_256_NROUNDS); /* buf := P_M */
+ adiantum_hash(h, pL, Llen, t, tlen, A->kt, A->kl, A->kn);
+ sub128(pR, buf, h);
+
+ explicit_memset(h, 0, sizeof h);
+ explicit_memset(buf, 0, sizeof buf);
+}
+
+#ifdef _KERNEL
+
+MODULE(MODULE_CLASS_MISC, adiantum, "aes");
+
+static int
+adiantum_modcmd(modcmd_t cmd, void *opaque)
+{
+
+ switch (cmd) {
+ case MODULE_CMD_INIT: {
+ int result = 0;
+ result |= addsub128_selftest();
+ result |= poly1305_selftest();
+ result |= nh_selftest();
+ result |= nhpoly1305_selftest();
+ result |= chacha_core_selftest();
+ result |= hchacha_selftest();
+ result |= xchacha_selftest();
+ result |= adiantum_selftest();
+ if (result)
+ panic("adiantum self-tests failed");
+ return 0;
+ }
+ case MODULE_CMD_FINI:
+ return 0;
+ default:
+ return ENOTTY;
+ }
+}
+
+#else /* !defined(_KERNEL) */
+
+#include <err.h>
+#include <stdio.h>
+#include <unistd.h>
+
+static int
+read_block(int fd, void *buf, size_t len)
+{
+ char *p = buf;
+ size_t n = len;
+ ssize_t nread;
+
+ for (;;) {
+ if ((nread = read(fd, p, n)) == -1)
+ err(1, "read");
+ if (nread == 0) {
+ if (n < len)
+ errx(1, "partial block");
+ return -1; /* eof */
+ }
+ if ((size_t)nread >= n)
+ break;
+ p += (size_t)nread;
+ n -= (size_t)nread;
+ }
+
+ return 0;
+}
+
+static void
+write_block(int fd, const void *buf, size_t len)
+{
+ const char *p = buf;
+ size_t n = len;
+ ssize_t nwrit;
+
+ for (;;) {
+ if ((nwrit = write(fd, p, n)) == -1)
+ err(1, "write");
+ if ((size_t)nwrit >= n)
+ break;
+ p += (size_t)nwrit;
+ n -= (size_t)nwrit;
+ }
+}
+
+#define SECSIZE 512
+
+static void
+process(void)
+{
+ static const uint8_t k[32] = {0};
+ static uint8_t buf[65536];
+ static struct adiantum C;
+ uint8_t blkno[8] = {0};
+ unsigned i;
+
+ adiantum_init(&C, k);
+ while (read_block(STDIN_FILENO, buf, sizeof buf) == 0) {
+ for (i = 0; i < sizeof buf; i += SECSIZE) {
+ adiantum_enc(buf + i, buf + i, SECSIZE, blkno, 8, &C);
+ le64enc(blkno, 1 + le32dec(blkno));
+ }
+ write_block(STDOUT_FILENO, buf, sizeof buf);
+ if (le64dec(blkno) == 1024*1024*1024/SECSIZE)
+ return;
+ }
+}
+
+int
+main(void)
+{
+ int result = 0;
+
+ result |= addsub128_selftest();
+ result |= poly1305_selftest();
+ result |= nh_selftest();
+ result |= nhpoly1305_selftest();
+ result |= chacha_core_selftest();
+ result |= hchacha_selftest();
+ result |= xchacha_selftest();
+ result |= adiantum_selftest();
+ if (result)
+ return result;
+
+ process();
+ return 0;
+}
+
+#endif /* _KERNEL */
diff -r 36794fee0d04 -r 9fde04e138c1 sys/crypto/adiantum/adiantum_selftest.c
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/sys/crypto/adiantum/adiantum_selftest.c Wed Jun 17 02:47:43 2020 +0000
@@ -0,0 +1,1835 @@
+/* $NetBSD$ */
+
+/*-
+ * Copyright (c) 2020 The NetBSD Foundation, Inc.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND CONTRIBUTORS
+ * ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
+ * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+ * POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <sys/cdefs.h>
+__KERNEL_RCSID(1, "$NetBSD$");
+
+#include <sys/types.h>
+
+#ifdef _KERNEL
+
+#include <sys/systm.h>
+
+#include <lib/libkern/libkern.h>
+
+#else
+
+#include <string.h>
+#include <stdio.h>
+
+#include <openssl/aes.h>
+
+struct aesenc {
+ AES_KEY enckey;
+};
+
+struct aesdec {
+ AES_KEY deckey;
+};
+
+static void
+hexdump(int (*prf)(const char *, ...) __printflike(1,2), const char *prefix,
+ const void *buf, size_t len)
+{
+ const uint8_t *p = buf;
+ size_t i;
+
+ (*prf)("%s (%zu bytes)\n", prefix, len);
+ for (i = 0; i < len; i++) {
+ if (i % 16 == 8)
+ (*prf)(" ");
+ else
+ (*prf)(" ");
+ (*prf)("%02hhx", p[i]);
+ if ((i + 1) % 16 == 0)
+ (*prf)("\n");
+ }
+ if (i % 16)
+ (*prf)("\n");
+}
+
+#endif
+
+#include "adiantum.h"
+
+/* https://github.com/google/adiantum/blob/aab35db7bfb6e05d5ad0b41b5088a9f5a840bde3/test_vectors/ours/Adiantum/Adiantum_XChaCha12_32_AES256.json */
+
+int
+adiantum_selftest(void)
+{
+ static const struct {
+ uint8_t k[32];
+ unsigned tlen;
+ uint8_t t[64];
+ unsigned len;
+ uint8_t p[4096];
+ uint8_t c[4096];
+ } C[] = {
+ [0] = {
+ .k = {
+ 0x7f,0xc7,0x15,0x2a, 0xe1,0xf5,0xfd,0xa4,
+ 0x17,0x67,0x69,0xae, 0xc9,0x2b,0xba,0x82,
+ 0xa3,0x14,0xe7,0xcf, 0xad,0xfd,0x85,0x40,
+ 0xda,0x7b,0x7d,0x24, 0xbd,0xf1,0x7d,0x07,
+ },
+ .tlen = 0,
+ .len = 16,
+ .p = {
+ 0x9b,0xe3,0x82,0xc6, 0x5a,0xc1,0x9f,0xad,
+ 0x46,0x59,0xb8,0x0b, 0xac,0xc8,0x57,0xa0,
+ },
+ .c = {
+ 0x82,0x0a,0xe4,0x44, 0x77,0xdd,0x9a,0x18,
+ 0x6f,0x80,0x28,0x8b, 0x25,0x07,0x0e,0x85,
+ },
+ },
+ [1] = {
+ .k = {
+ 0x26,0x6a,0xf9,0x4a, 0x21,0x49,0x6b,0x4e,
+ 0x3e,0xff,0x43,0x46, 0x9c,0xc1,0xfa,0x72,
+ 0x0e,0x77,0x9a,0xd5, 0x37,0x47,0x00,0x38,
+ 0xb3,0x6f,0x58,0x6c, 0xde,0xc0,0xa6,0x74,
+ },
+ .tlen = 0,
+ .len = 128,
+ .p = {
+ 0xdd,0x07,0xfe,0x61, 0x97,0x0c,0x31,0x48,
+ 0x09,0xbf,0xdb,0x9b, 0x4b,0x7d,0x9c,0x80,
+ 0xe6,0x11,0xe5,0x76, 0x5b,0xcc,0x76,0xdf,
+ 0x34,0xd5,0x23,0xcd, 0xe1,0xdc,0x4e,0x4f,
+ 0x65,0x20,0x58,0x8e, 0xe8,0x2c,0xc2,0x64,
+ 0x32,0x83,0x7a,0xbf, 0xe1,0xca,0x0b,0x4b,
+ 0xc6,0xec,0x0d,0xc5, 0x4a,0xb6,0x9b,0xa5,
+ 0xc4,0x01,0x54,0xf5, 0xb5,0xfa,0x8f,0x58,
+ 0x45,0x72,0x28,0xd8, 0x55,0x21,0xa2,0x5c,
+ 0x7d,0xc8,0x0c,0x3c, 0x3c,0x99,0xc4,0x1a,
+ 0xc2,0xe7,0x1c,0x0c, 0x14,0x72,0x1d,0xf8,
+ 0x45,0xb7,0x9c,0x97, 0x07,0x04,0x9b,0x91,
+ 0x5e,0x95,0xef,0x5f, 0xe6,0xad,0xbd,0xbb,
+ 0xe7,0xd1,0x22,0xc3, 0x98,0x44,0x89,0x05,
+ 0xe8,0x63,0x0d,0x44, 0xcb,0x36,0xd5,0x43,
+ 0xcc,0x05,0x7c,0x31, 0xd3,0xbc,0x17,0x7f,
+ },
+ .c = {
+ 0xba,0xd3,0xbf,0xbf, 0xb2,0x4e,0x1a,0xfd,
+ 0x59,0xbe,0x9d,0x40, 0xe0,0x27,0x94,0xdd,
+ 0x5c,0x08,0x1c,0xa5, 0xd0,0x25,0x87,0xca,
+ 0x15,0x6a,0x35,0xe9, 0x8a,0x05,0x67,0x53,
+ 0x04,0x4d,0xdf,0x35, 0x07,0x19,0x25,0xa0,
+ 0x44,0x1a,0x5b,0xd6, 0x8b,0x0f,0xd3,0x36,
+ 0x8a,0x60,0x8c,0x6b, 0x53,0xdb,0x69,0xb0,
+ 0x37,0x69,0xb5,0x1b, 0x1f,0xf5,0xd5,0xab,
+ 0x47,0x3a,0x45,0xb2, 0x37,0x6c,0xc3,0xc1,
+ 0x1f,0xdb,0x74,0x6b, 0x1f,0x3b,0x2c,0x1a,
+ 0xee,0xff,0xe9,0x28, 0xfe,0xa3,0x49,0x96,
+ 0x7a,0xb3,0x68,0x4e, 0xb1,0xc4,0x85,0xdc,
+ 0x18,0x87,0xfd,0xbf, 0x84,0x39,0xb2,0x20,
+ 0x29,0x46,0x8a,0x3e, 0xa9,0xf9,0xcc,0x56,
+ 0x6b,0x2f,0x43,0x4a, 0x1b,0x48,0x6b,0xd6,
+ 0x03,0x1d,0x66,0xa1, 0x49,0xba,0xe9,0xf5,
+ },
+ },
+ [2] = {
+ .k = {
+ 0x7c,0xab,0xc4,0x63, 0xc0,0x40,0x5e,0xad,
+ 0x8f,0x02,0x5a,0xa9, 0xba,0x68,0x58,0xe3,
+ 0xb6,0xbb,0x03,0xc9, 0xe6,0x1e,0xe7,0xc3,
+ 0xd7,0x2c,0xf7,0x7a, 0xf7,0x2c,0xd1,0x07,
+ },
+ .tlen = 0,
+ .len = 512,
+ .p = {
+ 0x4f,0xc9,0x8f,0xa7, 0x81,0x81,0x3a,0xb7,
+ 0x3c,0x55,0x8f,0x8f, 0x18,0xc4,0x7a,0xd2,
+ 0x13,0x70,0x94,0x0f, 0x46,0xb2,0x0f,0x53,
+ 0xde,0xdf,0x06,0xf8, 0x60,0x34,0xad,0x39,
+ 0xe9,0x47,0x23,0x31, 0x94,0xf3,0x59,0x88,
+ 0x96,0x14,0x52,0x3b, 0x88,0xb7,0x55,0xe9,
+ 0x4a,0xbc,0x41,0xea, 0x24,0x03,0x35,0x78,
+ 0xb7,0x4b,0x9f,0x8b, 0xe4,0x36,0x77,0x0a,
+ 0x70,0x19,0x90,0x9b, 0xb1,0x70,0x27,0x23,
+ 0x31,0xd9,0xe5,0x26, 0x36,0x71,0x06,0xc7,
+ 0xd3,0xb1,0xb8,0x52, 0x6a,0xe1,0x95,0x86,
+ 0x76,0xc3,0x02,0x2c, 0xd2,0xe7,0xc2,0x1c,
+ 0x6f,0xcb,0x61,0x56, 0xfc,0x5e,0xf2,0x57,
+ 0x90,0x46,0xfb,0x6a, 0xc1,0x5e,0x56,0x5b,
+ 0x18,0x8d,0x0e,0x4f, 0x4e,0x14,0x4c,0x6d,
+ 0x97,0xf9,0x73,0xed, 0xc5,0x41,0x94,0x24,
+ 0xaa,0x35,0x2f,0x01, 0xef,0x8f,0xb2,0xfd,
+ 0xc2,0xc7,0x8b,0x9c, 0x9b,0x10,0x89,0xec,
+ 0x64,0xbb,0x54,0xa5, 0x01,0xdc,0x51,0x57,
+ 0xc8,0x5a,0x03,0xcb, 0x91,0x73,0xb2,0x08,
+ 0xc3,0xcc,0x3c,0x1b, 0xae,0x3e,0x0f,0xf3,
+ 0x93,0xb9,0xc3,0x27, 0xd7,0x88,0x66,0xa2,
+ 0x40,0xf9,0xfd,0x02, 0x61,0xe1,0x2b,0x5d,
+ 0xc9,0xe8,0xd6,0xac, 0xf0,0xd0,0xe3,0x79,
+ 0x94,0xff,0x50,0x09, 0x4e,0x68,0xe8,0x5e,
+ 0x3f,0x58,0xc8,0xb8, 0x0f,0xd7,0xc2,0x2d,
+ 0x91,0x3e,0x47,0x10, 0x50,0x98,0xa6,0xf9,
+ 0x37,0xd6,0x90,0xed, 0xb7,0x5e,0x3a,0xd0,
+ 0xd7,0x50,0xc4,0x69, 0xe6,0x29,0xb8,0x9a,
+ 0xc1,0x5c,0x2b,0x34, 0x6d,0x44,0x58,0xd6,
+ 0xd4,0x7e,0xe2,0x42, 0x67,0x45,0xe5,0x64,
+ 0x48,0xac,0x00,0xe9, 0xb6,0xd0,0xc3,0xc5,
+ 0x5d,0x9e,0x95,0x4e, 0x10,0x18,0x29,0x86,
+ 0xaa,0x37,0xa3,0x3c, 0xe1,0xd6,0x5d,0x6d,
+ 0x4a,0xca,0xc3,0xe2, 0x25,0xb7,0x49,0x4a,
+ 0x36,0x67,0xc0,0xe1, 0x02,0x45,0xcc,0xd4,
+ 0x11,0x37,0x11,0x8e, 0x54,0xf5,0xea,0x80,
+ 0x04,0x72,0x06,0x36, 0x8f,0xf9,0x1e,0xed,
+ 0x91,0x14,0x9d,0x42, 0x59,0xc1,0x87,0xb8,
+ 0xf1,0xce,0xb2,0x17, 0x42,0xa1,0x2f,0x96,
+ 0xa3,0x50,0xe9,0x01, 0x24,0x9e,0xe5,0xbb,
+ 0x97,0x83,0x31,0x12, 0xa8,0x7c,0xca,0x7b,
+ 0x90,0x33,0xad,0x1c, 0x99,0x81,0x1a,0xb8,
+ 0xa1,0xe0,0xf1,0x5a, 0xbc,0x08,0xde,0xab,
+ 0x69,0x0a,0x89,0xa0, 0x9f,0x02,0x5e,0x3a,
+ 0xf3,0xba,0xb9,0x6e, 0x34,0xdf,0x15,0x13,
+ 0x64,0x51,0xa9,0x55, 0x67,0xa3,0xba,0x6b,
+ 0x35,0xb0,0x8a,0x05, 0xf5,0x79,0x84,0x97,
+ 0x92,0x8e,0x11,0xeb, 0xef,0xec,0x65,0xb5,
+ 0xe6,0x42,0xfb,0x06, 0x33,0x93,0x6b,0xff,
+ 0xc2,0x49,0x15,0x71, 0xb0,0xca,0x62,0xd1,
+ 0x81,0x40,0xd2,0xab, 0x0b,0x7d,0x7e,0x1a,
+ 0xe9,0xec,0xfc,0xde, 0xdb,0xd5,0xa7,0x56,
+ 0x83,0x25,0x0e,0x5e, 0xac,0x0c,0x42,0x26,
+ 0x00,0x59,0x55,0x17, 0x8b,0x5a,0x03,0x7b,
+ 0x85,0xe9,0xc1,0xa3, 0xe4,0xeb,0xd3,0xde,
+ 0xd8,0x81,0xf5,0x31, 0x2c,0xda,0x21,0xbc,
+ 0xb5,0xd9,0x7a,0xd0, 0x1e,0x2a,0x6b,0xcf,
+ 0xad,0x06,0x3c,0xf2, 0xf7,0x5c,0x3a,0xf1,
+ 0xa7,0x0f,0x5f,0x53, 0xe9,0x3f,0x3c,0xf1,
+ 0xb7,0x47,0x53,0x16, 0x19,0xd9,0xef,0xf0,
+ 0xcb,0x16,0xe4,0xc9, 0xa3,0x8f,0xd6,0x3f,
+ 0xf8,0xb2,0x22,0x65, 0xf9,0xa1,0xa3,0x03,
+ 0xe4,0x06,0x75,0x69, 0xf5,0x32,0x48,0x80,
+ },
+ .c = {
+ 0x66,0x3f,0xf7,0x7a, 0x20,0xa4,0x35,0xd6,
+ 0x0e,0xe8,0x17,0x32, 0x84,0xae,0xee,0x18,
+ 0x0f,0x64,0x83,0x66, 0xa4,0xf4,0x24,0x53,
+ 0xe6,0x58,0x2e,0xd5, 0x61,0x58,0xdd,0x5f,
+ 0x1d,0xb9,0xba,0x34, 0xd0,0xd3,0x64,0xde,
+ 0x99,0x47,0x92,0x3a, 0x26,0x90,0xbb,0x98,
+ 0xb0,0xbd,0xf4,0x5e, 0x26,0x57,0xe0,0xe1,
+ 0x09,0x27,0xc1,0xc4, 0x86,0x2b,0x4b,0x48,
+ 0xbb,0xcd,0xec,0x2f, 0xd1,0x54,0xe9,0x21,
+ 0xa0,0x40,0x76,0x01, 0x2d,0xb1,0xe7,0x75,
+ 0xa1,0xd7,0x04,0x23, 0x9d,0xd3,0x0f,0x3b,
+ 0x7e,0xb8,0xd0,0x37, 0xe4,0xd9,0x48,0xaa,
+ 0xe1,0x4d,0x0f,0xf6, 0xae,0x29,0x20,0xae,
+ 0xda,0x35,0x18,0x97, 0x2c,0xc2,0xa9,0xdd,
+ 0x6e,0x50,0x73,0x52, 0x0a,0x8a,0x2a,0xd2,
+ 0x2a,0xf4,0x12,0xe9, 0x7d,0x88,0x37,0xae,
+ 0x12,0x81,0x92,0x96, 0xbe,0xea,0x15,0xa4,
+ 0x3c,0x53,0xad,0x1f, 0x75,0x54,0x24,0x81,
+ 0xaa,0x1b,0x92,0x84, 0x7c,0xb2,0xd7,0x10,
+ 0x5e,0xb6,0xab,0x83, 0x25,0xf7,0x03,0x2b,
+ 0xd9,0x53,0x4d,0xf9, 0x41,0x21,0xef,0xef,
+ 0x40,0x3a,0x2d,0x54, 0xa9,0xf0,0x72,0xff,
+ 0x03,0x59,0x2e,0x91, 0x07,0xff,0xe2,0x86,
+ 0x33,0x59,0x98,0xdf, 0xa4,0x7d,0x9e,0x52,
+ 0x95,0xd9,0x77,0x4b, 0xdf,0x93,0xc8,0x2d,
+ 0xbc,0x81,0x2b,0x77, 0x89,0xae,0x52,0xdc,
+ 0xfc,0xb7,0x22,0xf0, 0x1a,0x9d,0xc1,0x28,
+ 0x70,0xe2,0x15,0xe4, 0x77,0x11,0x49,0x09,
+ 0x89,0xf4,0x06,0x00, 0x64,0x78,0xb6,0x3f,
+ 0x63,0x36,0xfd,0x9f, 0x35,0x33,0x85,0x52,
+ 0x18,0x26,0xc1,0x0d, 0xf7,0xab,0x5a,0x06,
+ 0x9c,0x3a,0xab,0x5f, 0x81,0x36,0x39,0xe3,
+ 0xe6,0xf7,0x33,0xb0, 0xec,0xe6,0x8d,0x05,
+ 0xbd,0xc7,0xbd,0x20, 0x5f,0x74,0xdf,0x98,
+ 0x3a,0xa9,0xde,0xae, 0x89,0xee,0xcc,0x60,
+ 0x8b,0x23,0xed,0x0f, 0x55,0x4d,0x56,0xd2,
+ 0x69,0xa5,0xf8,0xff, 0x94,0x62,0x99,0xc6,
+ 0xd4,0x02,0x0b,0xcf, 0xe4,0x86,0x23,0x5e,
+ 0xed,0x12,0x12,0x2e, 0x0a,0x0f,0xda,0x12,
+ 0x0a,0x68,0x56,0xea, 0x16,0x92,0xa5,0xdb,
+ 0xf5,0x9d,0x0e,0xe6, 0x39,0x5d,0x76,0x50,
+ 0x41,0x85,0xb4,0xcc, 0xb3,0x9e,0x84,0x46,
+ 0xd3,0x93,0xcf,0xa1, 0xee,0x5b,0x51,0x94,
+ 0x05,0x46,0x16,0xbb, 0xd1,0xae,0x94,0xe4,
+ 0x1c,0x3d,0xeb,0xf4, 0x09,0x00,0xf7,0x86,
+ 0x57,0x60,0x49,0x94, 0xf5,0xa7,0x7e,0x4b,
+ 0x32,0x4a,0x6a,0xae, 0x2c,0x5f,0x30,0x2d,
+ 0x7c,0xa1,0x71,0x5e, 0x63,0x7a,0x70,0x56,
+ 0x1f,0xaf,0x3e,0xf3, 0x46,0xb5,0x68,0x61,
+ 0xe2,0xd4,0x16,0x6b, 0xaf,0x94,0x07,0xa9,
+ 0x5d,0x7a,0xee,0x4c, 0xad,0x85,0xcc,0x3e,
+ 0x99,0xf3,0xfa,0x21, 0xab,0x9d,0x12,0xdf,
+ 0x33,0x32,0x23,0x68, 0x96,0x8f,0x8f,0x78,
+ 0xb3,0x63,0xa0,0x83, 0x16,0x06,0x64,0xbd,
+ 0xea,0x1f,0x69,0x73, 0x9c,0x54,0xe1,0x60,
+ 0xe8,0x98,0xc9,0x94, 0xe9,0xdf,0x0c,0xee,
+ 0xf4,0x38,0x1e,0x9f, 0x26,0xda,0x3f,0x4c,
+ 0xfd,0x6d,0xf5,0xee, 0x75,0x91,0x7c,0x4f,
+ 0x4d,0xc2,0xe8,0x1a, 0x7b,0x1b,0xa9,0x52,
+ 0x1e,0x24,0x22,0x5a, 0x73,0xa5,0x10,0xa2,
+ 0x37,0x39,0x1e,0xd2, 0xf7,0xe0,0xab,0x77,
+ 0xb7,0x93,0x5d,0x30, 0xd2,0x5a,0x33,0xf4,
+ 0x63,0x98,0xe8,0x6d, 0x3f,0x34,0x4a,0xb9,
+ 0x44,0x57,0x39,0xe7, 0xa9,0xdd,0xac,0x91,
+ },
+ },
+ [3] = {
+ .k = {
+ 0xac,0x95,0xec,0x00, 0xa5,0x57,0x8e,0x99,
+ 0x14,0x54,0x95,0x60, 0xdc,0xae,0x56,0x66,
+ 0x03,0x22,0xa1,0x55, 0xbf,0xa5,0x2b,0x1c,
+ 0x02,0xc9,0x0c,0x2f, 0xa1,0x5d,0x1b,0x84,
+ },
+ .tlen = 0,
+ .len = 1536,
+ .p = {
+ 0xd2,0x80,0x06,0x95, 0xcd,0xe1,0x71,0x2c,
+ 0xcf,0x89,0xa6,0xc7, 0x8b,0xa7,0xe3,0xcb,
+ 0x66,0x3e,0x6b,0x58, 0x2a,0x20,0xd1,0xc4,
+ 0x07,0xd6,0x3b,0x03, 0xdc,0x26,0xda,0x1b,
+ 0xe0,0x51,0xd5,0x1c, 0x4c,0xed,0xd0,0xf5,
+ 0xe2,0x7f,0x89,0xe8, 0x3d,0x41,0x1a,0xa0,
+ 0xb1,0xed,0x61,0xa8, 0xc7,0x0a,0xe8,0x69,
+ 0x4d,0xb8,0x18,0x81, 0x6c,0x76,0x67,0x83,
+ 0x8a,0x47,0xa2,0x4b, 0xfb,0xfd,0x6f,0x65,
+ 0x88,0xa8,0xf6,0x6d, 0x9f,0x71,0x6e,0x33,
+ 0x4f,0x82,0xee,0x8f, 0x38,0x5c,0xe4,0x9b,
+ 0x45,0x29,0xca,0xda, 0x9b,0x5d,0x65,0x06,
+ 0xab,0xf5,0x86,0x28, 0x8c,0x3e,0x20,0x38,
+ 0x1a,0x4c,0xb2,0xd9, 0x1f,0xc0,0x10,0x59,
+ 0x6b,0x2c,0xb5,0x41, 0x41,0xc5,0xd9,0xb7,
+ 0x4f,0xc3,0x36,0x08, 0xd4,0xdc,0xff,0x57,
+ 0xd7,0x97,0x77,0x45, 0xc4,0x28,0x93,0x2c,
+ 0xbe,0xdc,0xae,0x1d, 0x18,0xc8,0xfa,0x9a,
+ 0xd4,0x41,0x2e,0x5a, 0x26,0x03,0xae,0x7a,
+ 0xb2,0x6a,0xc0,0x0c, 0xb6,0x3e,0xf0,0x73,
+ 0x36,0xed,0xea,0xc1, 0xae,0x9d,0xc9,0xa1,
+ 0x85,0x4c,0x57,0x14, 0xb0,0xf3,0xf8,0x4e,
+ 0x91,0x99,0x06,0x65, 0x17,0x66,0xc2,0x9a,
+ 0x7a,0x4f,0x39,0x77, 0x32,0x44,0xc8,0x3f,
+ 0xe2,0x3c,0xc2,0x31, 0x0b,0x40,0x84,0xee,
+ 0xa1,0xeb,0xc6,0xc2, 0xb4,0x48,0xe6,0x09,
+ 0xc5,0xf5,0x3d,0x96, 0x90,0xa2,0x1d,0xf2,
+ 0x89,0x26,0x9f,0x10, 0x49,0x30,0x0f,0xe1,
+ 0x5e,0xca,0x1c,0x3f, 0x82,0xda,0xcb,0x8d,
+ 0x91,0x6d,0x08,0x96, 0x9e,0x57,0x88,0x16,
+ 0xee,0xa7,0x9e,0xe8, 0x1b,0xc1,0x63,0xb0,
+ 0x57,0xfa,0xfd,0x56, 0x49,0xec,0x51,0x1d,
+ 0x34,0x2e,0xc6,0xda, 0xc0,0x1d,0x02,0x3e,
+ 0x52,0xaf,0x44,0x24, 0xc6,0x80,0x12,0x64,
+ 0xbe,0x44,0xa8,0x46, 0xb5,0x8d,0x80,0xfd,
+ 0x95,0x4a,0xeb,0x3d, 0x4f,0x85,0x1f,0x1c,
+ 0xa4,0x3f,0x5c,0x0c, 0x71,0xed,0x96,0x41,
+ 0xde,0xb0,0xbd,0x08, 0xf3,0x4d,0x37,0xd2,
+ 0xb1,0x4f,0x71,0x04, 0xf1,0x14,0x66,0x4a,
+ 0x59,0x73,0xdc,0x98, 0x5b,0x61,0x56,0xfd,
+ 0x50,0xe5,0x76,0xd9, 0x6a,0x9f,0x30,0x82,
+ 0x6f,0xdf,0x6e,0x7b, 0x91,0xc2,0x5e,0x4f,
+ 0x74,0x92,0x92,0xb8, 0x24,0xd3,0x30,0x21,
+ 0x5d,0x4b,0xb1,0x01, 0xf7,0x62,0x27,0x94,
+ 0xb3,0x88,0x86,0x75, 0xe8,0xab,0xe8,0x42,
+ 0x50,0x15,0xb7,0xde, 0xc0,0xc4,0x8d,0x4e,
+ 0x08,0x17,0xcb,0xf9, 0x4a,0x2e,0xe3,0x69,
+ 0xbd,0xe7,0xdb,0xd1, 0xf1,0xfa,0x47,0xed,
+ 0x78,0xa9,0x26,0xf0, 0xd1,0xbb,0x02,0xa1,
+ 0x07,0x5c,0x1f,0xe8, 0x2f,0x52,0xd8,0x95,
+ 0xd7,0xa9,0x2b,0x79, 0x77,0xf4,0xee,0xee,
+ 0xbc,0x1f,0xaa,0x46, 0xe7,0x66,0x75,0xb1,
+ 0x43,0x01,0x35,0xac, 0xc6,0x85,0xad,0x44,
+ 0x23,0x59,0x50,0x0b, 0x39,0x47,0x51,0x54,
+ 0x68,0x92,0x89,0x00, 0x08,0xa3,0xaa,0x24,
+ 0x03,0x3f,0xf6,0xab, 0x19,0x42,0xff,0x0c,
+ 0xc5,0xa3,0x96,0xcb, 0xd9,0x6d,0xa0,0xcc,
+ 0x24,0x9e,0x71,0xb1, 0x87,0x95,0x7a,0x2e,
+ 0x31,0x5e,0x17,0x26, 0x5a,0x1b,0xa1,0x33,
+ 0x10,0x3f,0xd7,0xce, 0xa0,0xd9,0xbc,0xd8,
+ 0x72,0xbe,0x75,0xc4, 0x78,0x3b,0x67,0xf5,
+ 0xc3,0x82,0x2d,0x21, 0x49,0x74,0x2e,0xd5,
+ 0x63,0xaa,0xa2,0x54, 0xc5,0xe2,0x98,0x82,
+ 0x39,0xd9,0xda,0x14, 0x3c,0x75,0x18,0xc8,
+ 0x75,0x6a,0xa1,0x7d, 0xfa,0x72,0x0f,0x9b,
+ 0x5a,0xb3,0x7c,0x15, 0xc2,0xa5,0x6d,0x98,
+ 0x02,0x6c,0xa2,0x26, 0xaa,0xc0,0x69,0xc5,
+ 0xa7,0xa2,0xca,0xf5, 0xf3,0x8c,0x80,0x4e,
+ 0x7e,0x47,0xc9,0x87, 0x47,0x36,0xd6,0xc6,
+ 0xe8,0x49,0xb5,0x97, 0xa8,0xdc,0x4a,0x55,
+ 0x6f,0x02,0x79,0x83, 0xe4,0x7c,0x4c,0x69,
+ 0xa6,0x4d,0x4f,0x8a, 0x48,0x18,0x00,0xf9,
+ 0xad,0xd1,0xb2,0xca, 0xc4,0x50,0x47,0x21,
+ 0x4e,0xa7,0xce,0x6e, 0xdf,0xbd,0x2a,0x4d,
+ 0xca,0x13,0x33,0xde, 0xa2,0x30,0xe1,0x03,
+ 0xcd,0x2c,0x74,0xd3, 0x30,0x0d,0x61,0xe6,
+ 0x9d,0xf3,0x09,0xc5, 0x27,0x99,0x0e,0x23,
+ 0xbc,0x21,0xdb,0xdb, 0xeb,0x77,0xea,0xd4,
+ 0x4b,0xbf,0x9b,0x49, 0x30,0xd4,0xc2,0xe7,
+ 0x5e,0x85,0xe8,0xb6, 0xa5,0xe3,0x4e,0x64,
+ 0xf0,0x45,0x95,0x04, 0x9a,0xed,0xaa,0x4d,
+ 0xbd,0x5e,0x03,0x9f, 0xd4,0x2b,0xae,0x14,
+ 0x1a,0x3d,0x49,0x92, 0xd6,0x6f,0x64,0xc7,
+ 0xca,0x18,0x32,0x16, 0xf6,0x07,0x00,0x22,
+ 0xfd,0xe1,0x45,0xe6, 0x19,0x24,0x5b,0x6e,
+ 0xd3,0x67,0xf2,0x60, 0x36,0xf5,0x22,0xeb,
+ 0x5f,0x42,0xba,0x70, 0x38,0xfc,0x98,0x96,
+ 0x58,0x72,0xbf,0x13, 0x60,0xcc,0x32,0x45,
+ 0x8d,0x00,0x44,0x60, 0xaf,0x7a,0x19,0xd6,
+ 0xc0,0x14,0x33,0x96, 0xf3,0x33,0xc3,0xa8,
+ 0x34,0x77,0x69,0x0c, 0x50,0xe5,0xfc,0x1b,
+ 0x42,0x39,0x96,0x24, 0x3a,0x3a,0x47,0x0e,
+ 0x27,0x66,0xa8,0x18, 0x50,0xdf,0x6d,0xa7,
+ 0xad,0x4f,0xe5,0x88, 0x79,0xea,0x30,0xe2,
+ 0xcd,0x27,0x05,0x36, 0x0c,0x3c,0x97,0x12,
+ 0x69,0xa6,0xc0,0xa2, 0xa7,0x58,0x82,0x20,
+ 0x68,0xfc,0xd0,0x81, 0x49,0xc0,0xcf,0xba,
+ 0x90,0xe1,0x03,0xce, 0x70,0xd6,0x94,0x1a,
+ 0xc0,0x22,0x3b,0xdc, 0x7f,0x63,0x6b,0xc4,
+ 0x91,0xc2,0x21,0xdc, 0x84,0x42,0x80,0x04,
+ 0x6f,0x14,0xc3,0x2c, 0x79,0x49,0x3c,0xb1,
+ 0x5f,0xc7,0x69,0x4a, 0x4f,0xf5,0xd5,0x4b,
+ 0x7c,0xe7,0x83,0x79, 0x30,0xff,0x74,0xe0,
+ 0xf7,0xd3,0x6c,0x95, 0xef,0x77,0xe8,0x7b,
+ 0x1f,0x54,0xad,0xc7, 0x4b,0xe8,0x5a,0x37,
+ 0xd7,0xe9,0xfe,0xcb, 0x11,0x7b,0x54,0xb8,
+ 0xd2,0xc7,0x80,0x1d, 0x80,0x17,0xdd,0x21,
+ 0xa6,0xed,0x20,0x2c, 0x8a,0xa1,0x0b,0x3a,
+ 0x08,0xde,0x34,0xe4, 0xa0,0xff,0x68,0xfa,
+ 0x4a,0x01,0xcc,0x4f, 0x57,0x5f,0x84,0x95,
+ 0x88,0xe2,0x7f,0xb7, 0x5d,0x35,0x36,0xe2,
+ 0xa1,0xca,0xc0,0x9b, 0x4a,0xb0,0x6f,0x35,
+ 0xef,0x08,0xd7,0x5a, 0xec,0x4f,0x97,0x20,
+ 0x92,0x2a,0x63,0x1d, 0x15,0x07,0x73,0x1f,
+ 0x97,0xcf,0x28,0x41, 0x65,0x0d,0x41,0xee,
+ 0xca,0xd8,0x90,0x65, 0xaa,0x3d,0x04,0x7f,
+ 0x35,0x4b,0x9e,0xe9, 0x96,0xa9,0x61,0xcb,
+ 0x43,0xc9,0xfa,0x1d, 0xc8,0x85,0x40,0x64,
+ 0x88,0x89,0xea,0xb5, 0xf7,0xe5,0xe4,0xfe,
+ 0xaf,0x8e,0x52,0xf9, 0x7e,0x7d,0x83,0x92,
+ 0x90,0x51,0x4c,0xf0, 0x49,0x52,0x5e,0x56,
+ 0xc9,0xb7,0x4c,0xca, 0x57,0x01,0x3d,0x28,
+ 0xe2,0x7d,0xaa,0x96, 0xd7,0xad,0xad,0xd9,
+ 0xd5,0x1a,0xd5,0xc2, 0xd0,0x5a,0xd3,0x7a,
+ 0x9a,0x91,0xa0,0xb8, 0x6f,0x28,0xff,0xa0,
+ 0x1c,0x1d,0xf1,0x5e, 0x45,0x53,0x3f,0x85,
+ 0x1b,0xc2,0x76,0x51, 0xbf,0x25,0x02,0xf7,
+ 0x10,0xde,0xb7,0x1a, 0x04,0x6c,0x9a,0xeb,
+ 0xb9,0x4b,0x67,0xfb, 0xa1,0x5b,0xa8,0x02,
+ 0x01,0x1f,0x38,0xa9, 0x9d,0x96,0x50,0x07,
+ 0xef,0xa7,0xc3,0xb4, 0x0f,0xcd,0x1b,0x9f,
+ 0xd2,0x08,0x87,0xca, 0xd5,0x65,0x1a,0x5e,
+ 0x1a,0xff,0x97,0xb0, 0x4b,0x43,0x67,0x51,
+ 0x22,0xfd,0x49,0xcd, 0x54,0x2f,0xf8,0x9b,
+ 0xed,0x46,0x7e,0x00, 0x5b,0x67,0x06,0xeb,
+ 0xb7,0x4d,0x1c,0x72, 0x74,0xdd,0xbd,0xb1,
+ 0x71,0x0a,0x28,0xc7, 0x7b,0xa8,0x12,0xac,
+ 0x58,0x53,0xa4,0xfb, 0x41,0x74,0xb4,0x52,
+ 0x95,0x99,0xf6,0x38, 0x53,0xff,0x2d,0x26,
+ 0xef,0x12,0x91,0xc6, 0x52,0xe1,0xa9,0x50,
+ 0xfa,0x8e,0x2e,0x82, 0x8b,0x4f,0xb7,0xad,
+ 0xe1,0x74,0x0d,0xbf, 0x73,0x04,0xdf,0x3f,
+ 0xf6,0xf8,0x09,0x9d, 0xdf,0x18,0x07,0x13,
+ 0xe6,0x60,0xf0,0x6a, 0x98,0x22,0x15,0xdf,
+ 0x0c,0x72,0x6a,0x9d, 0x6e,0x67,0x76,0x61,
+ 0xda,0xbe,0x10,0xd6, 0xf0,0x5f,0x06,0x74,
+ 0x76,0xce,0x63,0xee, 0x91,0x39,0x24,0xa9,
+ 0xcf,0xc7,0xca,0xd5, 0xb4,0xff,0x30,0x6e,
+ 0x05,0x32,0x0c,0x9d, 0xeb,0xfb,0xc6,0x3e,
+ 0xe4,0xc6,0x20,0xc5, 0x3e,0x1d,0x5c,0xd6,
+ 0x05,0xbe,0xb8,0xc3, 0x44,0xe3,0xc9,0xc1,
+ 0x38,0xaa,0xc5,0xc8, 0xe3,0x11,0x8d,0xde,
+ 0xdc,0x48,0x8e,0xe9, 0x38,0xe5,0x80,0xec,
+ 0x82,0x17,0xf2,0xcf, 0x26,0x55,0xf7,0xdc,
+ 0x78,0x7f,0xfb,0xc1, 0xb4,0x6c,0x80,0xcc,
+ 0xf8,0x5a,0xbc,0x8f, 0x9d,0x62,0xfe,0x35,
+ 0x17,0x7c,0x10,0xb7, 0x4a,0x0f,0x81,0x43,
+ 0x11,0xbd,0x33,0x47, 0x9c,0x61,0x02,0xec,
+ 0xab,0xde,0xb2,0x3f, 0x73,0x48,0xfb,0x5c,
+ 0x84,0x4a,0xeb,0xab, 0x58,0x07,0x18,0xdc,
+ 0x57,0x85,0xb8,0xe7, 0xff,0x9c,0xc2,0xc8,
+ 0xb3,0xef,0x5b,0x50, 0x16,0xb1,0x38,0x6e,
+ 0xa7,0xd7,0x9c,0xb1, 0x29,0x6b,0x74,0x9c,
+ 0x50,0xcc,0x90,0xee, 0x86,0x2a,0x7c,0x07,
+ 0xd4,0xcb,0xc2,0x24, 0x53,0xb0,0x3f,0x4f,
+ 0x9b,0xc4,0x62,0x73, 0x85,0x3d,0x1e,0x54,
+ 0x86,0xda,0x1e,0x5e, 0x70,0x73,0x6a,0x2a,
+ 0x29,0x75,0xb7,0x18, 0x1a,0x72,0x81,0x64,
+ 0x58,0xa0,0xb3,0x70, 0x61,0x9f,0x22,0x37,
+ 0xac,0xdc,0xe8,0xaf, 0xe2,0x74,0xe4,0xa7,
+ 0xed,0x92,0x5c,0x47, 0xff,0xc3,0xaf,0x9e,
+ 0x59,0xe1,0x09,0x22, 0x72,0x18,0x96,0x35,
+ 0x23,0x91,0x00,0xa3, 0x7d,0x95,0x25,0x95,
+ 0xd5,0xad,0xf8,0x6e, 0xcc,0x14,0x31,0xb2,
+ 0x52,0x20,0x2a,0x41, 0xf1,0xaf,0x9a,0xaf,
+ 0xdd,0xbd,0x04,0x5a, 0xcd,0x1a,0x86,0xb1,
+ 0x45,0x1b,0x6f,0x7a, 0x02,0x45,0x05,0xef,
+ 0x74,0xdf,0xe8,0x72, 0x1c,0x82,0x57,0xea,
+ 0x2a,0x24,0x1b,0x46, 0x3f,0x66,0x89,0x9f,
+ 0x00,0xb9,0xec,0xf7, 0x59,0x6d,0xeb,0xac,
+ 0xca,0x82,0x14,0x79, 0xbf,0x7f,0xd5,0x18,
+ 0x26,0x6b,0xee,0x34, 0x44,0xee,0x6d,0x8a,
+ 0x82,0x8f,0x4f,0xa3, 0x1a,0xc3,0x9b,0x2e,
+ 0x57,0x83,0xb8,0x7d, 0xa0,0x21,0xc6,0x66,
+ 0x96,0x7d,0x30,0x81, 0x29,0xc7,0x05,0x46,
+ 0x99,0xd4,0x35,0x7b, 0x40,0xe8,0x87,0x60,
+ 0x13,0xa5,0xa6,0xb9, 0x24,0x59,0xca,0xa8,
+ 0xcd,0x62,0xeb,0xc5, 0x22,0xff,0x49,0x64,
+ 0x03,0x2d,0x42,0x01, 0xa2,0x09,0x4a,0x45,
+ 0x41,0x34,0x88,0x44, 0xf4,0xe1,0xa3,0x48,
+ 0xcf,0x2d,0xee,0xee, 0xbf,0x83,0x1a,0x42,
+ 0x8d,0xa4,0x15,0x3d, 0xfc,0x92,0x67,0x91,
+ },
+ .c = {
+ 0x5c,0xb9,0xab,0x7c, 0xe4,0x0b,0xbe,0xa5,
+ 0x17,0x18,0xdf,0xd7, 0x17,0x13,0x98,0xbd,
+ 0xcb,0x1c,0xa3,0x39, 0x9c,0xbc,0x19,0x1f,
+ 0xca,0xcb,0x50,0x89, 0x1d,0x69,0xc3,0xcb,
+ 0xd1,0x76,0x70,0x6b, 0x7c,0x62,0x49,0xe8,
+ 0xb1,0xa8,0xb7,0x58, 0x87,0xf6,0x79,0xf7,
+ 0xf2,0xc1,0xd8,0xb2, 0x1d,0xd2,0x1a,0xf5,
+ 0xa0,0x41,0xda,0x17, 0x3f,0xaa,0xdb,0xf6,
+ 0xa9,0xf2,0x49,0x1c, 0x6f,0x20,0xf3,0xae,
+ 0x4a,0x5e,0x55,0xdd, 0xa6,0x9e,0xc4,0x03,
+ 0x07,0x22,0xc0,0xbe, 0x5e,0x58,0xdd,0xf0,
+ 0x7e,0xfe,0xcf,0x2c, 0x96,0x33,0x32,0xbd,
+ 0xe8,0xdf,0x84,0x71, 0x45,0x35,0x40,0x48,
+ 0xcf,0x10,0x45,0x47, 0x97,0x4c,0x20,0x6b,
+ 0x3a,0xdd,0x73,0xd0, 0xce,0x0c,0x4c,0xf1,
+ 0x78,0xcd,0x93,0xd2, 0x21,0x70,0xeb,0x2f,
+ 0x23,0x99,0x64,0xbb, 0x97,0x28,0xe9,0xde,
+ 0xef,0x9c,0xf2,0x7f, 0x4b,0x4d,0x2c,0x66,
+ 0x7b,0x6e,0x70,0xf7, 0x25,0x68,0xea,0x93,
+ 0x3a,0x27,0xbd,0x04, 0x8b,0xcd,0xd9,0xed,
+ 0x1a,0x9d,0xca,0x8f, 0x15,0x2d,0xa1,0x25,
+ 0xb8,0x66,0x1b,0x3d, 0xd4,0xd4,0x9b,0xab,
+ 0x3a,0xa8,0xe8,0x88, 0xc6,0xd2,0x5a,0x28,
+ 0x51,0x4d,0x11,0xb6, 0x4a,0x2b,0x6d,0xe4,
+ 0xc9,0xc1,0x20,0x6f, 0xba,0x23,0x72,0xc9,
+ 0x6d,0x44,0xf0,0xaa, 0x06,0x8c,0x9b,0xbb,
+ 0x4b,0xd2,0xa0,0x94, 0x5f,0x0b,0xc8,0xa3,
+ 0x4c,0xe9,0xe2,0x8a, 0xe5,0xf9,0xe3,0x2c,
+ 0xc7,0x87,0x75,0xc1, 0xc9,0x62,0xb5,0xb4,
+ 0x04,0x86,0x6a,0x31, 0x54,0x0e,0x31,0xf7,
+ 0xad,0xea,0xbb,0xa6, 0x8e,0x6c,0xac,0x24,
+ 0x52,0x2c,0x9d,0x1f, 0xde,0x70,0xfd,0xc4,
+ 0x93,0x8b,0x75,0x6c, 0xef,0xa7,0x89,0xaf,
+ 0x2c,0x4c,0xf6,0x38, 0xdd,0x79,0xfa,0x70,
+ 0x54,0x1e,0x92,0xd4, 0xb4,0x04,0x69,0x8e,
+ 0x6b,0x9e,0x12,0xfe, 0x15,0x15,0xf7,0x99,
+ 0xb6,0x2f,0xfc,0xfa, 0x66,0xe9,0x40,0xb5,
+ 0xd3,0x10,0xbb,0x42, 0xf9,0x68,0x64,0xd4,
+ 0x2a,0xcd,0x43,0x75, 0xb0,0x9c,0x61,0x34,
+ 0xc1,0xc4,0x42,0xf3, 0xf1,0xa7,0x65,0xf4,
+ 0xcb,0x42,0xe9,0xc2, 0x5a,0x05,0xdf,0x98,
+ 0xa3,0xba,0xf7,0xe0, 0x15,0xa1,0xdf,0xf7,
+ 0xce,0xd5,0xf0,0x62, 0x89,0xe1,0x44,0x3a,
+ 0x4f,0x6f,0x75,0x3e, 0xfc,0x19,0xe3,0x5f,
+ 0x36,0x48,0xc1,0x95, 0x08,0x22,0x09,0xf9,
+ 0x07,0x74,0x1c,0xa4, 0x1b,0x7e,0xa8,0x82,
+ 0xca,0x0b,0xd9,0x1e, 0xe3,0x5b,0x1c,0xb5,
+ 0x57,0x13,0x7d,0xbd, 0xbd,0x16,0x88,0xd4,
+ 0xb1,0x8e,0xdb,0x6f, 0x2f,0x7b,0x55,0x72,
+ 0x79,0xc9,0x49,0x7b, 0xf7,0x86,0xa9,0x3d,
+ 0x2d,0x11,0x33,0x7d, 0x82,0x38,0xc7,0xb5,
+ 0x7c,0x6b,0x0b,0x28, 0x42,0x50,0x47,0x69,
+ 0xd8,0x48,0xc6,0x85, 0x0b,0x1b,0xca,0x08,
+ 0x85,0x36,0x6d,0x97, 0xe9,0x3e,0xeb,0xe2,
+ 0x28,0x6a,0x17,0x61, 0x7d,0xcb,0xb6,0xb3,
+ 0x23,0x44,0x76,0xd3, 0x57,0x39,0x9b,0x1d,
+ 0x69,0x30,0xd8,0x3f, 0x21,0xe8,0x68,0x94,
+ 0x82,0x85,0x97,0xb1, 0x1f,0x0c,0x99,0x6e,
+ 0x6e,0x44,0xa6,0x82, 0xd0,0xa2,0xe6,0xfe,
+ 0xff,0x08,0x41,0x49, 0x54,0x18,0x51,0x88,
+ 0x23,0xd5,0x14,0xbd, 0xfe,0xea,0x5d,0x15,
+ 0xd4,0x0b,0x2d,0x92, 0x94,0x8d,0xd4,0xe5,
+ 0xaf,0x60,0x88,0x2b, 0x67,0xae,0xbb,0xa8,
+ 0xec,0xae,0x9b,0x35, 0xa2,0xd7,0xe8,0xb6,
+ 0xe5,0xaa,0x12,0xd5, 0xef,0x05,0x5a,0x64,
+ 0xe0,0xff,0x79,0x16, 0xb6,0xa3,0xdb,0x1e,
+ 0xee,0xe8,0xb7,0xd6, 0x71,0xbd,0x76,0xbf,
+ 0x66,0x2a,0x9c,0xec, 0xbe,0x8c,0xb5,0x8e,
+ 0x8e,0xc0,0x89,0x07, 0x5d,0x22,0xd8,0xe0,
+ 0x27,0xcf,0x58,0x8a, 0x8c,0x4d,0xc7,0xa4,
+ 0x45,0xfc,0xe5,0xa4, 0x32,0x7c,0xbf,0x86,
+ 0xf0,0x82,0x96,0x05, 0x1e,0x86,0x03,0x0f,
+ 0x1f,0x0d,0xf2,0xfc, 0x28,0x62,0x90,0x53,
+ 0xfe,0xd4,0x28,0x52, 0x4f,0xa6,0xbc,0x4d,
+ 0xba,0x5d,0x04,0xc0, 0x83,0x61,0xf6,0x41,
+ 0xc8,0x58,0x40,0x49, 0x1d,0x27,0xd5,0x9f,
+ 0x93,0x4f,0xb5,0x7a, 0xea,0x7b,0x86,0x31,
+ 0x2b,0xe5,0x92,0x51, 0x3e,0x7a,0xbe,0xdb,
+ 0x04,0xae,0x21,0x71, 0x5a,0x70,0xf9,0x9b,
+ 0xa8,0xb6,0xdb,0xcd, 0x21,0x56,0x75,0x2e,
+ 0x98,0x38,0x78,0x4d, 0x51,0x4a,0xa6,0x03,
+ 0x8a,0x84,0xb2,0xf9, 0x6b,0x98,0x6d,0xf3,
+ 0x12,0xaa,0xd4,0xea, 0xb3,0x7c,0xb0,0xd9,
+ 0x5e,0x1c,0xb0,0x69, 0x48,0x67,0x13,0x26,
+ 0xf0,0x25,0x04,0x93, 0x6d,0xc6,0x6c,0xb2,
+ 0xcd,0x7c,0x36,0x62, 0x6d,0x38,0x44,0xe9,
+ 0x6b,0xe2,0x7f,0xc1, 0x40,0xdb,0x55,0xe1,
+ 0xa6,0x71,0x94,0x0a, 0x13,0x5f,0x9e,0x66,
+ 0x3b,0xb3,0x11,0x90, 0xbb,0x68,0xd4,0x11,
+ 0xf2,0xb7,0x61,0xbd, 0xac,0x4a,0x56,0xf4,
+ 0x9e,0xe2,0xd0,0x1e, 0xb4,0xa1,0xb8,0x4e,
+ 0xbb,0xc2,0x73,0x63, 0x04,0x99,0x97,0x9f,
+ 0x76,0x18,0x82,0x11, 0x7e,0xe1,0xcc,0x58,
+ 0xb7,0xb5,0x37,0x78, 0x60,0x19,0x6c,0x2b,
+ 0x6e,0x65,0x15,0x10, 0x3c,0x93,0xf0,0xc5,
+ 0x3d,0x9e,0xeb,0x77, 0x72,0x25,0x95,0xf0,
+ 0x27,0xe8,0xbd,0x81, 0x9c,0x22,0x38,0xa7,
+ 0x8d,0xe9,0x94,0xf2, 0x27,0x8d,0x3a,0x34,
+ 0x36,0xba,0x26,0xa0, 0xd7,0x3e,0xd8,0xbe,
+ 0x60,0xd1,0x53,0x58, 0x56,0xe6,0xf3,0xa1,
+ 0x0d,0x62,0x5e,0x44, 0xd3,0x7c,0xc9,0x25,
+ 0x87,0xc8,0x1a,0x57, 0x7f,0xfa,0x79,0x4a,
+ 0x15,0xf6,0x3e,0x2e, 0xd0,0x6b,0x83,0x9b,
+ 0xe6,0xfe,0x6c,0xd3, 0x8e,0x40,0x4a,0x12,
+ 0x57,0x41,0xc9,0x5a, 0x42,0x91,0x0b,0x28,
+ 0x56,0x38,0xfc,0x45, 0x4b,0x26,0xbf,0x3a,
+ 0xa3,0x46,0x75,0x73, 0xde,0x7e,0x18,0x7c,
+ 0x82,0x92,0x73,0xe6, 0xb5,0xd2,0x1f,0x1c,
+ 0xdd,0xb3,0xd5,0x71, 0x9f,0xd2,0xa5,0xf4,
+ 0xf1,0xcb,0xfe,0xfb, 0xd3,0xb6,0x32,0xbd,
+ 0x8e,0x0d,0x73,0x0a, 0xb6,0xb1,0xfd,0x31,
+ 0xa5,0xa4,0x7a,0xb1, 0xa1,0xbb,0xf0,0x0b,
+ 0x97,0x21,0x27,0xe1, 0xbb,0x6a,0x2a,0x5b,
+ 0x95,0xda,0x01,0xd3, 0x06,0x8e,0x53,0xd8,
+ 0x23,0xa3,0xa9,0x82, 0x8a,0xa2,0x8f,0xdb,
+ 0x87,0x37,0x41,0x41, 0x2b,0x36,0xf3,0xb3,
+ 0xa6,0x32,0x5f,0x3e, 0xbf,0x70,0x3a,0x13,
+ 0xba,0x11,0xa1,0x4e, 0x11,0xa8,0xc0,0xb7,
+ 0xb2,0x1b,0xab,0xc8, 0xcb,0x38,0x35,0x2e,
+ 0x76,0xa7,0x0b,0x5a, 0x6c,0x53,0x83,0x60,
+ 0x4f,0xee,0x91,0xe8, 0xca,0x1e,0x7f,0x76,
+ 0x2b,0x4c,0xe7,0xd4, 0xcb,0xf8,0xeb,0x94,
+ 0x76,0x17,0x68,0x23, 0x95,0x93,0x7f,0x60,
+ 0x80,0x7a,0x85,0x70, 0x95,0x56,0xb9,0x76,
+ 0x76,0xb6,0x8f,0xe2, 0x93,0x60,0xfc,0x70,
+ 0x57,0x4a,0x27,0xc0, 0xfb,0x49,0x2f,0xac,
+ 0xde,0x87,0x2f,0x1a, 0x80,0xca,0x68,0x5e,
+ 0xc6,0x18,0x4e,0x3a, 0x4b,0x36,0xdc,0x24,
+ 0x78,0x7e,0xb0,0x58, 0x85,0x4d,0xa9,0xbc,
+ 0x0d,0x87,0xdd,0x02, 0xa6,0x0d,0x46,0xae,
+ 0xf7,0x2f,0x8e,0xeb, 0xf4,0x29,0xe0,0xbc,
+ 0x9a,0x34,0x30,0xc3, 0x29,0xea,0x2c,0xb3,
+ 0xb4,0xa2,0x9c,0x45, 0x6e,0xcb,0xa4,0x9d,
+ 0x22,0xe6,0x71,0xe0, 0xcb,0x9f,0x05,0xef,
+ 0x2f,0xf7,0x12,0xfd, 0x5d,0x48,0x6c,0x9e,
+ 0x8b,0xaa,0x90,0xb6, 0xa8,0x78,0xeb,0xde,
+ 0xeb,0x4c,0xce,0x7b, 0x62,0x60,0x69,0xc0,
+ 0x54,0xc3,0x13,0x76, 0xdc,0x7e,0xd1,0xc3,
+ 0x8e,0x24,0x58,0x43, 0x3c,0xbc,0xa0,0x75,
+ 0xf2,0x7c,0x2d,0x1e, 0x94,0xec,0x40,0x15,
+ 0xe1,0x78,0xac,0x4a, 0x93,0xef,0x87,0xec,
+ 0x99,0x94,0xcb,0x65, 0xde,0xcb,0x38,0xd7,
+ 0x89,0x90,0xa2,0x68, 0xcf,0xfd,0x98,0xf8,
+ 0x1f,0x06,0xd5,0x6c, 0x53,0x1d,0xd3,0xa7,
+ 0x06,0x0b,0xa9,0x92, 0xbb,0x6e,0x6f,0xaa,
+ 0x5a,0x54,0x71,0xb7, 0x90,0x00,0x06,0x6b,
+ 0xf9,0x34,0xba,0x41, 0x73,0x58,0x98,0xfc,
+ 0xca,0x98,0xbd,0xd3, 0x7d,0xa4,0x49,0xcc,
+ 0xa8,0x19,0xc1,0x40, 0x75,0x81,0x02,0x33,
+ 0xac,0x90,0xcd,0x58, 0xeb,0x1b,0xb4,0x4e,
+ 0xe0,0x8a,0xa9,0x0f, 0x15,0x8e,0x51,0x85,
+ 0x06,0x09,0x92,0x40, 0xe3,0x75,0x60,0x64,
+ 0xcf,0x9b,0x88,0xc7, 0xb0,0xab,0x37,0x5d,
+ 0x43,0x21,0x18,0x09, 0xff,0xec,0xa0,0xb3,
+ 0x47,0x09,0x22,0x4c, 0x55,0xc2,0x2d,0x2b,
+ 0xce,0xb9,0x3a,0xcc, 0xd7,0x0c,0xb2,0x9a,
+ 0xff,0x2a,0x73,0xac, 0x7a,0xf2,0x11,0x73,
+ 0x94,0xd9,0xbe,0x31, 0x9f,0xae,0x62,0xab,
+ 0x03,0xac,0x5f,0xe2, 0x99,0x90,0xfb,0xa5,
+ 0x74,0xc0,0xfa,0xb9, 0x3c,0x96,0x7c,0x36,
+ 0x25,0xab,0xff,0x2f, 0x24,0x65,0x73,0x21,
+ 0xc3,0x21,0x73,0xc9, 0x23,0x06,0x22,0x6c,
+ 0xb2,0x22,0x26,0x1d, 0x88,0x6f,0xd3,0x5f,
+ 0x6f,0x4d,0xf0,0x6d, 0x13,0x70,0x7d,0x67,
+ 0xe8,0x5c,0x3b,0x35, 0x27,0x8a,0x8c,0x65,
+ 0xae,0x50,0x78,0xe1, 0x26,0x07,0xf8,0x18,
+ 0xfc,0xea,0xa3,0x58, 0x73,0x2b,0xca,0x92,
+ 0x10,0xdc,0xb5,0x39, 0xd5,0x2d,0x21,0xfe,
+ 0x79,0xac,0x7d,0xe8, 0x0c,0xe9,0x6d,0x3e,
+ 0xb4,0x8a,0x23,0x65, 0x08,0xbc,0x57,0x51,
+ 0xe1,0xf8,0x8d,0x5b, 0xe4,0xfe,0x14,0x60,
+ 0x02,0xe7,0xd1,0xc2, 0xd2,0x2c,0x3f,0x4d,
+ 0x08,0xd1,0xd0,0xe7, 0x3b,0xcb,0x85,0x84,
+ 0x32,0xd6,0xb9,0xfb, 0xf7,0x45,0xa1,0xaf,
+ 0x9c,0xa3,0x8d,0x37, 0xde,0x03,0x6b,0xf4,
+ 0xae,0x58,0x03,0x26, 0x58,0x4f,0x73,0x49,
+ 0xc8,0x7f,0xa3,0xdd, 0x51,0xf2,0xec,0x34,
+ 0x8f,0xd5,0xe0,0xc2, 0xe5,0x33,0xf7,0x31,
+ 0x33,0xe7,0x98,0x5f, 0x26,0x14,0x4f,0xbb,
+ 0x88,0x1f,0xb3,0x92, 0x4e,0x97,0x2d,0xee,
+ 0x08,0x5f,0x9c,0x14, 0x5f,0xaf,0x6c,0x10,
+ 0xf9,0x47,0x41,0x81, 0xe9,0x99,0x49,0x52,
+ 0x86,0x29,0x55,0xba, 0x2e,0xb6,0x62,0x24,
+ 0x58,0xf7,0x4d,0x99, 0xce,0x75,0xa8,0x45,
+ 0x66,0x27,0x48,0x3f, 0x78,0xe3,0x48,0x7c,
+ 0xd7,0x1a,0x6c,0x89, 0x9d,0xb2,0x6a,0x23,
+ 0x9d,0xd7,0xed,0x82, 0x31,0x94,0x40,0x66,
+ 0xc8,0x28,0x52,0x23, 0xe7,0x61,0xde,0x71,
+ 0x69,0xf2,0x53,0x43, 0x30,0xce,0x6a,0x1a,
+ 0xfe,0x1e,0xeb,0xc2, 0x9f,0x61,0x81,0x94,
+ 0x18,0xed,0x58,0xbb, 0x01,0x13,0x92,0xb3,
+ 0xa6,0x90,0x7f,0xb5, 0xf4,0xbd,0xff,0xae,
+ },
+ },
+ [4] = {
+ .k = {
+ 0x7f,0x56,0x7d,0x15, 0x77,0xe6,0x83,0xac,
+ 0xd3,0xc5,0xb7,0x39, 0x9e,0x9f,0xf9,0x17,
+ 0xc7,0xff,0x50,0xb0, 0x33,0xee,0x8f,0xd7,
+ 0x3a,0xab,0x0b,0xfe, 0x6d,0xd1,0x41,0x8a,
+ },
+ .tlen = 0,
+ .len = 4096,
+ .p = {
+ 0x95,0x96,0x98,0xef, 0x73,0x92,0xb5,0x20,
+ 0xec,0xfc,0x4d,0x91, 0x54,0xbf,0x8d,0x9d,
+ 0x54,0xbc,0x4f,0x0f, 0x94,0xfc,0x94,0xcf,
+ 0x07,0xf6,0xef,0xbb, 0xed,0x3f,0xd3,0x60,
+ 0xba,0x85,0x1d,0x04, 0x08,0x54,0x92,0x08,
+ 0x06,0x52,0x7f,0x33, 0xfd,0xf3,0xdf,0x2a,
+ 0x17,0x2d,0xda,0x73, 0x03,0x56,0x21,0xa9,
+ 0xa3,0xab,0xf7,0x24, 0x17,0x39,0x7e,0x0f,
+ 0x00,0xdd,0xac,0x55, 0xb0,0x8b,0x2d,0x72,
+ 0x3b,0x9a,0x36,0x5a, 0xd9,0x0a,0x8e,0x0f,
+ 0xe2,0x1d,0xe8,0x85, 0xc3,0xc1,0x17,0x11,
+ 0xa7,0x2c,0x87,0x77, 0x9d,0x6c,0x3a,0xa6,
+ 0x90,0x59,0x10,0x24, 0xb0,0x92,0xe1,0xb6,
+ 0xa9,0x89,0x7c,0x95, 0x0a,0xf2,0xb2,0xa3,
+ 0x4a,0x40,0x88,0x35, 0x71,0x4e,0xa5,0xc9,
+ 0xde,0xba,0xd7,0x62, 0x56,0x46,0x40,0x1e,
+ 0xda,0x80,0xaf,0x28, 0x5d,0x40,0x36,0xf6,
+ 0x09,0x06,0x29,0x6e, 0xaa,0xca,0xe3,0x9e,
+ 0x9a,0x4f,0x4c,0x7e, 0x71,0x81,0x6f,0x9e,
+ 0x50,0x05,0x91,0x58, 0x13,0x6c,0x75,0x6a,
+ 0xd3,0x0e,0x7e,0xaf, 0xe1,0xbc,0xd9,0x38,
+ 0x18,0x47,0x73,0x3a, 0xf3,0x78,0x6f,0xcc,
+ 0x3e,0xea,0x52,0x82, 0xb9,0x0a,0xc5,0xfe,
+ 0x77,0xd6,0x25,0x56, 0x2f,0xec,0x04,0x59,
+ 0xda,0xd0,0xc9,0x22, 0xb1,0x01,0x60,0x7c,
+ 0x48,0x1a,0x31,0x3e, 0xcd,0x3d,0xc4,0x87,
+ 0xe4,0x83,0xc2,0x06, 0x91,0xf7,0x02,0x86,
+ 0xd2,0x9b,0xfd,0x26, 0x5b,0x9b,0x32,0xd1,
+ 0x5c,0xfd,0xb4,0xa8, 0x58,0x3f,0xd8,0x10,
+ 0x8a,0x56,0xee,0x04, 0xd0,0xbc,0xaa,0xa7,
+ 0x62,0xfd,0x9a,0x52, 0xec,0xb6,0x80,0x52,
+ 0x39,0x9e,0x07,0xc8, 0xb4,0x50,0xba,0x5a,
+ 0xb4,0x9a,0x27,0xdb, 0x93,0xb6,0x98,0xfe,
+ 0x52,0x08,0xa9,0x45, 0xeb,0x03,0x28,0x89,
+ 0x26,0x3c,0x9e,0x97, 0x0f,0x0d,0x0b,0x67,
+ 0xb0,0x00,0x01,0x71, 0x4b,0xa0,0x57,0x62,
+ 0xfe,0xb2,0x6d,0xbb, 0xe6,0xe4,0xdf,0xe9,
+ 0xbf,0xe6,0x21,0x58, 0xd7,0xf6,0x97,0x69,
+ 0xce,0xad,0xd8,0xfa, 0xce,0xe6,0x80,0xa5,
+ 0x60,0x10,0x2a,0x13, 0xb2,0x0b,0xbb,0x88,
+ 0xfb,0x64,0x66,0x00, 0x72,0x8c,0x4e,0x21,
+ 0x47,0x33,0x00,0x1f, 0x85,0xa6,0x3a,0xd3,
+ 0xe2,0x6c,0xc7,0x42, 0xb6,0x7b,0xc0,0x56,
+ 0x75,0xe2,0x61,0x72, 0x15,0xd1,0x88,0x08,
+ 0x3f,0x4d,0xfd,0xe2, 0x68,0x64,0xe5,0x7a,
+ 0x23,0x9b,0x3f,0x6c, 0xc3,0xd6,0x51,0x08,
+ 0x24,0x33,0x24,0x47, 0x7e,0xea,0x23,0xdc,
+ 0x07,0x41,0x66,0xa2, 0xa4,0xeb,0x23,0xa1,
+ 0x37,0x31,0xc0,0x7a, 0xe6,0xa4,0x63,0x05,
+ 0x20,0x44,0xe2,0x70, 0xd3,0x3e,0xee,0xd8,
+ 0x24,0x34,0x5d,0x80, 0xde,0xc2,0x34,0x66,
+ 0x5a,0x2b,0x6a,0x20, 0x4c,0x99,0x0d,0xbc,
+ 0x37,0x59,0xc5,0x8b, 0x70,0x4d,0xb4,0x0e,
+ 0x51,0xec,0x59,0xf6, 0x4f,0x08,0x1e,0x54,
+ 0x3d,0x45,0x31,0x99, 0x4d,0x5e,0x29,0x5f,
+ 0x12,0x57,0x46,0x09, 0x33,0xb9,0xf2,0x66,
+ 0xb4,0xc2,0xfa,0x63, 0xbe,0x42,0x6c,0x21,
+ 0x68,0x33,0x40,0xc6, 0xbd,0xd8,0x8a,0x55,
+ 0xd7,0x90,0x27,0x25, 0x7d,0x1e,0xed,0x02,
+ 0x50,0xd8,0xb1,0xac, 0xfa,0xd9,0xd4,0xcb,
+ 0x1c,0xc9,0x43,0x60, 0x44,0xab,0xd8,0x97,
+ 0x04,0xac,0xef,0x72, 0xa3,0x88,0xdc,0xb0,
+ 0xb0,0xb6,0xc6,0xd4, 0xd0,0x38,0xaf,0xc7,
+ 0xcd,0x8d,0x2a,0xa4, 0x13,0x53,0xd9,0xfd,
+ 0x2d,0x0b,0x91,0xb4, 0x3c,0x3a,0x72,0x11,
+ 0x6c,0x8b,0x96,0xa3, 0xc6,0x0b,0xd6,0x9a,
+ 0xa2,0xb9,0xae,0x76, 0xad,0xfd,0x01,0x90,
+ 0xab,0x93,0x9c,0x4b, 0xde,0x7e,0xf2,0x82,
+ 0x96,0xb9,0x98,0x55, 0xe2,0x68,0xe0,0xd8,
+ 0x61,0xb8,0x91,0x9a, 0xaf,0x92,0xd7,0xe5,
+ 0xeb,0x88,0xc5,0xb0, 0xcb,0x75,0x55,0xa9,
+ 0x94,0x7c,0x9c,0x11, 0x14,0x81,0x1a,0x09,
+ 0x61,0xd8,0x22,0x44, 0x13,0xba,0xe8,0x06,
+ 0x78,0xfd,0xd5,0x82, 0x73,0x19,0x9a,0xd1,
+ 0x5d,0x16,0xf5,0xd8, 0x86,0x7e,0xe3,0xcd,
+ 0xdc,0xe8,0x6a,0x18, 0x05,0xba,0x10,0xe4,
+ 0x06,0xc7,0xb2,0xf3, 0xb2,0x3e,0x1c,0x74,
+ 0x86,0xdd,0xad,0x8c, 0x82,0xf0,0x73,0x15,
+ 0x34,0xac,0x1d,0x95, 0x5e,0xba,0x2a,0xba,
+ 0xf8,0xac,0xbd,0xd7, 0x28,0x74,0x28,0xc7,
+ 0x29,0xa0,0x00,0x11, 0xda,0x31,0x7c,0xab,
+ 0x66,0x4d,0xb2,0x5e, 0xae,0x71,0xc5,0x31,
+ 0xcc,0x2b,0x9f,0x36, 0x2e,0xe6,0x97,0xa4,
+ 0xe1,0xb8,0x4b,0xc9, 0x00,0x87,0x7b,0x54,
+ 0xaa,0xeb,0xff,0x1a, 0x15,0xe8,0x3e,0x11,
+ 0xf7,0x25,0x3a,0xce, 0x94,0x23,0x27,0x44,
+ 0x77,0x80,0x6e,0xdd, 0x3f,0x8e,0x5a,0x92,
+ 0xae,0xee,0xb9,0x00, 0x79,0xc3,0x1d,0xab,
+ 0x17,0xb8,0x2b,0xff, 0x0d,0x64,0x29,0xb7,
+ 0x61,0x4d,0xd0,0x8d, 0x3d,0x36,0x3d,0x13,
+ 0xed,0x12,0xe8,0x08, 0xdd,0x4b,0x37,0xf7,
+ 0x2b,0xe7,0xeb,0x92, 0x78,0x98,0xc2,0xd6,
+ 0x13,0x15,0x94,0xff, 0xef,0xdc,0xda,0x27,
+ 0x7b,0xf9,0x58,0x5b, 0x90,0xf3,0xcd,0x1b,
+ 0x38,0x8a,0x00,0x38, 0x9b,0x95,0xcb,0x18,
+ 0x1f,0x97,0xd2,0x1f, 0x60,0x9d,0x6c,0xac,
+ 0xb8,0x72,0x08,0xd9, 0xc1,0xf4,0x98,0x72,
+ 0xf9,0x44,0xf2,0x2b, 0xe1,0x6e,0x76,0x15,
+ 0x63,0xfc,0x57,0x12, 0x23,0x4a,0xff,0xd3,
+ 0x1f,0x0d,0x0c,0xb9, 0x14,0xf9,0x98,0x52,
+ 0xce,0x90,0x34,0x8c, 0xd4,0x54,0x14,0x9e,
+ 0xf7,0x2c,0xba,0x5f, 0x80,0xb0,0x02,0x68,
+ 0x4f,0xca,0xb0,0xda, 0x44,0x11,0xb4,0xbd,
+ 0x12,0x14,0x80,0x6b, 0xc1,0xce,0xa7,0xfe,
+ 0x0e,0x16,0x69,0x19, 0x3c,0xe7,0xb6,0xfe,
+ 0x5a,0x59,0x02,0xf6, 0x78,0x3e,0xa4,0x65,
+ 0x57,0xa1,0xf2,0x65, 0xad,0x64,0xfc,0xba,
+ 0xd8,0x47,0xc8,0x8d, 0x11,0xf9,0x6a,0x25,
+ 0x22,0xa7,0x7f,0xa9, 0x43,0xe4,0x07,0x6b,
+ 0x49,0x26,0x42,0xe4, 0x03,0x1f,0x56,0xcd,
+ 0xf1,0x49,0xf8,0x0d, 0xea,0x1d,0x4f,0x77,
+ 0x5c,0x3c,0xcd,0x6d, 0x58,0xa8,0x92,0x6d,
+ 0x50,0x4a,0x81,0x6e, 0x09,0x2a,0x15,0x9e,
+ 0x3b,0x56,0xd3,0xb4, 0xef,0xe6,0x12,0xaf,
+ 0x60,0x3b,0x73,0xe7, 0xd8,0x2e,0xab,0x13,
+ 0xfb,0x7e,0xea,0xb1, 0x7b,0x54,0xc5,0x26,
+ 0x41,0x93,0x31,0xda, 0xb5,0x7a,0xe3,0x46,
+ 0x7a,0x8a,0xb0,0x81, 0xab,0xd5,0x90,0x85,
+ 0x4b,0xef,0x30,0x11, 0xb8,0x00,0x19,0x39,
+ 0xd3,0x11,0x54,0x53, 0x48,0x7a,0x7e,0xc5,
+ 0x4e,0x52,0xe5,0x4c, 0xeb,0xa2,0x9f,0x7a,
+ 0xdc,0xb5,0xc8,0x4e, 0x3b,0x5c,0x92,0x0f,
+ 0x19,0xcb,0x0a,0x9d, 0xda,0x01,0xfc,0x17,
+ 0x62,0xc3,0x46,0x63, 0x8b,0x4e,0x85,0x92,
+ 0x75,0x01,0x00,0xb3, 0x74,0xa8,0x23,0xd1,
+ 0xd2,0x91,0x53,0x0f, 0xd0,0xe9,0xed,0x90,
+ 0xde,0x9c,0x8c,0xb7, 0xf1,0x6a,0xd6,0x49,
+ 0x3c,0x22,0x2b,0xd7, 0x73,0x76,0x38,0x79,
+ 0xb5,0x88,0x1e,0xee, 0xdf,0xed,0x9f,0xfd,
+ 0x1a,0x0e,0xe7,0xd5, 0xc6,0xc9,0xfb,0x03,
+ 0xcc,0x84,0xb5,0xd2, 0x49,0xca,0x49,0x0a,
+ 0x1b,0x7c,0x78,0xe4, 0xd1,0x2e,0x7c,0x14,
+ 0x80,0x38,0x9d,0xba, 0x64,0x13,0xd3,0xf8,
+ 0x8e,0x05,0x4a,0xd6, 0x0d,0x73,0x09,0x1e,
+ 0xf1,0x75,0x63,0x59, 0xed,0xfc,0xbe,0x83,
+ 0x56,0x91,0x22,0x84, 0xd2,0x1e,0xf2,0x61,
+ 0x12,0x3d,0x50,0x6c, 0x9f,0xea,0x6b,0xcd,
+ 0x8c,0xac,0x28,0x0d, 0xad,0xf4,0xfd,0x77,
+ 0x45,0x68,0x17,0xb6, 0x03,0x13,0x54,0x7a,
+ 0xc0,0x8e,0x6b,0x56, 0x8a,0xd2,0xc6,0x1b,
+ 0xb3,0x3e,0x4f,0x68, 0x91,0x2e,0x2d,0x35,
+ 0x2a,0x32,0x27,0x86, 0x67,0x36,0x73,0xb8,
+ 0xfc,0x08,0xb8,0xf8, 0x1f,0x67,0x0b,0x32,
+ 0x89,0x00,0xfb,0x2d, 0xbe,0x74,0xae,0x41,
+ 0x3a,0xd3,0xed,0xf1, 0x67,0xee,0xe5,0x26,
+ 0xd4,0x59,0xdc,0x3b, 0x6b,0xf7,0x33,0x67,
+ 0xed,0xef,0xb0,0x5d, 0x5e,0x43,0x34,0xa2,
+ 0x3d,0x55,0x16,0x99, 0x4b,0x90,0x49,0x40,
+ 0x82,0x35,0x0d,0x82, 0xa6,0x16,0xd2,0x41,
+ 0xc8,0x65,0xd4,0xe7, 0x1a,0xdb,0xad,0xe6,
+ 0x48,0x5e,0xeb,0x94, 0xa6,0x9f,0x97,0x1e,
+ 0xd4,0x38,0x5d,0xff, 0x6e,0x17,0x0c,0xd0,
+ 0xb3,0xd5,0xb4,0x06, 0xd7,0xcb,0x8e,0xa3,
+ 0x27,0x75,0x24,0xb5, 0x14,0xe9,0x55,0x94,
+ 0x51,0x14,0xaf,0x15, 0x02,0xd3,0x9c,0x5f,
+ 0x43,0xfe,0x97,0xf4, 0x0b,0x4e,0x4d,0x89,
+ 0x15,0x33,0x4a,0x04, 0x10,0xf3,0xeb,0x13,
+ 0x71,0x86,0xb4,0x8a, 0x2c,0x75,0x04,0x47,
+ 0xb9,0x60,0xe9,0x2a, 0x5a,0xe8,0x7e,0x8b,
+ 0x91,0xa7,0x01,0x49, 0xcf,0xfc,0x48,0x83,
+ 0xa7,0x42,0xc8,0x2f, 0x80,0x92,0x04,0x64,
+ 0x03,0xf7,0x9f,0x1d, 0xc2,0x82,0x0b,0x14,
+ 0x65,0x4d,0x04,0x09, 0x13,0x5f,0xb8,0x66,
+ 0x19,0x14,0x7a,0x09, 0xa7,0xf8,0x73,0x2d,
+ 0x4d,0x90,0x86,0x14, 0x25,0xd6,0xd6,0xf5,
+ 0x82,0x9c,0x32,0xab, 0x5c,0x37,0x12,0x28,
+ 0xd1,0xfe,0xfa,0x0d, 0x90,0x8d,0x28,0x20,
+ 0xb1,0x1e,0xbe,0x30, 0x80,0xd7,0xb1,0x63,
+ 0xd9,0x23,0x83,0x0b, 0x9d,0xf5,0x0e,0x9c,
+ 0xa2,0x88,0x5f,0x2c, 0xf2,0xa6,0x9d,0x23,
+ 0x45,0x1c,0x9b,0x7a, 0xd2,0x60,0xa6,0x0f,
+ 0x44,0xba,0x91,0x3d, 0xc6,0xf7,0xef,0x2f,
+ 0x5c,0xa8,0x5e,0x2b, 0x50,0xd3,0xd1,0x85,
+ 0xfd,0xed,0x52,0x48, 0xe2,0xd9,0xd2,0x12,
+ 0x4e,0x03,0xc9,0x3d, 0x8f,0x8d,0x1f,0x8e,
+ 0x6b,0xd8,0xe3,0x32, 0xa7,0x5b,0x39,0x57,
+ 0x91,0x08,0x52,0x09, 0xa4,0x7a,0x40,0xc6,
+ 0xcf,0xcf,0x68,0xba, 0xb1,0x97,0xf8,0x38,
+ 0x94,0x1d,0x18,0x69, 0x80,0x6a,0x11,0x15,
+ 0xc2,0xfb,0x2d,0x6c, 0xd1,0xd4,0x88,0x50,
+ 0xbb,0xca,0x8c,0x56, 0x36,0xb6,0xc4,0x41,
+ 0x97,0xe6,0xb0,0x5c, 0x7f,0x51,0x00,0x6f,
+ 0x17,0xe5,0xde,0x27, 0xf7,0xb4,0x85,0x3b,
+ 0xc5,0xa1,0x60,0x1c, 0xba,0x21,0xd6,0xed,
+ 0xd5,0x08,0x62,0x80, 0xb4,0x85,0x52,0x15,
+ 0x5c,0x94,0x19,0x3a, 0x10,0x92,0xa4,0x06,
+ 0xf1,0x86,0x02,0xce, 0x94,0xd3,0xd5,0x33,
+ 0xe7,0x59,0x47,0x72, 0x12,0xf4,0x8b,0x06,
+ 0x29,0xa3,0xb0,0x39, 0x78,0x8f,0x46,0x56,
+ 0x4a,0x42,0x4f,0x89, 0x1b,0x3f,0x09,0x12,
+ 0xc4,0x24,0x0b,0x22, 0xf0,0x27,0x04,0x4d,
+ 0x39,0xd8,0x59,0xc8, 0x7c,0x59,0x18,0x0a,
+ 0x36,0xa8,0x3c,0xba, 0x42,0xe2,0xf7,0x7a,
+ 0x23,0x90,0x73,0xff, 0xd6,0xa3,0xb2,0xcf,
+ 0x60,0xc6,0x62,0x76, 0x61,0xa3,0xcd,0x53,
+ 0x94,0x37,0x3c,0x24, 0x4b,0xc1,0xc5,0x3b,
+ 0x26,0xf8,0x67,0x1d, 0xca,0xdd,0x08,0xcb,
+ 0xdb,0x00,0x96,0x34, 0xd0,0x5d,0xef,0x4e,
+ 0x64,0x18,0xb1,0xdc, 0x46,0x13,0xc1,0x8c,
+ 0x87,0xbf,0xa3,0xfe, 0xd7,0x49,0x7e,0xb3,
+ 0x94,0xe4,0x38,0x70, 0x2a,0xde,0xaf,0x73,
+ 0x46,0xda,0xff,0xec, 0xfc,0x18,0xe2,0x02,
+ 0x64,0x5f,0x9b,0xd2, 0xdf,0x8b,0xa8,0xd0,
+ 0x4c,0xd7,0x5c,0xc7, 0x80,0x59,0x4d,0x66,
+ 0x68,0xd3,0x4a,0x51, 0xc3,0x68,0xe2,0x0a,
+ 0x17,0x31,0x4b,0xd7, 0x23,0x28,0x25,0x26,
+ 0x4a,0xef,0x02,0xd7, 0x3a,0x53,0xdb,0x09,
+ 0x19,0x85,0x68,0xab, 0xa9,0x8c,0xff,0x7e,
+ 0x30,0xfb,0x42,0x08, 0xa1,0x5a,0xd1,0xc9,
+ 0x3f,0xc9,0x00,0xfb, 0xd4,0x3e,0xb0,0x1c,
+ 0x99,0xba,0xdc,0xb4, 0x69,0xe7,0xe1,0xb0,
+ 0x67,0x53,0x46,0xa6, 0xc6,0x34,0x5c,0x94,
+ 0xfa,0xd3,0x9b,0x48, 0x92,0xa1,0xd3,0xe5,
+ 0xa7,0xea,0xe1,0x86, 0x5e,0x90,0x26,0x2d,
+ 0x4b,0x85,0xe1,0x68, 0xee,0xc2,0xf1,0x25,
+ 0xb7,0xff,0x01,0x96, 0x61,0x54,0xba,0xf3,
+ 0x09,0x62,0x7f,0xa3, 0x92,0x6b,0xe7,0x00,
+ 0xfc,0xd4,0x04,0xfd, 0x2d,0x42,0x7e,0x56,
+ 0x91,0x33,0x6e,0xf8, 0x08,0x94,0xff,0xce,
+ 0x03,0x7e,0x4d,0x0a, 0x91,0x41,0x4f,0xaa,
+ 0xdd,0xd1,0x8c,0x34, 0x99,0x46,0xb5,0xfb,
+ 0x0e,0x09,0x26,0xcc, 0x6d,0x35,0x58,0x0a,
+ 0xc6,0xc0,0x89,0xa0, 0xbd,0xb6,0x89,0xd1,
+ 0x51,0x64,0x85,0x96, 0x4d,0x6a,0x16,0x26,
+ 0x30,0xb7,0xb3,0xe4, 0x80,0x46,0xaa,0x37,
+ 0x4c,0x9b,0x2b,0xa3, 0x76,0x5e,0x8b,0x52,
+ 0x13,0x42,0xe5,0xe3, 0xa8,0xe9,0xaf,0x83,
+ 0x60,0xc0,0xb0,0xf8, 0x3d,0x82,0x0a,0x21,
+ 0x60,0xd2,0x3f,0x1c, 0xb4,0xb5,0x53,0x31,
+ 0x2e,0x16,0xfd,0xf3, 0xc3,0x46,0xfa,0xcc,
+ 0x45,0x1f,0xd1,0xac, 0x22,0xe2,0x41,0xb5,
+ 0x21,0xf3,0xdd,0x1f, 0x81,0xbf,0x03,0xaf,
+ 0xd6,0x31,0xc1,0x6a, 0x2e,0xff,0xc1,0x2d,
+ 0x44,0x53,0xd0,0xb5, 0xa2,0x7c,0x5f,0xf4,
+ 0x47,0xf7,0x4d,0x1e, 0x77,0xe2,0x29,0xcc,
+ 0xd2,0x46,0x85,0xfa, 0xdb,0x7f,0x46,0xf5,
+ 0xc9,0x60,0x4a,0x2c, 0xb7,0xf2,0xa2,0x2c,
+ 0x9d,0x76,0xcd,0x82, 0x67,0xae,0xbb,0xe0,
+ 0x92,0x56,0x48,0xcb, 0xe5,0xf5,0x3c,0x2c,
+ 0xe0,0xe8,0x6a,0x6a, 0x5a,0x0a,0x20,0x7c,
+ 0xa6,0x9d,0x8e,0x84, 0xfa,0xfe,0x61,0x13,
+ 0x54,0x79,0xe0,0x83, 0xd2,0x15,0xe0,0x33,
+ 0xe4,0xf9,0xad,0xb8, 0x1e,0x75,0x35,0xd3,
+ 0xee,0x7e,0x4a,0x63, 0x2f,0xeb,0xf1,0xe6,
+ 0x22,0xac,0x77,0x74, 0xa1,0xc0,0xa0,0x21,
+ 0x66,0x59,0x7c,0x48, 0x7f,0xaa,0x05,0xe8,
+ 0x51,0xd9,0xc7,0xed, 0xb9,0xea,0x7a,0xdd,
+ 0x23,0x53,0xea,0x8f, 0xef,0xaa,0xe6,0x9e,
+ 0x19,0x21,0x84,0x27, 0xc5,0x78,0x2e,0x8c,
+ 0x52,0x40,0x15,0x1c, 0x2b,0x91,0xb3,0x4c,
+ 0xe8,0xfa,0xd3,0x64, 0x0f,0xf9,0xf4,0xb8,
+ 0x59,0x4d,0x6b,0x2d, 0x44,0x6c,0x8d,0xb2,
+ 0xdb,0x73,0x29,0x66, 0xb1,0xc2,0x28,0xfc,
+ 0x85,0xba,0x60,0x5e, 0x27,0x8f,0xfb,0xb3,
+ 0xc9,0x20,0x43,0xb1, 0x3e,0x18,0x97,0x42,
+ 0x63,0x2d,0x0c,0x97, 0xf2,0xcc,0xcd,0x90,
+ 0x46,0x5f,0x1a,0x85, 0xca,0x44,0x2a,0x1a,
+ 0x52,0xf7,0xbb,0x4e, 0xd1,0xab,0xd5,0xa3,
+ 0x58,0x6b,0xb6,0x5a, 0x88,0x1c,0x9d,0x3b,
+ 0xe2,0x46,0xe4,0x3b, 0x33,0x64,0x6c,0xfd,
+ 0xeb,0x36,0x8e,0x32, 0x1f,0x71,0xbd,0x95,
+ 0xb6,0xfd,0x1a,0xcb, 0xfb,0x4a,0x88,0x27,
+ 0xd6,0x28,0x7b,0x5e, 0xa3,0x8a,0x0c,0x36,
+ 0xa8,0x5d,0x2f,0x28, 0xa9,0xad,0xb2,0x88,
+ 0x9e,0x62,0x9d,0x4a, 0x07,0x74,0x00,0x04,
+ 0x0c,0xc1,0x6a,0x09, 0xe1,0x0b,0xfa,0xf3,
+ 0xd1,0x41,0xdd,0x94, 0x52,0x06,0xb8,0x9e,
+ 0xba,0x81,0xe0,0x52, 0xdf,0x52,0x5d,0x74,
+ 0x40,0x59,0x36,0x05, 0xf2,0x30,0xc4,0x84,
+ 0x85,0xdc,0xb8,0xba, 0xd9,0xf4,0x5f,0x11,
+ 0x83,0xce,0x25,0x57, 0x97,0xf5,0x0f,0xb5,
+ 0x0b,0xd6,0x6d,0x1c, 0xfb,0xf2,0x30,0xda,
+ 0xc2,0x05,0xa8,0xe1, 0xc2,0x57,0x0a,0x05,
+ 0x2d,0x4c,0x8b,0xb7, 0x5a,0xc0,0x8a,0xba,
+ 0xa9,0x85,0x7c,0xf0, 0xb8,0xce,0x72,0x79,
+ 0xf5,0x27,0x99,0xd7, 0xed,0xcf,0x85,0xfa,
+ 0x92,0x15,0xf1,0x47, 0x02,0x24,0x39,0x07,
+ 0x89,0xb6,0xdd,0x4a, 0xb8,0xbc,0xd5,0x9d,
+ 0x4c,0x03,0x8b,0x1d, 0x45,0x58,0x1c,0x86,
+ 0x46,0x71,0x0a,0x0d, 0x7c,0x5b,0xf9,0xdc,
+ 0x60,0xb5,0xb0,0x00, 0x70,0x47,0x83,0xa6,
+ 0x8e,0x79,0xba,0x1d, 0x21,0x20,0xc0,0x24,
+ 0x56,0x35,0x6a,0x49, 0xb6,0xa3,0x58,0x87,
+ 0x16,0xae,0xd9,0x77, 0x62,0xa0,0x61,0xce,
+ 0x3d,0xe6,0x77,0x9e, 0x83,0xec,0xc2,0x04,
+ 0x8c,0xba,0x62,0xac, 0x32,0xda,0xf0,0x89,
+ 0x7b,0x2b,0xb0,0xa3, 0x3a,0x5f,0x8b,0x0d,
+ 0xbd,0xe9,0x14,0xcd, 0x5b,0x7a,0xde,0xd5,
+ 0x0d,0xc3,0x4b,0x38, 0x92,0x31,0x97,0xd8,
+ 0xae,0x89,0x17,0x2c, 0xc9,0x54,0x96,0x66,
+ 0xd0,0x9f,0x60,0x7a, 0x7d,0x63,0x67,0xfc,
+ 0xb6,0x02,0xce,0xcc, 0x97,0x36,0x9c,0x3c,
+ 0x1e,0x69,0x3e,0xdb, 0x54,0x84,0x0a,0x77,
+ 0x6d,0x0b,0x6e,0x10, 0x9f,0xfb,0x2a,0xb1,
+ 0x49,0x31,0x71,0xf2, 0xd1,0x1e,0xea,0x87,
+ 0xb9,0xd6,0x4a,0x4c, 0x57,0x17,0xbc,0x8b,
+ 0x38,0x66,0x2d,0x5f, 0x25,0xca,0x6d,0x10,
+ 0xc6,0x2e,0xd7,0x2c, 0x89,0xf1,0x4c,0x1d,
+ 0xc9,0x9c,0x02,0x23, 0xc6,0x1f,0xd6,0xc3,
+ 0xb8,0xc7,0x85,0x29, 0x75,0x40,0x1e,0x04,
+ 0x6e,0xc7,0xb4,0x60, 0xfc,0xea,0x30,0x8b,
+ 0x4d,0x9d,0xb7,0x5d, 0x91,0xfb,0x8e,0xb8,
+ 0xc2,0x54,0xdf,0xdb, 0x79,0x58,0x32,0xda,
+ 0xd0,0xa1,0xd6,0xd6, 0xc4,0xc8,0xa4,0x16,
+ 0x95,0xbb,0xe5,0x58, 0xd2,0xb6,0x83,0x76,
+ 0x1d,0xd7,0x45,0xbc, 0xb8,0x14,0x79,0x3b,
+ 0x4e,0x1a,0x0b,0x5c, 0xfc,0xa5,0xa0,0xc3,
+ 0xf1,0x64,0x74,0xb0, 0x0d,0x82,0x90,0x62,
+ 0x87,0x02,0x0f,0x71, 0xc7,0xab,0x7d,0x2b,
+ 0x70,0xf1,0x9b,0x9e, 0xe7,0x6b,0x99,0x18,
+ 0x6c,0x54,0x17,0x0b, 0xf5,0x44,0x58,0x54,
+ 0x44,0x9b,0x54,0x30, 0x5e,0xaf,0xa6,0xfa,
+ 0x42,0x37,0xe8,0x67, 0xbf,0xf7,0x6c,0x1e,
+ 0x73,0xd8,0xc7,0x5c, 0xfa,0x51,0xd5,0x1f,
+ 0xab,0xfc,0x91,0x03, 0xc1,0xc1,0x22,0x58,
+ 0xc7,0xe8,0x60,0xae, 0xb6,0x58,0x44,0xad,
+ 0x1e,0x07,0x5d,0x3c, 0x90,0x33,0x43,0xe0,
+ 0x67,0x44,0x9f,0x8c, 0xf3,0xef,0xce,0x3a,
+ 0x22,0x2b,0x1b,0x97, 0x83,0x6f,0x9f,0xd3,
+ 0x46,0xc3,0xa1,0xdf, 0xde,0x60,0xf0,0x32,
+ 0x2e,0xcf,0xed,0x72, 0x27,0x0d,0xa7,0xd0,
+ 0x91,0x6a,0xf0,0x6d, 0x41,0xfa,0x77,0x2e,
+ 0xd8,0x43,0xce,0xe2, 0xf5,0x7a,0x9e,0x04,
+ 0x30,0x4c,0xe7,0x08, 0xf3,0x2e,0x13,0x05,
+ 0x5e,0xfa,0x16,0x2c, 0x6c,0x53,0x02,0xb5,
+ 0x2f,0x2c,0x7d,0x86, 0x61,0x0e,0x5f,0x96,
+ 0xe1,0x1c,0x37,0x87, 0xf0,0x84,0xe4,0x1d,
+ 0x53,0x4d,0xb1,0x13, 0xe2,0xcb,0x71,0x6e,
+ 0x86,0x7b,0xad,0x97, 0x3e,0x16,0xb3,0xb4,
+ 0x0f,0x32,0x01,0x69, 0x31,0x1f,0x49,0x99,
+ 0x7a,0x46,0xd9,0x9b, 0x5f,0x17,0x3d,0xcb,
+ 0xe4,0xfd,0xbc,0xbb, 0xe3,0xec,0x8c,0x54,
+ 0xc4,0x14,0x44,0x89, 0xa3,0x65,0x25,0xc0,
+ 0x06,0x9b,0x7d,0x9b, 0x7f,0x15,0x8f,0x84,
+ 0xe1,0x08,0x0d,0x2c, 0x0a,0x91,0x9a,0x85,
+ 0x4e,0xa1,0x50,0xee, 0x72,0x70,0xf4,0xd2,
+ 0x1c,0x67,0x20,0x1f, 0xe6,0xb2,0x9d,0x95,
+ 0x85,0x7e,0xf2,0x9d, 0xf0,0x73,0x10,0xe7,
+ 0xfc,0x62,0x9d,0xea, 0x8d,0x63,0xdc,0x70,
+ 0xe0,0x2b,0x30,0x01, 0x7c,0xcd,0x24,0x22,
+ 0x03,0xf9,0x8b,0xe4, 0x77,0xef,0x2c,0xdc,
+ 0xa5,0xfb,0x29,0x66, 0x50,0x1c,0xd7,0x4e,
+ 0x8f,0x0f,0xbf,0x61, 0x0c,0xea,0xc0,0xe6,
+ 0xc6,0xc3,0xa1,0xae, 0xf3,0xea,0x4c,0xfb,
+ 0x21,0x96,0xd1,0x38, 0x64,0xe0,0xdd,0xa8,
+ 0xa4,0xd0,0x33,0x82, 0xf0,0xdd,0x91,0x6e,
+ 0x88,0x27,0xe1,0x0d, 0x8b,0xfb,0xc6,0x36,
+ 0xc5,0x9a,0x9d,0xbc, 0x32,0x8f,0x8a,0x3a,
+ 0xfb,0xd0,0x88,0x1e, 0xe5,0xb8,0x68,0x35,
+ 0x4b,0x22,0x72,0x55, 0x9e,0x77,0x39,0x1d,
+ 0x64,0x81,0x6e,0xfd, 0xe3,0x29,0xb8,0xa5,
+ 0x3e,0xc8,0x4c,0x6f, 0x41,0xc2,0xbd,0xb6,
+ 0x15,0xd1,0xd5,0xe9, 0x77,0x97,0xb6,0x54,
+ 0x9e,0x60,0xdd,0xf3, 0x48,0xdb,0x65,0x04,
+ 0x54,0xa2,0x93,0x12, 0xf0,0x66,0x6c,0xae,
+ 0xa2,0x2c,0xb9,0xeb, 0xf0,0x7c,0x9c,0xae,
+ 0x8e,0x49,0xf5,0x0f, 0xfc,0x4b,0x2a,0xdb,
+ 0xaf,0xff,0x96,0x0d, 0xa6,0x05,0xe9,0x37,
+ 0x81,0x43,0x41,0xb2, 0x69,0x88,0xd5,0x2c,
+ 0xa2,0xa9,0x9b,0xf2, 0xf1,0x77,0x68,0x05,
+ 0x84,0x0f,0x6a,0xee, 0xd0,0xb5,0x65,0x4b,
+ 0x35,0x18,0xeb,0x34, 0xba,0x09,0x4f,0xc3,
+ 0x5a,0xac,0x44,0x5b, 0x03,0xf5,0xf5,0x1d,
+ 0x10,0x04,0xfd,0xb5, 0xc4,0x26,0x84,0x13,
+ 0x8a,0xde,0x8d,0xbb, 0x51,0xd0,0x6f,0x58,
+ 0xc1,0xe5,0x9e,0x12, 0xe6,0xba,0x13,0x73,
+ 0x27,0x3e,0x3f,0xf0, 0x4f,0x0f,0x64,0x6c,
+ 0x0e,0x36,0xe9,0xcc, 0x38,0x93,0x9b,0xda,
+ 0xf9,0xfd,0xc2,0xe9, 0x44,0x7a,0x93,0xa6,
+ 0x73,0xf6,0x2a,0xc0, 0x21,0x42,0xbc,0x58,
+ 0x9e,0xe3,0x0c,0x6f, 0xa1,0xd0,0xdd,0x67,
+ 0x14,0x3d,0x49,0xf1, 0x5b,0xc3,0xc3,0xa4,
+ 0x52,0xa3,0xe7,0x0f, 0xb4,0x26,0xf4,0x62,
+ 0x73,0xf5,0x9f,0x75, 0x5b,0x6e,0x38,0xc8,
+ 0x4a,0xcc,0xf6,0xfa, 0xcf,0xfb,0x28,0x02,
+ 0x8a,0xdb,0x6b,0x63, 0x52,0x17,0x94,0x87,
+ 0x71,0xa2,0xf5,0x5a, 0x1d,0x94,0xe3,0xcd,
+ 0x28,0x70,0x96,0xd5, 0xb1,0xaf,0xec,0xd6,
+ 0xea,0xf4,0xfc,0xe9, 0x10,0x66,0xd9,0x8a,
+ 0x1e,0x03,0x03,0xf1, 0x54,0x2d,0xc5,0x8c,
+ 0x85,0x71,0xed,0xa7, 0xa4,0x1e,0x5a,0xff,
+ 0xab,0xb8,0x07,0xb3, 0x0b,0x84,0x00,0x0a,
+ 0x7f,0xa5,0x38,0x20, 0x66,0x33,0x84,0x2f,
+ 0xec,0x16,0x94,0x78, 0xa8,0x42,0x98,0x55,
+ 0xa3,0xe5,0xd3,0x62, 0x2a,0xfc,0xed,0xec,
+ 0x7a,0x96,0x41,0x35, 0xc0,0xd2,0xe6,0x53,
+ 0xf8,0x0f,0x59,0x94, 0x0a,0xa0,0x50,0xef,
+ 0x0d,0x9f,0x04,0x1c, 0x5f,0x48,0xfe,0x33,
+ 0x20,0xca,0x8d,0x09, 0xdd,0x0b,0xf8,0x59,
+ 0xd3,0x63,0x8a,0xa4, 0xf5,0x73,0x6b,0x3e,
+ 0x7e,0x0f,0xff,0xdb, 0x96,0x62,0x4d,0x3a,
+ 0xdb,0x8d,0x8c,0x9b, 0x8c,0xb3,0xa1,0xff,
+ 0x16,0xb9,0x2c,0x8c, 0xf6,0xbb,0x0d,0x9e,
+ 0x6f,0xff,0x24,0x6f, 0x59,0xee,0x02,0xe6,
+ 0x57,0x38,0xbd,0x5f, 0xbd,0xd4,0xe5,0x74,
+ 0x14,0xea,0x85,0xbb, 0x0c,0xfe,0xad,0xad,
+ 0x98,0x82,0x8a,0x81, 0x0b,0x37,0xdc,0x7d,
+ 0xda,0x13,0x74,0x8a, 0xa5,0xaf,0x74,0x82,
+ 0x95,0x35,0x1f,0x0b, 0x03,0x88,0x17,0xf3,
+ 0x67,0x11,0x40,0xd1, 0x9d,0x48,0xec,0x9b,
+ 0xc8,0xb2,0xcc,0xb4, 0x93,0xd2,0x0b,0x0a,
+ 0xd6,0x6f,0x34,0x32, 0xd1,0x9a,0x0d,0x89,
+ 0x93,0x1f,0x96,0x5a, 0x7a,0x57,0x06,0x02,
+ 0x1d,0xbf,0x57,0x3c, 0x9e,0xca,0x5d,0x68,
+ 0xe8,0x4e,0xea,0x4f, 0x0b,0x11,0xf0,0x35,
+ 0x73,0x5a,0x77,0x24, 0x29,0xc3,0x60,0x51,
+ 0xf0,0x15,0x93,0x45, 0x6b,0xb1,0x70,0xe0,
+ 0xda,0xf7,0xf4,0x0a, 0x70,0xd1,0x73,0x3f,
+ 0x9c,0x9d,0x07,0x19, 0xad,0xb2,0x28,0xae,
+ 0xf2,0xe2,0xb6,0xf4, 0xbc,0x71,0x63,0x00,
+ 0xde,0xe3,0xdc,0xb1, 0xa3,0xd5,0x4c,0x34,
+ 0xf8,0x6b,0x68,0x4c, 0x73,0x84,0xab,0xd4,
+ 0x89,0xae,0x07,0x1a, 0x0d,0x3d,0x8e,0xaa,
+ 0x6c,0xa2,0x54,0xb3, 0xd9,0x46,0x81,0x87,
+ 0xe2,0xdc,0x49,0xb1, 0x14,0x5c,0xcc,0x72,
+ 0x56,0xf0,0x0f,0xa9, 0x3d,0x31,0x2f,0x08,
+ 0xbc,0x15,0xb7,0xd3, 0x0d,0x4f,0xd1,0xc9,
+ 0x4e,0xde,0x1c,0x03, 0xd1,0xae,0xaf,0x14,
+ 0x62,0xbc,0x1f,0x33, 0x5c,0x00,0xeb,0xf4,
+ 0x8e,0xf6,0x3e,0x13, 0x6a,0x64,0x42,0x07,
+ 0x60,0x71,0x35,0xf1, 0xd0,0xff,0x8d,0x1f,
+ 0x88,0xc0,0x1c,0x3c, 0x6c,0x1c,0x54,0x71,
+ 0x6b,0x65,0x4a,0xe2, 0xe3,0x5f,0x77,0x56,
+ 0x1c,0x8d,0x2a,0x8d, 0xef,0x92,0x4a,0xa9,
+ 0xf6,0xcf,0xa5,0x67, 0x89,0x8e,0x5a,0xd9,
+ 0x60,0xaa,0x94,0x14, 0x55,0x66,0x8a,0xb0,
+ 0x18,0x4f,0x9e,0x8e, 0xf4,0xdb,0xc1,0x88,
+ 0x9b,0xf0,0x84,0x33, 0x2f,0xcd,0x2c,0xeb,
+ 0x65,0xe6,0x5d,0xde, 0x30,0x97,0xad,0xe6,
+ 0xbc,0xcb,0x83,0x93, 0xf3,0xfd,0x65,0xdc,
+ 0x07,0x27,0xf9,0x0f, 0x4a,0x56,0x5c,0xf7,
+ 0xff,0xa3,0xd1,0xad, 0xd4,0xd1,0x38,0x13,
+ 0x71,0xc9,0x42,0x0f, 0x0d,0x35,0x12,0x32,
+ 0xd2,0x2d,0x2b,0x96, 0xe4,0x01,0xdc,0x55,
+ 0xd8,0x71,0x2c,0x0c, 0xc4,0x55,0x3f,0x16,
+ 0xe8,0xaa,0xe7,0xe8, 0x45,0xfa,0x23,0x23,
+ 0x5e,0x21,0x02,0xab, 0xc8,0x6b,0x88,0x5e,
+ 0xdc,0x90,0x13,0xb5, 0xe7,0x47,0xfa,0x12,
+ 0xd5,0xa7,0x0a,0x06, 0xd2,0x7c,0x62,0x80,
+ 0xb7,0x8e,0x4f,0x77, 0x88,0xb7,0xa2,0x12,
+ 0xdb,0x19,0x1f,0xd8, 0x00,0x82,0xf5,0xf2,
+ 0x59,0x34,0xec,0x91, 0xa8,0xc1,0xd7,0x6e,
+ 0x76,0x10,0xf3,0x15, 0xa6,0x86,0xfa,0xfd,
+ 0x45,0x2f,0x86,0x18, 0x16,0x83,0x16,0x8c,
+ 0x6e,0x99,0x7e,0x43, 0x3f,0x0a,0xba,0x32,
+ 0x94,0x5b,0x15,0x32, 0x66,0xc2,0x3a,0xdc,
+ 0xf3,0xd3,0x1d,0xd1, 0x5d,0x6f,0x5f,0x9a,
+ 0x7f,0xa2,0x90,0xf1, 0xa1,0xd0,0x17,0x33,
+ 0xdf,0x9a,0x2e,0xa2, 0xdc,0x89,0xe6,0xb0,
+ 0xda,0x23,0x2b,0xf6, 0xe9,0x1f,0x82,0x3c,
+ 0x07,0x90,0xab,0x3a, 0xb9,0x87,0xb0,0x02,
+ 0xcc,0xb9,0xe7,0x2e, 0xe7,0xc6,0xee,0xfa,
+ 0xe2,0x16,0xc8,0xc3, 0xd0,0x40,0x15,0xc5,
+ 0xa7,0xc8,0x20,0x42, 0xb7,0x09,0xf8,0x66,
+ 0xeb,0x0e,0x4b,0xd7, 0x91,0x74,0xa3,0x8b,
+ 0x17,0x2a,0x0c,0xee, 0x7f,0xc1,0xea,0x63,
+ 0xc6,0x3c,0x1e,0xea, 0x8b,0xa2,0xd1,0x2e,
+ 0xf3,0xa6,0x0f,0x36, 0xff,0xdd,0x81,0x06,
+ 0xe3,0x63,0xfc,0x0c, 0x38,0xb0,0x23,0xfb,
+ 0x83,0x66,0x81,0x73, 0x5c,0x0b,0x9c,0xd4,
+ 0x23,0xdc,0x7f,0x5c, 0x00,0x8c,0xa6,0xa7,
+ 0x52,0xd4,0xc1,0x00, 0xea,0x99,0x6b,0x59,
+ 0x19,0x8e,0x34,0x32, 0x24,0xea,0x0c,0x61,
+ 0x95,0x9d,0xdb,0xf0, 0x63,0xcc,0xa9,0xfd,
+ 0x1b,0xeb,0xd7,0xbc, 0x0c,0xa4,0x74,0x24,
+ 0xfd,0xfa,0x32,0x58, 0xe3,0x74,0x1c,0x8f,
+ 0x76,0xa6,0x53,0x0d, 0xea,0xde,0x50,0x92,
+ 0xbd,0x3f,0x3d,0x56, 0x8f,0x48,0x4e,0xb7,
+ 0x8c,0x5e,0x83,0x2c, 0xf7,0xec,0x04,0x2c,
+ 0x35,0xdf,0xa9,0x72, 0xc0,0x77,0xf5,0x44,
+ 0xe5,0xa7,0x56,0x3e, 0xa4,0x8d,0xb8,0x6e,
+ 0x31,0x86,0x15,0x1d, 0xc4,0x66,0x86,0x75,
+ 0xf8,0x1a,0xea,0x2f, 0x3a,0xb7,0xbf,0x97,
+ 0xe9,0x11,0x53,0x64, 0xa8,0x71,0xc6,0x78,
+ 0x8a,0x70,0xb5,0x18, 0xd7,0x9c,0xe3,0x44,
+ 0x1a,0x7c,0x6b,0x1b, 0x41,0xe1,0x1c,0x0d,
+ 0x98,0x43,0x67,0x28, 0xb8,0x14,0xb4,0x48,
+ 0x01,0x85,0x79,0x20, 0x94,0x36,0x25,0x3a,
+ 0x5c,0x48,0xd2,0x2e, 0x91,0x91,0xfd,0x85,
+ 0x38,0xc1,0xc5,0xa5, 0x4d,0x52,0x1f,0xb4,
+ 0xe7,0x44,0x7a,0xff, 0xb1,0x65,0xdf,0x53,
+ 0x86,0x2a,0xff,0x25, 0x2b,0xeb,0x3e,0xdc,
+ 0x3d,0xec,0x72,0xae, 0xa9,0xd1,0xdf,0xe9,
+ 0x4a,0x3e,0xe8,0xf1, 0x74,0xe0,0xee,0xd6,
+ 0x0b,0xba,0x9b,0x14, 0x9b,0x0c,0x4a,0xf9,
+ 0x55,0xee,0x7e,0x82, 0xa4,0xb5,0xa5,0xb7,
+ 0x2f,0x75,0x48,0x51, 0x60,0xcc,0x41,0x8e,
+ 0x65,0xe3,0xb7,0x29, 0xe0,0x32,0xe7,0x1b,
+ 0x2f,0xa0,0x80,0xce, 0x73,0x28,0x6c,0xf4,
+ 0xd0,0xc7,0x05,0x69, 0xbd,0x3e,0x2e,0x77,
+ 0x1a,0x7f,0x9a,0x98, 0x60,0x31,0xdb,0x47,
+ 0xc2,0xa2,0x12,0xcb, 0x8c,0x35,0xff,0x58,
+ 0xe3,0x07,0x22,0xe4, 0x2f,0x26,0x87,0x30,
+ 0x16,0xea,0x64,0x4f, 0x44,0x64,0x3d,0xe4,
+ 0x7b,0x41,0x06,0xca, 0xee,0x02,0xcf,0xf3,
+ 0x26,0x4c,0xfe,0x9c, 0xf6,0x64,0x96,0xd4,
+ 0xd9,0x7e,0x04,0x47, 0x1d,0xdb,0xc7,0x8c,
+ 0xae,0xd7,0x9d,0xea, 0xe3,0x3a,0xee,0x24,
+ 0xa9,0x2d,0x65,0xba, 0xd5,0x9f,0x38,0x81,
+ 0x61,0x42,0x15,0xdf, 0xcc,0x29,0xd9,0xf7,
+ 0xd4,0x30,0xb9,0xc9, 0x86,0x76,0xdc,0xee,
+ 0xa5,0x27,0xa6,0x27, 0xa3,0xbb,0x8f,0x3b,
+ 0xaa,0xca,0x01,0x52, 0x37,0x12,0xc0,0x55,
+ 0x39,0x4a,0xb2,0xce, 0x85,0x73,0xf2,0x10,
+ 0x9c,0x7f,0xa6,0x34, 0x7f,0x0f,0x69,0x63,
+ 0x03,0xc4,0xde,0xe2, 0x7b,0x10,0xbf,0x91,
+ 0x3e,0x7e,0xad,0xb7, 0xa8,0x85,0xc7,0x99,
+ 0xae,0x8e,0x7c,0x2e, 0x02,0x25,0x5b,0xd5,
+ 0xf4,0x46,0xd1,0x49, 0x48,0xa0,0x12,0x6a,
+ 0x6a,0x01,0x23,0xb9, 0x7e,0x67,0x8b,0x48,
+ 0xac,0xf7,0x88,0x88, 0xeb,0xd9,0x39,0x3a,
+ 0xc8,0xa0,0x06,0xd9, 0x0b,0x80,0xc4,0x84,
+ },
+ .c = {
+ 0x10,0x46,0xb6,0xc8, 0xaa,0x83,0x67,0x7b,
+ 0xc5,0x9a,0x9a,0x0d, 0xe2,0xec,0x6f,0x9a,
+ 0x3e,0x74,0xa7,0xfa, 0x43,0x93,0x9d,0xc5,
+ 0x23,0x27,0xad,0x99, 0x74,0xb4,0xc0,0xe4,
+ 0xd7,0x70,0x5c,0x95, 0x58,0xe3,0x8f,0x72,
+ 0xe3,0x03,0x3d,0xc2, 0xd9,0x69,0x37,0x3e,
+ 0x8e,0x2a,0x0c,0x2b, 0x75,0x59,0x05,0x18,
+ 0x4a,0x50,0x67,0xd4, 0xf5,0x4b,0xb0,0x59,
+ 0x08,0xaf,0xbc,0x6f, 0xb1,0x95,0xa1,0x32,
+ 0xe7,0x77,0x1a,0xfd, 0xaf,0xe8,0x4d,0x32,
+ 0x87,0x9c,0x87,0x90, 0x5e,0xe8,0x08,0xc3,
+ 0xb4,0x0c,0x80,0x9a, 0x9e,0x23,0xeb,0x5a,
+ 0x5c,0x18,0x4a,0x7c, 0xd0,0x4a,0x91,0x57,
+ 0x7e,0x6c,0x53,0xde, 0x98,0xc0,0x09,0x80,
+ 0x8d,0x41,0x0b,0xbc, 0x56,0x5e,0x69,0x61,
+ 0xd3,0x56,0x48,0x43, 0x19,0x49,0x49,0xaf,
+ 0xcf,0xad,0x98,0x3e, 0x88,0x4b,0x44,0x69,
+ 0x73,0xd2,0xcb,0xdf, 0x30,0xdb,0x76,0x1d,
+ 0xfb,0x4b,0xc5,0x66, 0x22,0x34,0x6f,0x07,
+ 0x0b,0xcd,0x1c,0xed, 0x88,0xd9,0x0d,0x30,
+ 0xe9,0x96,0xcb,0xf5, 0xde,0x57,0x5f,0x0b,
+ 0x12,0x11,0xcf,0x52, 0xf5,0x0d,0xf8,0x29,
+ 0x39,0x87,0xb2,0xa5, 0x7f,0x7a,0x2b,0x9d,
+ 0x66,0x11,0x32,0xf4, 0xd4,0x37,0x16,0x75,
+ 0xe3,0x0b,0x55,0x98, 0x44,0x6f,0xc7,0x5c,
+ 0xd4,0x89,0xf8,0xb3, 0xee,0xe4,0x5e,0x45,
+ 0x34,0xc2,0xc0,0xef, 0xdd,0x4d,0xbb,0xb4,
+ 0x0a,0x7b,0xda,0xe3, 0x6e,0x41,0xe1,0xb4,
+ 0x73,0xf8,0x9b,0x65, 0x1c,0x5f,0xdf,0x9c,
+ 0xd7,0x71,0x91,0x72, 0x6f,0x9e,0x8f,0x96,
+ 0x5d,0x45,0x11,0xd1, 0xb9,0x99,0x63,0x50,
+ 0xda,0x36,0xe9,0x75, 0x21,0x9a,0xce,0xc5,
+ 0x1a,0x8a,0x12,0x81, 0x8b,0xeb,0x51,0x7c,
+ 0x00,0x5f,0x58,0x5a, 0x3e,0x65,0x10,0x9e,
+ 0xe3,0x9e,0xf0,0x6b, 0xfe,0x49,0x50,0x2a,
+ 0x2a,0x3b,0xa5,0x42, 0x1b,0x15,0x2b,0x5b,
+ 0x88,0xb8,0xfb,0x6f, 0x0c,0x5d,0x16,0x76,
+ 0x48,0x77,0x4d,0x22, 0xb9,0xf0,0x0a,0x3f,
+ 0xa6,0xdd,0xc8,0x32, 0xcc,0x98,0x76,0x41,
+ 0x84,0x36,0x24,0x6d, 0x88,0x62,0x65,0x40,
+ 0xa4,0x55,0xdc,0x39, 0x74,0xed,0x0f,0x50,
+ 0x08,0xcf,0x69,0x5f, 0x1d,0x31,0xd6,0xb4,
+ 0x39,0x94,0x5b,0x18, 0x88,0x0f,0xcb,0x56,
+ 0xfb,0xf7,0x19,0xe0, 0x80,0xe0,0x4f,0x67,
+ 0x9c,0xab,0x35,0x78, 0xc9,0xca,0x95,0xfa,
+ 0x31,0xf0,0x5f,0xa6, 0xf9,0x71,0xbd,0x7f,
+ 0xb1,0xe2,0x42,0x67, 0x9d,0xfb,0x7f,0xde,
+ 0x41,0xa6,0x7f,0xc7, 0x7f,0x75,0xd8,0x8d,
+ 0x43,0xce,0xe6,0xeb, 0x74,0xee,0x4e,0x35,
+ 0xbc,0x7b,0x7c,0xfc, 0x8b,0x4f,0x1f,0xa2,
+ 0x5e,0x34,0x3b,0x5f, 0xd0,0x05,0x9d,0x4f,
+ 0xfe,0x47,0x59,0xa3, 0xf6,0xb7,0x27,0xb0,
+ 0xa1,0xec,0x1d,0x09, 0x86,0x70,0x48,0x00,
+ 0x03,0x0a,0x15,0x98, 0x2e,0x6d,0x48,0x2a,
+ 0x81,0xa2,0xde,0x11, 0xe4,0xde,0x8b,0xb0,
+ 0x06,0x28,0x03,0x82, 0xe4,0x6e,0x40,0xfb,
+ 0x3c,0x35,0x2d,0x1b, 0x62,0x56,0x87,0xd4,
+ 0xd6,0x06,0x36,0xce, 0x70,0x26,0x2f,0x21,
+ 0xf5,0x47,0x3f,0xf8, 0x57,0x17,0xa9,0x15,
+ 0x30,0xfd,0x1f,0xa6, 0x7a,0x24,0x1c,0xf8,
+ 0x33,0xf3,0xef,0xe1, 0x6c,0xb5,0x0b,0x04,
+ 0x21,0x5d,0xb5,0xff, 0x4f,0xdb,0xd1,0x3d,
+ 0x8f,0x01,0x56,0x7f, 0x0b,0xa4,0xf1,0xf9,
+ 0xdd,0xa3,0x38,0xcb, 0xa9,0xd3,0xdd,0xe3,
+ 0x29,0x5b,0x2b,0x22, 0xd7,0xe8,0x4f,0x02,
+ 0xb1,0x73,0x83,0x80, 0xda,0xd0,0x8e,0x11,
+ 0x9f,0x4d,0xd4,0x0a, 0x86,0x45,0x11,0xa1,
+ 0x9e,0x2e,0xa9,0x59, 0x6d,0x95,0x49,0xc5,
+ 0xc9,0xcd,0x7c,0x71, 0x81,0xac,0x6b,0xb8,
+ 0x1b,0x94,0xe8,0xe3, 0xb2,0xb7,0x8a,0x9b,
+ 0xda,0x5b,0xb7,0xc6, 0x00,0xcb,0x40,0x47,
+ 0x0c,0x38,0x75,0xb8, 0xba,0x6f,0x2b,0x9d,
+ 0x01,0xf3,0xf2,0xc8, 0xf7,0xde,0xcf,0xfb,
+ 0x82,0xa8,0x8f,0x10, 0x75,0x0e,0x27,0xc5,
+ 0x4b,0x9f,0xfe,0x1d, 0x60,0x84,0x69,0x96,
+ 0xac,0xb1,0xd3,0xdd, 0x07,0x4c,0x50,0x94,
+ 0xb1,0x17,0x53,0x23, 0x98,0xbf,0x22,0xf9,
+ 0x2c,0xb0,0x3f,0x62, 0x16,0xa7,0x8f,0xea,
+ 0x43,0x25,0xfb,0x21, 0x18,0xec,0x1a,0xf6,
+ 0x5e,0x64,0xbd,0x3d, 0xcf,0x27,0xf5,0x02,
+ 0xf2,0xaf,0x1b,0x2d, 0x2c,0xcb,0xaa,0x6d,
+ 0x7d,0xa0,0xae,0x31, 0x05,0x51,0x80,0x7f,
+ 0x99,0xcf,0xbd,0x0f, 0x12,0x5a,0xda,0x4a,
+ 0x56,0x22,0xd4,0x22, 0x95,0x2c,0x46,0x5a,
+ 0xb3,0x5a,0x5e,0xd4, 0x27,0x7f,0x06,0xbd,
+ 0x3c,0xf6,0xf2,0x0f, 0x9d,0xbb,0x0c,0x14,
+ 0x8c,0xb1,0x72,0xf2, 0xb0,0xaf,0xda,0xf7,
+ 0x05,0x33,0x78,0x9c, 0x79,0xe9,0xe0,0xc5,
+ 0x8c,0x4b,0x23,0x65, 0xd1,0x70,0x81,0x3d,
+ 0x74,0xfa,0xb6,0xff, 0xf2,0x65,0x21,0x3f,
+ 0xe4,0xc2,0x9e,0x9d, 0x49,0x0e,0xad,0xaf,
+ 0xc2,0x21,0x18,0xa8, 0x19,0xa8,0x69,0x32,
+ 0xcb,0x8e,0xc2,0x9d, 0xf5,0xbd,0x50,0x60,
+ 0x72,0xa2,0xa6,0xad, 0xe6,0x6b,0xd2,0x01,
+ 0x52,0xf9,0xac,0x18, 0xfa,0xe8,0x8d,0x4a,
+ 0x98,0x25,0xd3,0xa8, 0x0e,0x97,0x2d,0xa3,
+ 0xf6,0xf1,0x34,0x7c, 0xf0,0x15,0x06,0x05,
+ 0x31,0xdf,0xc7,0x86, 0x54,0xfb,0x62,0xe2,
+ 0xd5,0x3b,0x72,0xd2, 0x70,0x7c,0x3c,0x62,
+ 0x2f,0xbd,0x47,0x0d, 0x20,0x97,0xf5,0x1f,
+ 0xa1,0xe8,0x4c,0x3e, 0x13,0xec,0xb3,0xcc,
+ 0xc9,0x15,0x01,0x23, 0xe5,0x1f,0x3b,0x2e,
+ 0xc5,0xdd,0x71,0xe3, 0xfa,0x6a,0x44,0x07,
+ 0x25,0x64,0xa5,0xa5, 0x16,0x64,0x14,0xb8,
+ 0x86,0xb1,0xae,0x6f, 0xc5,0xdb,0x6b,0xfa,
+ 0x0f,0x8f,0xc5,0x89, 0x57,0x52,0xeb,0xb3,
+ 0xca,0x4e,0x23,0xac, 0xbd,0xad,0xf5,0x77,
+ 0x58,0x72,0x18,0x2c, 0xb8,0x37,0x0b,0xfd,
+ 0xfd,0x04,0x49,0x4a, 0x7b,0x11,0x82,0x1b,
+ 0xc4,0x5f,0x54,0x46, 0x97,0xe9,0xac,0x64,
+ 0xa7,0x13,0x04,0x56, 0x5a,0x3b,0x17,0x2c,
+ 0x08,0xff,0xa4,0xe2, 0xe4,0x43,0x05,0xfa,
+ 0x94,0x3a,0xbc,0x24, 0xec,0xa8,0x89,0x02,
+ 0xd0,0xbc,0xcf,0x4a, 0xef,0x0f,0x90,0x50,
+ 0xfb,0x6a,0x25,0x4f, 0xdb,0x67,0x5b,0xd8,
+ 0xa1,0x1e,0x95,0x4d, 0xe5,0xd6,0xf3,0x22,
+ 0x2e,0x6f,0x01,0x50, 0xd8,0x2f,0x91,0x47,
+ 0x82,0x0e,0xae,0x18, 0xbf,0x3a,0xc9,0x5a,
+ 0x71,0xcf,0x5e,0xbf, 0x9e,0xec,0x1d,0x11,
+ 0x96,0x33,0x32,0x5e, 0x5e,0xee,0xc8,0xee,
+ 0x52,0x03,0xbc,0x8d, 0x97,0xd2,0x55,0xc5,
+ 0xaf,0x52,0xb0,0x55, 0x8f,0xb8,0x9b,0x83,
+ 0x60,0x9f,0x60,0x92, 0x47,0x1d,0xf2,0x6e,
+ 0xd1,0x93,0xfe,0xc2, 0x77,0x8c,0xb6,0x49,
+ 0x5e,0x3e,0xdb,0xb9, 0x7a,0x58,0x4d,0x18,
+ 0x66,0xc8,0xc2,0x67, 0xf8,0x37,0x7d,0x06,
+ 0x50,0xcc,0x42,0xab, 0x08,0x27,0x8e,0x81,
+ 0x6f,0xb3,0x03,0xbd, 0x41,0x11,0xeb,0x13,
+ 0xf1,0xaf,0xee,0x56, 0xae,0xb3,0x36,0x41,
+ 0xb8,0xc9,0x0a,0x96, 0x88,0x1d,0x98,0x25,
+ 0xc6,0x45,0xeb,0x76, 0x07,0xc1,0xfe,0xae,
+ 0xbc,0x26,0x1f,0xc4, 0x5f,0x70,0x0c,0xae,
+ 0x70,0x00,0xcf,0xc6, 0x77,0x5c,0x9c,0x24,
+ 0x8b,0x4b,0x83,0x32, 0x09,0xb7,0xb1,0x43,
+ 0x4a,0x01,0x42,0x04, 0x4d,0xca,0x5f,0x4e,
+ 0x9b,0x2b,0xa9,0xcb, 0x99,0x0b,0x0e,0x57,
+ 0x09,0xd6,0xe2,0xa0, 0xc1,0x12,0x79,0xf2,
+ 0x6f,0xe1,0x6c,0x7f, 0x0a,0x1a,0xec,0xc1,
+ 0x82,0x4a,0xf8,0x98, 0x22,0xc9,0x81,0x81,
+ 0x5d,0xf8,0x7d,0x9d, 0x86,0x97,0xdd,0x9e,
+ 0x8a,0xb5,0xce,0x6c, 0xfb,0x06,0xc3,0x8a,
+ 0x0d,0x53,0xda,0x12, 0x0c,0x4b,0x6f,0xa0,
+ 0x3f,0x8d,0xc3,0x07, 0x27,0x10,0xaf,0xc5,
+ 0x27,0xfe,0x64,0x17, 0x18,0xa5,0x3a,0xfe,
+ 0x9b,0x91,0xae,0xd0, 0x2d,0x34,0x34,0x9e,
+ 0x9f,0x31,0x5d,0x3e, 0x4c,0x26,0x1e,0xcb,
+ 0x62,0x05,0xd2,0x83, 0x8d,0x71,0xb8,0x57,
+ 0xef,0x3a,0x94,0xb3, 0x3a,0x67,0x1b,0x21,
+ 0x33,0x1f,0x7f,0x10, 0xd8,0xd7,0x89,0x1b,
+ 0x4f,0x51,0x74,0x97, 0x4a,0x0e,0x74,0x59,
+ 0x74,0x66,0xef,0xdd, 0x26,0xb6,0xa1,0x53,
+ 0xd4,0x2f,0xd7,0x76, 0x51,0x27,0xcc,0xe4,
+ 0x94,0xe3,0xed,0x26, 0x13,0x4e,0xe8,0x2c,
+ 0x11,0x6e,0xb3,0x63, 0x51,0x36,0x9c,0x91,
+ 0x2d,0x66,0x2c,0x3e, 0x0a,0xf7,0xa4,0x97,
+ 0x70,0x6d,0x04,0xaa, 0x89,0xe8,0x2c,0x5e,
+ 0xdd,0x01,0x46,0xfc, 0x99,0xce,0xe6,0x32,
+ 0x8a,0x85,0xe6,0x07, 0x1e,0x71,0x5d,0x29,
+ 0x07,0x16,0x0e,0xf9, 0xd4,0xdf,0x54,0xb4,
+ 0x7b,0x7b,0x3f,0xe0, 0xeb,0x73,0xe0,0xe1,
+ 0x92,0x51,0x50,0x74, 0xb5,0x6e,0x08,0x7e,
+ 0x57,0x70,0xb2,0x1b, 0x9c,0xf2,0xa2,0x6b,
+ 0x52,0xa3,0x35,0xf7, 0x22,0x40,0xa6,0x11,
+ 0x30,0xd3,0x5b,0x4b, 0x78,0xc9,0xd7,0x84,
+ 0x9a,0x88,0x9a,0x44, 0xb4,0x88,0xfe,0x8c,
+ 0x3f,0x10,0xab,0xc7, 0xc9,0xb6,0x59,0x9a,
+ 0xf3,0xe6,0xe6,0x4d, 0xea,0x3e,0xe0,0xeb,
+ 0x9e,0xb4,0x41,0xf6, 0xcb,0xfc,0x04,0x73,
+ 0x7d,0xc8,0x00,0xc6, 0xf2,0x10,0x00,0xcf,
+ 0x59,0xed,0x05,0x2a, 0x6a,0xde,0x7a,0xdf,
+ 0x7d,0xa9,0x25,0xc8, 0x6e,0x08,0x60,0xf9,
+ 0xd8,0x23,0x9b,0x20, 0xe5,0x93,0x9c,0x90,
+ 0x3d,0xe0,0xd0,0x33, 0x2d,0xce,0x86,0x93,
+ 0xdc,0xb3,0x9c,0x40, 0x33,0x9a,0xf0,0x71,
+ 0x47,0x0e,0xc4,0xb9, 0x58,0xc4,0x36,0xf1,
+ 0x4c,0x82,0xcf,0x91, 0x9f,0x16,0xce,0x43,
+ 0x58,0x72,0x54,0x51, 0x0d,0x8e,0x1e,0x3d,
+ 0x5e,0x67,0x7e,0x96, 0x6e,0x12,0xb8,0xee,
+ 0x1f,0x8b,0x15,0x3b, 0x49,0x95,0x2f,0xd9,
+ 0xec,0x63,0x56,0xec, 0x4e,0x88,0x37,0x2f,
+ 0xa7,0xd5,0xe5,0x4a, 0x97,0x1f,0x6f,0xa0,
+ 0x40,0x68,0x69,0xee, 0x6a,0xc6,0xbe,0x83,
+ 0xba,0x69,0xb8,0x08, 0x0a,0x5c,0x2f,0xd2,
+ 0x3e,0x3b,0x73,0x40, 0x9c,0x62,0xcc,0xe1,
+ 0x99,0x44,0xa2,0xaa, 0xb8,0xe9,0x48,0xf4,
+ 0x79,0x07,0xe8,0xe8, 0x16,0x99,0x84,0x7b,
+ 0x3d,0x53,0xb2,0x5d, 0x2d,0xa4,0xb0,0x12,
+ 0xb9,0xa9,0x0d,0x77, 0x98,0xa1,0x98,0x90,
+ 0x4e,0xe2,0x14,0xd4, 0x15,0x35,0xd0,0x85,
+ 0xbf,0xa1,0x0f,0x54, 0x05,0xa0,0x90,0x2a,
+ 0x74,0xe3,0xd3,0x1b, 0x5e,0x16,0x07,0xcf,
+ 0x36,0xbd,0xea,0x9b, 0x2d,0x35,0x47,0xea,
+ 0xea,0xb7,0xd1,0xda, 0x66,0x47,0x42,0x47,
+ 0x4e,0x76,0xe5,0x90, 0x0c,0x82,0x15,0x3f,
+ 0x17,0x1b,0xa6,0x04, 0xb6,0x58,0x67,0x42,
+ 0xfb,0x19,0x2a,0xc2, 0xd7,0x6a,0x48,0x36,
+ 0x87,0x53,0x90,0x95, 0x53,0xb7,0xf1,0xbe,
+ 0x0d,0x9f,0xa3,0x74, 0x5f,0x3d,0x89,0xef,
+ 0x29,0x07,0xe1,0xc1, 0x13,0xe0,0xc7,0xf6,
+ 0x53,0xc2,0xe5,0x7e, 0x96,0xdf,0x1f,0x12,
+ 0x98,0xd6,0x7b,0x2d, 0xdb,0x3e,0x01,0x03,
+ 0x05,0xbe,0x66,0x29, 0x42,0xeb,0x5d,0xab,
+ 0xa8,0x13,0x78,0x7f, 0x1e,0x0e,0xfd,0x7f,
+ 0xf1,0xd2,0x59,0xb2, 0x46,0x13,0x1c,0xb8,
+ 0x42,0x4f,0x87,0xb3, 0x26,0x0b,0xed,0x26,
+ 0xb2,0xd5,0x27,0xfc, 0xf1,0xec,0x32,0x66,
+ 0xe1,0x2d,0x27,0x2a, 0xe2,0x80,0xf2,0x72,
+ 0x90,0x3c,0x54,0xfa, 0xaa,0xe6,0x31,0xb0,
+ 0xb7,0xdd,0x97,0x0d, 0x22,0xb5,0x16,0x46,
+ 0x66,0x6d,0x02,0x13, 0x9a,0x7c,0x52,0xfc,
+ 0xf8,0x73,0x0c,0x81, 0xac,0xa3,0x8f,0x40,
+ 0x50,0x2e,0x80,0x3b, 0xb6,0xdf,0x88,0xbb,
+ 0xb5,0xa8,0x13,0xfa, 0xd2,0xd6,0xb8,0x07,
+ 0x47,0x7b,0xa0,0x09, 0x9f,0xc3,0x42,0xab,
+ 0xb8,0xd6,0xca,0xfa, 0x41,0xdc,0x9a,0xb5,
+ 0x96,0xf4,0xfa,0xfd, 0x09,0xca,0x8e,0x47,
+ 0x1d,0x8f,0x8d,0x54, 0x3f,0xbf,0xfd,0x22,
+ 0x30,0x25,0xbd,0xea, 0xb3,0xf6,0x90,0x68,
+ 0x6e,0x2b,0x78,0x8e, 0xc4,0x58,0x1c,0xbd,
+ 0x6b,0x36,0xdc,0x9d, 0x9f,0x27,0xce,0xf6,
+ 0x4f,0x1b,0xeb,0x41, 0x2c,0x07,0xa1,0x1f,
+ 0xaa,0xc3,0x65,0xe0, 0x78,0x85,0x80,0x22,
+ 0x00,0x94,0x1a,0x9f, 0x34,0x2b,0x2b,0x51,
+ 0x94,0x93,0x23,0x20, 0x48,0x2e,0x16,0xd6,
+ 0xdf,0x09,0xa2,0xfa, 0xb8,0x9b,0xf0,0x64,
+ 0x18,0x36,0x78,0xbc, 0xb8,0x5b,0x87,0x90,
+ 0xba,0xd2,0x2e,0x30, 0xe6,0xc5,0xe0,0x0c,
+ 0x81,0x32,0x69,0x9a, 0x8a,0x5a,0x3d,0x6f,
+ 0x06,0xe1,0x3f,0xa9, 0xf2,0x0e,0x21,0xfe,
+ 0x9e,0x63,0x31,0xa9, 0xc3,0x3e,0xb4,0xcd,
+ 0xcb,0x60,0xd9,0x45, 0xc6,0x5f,0xc5,0xca,
+ 0x9e,0xd8,0x40,0x72, 0x39,0x04,0x59,0x2d,
+ 0x4c,0xac,0xdf,0xea, 0x4a,0x78,0xa9,0xd5,
+ 0x87,0xb1,0xd6,0x59, 0x77,0x58,0x4d,0xa7,
+ 0xd3,0x9b,0xfc,0xe3, 0xdd,0x8d,0xf5,0x57,
+ 0x06,0xb3,0x96,0xf1, 0xbe,0xd9,0x07,0x54,
+ 0x36,0xa4,0x8b,0xaa, 0x0b,0xcb,0xd3,0x80,
+ 0x13,0xa6,0x53,0x8e, 0xcc,0x23,0x15,0x02,
+ 0x1e,0x1b,0x2f,0x0a, 0x02,0x5b,0xca,0x50,
+ 0x11,0x28,0x27,0x0e, 0xbe,0xfe,0x76,0x60,
+ 0x1b,0x78,0x58,0x9b, 0xe6,0x0a,0x0a,0xef,
+ 0xa3,0xa5,0x33,0x0d, 0x5b,0x65,0xe1,0x03,
+ 0x38,0xdd,0xf8,0x22, 0x92,0xcd,0x50,0x87,
+ 0x02,0xbc,0x91,0x16, 0xfd,0x05,0x9c,0xcd,
+ 0x72,0xae,0x4c,0xd7, 0xef,0xb3,0x57,0x1a,
+ 0x3f,0x79,0x23,0xfd, 0xf0,0xc3,0xfb,0x68,
+ 0xb4,0xc9,0x93,0x22, 0x33,0xd3,0x01,0x74,
+ 0xe3,0x00,0x31,0xcf, 0x0f,0x23,0xc5,0xf7,
+ 0x09,0x95,0x5a,0xa0, 0x56,0xf9,0xb0,0x20,
+ 0xb1,0xcc,0x8d,0x88, 0xd6,0x27,0x97,0x8d,
+ 0x0e,0xa3,0x3d,0x33, 0x94,0x04,0x44,0x93,
+ 0x67,0x10,0xb6,0xa0, 0x0c,0x2a,0x28,0xd4,
+ 0x1b,0x41,0x86,0xe7, 0x29,0x2c,0x68,0x2a,
+ 0x94,0xf3,0x4f,0x20, 0xa1,0xb4,0x6c,0x9d,
+ 0x85,0x6b,0xa0,0x31, 0xa2,0xbd,0x74,0xf0,
+ 0x0b,0xe5,0x2f,0xb7, 0x8a,0x33,0xd9,0x1f,
+ 0xf2,0xb5,0xad,0x85, 0xc3,0xad,0x47,0x2f,
+ 0x27,0x2a,0xc9,0x32, 0xd8,0xd9,0x05,0xc2,
+ 0x9d,0xbf,0x21,0x88, 0x02,0x05,0x12,0x6e,
+ 0x0f,0xb6,0x64,0x43, 0xa8,0xc3,0x87,0xea,
+ 0xb0,0x81,0x5b,0x51, 0x51,0xf1,0x83,0x7d,
+ 0x94,0x46,0x7f,0x0a, 0x9a,0xef,0xcc,0x68,
+ 0x73,0xef,0x9d,0x3c, 0x0e,0xfc,0x37,0x91,
+ 0xca,0x36,0x2d,0x1d, 0x72,0x7e,0x39,0x9e,
+ 0xad,0xd3,0x55,0x1b, 0x10,0x1e,0xff,0x00,
+ 0xc1,0x45,0x80,0xe7, 0xb4,0xcc,0xc8,0xb0,
+ 0x62,0xbd,0xf9,0xa5, 0x8f,0x05,0xaa,0x3b,
+ 0x86,0x73,0x14,0xf9, 0xee,0x95,0xd0,0xfd,
+ 0x95,0x30,0x68,0x22, 0xc9,0x70,0x66,0x1d,
+ 0x91,0x3f,0xc0,0x19, 0x93,0x07,0x19,0x2d,
+ 0x3c,0x21,0x6b,0xc1, 0x2a,0xeb,0xaa,0xf2,
+ 0xa4,0x45,0x35,0xff, 0x8f,0x24,0x46,0x2c,
+ 0xc8,0x75,0x58,0x68, 0x0f,0x3b,0x87,0x11,
+ 0xcb,0x9f,0xf7,0x28, 0xbd,0x66,0x91,0x01,
+ 0xeb,0x70,0x8e,0x8d, 0xe6,0x01,0xc8,0x48,
+ 0x94,0xfe,0x4e,0xa8, 0xeb,0x90,0xbf,0xd1,
+ 0xcd,0x89,0xc2,0x98, 0x34,0x92,0xf9,0x08,
+ 0xb9,0xbc,0xd4,0x34, 0x1a,0x59,0xcc,0x80,
+ 0x9a,0xe6,0xbc,0xbb, 0x23,0x12,0x9c,0xa4,
+ 0x5b,0x79,0xc6,0x8a, 0xc0,0x03,0x2b,0x16,
+ 0xe5,0x1c,0x0f,0x02, 0x37,0x4f,0x3e,0xc2,
+ 0xf3,0x4d,0x7c,0xcb, 0xde,0x9b,0x66,0x52,
+ 0xf3,0xdd,0x86,0x42, 0x4a,0x81,0x5b,0x96,
+ 0x83,0x2a,0xb1,0x48, 0x31,0x42,0x16,0x16,
+ 0xf8,0x97,0xa3,0x52, 0xeb,0xb6,0xbe,0x99,
+ 0xe1,0xbc,0xa1,0x3a, 0xdd,0xea,0x00,0xfa,
+ 0x11,0x2f,0x0b,0xf8, 0xc7,0xcc,0xba,0x1a,
+ 0xf3,0x36,0x20,0x3f, 0x59,0xea,0xf1,0xc8,
+ 0x08,0xd0,0x6d,0x8e, 0x91,0x1e,0x90,0x91,
+ 0x7b,0x80,0xdc,0xcb, 0x5c,0x94,0x74,0x26,
+ 0xd3,0x5d,0x1a,0x2d, 0xad,0xcf,0xef,0xfa,
+ 0xe9,0xa0,0x17,0xb7, 0x2b,0x7c,0x37,0x83,
+ 0x31,0x78,0x1a,0xcf, 0x04,0xa0,0xe7,0x83,
+ 0x66,0x12,0x4f,0x9d, 0x31,0x6b,0x4d,0xc5,
+ 0x31,0x1b,0x3a,0xd9, 0x79,0x76,0x49,0xc3,
+ 0x19,0xf0,0x3f,0xb5, 0xbc,0x7d,0xa4,0xa7,
+ 0x24,0x44,0x75,0xbb, 0x6d,0x65,0x59,0xf8,
+ 0xe0,0xb9,0xd7,0x29, 0x79,0xce,0x14,0x32,
+ 0xd2,0x3e,0xb8,0x22, 0x4a,0x0a,0x2a,0x6c,
+ 0xb2,0xbd,0xa5,0xd4, 0xc4,0xc5,0x68,0xb3,
+ 0x63,0xe7,0x46,0x05, 0x3a,0x18,0xa5,0xad,
+ 0xcc,0x61,0xc3,0xec, 0x3d,0x42,0xb0,0xa7,
+ 0x23,0x72,0x1e,0x14, 0xd8,0x7e,0x68,0x60,
+ 0xec,0xe9,0x1d,0x5b, 0x1f,0x86,0xda,0x5e,
+ 0x34,0x74,0x00,0xd3, 0x98,0x98,0x7e,0xbd,
+ 0x6a,0x8b,0xd3,0x6f, 0x31,0xf1,0x62,0xb3,
+ 0xa3,0x86,0x95,0x02, 0x76,0x7d,0x58,0xbc,
+ 0xf8,0xb1,0x52,0xc3, 0x0b,0xd5,0x6b,0x74,
+ 0xa5,0x84,0xef,0xf2, 0x31,0xc1,0xe4,0x83,
+ 0x42,0x12,0xb5,0xe7, 0x61,0xdd,0xba,0x43,
+ 0x39,0xf2,0x44,0x0a, 0xb4,0x62,0x06,0x32,
+ 0x5b,0x33,0x67,0x2e, 0x7a,0x93,0x85,0x1a,
+ 0x07,0x36,0x9f,0xab, 0xf7,0x2a,0x6e,0x3d,
+ 0x3e,0xe3,0x59,0x1b, 0xf8,0xd3,0xe8,0x5f,
+ 0xe5,0x24,0xb3,0x59, 0x80,0xd5,0x11,0x14,
+ 0x98,0x3a,0xb4,0x7d, 0x8f,0x37,0x18,0xb2,
+ 0xa7,0x25,0xf4,0x31, 0x74,0x61,0x3a,0x42,
+ 0x62,0x77,0x37,0x3d, 0x72,0x1b,0x67,0x87,
+ 0xb3,0x59,0x4b,0x08, 0x07,0xdb,0x0b,0x57,
+ 0xfd,0x61,0x99,0x28, 0x3b,0xe5,0x7a,0xb4,
+ 0x6c,0x06,0x95,0x65, 0x2c,0x1c,0x41,0x71,
+ 0x21,0xd7,0x94,0x51, 0x1c,0x8d,0xe6,0x38,
+ 0xc5,0x95,0x7f,0x30, 0xd5,0xc5,0xcc,0xd2,
+ 0x03,0x7f,0x69,0x2e, 0xae,0xc7,0x28,0x2e,
+ 0xc6,0xa9,0x28,0x4b, 0x77,0xc3,0xcf,0xa3,
+ 0xc3,0xd3,0x2d,0x43, 0x47,0x87,0xde,0x38,
+ 0xeb,0x3a,0xb6,0xf9, 0xe7,0x3c,0xb6,0x92,
+ 0x19,0x42,0xf8,0xc2, 0x87,0x50,0xed,0xe6,
+ 0x3d,0x2b,0xb5,0xf8, 0x89,0x14,0x42,0xf7,
+ 0x2c,0x7a,0xbe,0xdc, 0x2f,0x5d,0x49,0x83,
+ 0xf5,0x60,0xe0,0xcf, 0xbc,0x23,0x13,0x4f,
+ 0xb3,0x16,0xd7,0x9a, 0xca,0x16,0x8b,0xa5,
+ 0x08,0x80,0xcf,0x21, 0xbb,0xd8,0x32,0x5e,
+ 0x07,0x8a,0xb3,0x48, 0xba,0x99,0xd4,0xd7,
+ 0x6a,0xae,0x4b,0x9b, 0xb4,0xd7,0x2f,0x87,
+ 0xb0,0x0a,0xd1,0x1b, 0xf1,0x8b,0xf6,0x21,
+ 0x81,0x8e,0xc4,0x79, 0x9a,0x5c,0x75,0xbe,
+ 0x87,0x99,0xe5,0x11, 0xf9,0x9a,0xe1,0xf9,
+ 0x76,0xa2,0x92,0xc6, 0xc0,0xd8,0x05,0xc9,
+ 0x7d,0x8c,0x27,0xc2, 0x7f,0xf4,0xe9,0x4f,
+ 0xb7,0xbc,0xa3,0x3e, 0x66,0x3b,0xaf,0xed,
+ 0x7a,0xd9,0x78,0x20, 0x6b,0xd5,0xe1,0xfe,
+ 0xd5,0x06,0x65,0x11, 0x49,0xac,0x22,0x38,
+ 0x02,0x80,0xec,0x91, 0x11,0x18,0x1a,0x61,
+ 0x3c,0x59,0x4e,0x7a, 0xd8,0xca,0xda,0xd4,
+ 0x27,0xbd,0xf4,0x00, 0x9c,0x1b,0xde,0xf3,
+ 0x6c,0x1f,0x20,0x9a, 0x30,0xc9,0x9b,0x3c,
+ 0xe5,0x55,0xb7,0xb3, 0xc8,0x52,0x9c,0x05,
+ 0xad,0xe8,0x13,0x9e, 0x31,0xc2,0x2c,0xd4,
+ 0x3f,0x18,0x00,0xc4, 0xcf,0x08,0x05,0x7b,
+ 0x5e,0x2a,0x8e,0x11, 0x61,0x03,0xc8,0x39,
+ 0x2b,0x54,0x1a,0xd9, 0x08,0x04,0xc6,0xe9,
+ 0xda,0x69,0xb3,0x0c, 0x83,0x44,0xcd,0xe8,
+ 0x50,0x04,0x72,0xa2, 0xb4,0x10,0x17,0x39,
+ 0x68,0x32,0xdb,0xab, 0xe3,0xee,0x57,0x1b,
+ 0x05,0x45,0x1f,0x5a, 0xdc,0xdc,0x56,0x81,
+ 0x98,0x20,0xfe,0x69, 0x0a,0xa4,0xd6,0x9d,
+ 0x25,0xdd,0x7e,0xd0, 0x2b,0x33,0x41,0x75,
+ 0xf6,0x59,0xa8,0xa3, 0x3c,0xdd,0xd9,0x6b,
+ 0xa8,0xcd,0x1d,0x1f, 0xc5,0x78,0x5b,0x93,
+ 0xdf,0x10,0x71,0xeb, 0xcc,0xbd,0x35,0x4c,
+ 0x07,0x21,0x5f,0xb7, 0x47,0x21,0x6d,0x55,
+ 0x8b,0x72,0x0e,0x4a, 0x2c,0x17,0xfc,0x75,
+ 0x21,0xdd,0x76,0xfd, 0x34,0xfc,0x0f,0x1b,
+ 0xa6,0x77,0x53,0xf9, 0xdb,0x09,0x07,0x58,
+ 0xb0,0x18,0x32,0x03, 0x98,0x79,0xdf,0x55,
+ 0xd3,0x95,0xba,0xa9, 0xb6,0x9f,0xad,0xc4,
+ 0x9d,0xba,0x76,0x36, 0x47,0xb1,0xde,0x78,
+ 0x18,0xa0,0x2f,0x16, 0x41,0xeb,0x4a,0x96,
+ 0x82,0xc4,0xa4,0xde, 0x4b,0xdf,0xee,0xc7,
+ 0x33,0xdf,0xb7,0xde, 0xd3,0xa7,0x0f,0xc7,
+ 0x23,0x61,0x6b,0xd9, 0x15,0xc8,0x09,0xf7,
+ 0xe7,0xf9,0x44,0xba, 0x14,0xdc,0x94,0x5e,
+ 0xd9,0xcc,0x74,0xb2, 0x3d,0xef,0x78,0x15,
+ 0xb5,0xb9,0x56,0xd5, 0xfb,0x47,0x49,0x3a,
+ 0xbc,0x53,0x71,0x8b, 0x72,0x8b,0xb2,0xe3,
+ 0x58,0xbf,0xea,0x47, 0x7a,0x76,0x03,0x48,
+ 0xdd,0x8c,0x30,0x99, 0x81,0x2c,0x5f,0xf6,
+ 0xd3,0x9b,0x8e,0x77, 0x1c,0xb7,0xbd,0x1e,
+ 0xd4,0x28,0x05,0xf7, 0xff,0xdf,0xd6,0xb9,
+ 0x83,0x99,0xbc,0x94, 0xb7,0x41,0x93,0xc4,
+ 0x66,0xff,0x29,0x4d, 0x5c,0xba,0x79,0xd9,
+ 0x6e,0x79,0x47,0x45, 0xd6,0x2d,0xcd,0x79,
+ 0xa1,0xfa,0x49,0xee, 0x8e,0x7f,0x2b,0x08,
+ 0x3f,0x60,0x56,0xcf, 0xcb,0xe8,0x0d,0x55,
+ 0xee,0xa5,0xaf,0x04, 0xde,0x01,0xde,0xce,
+ 0xb6,0x9c,0x68,0x4e, 0xb0,0x88,0xcd,0x89,
+ 0x83,0x6b,0x01,0xb5, 0x78,0xac,0x85,0x3c,
+ 0x2c,0xcf,0x39,0xb6, 0xc8,0x5f,0x0e,0xac,
+ 0x02,0x08,0x56,0xbe, 0xd1,0x8d,0x7d,0x55,
+ 0x69,0x0c,0x33,0x33, 0xff,0x1a,0xd6,0x0b,
+ 0xcf,0x57,0x18,0x01, 0x56,0x5f,0x9c,0x6f,
+ 0xe2,0x24,0xda,0xc3, 0x9f,0x81,0xc3,0x27,
+ 0x46,0x7a,0xb4,0xae, 0xec,0xa4,0x0e,0x41,
+ 0x8b,0xb7,0x16,0xe3, 0x9b,0x2e,0x32,0x75,
+ 0xd9,0x86,0xa2,0x13, 0x68,0x4e,0xbc,0x43,
+ 0xa2,0x78,0x64,0x1a, 0x7c,0xac,0x13,0x70,
+ 0x1c,0x23,0x15,0x5b, 0xda,0x99,0xa5,0x24,
+ 0x3d,0xcf,0x29,0xf7, 0xbc,0x1d,0x10,0xe8,
+ 0x95,0x1a,0x11,0xec, 0xfc,0xfb,0x20,0x1f,
+ 0x09,0x1b,0xe3,0x3d, 0xae,0x82,0x70,0xd7,
+ 0x9e,0xf3,0x18,0x97, 0x89,0xfa,0x42,0x67,
+ 0x70,0x9c,0xc8,0xbe, 0x62,0x98,0xf1,0x82,
+ 0xfc,0x2b,0xf0,0x40, 0xaa,0xdc,0x27,0xf9,
+ 0x21,0x5a,0xc1,0x25, 0x8b,0xef,0xd5,0x48,
+ 0x6c,0x68,0xae,0xbc, 0xcd,0xa9,0x3c,0x1e,
+ 0xe9,0xcf,0xe2,0xd1, 0xc0,0x98,0xa9,0x62,
+ 0x5d,0x1f,0x57,0x7a, 0xca,0x8a,0x0f,0xfb,
+ 0xe3,0xc9,0x7e,0x98, 0x44,0x84,0x67,0x12,
+ 0x60,0x60,0xe5,0xc7, 0xcc,0x72,0x90,0x64,
+ 0x67,0x30,0x6a,0xd8, 0xa1,0x11,0xd5,0x7e,
+ 0x5e,0x0c,0x74,0xa2, 0x6f,0x0a,0xff,0x41,
+ 0xd3,0x9a,0x30,0x56, 0xd4,0xec,0x9a,0x5f,
+ 0x22,0x71,0x6b,0x4e, 0xe6,0xe0,0x19,0x69,
+ 0x56,0x4a,0xba,0x9d, 0x50,0x8a,0x73,0x6a,
+ 0xf1,0x59,0x48,0xd6, 0xcd,0xfa,0xaa,0x0c,
+ 0xbb,0x7c,0xa4,0xbc, 0xf5,0x32,0x95,0x55,
+ 0x1c,0xe9,0x9a,0x60, 0x43,0x10,0xbd,0x27,
+ 0x88,0x2f,0x05,0xcf, 0xce,0x21,0x25,0x3a,
+ 0x07,0xab,0x37,0xfd, 0xf6,0x2f,0xd6,0x51,
+ 0xbe,0xe6,0xcc,0x58, 0x3a,0xab,0x60,0x23,
+ 0x45,0xa0,0xe5,0x79, 0xe5,0xaa,0xed,0xa4,
+ 0x28,0xd0,0x4d,0x37, 0x9c,0x6a,0xd7,0xc2,
+ 0x39,0x22,0xb9,0x3e, 0x0d,0xb8,0x94,0x65,
+ 0x48,0x4d,0x4c,0x02, 0x31,0x7e,0x9c,0xc9,
+ 0xb7,0xd6,0x23,0x1a, 0x94,0x5a,0x13,0x55,
+ 0x78,0x7a,0x29,0x4a, 0xa2,0xfd,0x37,0x24,
+ 0xd8,0xd0,0x9e,0x47, 0x24,0xab,0x26,0x34,
+ 0x28,0xb5,0x2d,0x82, 0x9a,0x4d,0xdd,0x17,
+ 0x68,0xe0,0x07,0x5d, 0xb9,0x2d,0xff,0xa9,
+ 0x0c,0x11,0x59,0x75, 0xda,0x98,0xe9,0xd5,
+ 0xfa,0xb5,0x18,0x16, 0x28,0x17,0x7c,0xad,
+ 0xab,0xee,0x65,0x10, 0x13,0x0d,0x26,0xfa,
+ 0x7f,0xac,0x06,0x43, 0x4d,0x5d,0x3a,0xf4,
+ 0x77,0xe7,0x03,0x17, 0x39,0x9f,0xbe,0x52,
+ 0x9b,0x68,0x2b,0x7f, 0xd3,0xa2,0x7e,0x5c,
+ 0x78,0x22,0xc5,0xe3, 0x17,0x73,0xc6,0x9e,
+ 0x68,0x17,0x74,0x50, 0xf4,0xc5,0xa8,0xc3,
+ 0x66,0xe1,0x05,0xed, 0xdd,0xdb,0xd3,0x11,
+ 0x16,0xad,0x05,0x3a, 0x38,0x55,0x1c,0xf0,
+ 0x93,0x0b,0x22,0x83, 0xc8,0x34,0xc5,0x43,
+ 0x4d,0x65,0x57,0xf3, 0x03,0x56,0x21,0xa9,
+ 0xbd,0x04,0x41,0x49, 0x62,0xfd,0xcc,0xc2,
+ 0x75,0x59,0x09,0xb9, 0x28,0x38,0xcf,0xfb,
+ 0x54,0x64,0x51,0xc2, 0x3e,0xad,0x35,0x3e,
+ 0x31,0x87,0x6e,0xfe, 0xf0,0x41,0xef,0x1d,
+ 0xb8,0x46,0xbe,0x85, 0xb9,0xff,0xa3,0xdb,
+ 0x87,0xf9,0x65,0x95, 0x60,0x53,0x7c,0x9d,
+ 0x26,0x83,0xfc,0xa7, 0xad,0x5a,0xcb,0x8d,
+ 0x81,0xec,0x28,0xeb, 0xdd,0x96,0x25,0x31,
+ 0x24,0x3f,0x59,0x28, 0x60,0x0b,0xc0,0x59,
+ 0xea,0x36,0x15,0xad, 0x70,0xd8,0x70,0xff,
+ 0x9b,0x15,0x76,0xc5, 0x84,0xe6,0x81,0x75,
+ 0x1a,0x1e,0xc9,0xec, 0x33,0xbe,0x10,0xd4,
+ 0x6f,0x10,0x1b,0xa2, 0xdb,0xc6,0x1b,0x0a,
+ 0xfb,0xe9,0x3f,0x4d, 0x04,0x4e,0x33,0x87,
+ 0xb3,0x21,0xad,0x41, 0xbe,0xce,0x26,0x0c,
+ 0x0c,0x84,0x0f,0x9a, 0xb9,0xa7,0xa2,0x36,
+ 0x70,0x49,0xce,0x25, 0x0f,0x69,0x4a,0x4a,
+ 0x3d,0xf5,0xa0,0x9e, 0xad,0x69,0x2d,0x79,
+ 0xdb,0x8b,0x85,0xf6, 0xb8,0x55,0xcd,0xf1,
+ 0xbb,0x04,0x35,0xad, 0xa8,0xb6,0x0d,0x3f,
+ 0x23,0xec,0x39,0xd7, 0xef,0x02,0x95,0x42,
+ 0x11,0xc9,0x70,0xc6, 0xa4,0x65,0x37,0x4d,
+ 0x9f,0x51,0x99,0xd6, 0x9e,0xb1,0x18,0xcf,
+ 0x31,0x81,0xde,0x95, 0x0a,0x8c,0x0c,0x80,
+ 0xdc,0xf7,0x19,0x5d, 0xdc,0x3e,0xee,0x0c,
+ 0x17,0xaf,0xc4,0x9c, 0xbf,0x65,0xf2,0xe1,
+ 0xc9,0xdb,0xc0,0x2a, 0xd0,0xbd,0xa1,0x7f,
+ 0x4b,0x9c,0x5b,0xe6, 0x91,0x98,0xa6,0xdb,
+ 0x72,0xef,0x14,0x38, 0x24,0x77,0x1e,0x71,
+ 0x74,0x63,0x0c,0xd9, 0x16,0x90,0x23,0x4a,
+ 0xe6,0xa4,0xc1,0x53, 0x8b,0xb4,0x7e,0x90,
+ 0x1b,0x68,0x32,0x48, 0x93,0xd8,0x72,0x43,
+ 0x8e,0x32,0x09,0x1e, 0x48,0xfc,0x3a,0xc6,
+ 0x15,0xb9,0x79,0x57, 0x02,0x61,0xc6,0x4b,
+ 0x56,0x1e,0x68,0x4e, 0x65,0x26,0xe5,0x1c,
+ 0xb1,0xd1,0x86,0x1d, 0xea,0x93,0x5a,0x88,
+ 0x4c,0x3b,0x10,0xd1, 0xf7,0x5a,0x4c,0xa3,
+ 0xe7,0x59,0xf5,0x04, 0x7d,0xd7,0xe3,0x2e,
+ 0x2c,0x3e,0x14,0x14, 0x83,0xed,0x3d,0x0b,
+ 0xa4,0xab,0x65,0xcf, 0x39,0xee,0xbe,0x0c,
+ 0x5e,0x4b,0x62,0x5e, 0xb4,0xd2,0x16,0xc7,
+ 0xe0,0x71,0x2b,0x92, 0x1e,0x21,0x45,0x02,
+ 0xfd,0xa1,0xda,0x0b, 0xbe,0xa6,0xe5,0x7f,
+ 0x31,0x8b,0x5a,0xcb, 0x8f,0xb8,0x0c,0xfb,
+ 0x7f,0x2d,0x7e,0xa2, 0x14,0xfd,0xe0,0xbb,
+ 0xa4,0x1b,0xce,0x81, 0x6f,0x25,0xbd,0x72,
+ 0x44,0x00,0x13,0x18, 0x75,0x04,0xf3,0x06,
+ 0xdc,0xf1,0x5b,0xa0, 0xb1,0x5a,0x9a,0xd8,
+ 0x4f,0xe7,0x94,0xe1, 0x65,0xe5,0xb2,0xd1,
+ 0x47,0x6d,0xd8,0x81, 0x22,0x96,0x09,0xd8,
+ 0x5e,0x12,0x73,0x62, 0xd6,0x2c,0xcb,0x45,
+ 0x71,0xa9,0xc1,0x21, 0x16,0x6f,0xf0,0xaa,
+ 0xce,0x19,0x1f,0x68, 0xee,0x17,0x07,0x94,
+ 0x4f,0x93,0x9a,0x12, 0xf7,0x91,0xe1,0xc6,
+ 0x9c,0x29,0xe5,0x06, 0x7a,0x40,0xf5,0xf6,
+ 0x51,0xc8,0x32,0x94, 0x52,0xd9,0x6b,0x9b,
+ 0x3e,0xb5,0xcf,0x1a, 0xf1,0x6c,0x7b,0x0a,
+ 0x16,0x47,0xee,0xa6, 0x46,0x0f,0xed,0xe0,
+ 0x1b,0x3f,0x39,0xfa, 0x4c,0x69,0xeb,0xfb,
+ 0xd0,0x36,0x3b,0x3a, 0x04,0x94,0xa4,0x2f,
+ 0x51,0xe1,0x1a,0x47, 0xc9,0xdb,0xf6,0x09,
+ 0xab,0x35,0x46,0x2c, 0x2f,0xb7,0x19,0xed,
+ 0x55,0x7e,0xa3,0x2c, 0xec,0xff,0x39,0xba,
+ 0x0f,0xfb,0x4f,0x8b, 0xfc,0x36,0x4e,0x5e,
+ 0xa1,0xe8,0x49,0x15, 0x65,0xd2,0xfb,0x11,
+ 0x4b,0x10,0xe6,0x07, 0x82,0x3a,0x5d,0x3f,
+ 0xeb,0xc0,0x0b,0x76, 0x66,0xb5,0xed,0x65,
+ 0xb3,0x9d,0x06,0x13, 0x3b,0x18,0x70,0x7a,
+ 0xbd,0xf7,0xd8,0x20, 0x81,0xc7,0x76,0x2e,
+ 0x21,0x6f,0xdb,0x8e, 0xba,0x83,0x42,0xb1,
+ },
+ },
+ [5] = {
+ .k = {
+ 0x79,0xce,0xb0,0x8e, 0xf8,0x7a,0x67,0xc6,
+ 0x48,0x2c,0x2a,0xc0, 0xa5,0x45,0x06,0x49,
+ 0xc8,0x90,0xb8,0xe9, 0xc6,0xb6,0xb3,0x50,
+ 0xbd,0x9e,0x46,0x56, 0x26,0xf2,0xb0,0x3b,
+ },
+ .tlen = 17,
+ .t = {
+ 0xe6,0x93,0xbe,0x89, 0xf5,0xee,0x40,0xde,
+ 0xf2,0x9c,0xb5,0xec, 0x6a,0x37,0x23,0x46,
+ 0x0e,
+ },
+ .len = 16,
+ .p = {
+ 0x5d,0x83,0x98,0x37, 0xc6,0x33,0x9e,0x7e,
+ 0x59,0xad,0xd2,0x5b, 0x8a,0x3a,0x9d,0x03,
+ },
+ .c = {
+ 0x96,0x23,0x2f,0x7d, 0x52,0xfc,0x98,0x63,
+ 0x98,0xa5,0x8b,0xdf, 0xca,0xbc,0x85,0x2f,
+ },
+ },
+ [6] = {
+ .k = {
+ 0x9f,0xd3,0x36,0xb1, 0x85,0x07,0xdf,0x19,
+ 0x01,0xea,0xf9,0x52, 0x68,0xbf,0xce,0xe7,
+ 0xd0,0x49,0xf3,0xba, 0x58,0xfb,0x87,0x18,
+ 0x9f,0xca,0x24,0xca, 0x61,0xa3,0xf0,0xda,
+ },
+ .tlen = 17,
+ .t = {
+ 0xea,0xc6,0x72,0x5e, 0x66,0xd4,0xc7,0xbd,
+ 0xa1,0x6e,0xab,0x09, 0xb5,0x58,0x39,0xae,
+ 0x40,
+ },
+ .len = 128,
+ .p = {
+ 0xc7,0xd6,0x73,0x65, 0xcb,0xf3,0xf5,0x3e,
+ 0xb9,0xa7,0xbf,0xb1, 0x54,0xcb,0xac,0x01,
+ 0xee,0xb5,0x94,0x17, 0x40,0x92,0xfd,0xad,
+ 0x8f,0xdb,0x27,0x22, 0x3d,0xb1,0x0b,0xf7,
+ 0xa7,0x46,0x70,0xd0, 0x31,0xdb,0xf9,0xdb,
+ 0xb9,0xb9,0x40,0x4a, 0x0a,0xba,0x77,0x6f,
+ 0x35,0x36,0x9e,0xeb, 0x68,0xe2,0x9e,0xd7,
+ 0xef,0xc2,0x5e,0x21, 0x0d,0xb3,0xb0,0x87,
+ 0xd6,0x43,0x35,0x6e, 0x22,0xa0,0xb7,0xec,
+ 0x26,0xe0,0x7d,0x48, 0xf5,0x5d,0x58,0xd3,
+ 0x29,0xb7,0x1f,0x7e, 0xe9,0x5a,0x02,0xa4,
+ 0xb1,0xde,0x10,0x9f, 0xe1,0xa8,0x5e,0x05,
+ 0xb6,0xa2,0x59,0xca, 0x3e,0xbc,0xd1,0x94,
+ 0x09,0x4e,0x1b,0x37, 0x29,0x9c,0x15,0xef,
+ 0x8c,0x72,0x53,0xbe, 0x6f,0x25,0x2c,0x68,
+ 0x88,0x08,0x0c,0x00, 0x80,0x7a,0x85,0x64,
+ },
+ .c = {
+ 0x49,0x36,0x97,0xd2, 0xde,0xa4,0xde,0x92,
+ 0x7d,0x30,0x08,0xc3, 0xd9,0x47,0xd4,0xcb,
+ 0x5b,0x41,0x27,0x2c, 0x06,0xb8,0x2b,0xef,
+ 0x7b,0x57,0x59,0xb7, 0x5b,0x81,0x38,0xb4,
+ 0xd1,0x81,0xb3,0xe8, 0xac,0xf0,0xa0,0x06,
+ 0xcb,0x74,0x31,0x01, 0xe1,0x3d,0xcf,0x6d,
+ 0x57,0xd1,0x65,0xcd, 0xe7,0x33,0x6c,0x03,
+ 0x54,0xf0,0x2c,0x41, 0xb8,0x75,0x07,0x1d,
+ 0x70,0xf0,0x9c,0xbd, 0x8f,0x6b,0xdb,0x76,
+ 0x86,0x5b,0xe0,0xfd, 0xad,0x61,0x7a,0x4c,
+ 0xd6,0xf1,0x85,0x0b, 0xfd,0x0b,0x3a,0x5f,
+ 0xcf,0xfc,0xb0,0x0b, 0x2b,0xc7,0x31,0x07,
+ 0x9d,0x75,0x82,0xd9, 0x14,0xd4,0x33,0xd3,
+ 0xff,0x20,0xf7,0x14, 0xcf,0xe4,0xda,0xca,
+ 0x11,0xcc,0x57,0x8f, 0x51,0x52,0x9d,0x90,
+ 0x01,0xc8,0x4e,0x1f, 0x2a,0x89,0xe2,0x52,
+ },
+ },
+ };
+ static struct adiantum A;
+ static uint8_t buf[4096];
+ unsigned i;
+ int result = 0;
+
+ for (i = 0; i < __arraycount(C); i++) {
+ adiantum_init(&A, C[i].k);
+ adiantum_enc(buf, C[i].p, C[i].len, C[i].t, C[i].tlen, &A);
+ if (memcmp(buf, C[i].c, C[i].len)) {
+ char prefix[16];
+ snprintf(prefix, sizeof prefix, "adiantum enc %u", i);
+ hexdump(printf, prefix, buf, C[i].len);
+ result = -1;
+ }
+ memset(buf, 0, sizeof buf); /* paranoia */
+ adiantum_dec(buf, C[i].c, C[i].len, C[i].t, C[i].tlen, &A);
+ if (memcmp(buf, C[i].p, C[i].len)) {
+ char prefix[16];
+ snprintf(prefix, sizeof prefix, "adiantum dec %u", i);
+ hexdump(printf, prefix, buf, C[i].len);
+ result = -1;
+ }
+ }
+
+ return result;
+}
diff -r 36794fee0d04 -r 9fde04e138c1 sys/crypto/adiantum/files.adiantum
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/sys/crypto/adiantum/files.adiantum Wed Jun 17 02:47:43 2020 +0000
@@ -0,0 +1,6 @@
+# $NetBSD$
+
+define adiantum
+
+file crypto/adiantum/adiantum.c adiantum
+file crypto/adiantum/adiantum_selftest.c adiantum
diff -r 36794fee0d04 -r 9fde04e138c1 sys/dev/cgd_crypto.c
--- a/sys/dev/cgd_crypto.c Mon Jun 15 22:55:59 2020 +0000
+++ b/sys/dev/cgd_crypto.c Wed Jun 17 02:47:43 2020 +0000
@@ -45,6 +45,7 @@
#include <dev/cgd_crypto.h>
+#include <crypto/adiantum/adiantum.h>
#include <crypto/aes/aes.h>
#include <crypto/blowfish/blowfish.h>
#include <crypto/des/des.h>
@@ -72,6 +73,10 @@ static cfunc_init cgd_cipher_bf_init;
static cfunc_destroy cgd_cipher_bf_destroy;
static cfunc_cipher cgd_cipher_bf_cbc;
+static cfunc_init cgd_cipher_adiantum_init;
+static cfunc_destroy cgd_cipher_adiantum_destroy;
+static cfunc_cipher cgd_cipher_adiantum_crypt;
+
static const struct cryptfuncs cf[] = {
{
.cf_name = "aes-xts",
@@ -97,6 +102,12 @@ static const struct cryptfuncs cf[] = {
.cf_destroy = cgd_cipher_bf_destroy,
.cf_cipher = cgd_cipher_bf_cbc,
},
+ {
+ .cf_name = "adiantum",
+ .cf_init = cgd_cipher_adiantum_init,
+ .cf_destroy = cgd_cipher_adiantum_destroy,
+ .cf_cipher = cgd_cipher_adiantum_crypt,
+ },
};
const struct cryptfuncs *
cryptfuncs_find(const char *alg)
@@ -409,3 +420,61 @@ cgd_cipher_bf_cbc(void *privdata, void *
panic("%s: unrecognised direction %d", __func__, dir);
}
}
+
+/*
+ * Adiantum
+ */
+
+static void *
+cgd_cipher_adiantum_init(size_t keylen, const void *key, size_t *blocksize)
+{
+ struct adiantum *A;
+
+ if (!blocksize)
+ return NULL;
+ if (keylen != 256)
+ return NULL;
+ if (*blocksize == (size_t)-1)
+ *blocksize = 128;
+ if (*blocksize != 128)
+ return NULL;
+
+ A = kmem_zalloc(sizeof(*A), KM_SLEEP);
+ adiantum_init(A, key);
+
+ return A;
+}
+
+static void
+cgd_cipher_adiantum_destroy(void *cookie)
+{
+ struct adiantum *A = cookie;
+
+ explicit_memset(A, 0, sizeof(*A));
+ kmem_free(A, sizeof(*A));
+}
+
+static void
+cgd_cipher_adiantum_crypt(void *cookie, void *dst, const void *src,
+ size_t nbytes, const void *blkno, int dir)
+{
+ /*
+ * Treat the block number as a 128-bit block. This is more
+ * than twice as big as the largest number of reasonable
+ * blocks, but it doesn't hurt (it would be rounded up to a
+ * 128-bit input anyway).
+ */
+ const unsigned tweaklen = 16;
+ struct adiantum *A = cookie;
+
+ switch (dir) {
+ case CGD_CIPHER_ENCRYPT:
+ adiantum_enc(dst, src, nbytes, blkno, tweaklen, A);
+ break;
+ case CGD_CIPHER_DECRYPT:
+ adiantum_dec(dst, src, nbytes, blkno, tweaklen, A);
+ break;
+ default:
+ panic("%s: unrecognised direction %d", __func__, dir);
+ }
+}
diff -r 36794fee0d04 -r 9fde04e138c1 sys/rump/kern/lib/libcrypto/Makefile
--- a/sys/rump/kern/lib/libcrypto/Makefile Mon Jun 15 22:55:59 2020 +0000
+++ b/sys/rump/kern/lib/libcrypto/Makefile Wed Jun 17 02:47:43 2020 +0000
@@ -1,7 +1,8 @@
# $NetBSD: Makefile,v 1.6 2019/12/05 03:57:55 riastradh Exp $
#
-.PATH: ${.CURDIR}/../../../../crypto/aes \
+.PATH: ${.CURDIR}/../../../../crypto/adiantum \
+ ${.CURDIR}/../../../../crypto/aes \
${.CURDIR}/../../../../crypto/blowfish \
${.CURDIR}/../../../../crypto/camellia \
${.CURDIR}/../../../../crypto/cast128 \
@@ -11,6 +12,10 @@
LIB= rumpkern_crypto
COMMENT=Cryptographic routines
+# Adiantum
+SRCS+= adiantum.c
+SRCS+= adiantum_selftest.c
+
# blowfish
SRCS+= bf_ecb.c bf_enc.c bf_cbc.c bf_skey.c bf_module.c
Home |
Main Index |
Thread Index |
Old Index