Is the intent that user and kernel should be able to use different pointer sizes? I think yes, at least in the MIPS case you can have a 64 bit kernel running 32 bit user code.
One place this breaks is ptrace with PT_IO opcode, which passes a ptrace_io_desc struct that has pointers in it. Replacing those by uint64 cures that issue. It also fixes the problem of trying to debug a 64 bit process with a 32 bit gdb, which otherwise would work.