Subject: bootstrap-pkgsrc on QNX Neutrino RTOS 6.2.1
To: None <tech-pkg@netbsd.org>
From: Michal Pasternak <michal@pasternak.w.lub.pl>
List: tech-pkg
Date: 02/06/2004 20:50:07
Hello,

I am able to compile bootstrap-pkgsrc on QNX Neutrino OS, but there are some
issues. I need your help & advice. 

"QNX" in the below document means "QNX Neutrino OS 6.2.1"

I am not really sure if I want to PR this, because this is quite specific
task, and there are still problems & issues. Would any of pkgsrc developers
like to cooperate to get this running? As usual, I offer my time, my CPU
time, and an account on a QNX box. Some trival things could be updated at
once, some non-trival things need some more time.

Please note, that almost everything mentioned below are BSDisms. QNX has
very good POSIX support. It can run, among other applications, things as
large as Mozilla or X11 (in a variety of ways), and things as useful as
PostgreSQL, MySQL, Python, PHP, Gimp, Emacs, gcc and many other.

QNX currently doesn't seem to have any building system available. It has
wonderful binary package installer, but I'd like to run pkgsrc on it.

bootstrap root & shell script
=============================
QNX has pax and sed, needs mtree and bsd-install.

QNX doesn't have whoami(1) and groups(1) utilities. Needed information could
be extracted from id(1) using sed. Any regexp-master would like to help?
Until then, non-root buildling of bootstrap-pkgsrc is disabled, but I don't
consider this to be a really bad thing. Please check the patch.

QNX doesn't have patch(1) program, which will be needed later. What about
importing patch to bootstrap-pkgsrc and making it optional for systems, which
don't have it? GNU patch compiles & works okay. I belive this part could be
done at once.

Patches: http://pasternak.w.lub.pl/bootstrap.diff

Current README.QNX: http://pasternak.w.lub.pl/README.QNX

bmake
=====
Pretty trival. <errno.h> and <signal.h> instead of <sys/errno.h> and
<sys/signal.h>. Don't define __BEGIN_DECLS and __END_DECLS in
missing/sys/cdefs.h, because bmake build will fail at the 2nd phase (when
-Werror is specified).

Patches: http://pasternak.w.lub.pl/bmake.diffs

libnbcompat
===========
Fun starts here. Really.

dirfd() is hardly possible to be emulated on QNX. See notes in the patch.
The problem is basically there is not a 1:1 mapping of DIR to fd's - a
directory actually may point to multiple fd's. Such situation occurs, when
QNX directory is an union directory (similar functionality like mount_union
on BSDs). In this case, DIR structure could "resolve" to one or more file
descriptors.

QNX has no fchdir(2) call. Same reasons, as above. As I have been told on
#qnx@freenode.net, it is "unreliable" to get the filename, basing on the
filedescriptor (I thought about emulating fchdir via
chdir(getFileNameOfTheFileDescriptor(fd)), but just have a look at lsof
sources, there doesn't seem to be a portable way to do this).

Solutions? Well, currently only fts emulation uses fchdir(2). fchdir(2)
doesn't seem to be a big problem, programs which can use it, also have code,
which will work when fchdir(2) is absent (chat/epic4, editors/vim).

fts... QNX has ftw. ftw is POSIX. fts is BSD.
You have ftw listed as TODO for example here: http://www.freebsd.org/projects/c99/ .

QNX fts emulation could be done as replacing fchdir(2) calls with just
chdir(2), but there will be problems if an application using fts will
explictly want to use file descriptors from the structure. Any ideas?

Anyway, read mtree comments. There are still things left to implement in
nbcompat.

Lame patches: http://pasternak.w.lub.pl/libnbcompat.lame-diff

tnftp
=====
progressbar.c wants SA_RESTART signal, which is declared in QNX's
<signal.h>, only commented out -- it is not currently implemented.

Any chances of getting tnftp to work without this?

You can still use other programs for downloading sources. On my FreeBSD box
I use fetch(1) from FreeBSD basesystem, it works quite okay. For QNX box, I
used GNU wget (3rd-party binary package), which als works. I have no idea
about tnftp internals, but as the file is called "progressbar.c", well,
maybe we could get along without progressbars for FTP downloads on some
architectures? :)

Lame patch: http://pasternak.w.lub.pl/tnftp.lame-diff

mtree
=====
pack_dev.c is kind of mystery to me.

makedev macro on QNX wants 3 arguments: node, major, minor. As pack_dev.c
seems to be quite portable, could anyone understanding what's going on there
help me with this?

... and here's the rest of functions, that are missing. Except fts (discussed
earlier) - and encryption, other parts seem to be easy to implement in
libnbcompat.

mtree/compare.c:276: undefined reference to lchmod'
mtree/compare.c:404: undefined reference to MD5File'
mtree/compare.c:421: undefined reference to RMD160File'
mtree/compare.c:438: undefined reference to SHA1File'

	Those encryption functions are really strange. compare.c has
	#ifdefs for defines like NO_MD5, NO_SHA1, but there are no
	entries in config.h or checks in configure for them. Is the
	configure process broken? Seems so.

mtree/create.c:142: undefined reference to fts_open'
mtree/create.c:146: undefined reference to fts_set'
mtree/create.c:171: undefined reference to fts_read'
mtree/create.c:172: undefined reference to fts_close'

	Discussed earlier.

mtree/create.c:199: undefined reference to user_from_uid'
mtree/create.c:206: undefined  reference to group_from_gid'
mtree/excludes.c:102: undefined reference to fparseln'

	Should be relativley easy to add them to libnbcompat,
	I can handle this.

mtree/mtree.c:91: undefined reference to setprogname'
mtree/mtree.c:237: undefined reference to getprogname'

	Why are those used / important?
	
	BTW, FYI: some /usr/share/examples/ .c source on NetBSD suggest using
	setprogname, because it somehow magically "raises portability" of a
	specific application (if I recall this properly). As you can see,
	not always :)

mtree/spec.c:197: undefined reference to strunvis'
mtree/spec.c:524: undefined reference to setmode'
mtree/spec.c:526: undefined reference to getmode'
mtree/stat_flags.c:108: undefined reference to strlcpy'

	I can handle them, propably.

mtree/getid.c:159: undefined reference to pwcache_groupdb'
mtree/getid.c:159: undefined reference to pwcache_userdb'

	Those are mystery. Why are they used?
	
	QNX has getpwent(), setpwent() and other pwent functions.

Patches: currently none.

pkgsrc itself
=============
Well, without mtree I can only see some packages compile, not install.

QNX has gcc-2.95.3, but it is somehow not discovered by pkgsrc. I suppose,
that it is because of version suffix:

	# uname -a
	QNX qnx 6.2.1 2003/01/08-14:50:46est x86pc x86 
	# gcc -dumpversion
	2.95.3qnx-nto
	# gcc -v
	Reading specs from /usr/lib/gcc-lib/ntox86/2.95.3qnx-nto/specs
	gcc version 2.95.3qnx-nto 20010315 (release)

I have modified _GCC2_PATTERNS in mk/compiler/gcc.mk, so they included
string "2.95.[0-3].*", but it did not help much. I guess, that it needs
changes somewhere else, but I have not had time to investigate. Anyone? I
guess this will be a piece of cake for senior pkgsrc developer ;)

dist and defs files (taken from FreeBSD dist and defs, so they might not be
exactly accurate at some points, anyway the defs.QNX is good until "TODO" line):

	http://pasternak.w.lub.pl/defs.QNX.mk
	http://pasternak.w.lub.pl/QNX.pkg.mk


Conclusions
===========
It is possible to port pkgsrc to QNX. In fact, I was able to do some
basic compilations of utilities from "pkgtools" directory. I was unable to
install them, because of lack of mtree(1).

Where do I need your help?
==========================
1) tnftp/progressbar.c
2) makedev in mtree/pack_dev.c
3) please import patch to othersrc/bootstrap-pkgsrc
4) please help with regexp for extracting information from id
5) please review mtree's configure for encryption stuff, mentioned above
6) gcc-2.95.3qnx-nto should be detected properly

Take care,
-- 
Michal Pasternak :: http://pasternak.w.lub.pl :: http://winsrc.sf.net
::don't use MySQL::details on http://sql-info.de/mysql/gotchas.html::