Subject: Re: pkg/20340: jwhois' config doesn't include .org
To: Wolfgang S. Rupprecht <wolfgang+gnus20030214T100936@wsrcc.com>
From: Greg A. Woods <woods@weird.com>
List: netbsd-bugs
Date: 02/14/2003 15:06:49
[ On , February 14, 2003 at 10:14:00 (-0800), Wolfgang S. Rupprecht wrote: ]
> Subject: Re: pkg/20340: jwhois' config doesn't include .org
>
> 
> > >How-To-Repeat:
> > look up any .org domain
> > >Fix:
> > upgrade to a later version
> 
> I dont' believe a newer version exists

A newer version of Jwhois does indeed exist, primarily for this very reason.

> but this should fix the *.org
> problem.  

No, not that it really helps.  For example this query has been sitting
there for almost twenty minutes with not a peep:

	$ jwhois NetBSD.org
	[Querying whois.publicinterestregistry.net]

They have a bug in their server -- it never disconnects!  If you telnet
to it directly then you'll see the results quite quickly but it'll sit
there forever (well I've waited over an hour once).  This causes jwhois
to appear to hang since it won't show any output until after it's got it
all (since it might be looking for redirects and such).

Even worse though all of their data is bogus right now and you really do
need to catch the redirect strings (which your patch did not provide):

	"whois\\.publicinterestregistry\\.net" {
		whois-redirect = ".*Whois Server:\\(.*\\)";
	}


The following patch will also apparently be necessary to deal with
results from that server, but it is not sufficient to deal with the
no-disconnect bug:

Index: src/lookup.c
===================================================================
RCS file: /cvsroot/jwhois/jwhois/src/lookup.c,v
retrieving revision 1.33
retrieving revision 1.34
diff -c -r1.33 -r1.34
*** src/lookup.c	2 Aug 2002 22:17:41 -0000	1.33
--- src/lookup.c	4 Feb 2003 21:21:16 -0000	1.34
***************
*** 391,397 ****
  	{
  	  memcpy(bptr, text, strlen(text)+1);
  
! 	  strptr = (char *)strtok(bptr, "\n");
  	  while (strptr)
  	    {
  	      rpb.allocated = 0;
--- 391,397 ----
  	{
  	  memcpy(bptr, text, strlen(text)+1);
  
! 	  strptr = (char *)strtok(bptr, "\r\n");
  	  while (strptr)
  	    {
  	      rpb.allocated = 0;
***************
*** 443,449 ****
  		}
  	      else if (ind == -2)
  		return -1;
! 	      strptr = (char *)strtok(NULL, "\n");
  	    }
  	}
      }
--- 443,449 ----
  		}
  	      else if (ind == -2)
  		return -1;
! 	      strptr = (char *)strtok(NULL, "\r\n");
  	    }
  	}
      }


> As a freebee it also fixes the problem with the NET-* tags
> that ARIN uses not being found.

Even the newest version from their public CVS server needs a few tweaks
to really be useful, especially for getting default English output where
possible.  Here's my diff:


Index: example/jwhois.conf
===================================================================
RCS file: /cvsroot/jwhois/jwhois/example/jwhois.conf,v
retrieving revision 1.97
diff -c -r1.97 jwhois.conf
*** example/jwhois.conf	14 Feb 2003 17:18:59 -0000	1.97
--- example/jwhois.conf	14 Feb 2003 19:42:13 -0000
***************
*** 1,4 ****
! # This is jwhois.conf - an example configuration file for jwhois
  #
  # Due to the ever-changing nature of the Internet, the configuration file
  # released with Jwhois quickly gets outdated. You can find the latest version
--- 1,4 ----
! # This is jwhois.conf - the configuration file for jwhois
  #
  # Due to the ever-changing nature of the Internet, the configuration file
  # released with Jwhois quickly gets outdated. You can find the latest version
***************
*** 26,31 ****
--- 26,32 ----
  	"^CORE-[0-9]+$" = "struct handles";
  	"^CO[CH]O-[0-9]+$" = "struct handles";
  	".*-[A-Z]+$" = "struct handles";
+ 	"^[^\.]*[A-Z]+$" = "struct handles";
  
  	#
  	# Catch AS numbers
***************
*** 45,50 ****
--- 46,56 ----
  	"^!?NET\\(BLK\\)?\\(-[A-Z0-9]+\\)+$" = "whois.arin.net";
  
  	#
+ 	# Catch Verio netblock records
+ 	#
+ 	"^!?NET\\(BLK\\)?-VRIO\\(-[A-Z0-9]+\\)+$" = "rwhois.verio.net";
+ 
+ 	#
  	# Redirect IPv6 numbers to the correct host
  	#
  	".*:.*" = "whois.6bone.net";
***************
*** 101,106 ****
--- 107,113 ----
  	"\\.org\\.au$" = "whois.ausregistry.net.au";
  	"\\.au$" = "whois.aunic.net";
  	"\\.az$" = "whois.ripe.net";
+ 	"\\.ba$" = "whois.ripe.net";
  	"\\.be$" = "whois.dns.be";
  	"\\.bg$" = "whois.ripe.net";
  	"\\.bi$" = "www.nic.bi";
***************
*** 179,184 ****
--- 186,192 ----
  	"\\.lv$" = "whois.nic.lv";
  	"\\.ma$" = "whois.ripe.net";
  	"\\.mil$" = "whois.nic.mil";
+ 	"\\.mk$" = "whois.ripe.net";
  	"\\.mm$" = "whois.nic.mm";
  	"\\.ms$" = "whois.adamsnames.tc";
  	"\\.mt$" = "www.um.edu.mt";
***************
*** 208,214 ****
  	"\\.ru$" = "whois.ripn.net";
  	"\\.rw$" = "www.nic.rw";
  	"\\.sa$" = "saudinic.net.sa";
! 	"\\.se$" = "www.nic-se.se"; # "whois.nic-se.se";
  	"\\.sg$" = "whois.nic.net.sg";
  	"\\.sh$" = "whois.nic.sh";
  	"\\.si$" = "whois.arnes.si";
--- 216,222 ----
  	"\\.ru$" = "whois.ripn.net";
  	"\\.rw$" = "www.nic.rw";
  	"\\.sa$" = "saudinic.net.sa";
! 	"\\.se$" = "whois.nic-se.se";
  	"\\.sg$" = "whois.nic.net.sg";
  	"\\.sh$" = "whois.nic.sh";
  	"\\.si$" = "whois.arnes.si";
***************
*** 249,254 ****
--- 257,264 ----
  	"\\.vu$" = "www.vunic.vu";
  	"\\.ws$" = "whois.worldsite.ws";
  	"\\.yu$" = "whois.ripe.net";
+ 	"\\.gb.com$" = "whois.centralnic.net";
+ 	"\\.gb.net$" = "whois.centralnic.net";
  	"\\.ac\\.za$" = "whois.ac.za";
  	"\\.org\\.za$" = "rwhois.org.za 4321";
  	"\\.co\\.za$" = "whois.co.za";
***************
*** 307,316 ****
  	"193.0.0.0/8" = "whois.ripe.net";
  	"194.0.0.0/8" = "whois.ripe.net";
  	"195.0.0.0/8" = "whois.ripe.net";
- 	"200.128.0.0/9" = "whois.nic.br";
  	"200.0.0.0/8" = "whois.lacnic.net";
  	"201.0.0.0/8" = "whois.lacnic.net";
! 	"202.0.0.0/7" = "whois.apnic.net";
  	"203.0.0.0/8" = "whois.apnic.net";
  	"210.0.0.0/8" = "whois.apnic.net";
  	"211.0.0.0/8" = "whois.apnic.net";
--- 317,326 ----
  	"193.0.0.0/8" = "whois.ripe.net";
  	"194.0.0.0/8" = "whois.ripe.net";
  	"195.0.0.0/8" = "whois.ripe.net";
  	"200.0.0.0/8" = "whois.lacnic.net";
+ 	"200.128.0.0/9" = "whois.nic.br";
  	"201.0.0.0/8" = "whois.lacnic.net";
! 	"202.0.0.0/8" = "whois.apnic.net";
  	"203.0.0.0/8" = "whois.apnic.net";
  	"210.0.0.0/8" = "whois.apnic.net";
  	"211.0.0.0/8" = "whois.apnic.net";
***************
*** 324,329 ****
--- 334,346 ----
  	"222.0.0.0/8" = "whois.apnic.net";
  	"223.0.0.0/8" = "whois.apnic.net";
  
+ 	# HANANET
+ 	# whois.apnic.net doesn't really have a usable redirect string....
+ 	"218.232.0.0/16" = "whois.nic.or.kr";
+ 	"218.233.0.0/16" = "whois.nic.or.kr";
+ 	"218.234.0.0/16" = "whois.nic.or.kr";
+ 
+ 
  # Note: JPNIC doesn't appear to publish lists of netblocks on their
  #       website - this data was derived by searching for "Japan Network
  #       Information Center" in the ARIN and APNIC databases.  It may
***************
*** 398,403 ****
--- 415,429 ----
  	"211.192.0.0/13" = "whois.nic.or.kr";
  
  	default = "whois.arin.net";
+ 	#
+ 	# NOTE:  I'd rather just do this instead, but you're currently
+ 	# not allowed to specify a block of values unless the key is
+ 	# of type=regex.
+ 	#
+ 	#default {
+ 	#	whois-server = "whois.arin.net";
+ 	#	query-format = "+$*";		# expand all records
+ 	#}
  }
  
  #
***************
*** 421,428 ****
  	".*-NORID$" = "whois.norid.no";
  	".*-GANDI$" = "whois.gandi.net";
  	".*-AU$" = "whois.aunic.net";
! 	".*-CKNIC" = "whois.nic.ck";
  	".*-IDNIC$" = "whois.idnic.net.id";
  	".*-KG$" = "whois.domain.kg";
  	".*-NICAT$" = "whois.nic.at";
  	".*-CZ$" = "whois.nic.cz";
--- 447,458 ----
  	".*-NORID$" = "whois.norid.no";
  	".*-GANDI$" = "whois.gandi.net";
  	".*-AU$" = "whois.aunic.net";
! 	".*-CKNIC$" = "whois.nic.ck";
  	".*-IDNIC$" = "whois.idnic.net.id";
+ 	"^[^\.]+JP$" {
+ 		whois-server = "whois.nic.ad.jp";
+  		query-format = "$* /e";
+ 	}
  	".*-KG$" = "whois.domain.kg";
  	".*-NICAT$" = "whois.nic.at";
  	".*-CZ$" = "whois.nic.cz";
***************
*** 451,456 ****
--- 481,490 ----
  	"rwhois\\.exodus\\.net" {
  		rwhois = true;
  	}
+ 	"rwhois\\.intermedia\\.com\\.ar" {
+ 		rwhois = true;
+ 		rwhois-display = "dump";
+ 	}
  	"whois\\.publicinterestregistry\\.net" {
  		whois-redirect = ".*Whois Server:\\(.*\\)";
  	}
***************
*** 470,475 ****
--- 504,512 ----
  	"whois\\.apnic\\.net" {
  		whois-redirect = ".*http://\\(whois\\.nic\\.or\\.kr\\)/";
  		whois-redirect = ".*at \\([Ww][Hh][Oo][Ii][Ss]\\.[A-Za-z]*\\.[Nn][Ee][Tt]\\)";
+ 		whois-redirect = ".* whois server at \\([Ww][Hh][Oo][Ii][Ss]\\.[^ ]*\\) ";
+ 		whois-redirect = "^remarks:[ 	]*http://\\([^/]*\\)";
+ 		whois-redirect = "^descr:[ 	]*\"http://\\([^/]*\\)";
  	}
  
  	"whois\\.arin\\.net" {
***************
*** 477,486 ****
  		# Content redirection for whois.arin.net - allows redirection
  		# of European and Asia-Pacific addresses to the appropriate
  		# servers.
! 		#
! 		whois-redirect = ".*at \\([Ww][Hh][Oo][Ii][Ss]\\.[A-Za-z]*\\.[Nn][Ee][Tt]\\)";
! 		whois-redirect = ".* \\([Rr]+[Ww][Hh][Oo][Ii][Ss]\\.[A-Za-z]*\\.[Nn][Ee][Tt]\\) \\([0-9]*\\)";
! 		whois-redirect = ".* rwhois://\\(.*\\):\\([0-9]*\\)/";
  	}
  
  	"whois\\.ncst\\.ernet\\.in" {
--- 514,529 ----
  		# Content redirection for whois.arin.net - allows redirection
  		# of European and Asia-Pacific addresses to the appropriate
  		# servers.
! 		#	
! 		whois-redirect = ".*[ 	:]\\([Ww][Hh][Oo][Ii][Ss]\\.[A-Za-z]*\\.[Nn][Ee][Tt]\\)";
! 	        whois-redirect = ".*[ 	:]\\([Rr]+[Ww][Hh][Oo][Ii][Ss]\\.[A-Za-z]*\\.[Nn][Ee][Tt]\\) \\([0-9]*\\)";
! 		whois-redirect = ".*[ 	:]rwhois://\\(.*\\):\\([0-9]*\\)/";
! 		# 
! 		# NOTE: I'd rather just do this for CIDR blocks, but
! 		# you're not currently allowed to specify a block of
! 		# settings if the key is of type=cidr.
! 		# 
!  		query-format = "+$*";		# expand all records
  	}
  
  	"whois\\.ncst\\.ernet\\.in" {
***************
*** 490,495 ****
--- 533,542 ----
  		query-format = "domain $*";
  	}
  
+ 	"whois\\.nic\\.ad\\.jp" {
+  		query-format = "$* /e";
+ 	}
+ 
  	".*\\.connect\\.com\\.au" {
  		#
  		# Referals from the net.au whois server.
***************
*** 504,509 ****
--- 551,564 ----
  		form-element = "Domain";			# <input> element
  	}
  
+ 	"www\\.dk-hostmaster\\.dk" {
+ 		http = "true";
+ 		http-method = "GET";
+ 		http-action = "/script/whois.php";
+ 		form-element = "query";
+ 		form-extra = "lang=eng";
+ 	}
+ 
  	"www\\.nic-se\\.se" {
  		http = "true";
  		http-method = "GET";
***************
*** 578,584 ****
  	}
  
  	"www\\.nic\\.kz" {
! 		# whois.nic.kz exists but wasn't checkable
  		http = "true";
  		http-method = "GET";
  		http-action = "/cgi-bin/whois";
--- 633,639 ----
  	}
  
  	"www\\.nic\\.kz" {
! 		# whois.nic.kz exists but refuses connections on port 43
  		http = "true";
  		http-method = "GET";
  		http-action = "/cgi-bin/whois";
***************
*** 679,684 ****
--- 734,741 ----
  
  	"whois\\.lacnic\\.net" {
  		whois-redirect = ".* \\([Rr]+[Ww][Hh][Oo][Ii][Ss]\\.[A-Za-z]*\\.[Nn][Ee][Tt]\\) port \\([0-9]*\\)";
+ 		whois-redirect = ".*reassignment information for this block is available at: \\([^ ]*\\) port \\([0-9]*\\)";
+ 		whois-redirect = ".*reassignment information for this block is available at: \\([^ ]*\\)$";
  	}
  
  	"www\\.nic\\.im" {
***************
*** 691,700 ****
  
  #
  # The cache feature is used to provide a local cache with Whois information.
! # By default, the location of the database is /usr/local/var/jwhois.db and the
  # default expire time is 7 days, but you can change those below.
  #
! #cachefile = "/var/lib/jwhois.db";
  
  #
  # This is the time after which an entry expires from the cache in hours.
--- 748,757 ----
  
  #
  # The cache feature is used to provide a local cache with Whois information.
! # By default, the location of the database is /var/db/jwhois.db and the
  # default expire time is 7 days, but you can change those below.
  #
! #cachefile = "/var/db/jwhois.db";
  
  #
  # This is the time after which an entry expires from the cache in hours.
***************
*** 710,716 ****
  #
  # Path to the browser to use for HTTP servers.
  #
! browser-pathname = "/usr/bin/lynx";
  
  #
  # Command line argument to browser to get output on stdout.
--- 767,775 ----
  #
  # Path to the browser to use for HTTP servers.
  #
! # XXX this should really just use $PATH and be just the command-name.
! #
! browser-pathname = "/usr/pkg/bin/lynx";
  
  #
  # Command line argument to browser to get output on stdout.


-- 
								Greg A. Woods

+1 416 218-0098;            <g.a.woods@ieee.org>;           <woods@robohack.ca>
Planix, Inc. <woods@planix.com>; VE3TCP; Secrets of the Weird <woods@weird.com>