I was playing with rsh and I see that it runs a command locally if it
doesn't connect.

I don't use rsh.
 Trying 127.0.0.1...
 rlogin: rcmd: connect to address ::1
 Connection refused

Here is my example -- it doesn't connect, gives an error, and then
executes "ls":

rainier:~$ rsh localhost ls /
stty: TIOCGETD: Operation not supported
altroot
bin
boot
dev
emul
etc
etc.new
home
kern

...

Is this normal behaviour (to run a command even though it doesn't
connect)? (It doesn't run when it fails to connect with "Connection
refused".)

Why this error "stty: TIOCGETD: Operation not supported"?

   Jeremy C. Reed
   http://www.reedmedia.net/