Re: Something better than www/urlget ?

[coypu%SDF.ORG@localhost]

On Wed, Jul 13, 2016 at 10:07:46AM +0800, Paul Goyette wrote:
> I've been using urlget for quite some time (to retrieve financial info from
> finance.yahoo.com) and it has worked reasonably well.  However, some changes
> on the source site have resulted in less-than-acceptable behavior!
> 
> Aside from the fact that urlget won't follow redirects (something to which I
> can manually adapt), the source site is now using javascript to present the
> required info.  And I've searched high and low for any alternative sources,
> without success.
> 
> So, I'm wondering if there's anything out there that can help me to retrieve
> the necessary data.
> 
> If anyone cares, the specific data I'm interested in is the Trailing and
> Forward Annual Dividend Rate data shown on the "Statistics" page for a
> particular stock.  A typical URL would be
> 
> 	http://finance.yahoo.com/quote/AAPL/key-statistics
> 
> Thanks in advance for any clues, pointers, suggestions, etc.
> 
> 
> 
> +------------------+--------------------------+------------------------+
> | Paul Goyette     | PGP Key fingerprint:     | E-mail addresses:      |
> | (Retired)        | FA29 0E3B 35AF E8AE 6651 | paul at whooppee.com   |
> | Kernel Developer | 0786 F758 55DE 53BA 7731 | pgoyette at netbsd.org |
> +------------------+--------------------------+------------------------+

Hi, you'll be able to continue getting data by using an alternate user
agent. try e.g. `curl -A 'Lynx'`.