Sabtu, 24 Februari 2018

[smf_addin] Digest Number 4307

15 Messages

Digest #4307
1b
Re: Problem with smfgetCSVFile by "Randy Harmelink" rharmelink
2a
Re: smfGetYahooHistory Issue by "Randy Harmelink" rharmelink
2c
Re: smfGetYahooHistory Issue by "Randy Harmelink" rharmelink
3a
smfgetCSVfile by navelhunt
3b
Re: smfgetCSVfile by "Randy Harmelink" rharmelink
4b
Re: SMFgetoptionstrikes data source? by "Randy Harmelink" rharmelink
4d
Re: SMFgetoptionstrikes data source? by "Randy Harmelink" rharmelink
4f
Re: SMFgetoptionstrikes data source? by "Randy Harmelink" rharmelink
5b
Re: smfGetYahooHistory vs RCHGetYahooHistory by "Randy Harmelink" rharmelink

Messages

Tue Feb 20, 2018 11:20 am (PST) . Posted by:

navelhunt

Randy,
I am using this function to retrieve comma delimited web page information and have set up a number of arrays in my excel spreadsheet that each link to different webpages.
However i run into a weird situation where this function sometimes stops working, and the cells in a particular array show blanks, even though the webpage and link are functional. The only way to solve this problem is to select all the cells in that particular array, and copy paste them into a brand new excel sheet - where the link reestablishes and the numbers are displayed again. I can then copy the whole lot back to the original place.


I wonder if this has anything to do with the function and perhaps the fact that it somehow uses the Yahoo Finance website (just looking through VBA code) - and given that Yahoo Finance has recently changed. I didnt have this problem previously (i.e. like 6-12 months ago).
Would you have any insight into or experience with this? Many thanks

Tue Feb 20, 2018 12:25 pm (PST) . Posted by:

"Randy Harmelink" rharmelink

Hmm. Is this always for a given source of data? Each time that function is
run, there is a new request for the data. The simplest explanation is that
the file isn't getting retrieved properly from time to time, which would
result in all blanks. Then, the web page and link may appear to be
functional, because they are just new requests, and as you say, a new
request using the add-in also does get the data.

On Tue, Feb 20, 2018 at 12:20 PM, navelhunt@
​...
wrote:

> I am using this function to retrieve comma delimited web page information
> and have set up a number of arrays in my excel spreadsheet that each link
> to different webpages.
>
> However i run into a weird situation where this function sometimes stops
> working, and the cells in a particular array show blanks, even though the
> webpage and link are functional. The only way to solve this problem is to
> select all the cells in that particular array, and copy paste them into a
> brand new excel sheet - where the link reestablishes and the numbers are
> displayed again. I can then copy the whole lot back to the original place.
>
> I wonder if this has anything to do with the function and perhaps the fact
> that it somehow uses the Yahoo Finance website (just looking through VBA
> code) - and given that Yahoo Finance has recently changed. I didnt have
> this problem previously (i.e. like 6-12 months ago).
>
> Would you have any insight into or experience with this? Many thanks
>

Tue Feb 20, 2018 2:04 pm (PST) . Posted by:

"Randy Harmelink" rharmelink

I disagree. Yahoo does show data on the website for both 1/1/18 and
1/15/18. Using this URL:

https://finance.yahoo.com/quote/SPY/history?period1=1028012400&period2=1518937200&interval=1d&filter=history&frequency=1d

....I see:

17-Jan-18 278.03 280.05 276.97 279.61 279.61 113,258,800
16-Jan-18 279.35 280.09 276.18 276.97 276.97 106,555,100
15-Jan-18 - - - - - -
12-Jan-18 276.42 278.11 276.08 277.92 277.92 90,816,100
11-Jan-18 274.75 276.12 274.56 276.12 276.12 62,361,500
10-Jan-18 273.68 274.42 272.92 274.12 274.12 69,574,300
9-Jan-18 274.4 275.25 274.08 274.54 274.54 57,254,000
8-Jan-18 273.31 274.1 272.98 273.92 273.92 57,319,200
5-Jan-18 272.51 273.56 271.95 273.42 273.42 83,524,000
4-Jan-18 271.2 272.16 270.54 271.61 271.61 80,636,400
3-Jan-18 268.96 270.64 268.96 270.47 270.47 90,070,400
2-Jan-18 267.84 268.81 267.4 268.77 268.77 86,655,700
1-Jan-18 - - - - - -
29-Dec-17 268.53 268.55 266.64 266.86 266.86 96,007,400
28-Dec-17 267.89 267.92 267.45 267.87 267.87 45,116,100
When the JSON data has days like those, it fills in the closing price from
the prior day for O/H/L/C and posts a volume of zero. These types of
missing days used to be a prevalent ETF issue that I had to put a fix in
for. These empty days are triggering that fix.

Hmmm. Appears to be happening randomly. Not sure what is going on. But
since it's toggling on and off in *both *the browser web page and with
smfGetYahooHistory(), I'm calling it a Yahoo glitch. :(

I suspect one or more servers have incorrect data. Maybe they didn't get
updates or fixes (yet?). So it would depend on which server is fulfilling
the data request.

On Sun, Feb 18, 2018 at 1:40 PM, lewglenn@yahoo.com [smf_addin] <
smf_addin@yahoogroups.com> wrote:

>
> I thought I had submitted this topic earlier, but I guess not.
>
> The attached file contains a simple vba code that uses smfGetYahooHistory
> to download the date and adjusted close for TLT from inception and place
> the results on the spreadsheet ReadTest in columns "A" and "B". Next, it
> does the same for SPY (for the same date period) and places the results
> in columns "C" and "D".
>
> All's well until row 3886, at which point the date for TLT is 1/2/2018
> and for SPY is 1/1/2018. Am extra row appears to have been inserted for
> SPY, with the adjusted close data repeated from the previous row (3885). Note
> that the Yahoo website does not display any historical data for SPY on
> 1/1/2018. The same thing happens again on row 3896, where SPY data are
> inserted for 1/15/2018; again, there are not SPY data for this date on
> the Yahoo website.
>
> Can you think of how this might have occurred? I see it happening in other
> instances as well (with other ETFs).
>

Tue Feb 20, 2018 8:32 pm (PST) . Posted by:

lewglenn

Thanks for the update , Randy. I also have found that the occurrences appear to be random and to occur not only with SPY, but with other ETFs as well, e.g., EFA.


I don't understand what a "zero volume" day for SPY means; clearly, there were no transactions for SPY on New Years Day, 2018 but that should be true for all other ETFs as well. As my example showed, when such a day appeared with SPY it did not appear with TLT so I don't know why these days shouldn't be ignored in the download. Of course I can always write a piece of code to do that; this is needed if one is trying to compare valuations between ETFs.



Wed Feb 21, 2018 12:47 am (PST) . Posted by:

"Randy Harmelink" rharmelink

Well, last year, the Yahoo historical data had multiple days of such
entries, on valid trading days:

2017-05-21 smfGetYahooHistory Change null values to zeroes and then forward
fill zeroes as possible
So those days exist in the JSON file as null values. There really is no
good way for me to determine whether they are valid trading days that need
data to be carried forward, or invalid trading days that need to be
deleted. The former used to be the only situation. The latter now is
causing a problem. But without a lot of overhead processing, I have no way
to determine which situation it is.

And, please, don't get me thinking of what else might be wrong with Yahoo
data... :(

At least this appears to be a glitch issue instead of a consistent issue. I
suspect one or two servers with bad data on them that will eventually get
fixed.

On Tue, Feb 20, 2018 at 5:02 PM, lewglenn@
​...
wrote:

>
> Thanks for the update , Randy. I also have found that the occurrences
> appear to be random and to occur not only with SPY, but with other ETFs as
> well, e.g., EFA.
>
> I don't understand what a "zero volume" day for SPY means; clearly, there
> were no transactions for SPY on New Years Day, 2018 but that should be true
> for all other ETFs as well. As my example showed, when such a day appeared
> with SPY it did not appear with TLT so I don't know why these days
> shouldn't be ignored in the download. Of course I can always write a piece
> of code to do that; this is needed if one is trying to compare valuations
> between ETFs.
>

Tue Feb 20, 2018 2:13 pm (PST) . Posted by:

navelhunt

Randy,
I've been having some problems with this functions lately. Have a page which is comma delimited, and which i could previously read out with no problems using the functions, but its not properly working as of this moment. I notice that in the VB code, the function calls the RCHgetYahooQuotes function, which uses the finance.yahoo.com website.
I have been using the function smfgetCSVfile only to read a CSV file but this has nothing to do with Yahoo or Yahoo finance, and i also understand that Yahoo Finance has changed its website recently. Can you explain if this has possibly something to do with it? Is there a way to create a smfgetCSVfile code that would completely avoid Yahoo Finance?


Many thanks,
N

Tue Feb 20, 2018 6:08 pm (PST) . Posted by:

"Randy Harmelink" rharmelink

Not all calls to RCHGetYahooQuotes() call Yahoo web pages. Since that
function was already a simplistic CSV processor, I didn't see a need to
rewrite or copy the code when I created smfGetCSVFile(). I just put a
couple of tweaks in RCHGetYahooQuotes() so that it could process a CSV file
from any source.

What it does is check to see which items the pItems parameter is
requesting. If that parameter is empty, it knows it wasn't a request for
quotes from Yahoo and uses the pTickers parameter as the URL for the CSV
file.

So, for example, if I just pass those two parameters:

=RCHGetYahooQuotes("
http://financials.morningstar.com/ajax/exportKR2CSV.html?t=GOOG","")

....would only use that URL *and **never do anything with Yahoo*.

What URLs are you having problems with?

On Tue, Feb 20, 2018 at 8:58 AM, navelhunt@
​...
wrote:

> I've been having some problems with this functions lately. Have a page
> which is comma delimited, and which i could previously read out with no
> problems using the functions, but its not properly working as of this
> moment. I notice that in the VB code, the function calls the
> RCHgetYahooQuotes function, which uses the finance.yahoo.com website.
>
> I have been using the function smfgetCSVfile only to read a CSV file but
> this has nothing to do with Yahoo or Yahoo finance, and i also understand
> that Yahoo Finance has changed its website recently. Can you explain if
> this has possibly something to do with it? Is there a way to create a
> smfgetCSVfile code that would completely avoid Yahoo Finance?
>
>
>

Thu Feb 22, 2018 11:02 am (PST) . Posted by:

pete_cook2001

SMFgetoptionstrikes stopped working for me recently. I had been using "B" as the data source. I tried 8, G, OX, and Y, and none work. Has there been some change recently?

Thu Feb 22, 2018 11:13 am (PST) . Posted by:

"Randy Harmelink" rharmelink

I just tried:

=smfGetOptionStrikes("MMM";,"2/23/2018","C","B")

....and got:

$215.00
$220.00
$222.50
$225.00
$227.50
$230.00
$232.50
$235.00
$237.50
$240.00
$242.50
$245.00
$247.50
$250.00
$252.50
$255.00
$257.50
Yahoo also worked. But Google and OptionsXpress are now dead as sources of
options data. In fact, OX should return an "Invalid Source" error message
if you try and use it. "8" requires me to login and generate a security
cookie, so I didn't try that one. But right now, "G" and "8" return "None"
after timing out.

But I am running with the most recent release of the add-in. Looks like the
last changes to that function were in October? If you didn't get "Invalid
Source" for OX, you are probably running an old version of the add-in.

P.S. Yahoo has been kind of wonky today. Missing entire blocks of
expirations and such.

On Thu, Feb 22, 2018 at 12:00 PM, peterscottcook@
​...
wrote:

> SMFgetoptionstrikes stopped working for me recently. I had been using "B"
> as the data source. I tried 8, G, OX, and Y, and none work. Has there been
> some change recently?
>

Thu Feb 22, 2018 11:34 am (PST) . Posted by:

pete_cook2001

Randy, I just realized it works with some stocks and not others.

Can you try =smfGetOptionStrikes("AAPL&quot;,"3/16/2018","C","B")

?

Thu Feb 22, 2018 11:56 am (PST) . Posted by:

"Randy Harmelink" rharmelink

Oops. I forgot that the smfGetOptionStrikes() function uses Yahoo when you
request Barchart, because option strikes were difficult to extract from the
Barchart web page, and I didn't want to have to use different sources when
invoking the function, so I had the add-in do it automatically.

And remember I said Yahoo was missing entire blocks of expirations? The
3/16/2018 expiration date for APPL is one of those that is missing from
Yahoo right now.

I may have to look again at Barchart. Maybe the JSON file will be an easier
extract. With both Google and OX gone now, another source would be nice. I
kind of stopped looking at Barchart as an options source after I found out
they only update the options data hourly.

Lately, I've actually been working on getting options data from a Wells
Fargo brokerage account, as I can get live option quotes there. But it
requires me to create a security cookie each time I am looking at possible
positions, as the security cookie expires after about 10 or 15 minutes. So
very inconvenient.

On Thu, Feb 22, 2018 at 12:34 PM, peterscottcook@gmail.com [smf_addin] <
smf_addin@yahoogroups.com> wrote:

>
> Randy, I just realized it works with some stocks and not others.
>
> Can you try =smfGetOptionStrikes("AAPL&quot;,"3/16/2018","C","B")
>
> ?
>

Thu Feb 22, 2018 12:04 pm (PST) . Posted by:

pete_cook2001

I see that now, looking at the yahoo options page.

Is yahoo the only data source left?

Thu Feb 22, 2018 12:44 pm (PST) . Posted by:

"Randy Harmelink" rharmelink

Look like it. I haven't tried "8" in a long time, because of the need to
create the security cookie.

On Thu, Feb 22, 2018 at 1:04 PM, peterscottcook@
​...
wrote:

>
> I see that now, looking at the yahoo options page.
>
> Is yahoo the only data source left?
>
>

Sat Feb 24, 2018 6:10 am (PST) . Posted by:

lexstar

1 Which is the preferred choice now? smfGetYahooHistory vs RCHGetYahooHistory()


2 Does using "dohlc" give adjusted or non-adjusted prices? At first, my impression is that it only gives non-adjusted prices, but looks like it's giving adjusted prices. What specific parameters should I use for non-adjusted prices vs adjusted prices?


3 If I'm using RCHGetYahooHistory and using "M" for monthly data, why am I getting today's data (ie. 2/23/2018) and addition to this month's data (ie 2/1/2018) follow by previous month's data (which is correct).
Example:


2/23/2018 <<< Which does this have to be in a separate line? Why not included in 2/1/2018 and be done w it
2/1/2018
1/1/2018
12/1/2017
11/1/2017
etc
etc
ect


Sat Feb 24, 2018 9:34 am (PST) . Posted by:

"Randy Harmelink" rharmelink

On Sat, Feb 24, 2018 at 7:10 AM, lexstar@... wrote:

> 1 Which is the preferred choice now? smfGetYahooHistory vs
> RCHGetYahooHistory()
>

​smfGetYahooHistory() is the new function. RCHGetYahooHistory() calls it.​

2 Does using "dohlc" give adjusted or non-adjusted prices? At first, my
> impression is that it only gives non-adjusted prices, but looks like it's
> giving adjusted prices. What specific parameters should I use for
> non-adjusted prices vs adjusted prices?
>
​All prices are now adjusted.​
​ But you can get an unadjusted closing price.​

3 If I'm using RCHGetYahooHistory and using "M" for monthly data, why am I
> getting today's data (ie. 2/23/2018) and addition to this month's data (ie
> 2/1/2018) follow by previous month's data (which is correct).
>
​Because that's the way Yahoo does it. Note that Yahoo still is doing
dividend adjustments incorrectly on monthly data.​

> Example:
>
> 2/23/2018 <<< Which does this have to be in a separate line? Why not
> included in 2/1/2018 and be done w it
>
> 2/1/2018
>
> 1/1/2018
>
> 12/1/2017
>
> 11/1/2017
>
> etc
>
> etc
>
> ect
>
>
>
For the Add-in, Documentation, Templates, Tips and FAQs, visit http://ogres-crypt.com/SMF

Tidak ada komentar:

Posting Komentar