* Map Life Facts - filtering?

Writing and using plugins for Version 5 and above.
User avatar
Ron Melby
Megastar
Posts: 790
Joined: 15 Nov 2016 15:40
Family Historian: V6.2

Re: Map Life Facts - filtering?

Post by Ron Melby » 28 Aug 2021 11:55

Mike, I spoke too soon. This time I did not get the error.

couple of things that may or may not be significant to your investigation.

I have for a very long time, had my sticky setting @ us bias. when I ran without error the second time, it came up uk bias, and it must have been so when I ran thru it the first time (at which time I received that error), because I got default map london instead of new york. I ran it a 3rd time (every time getting out of fh entirely and going back in, and us bias stuck).

additionally, and this is the big one:

I run place fields only and address fields only.

Statistics: 0 plotted 0 tentative 0 no plot 0 invalid 1139 no data 1139 total.
(thats address, place is same except for the number of no datas and totals)
I have went in a couple more times, without error, but no address or places are plotted.
I have confirmed that they are not available, by selecting locations on both places and addresses and they are greyed out....at least in my case, Mike-- you have appeared to have lost the plot as it were.

As I peruse the PLACMap Life Facts.loc the latlong is there.
As I peruse the ADDRMap Life Facts.loc most of the latlong is not there.

They both were fully propagated, before I ran the program.

I have ran MLF 5 times total (once the old one, and 4 times the new, the first time on the new getting the error, and havent gotten it on subsequent runs) Where the plots were lost, I cannot say. I hope this helps. Mike, ADDR are in sources, and are not lost in the source. though, I don't know if you still keep up ADDRMap when you do that or where you put it.
FH V.6.2.7 Win 10 64 bit

User avatar
tatewise
Megastar
Posts: 22393
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Map Life Facts - filtering?

Post by tatewise » 28 Aug 2021 14:51

The only difference between plugin v4.8.7 and v4.9 is that it forces the use of Google Maps API v3.45 which is the latest version that supports Internet Explorer without warning messages.
(The plugin uses IE in the map pane because that is the only browser engine currently available with Lua.)
So with such a tiny change, I don't understand why map plotted values would vanish.

I don't think the JavaScript error message is significant to losing plotted values.

Have you a recent Project Backup in which the ...\Plugin Data\Map Life Facts.loc file has all then plotted values?
Can you recover the plot data from there?
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

User avatar
Ron Melby
Megastar
Posts: 790
Joined: 15 Nov 2016 15:40
Family Historian: V6.2

Re: Map Life Facts - filtering?

Post by Ron Melby » 28 Aug 2021 15:13

No, Mike I don't. But I do not mind rerunning it. So, you do retain ADDR latlong there even when you stick it in the sources. I wish I had a backup of 4.8.7...lol. When I was debugging something odd in my save restore stuff, (which I am still working on-- paths upgrade which are a wellspring, I was chasing a spurious error that actually eminated from loadfile but threw another one of my error statements. I can live without MLF a few days, don't throw your back out over this, I have confidence you will find the little. But as I say, something else is afoot in that my places are not showing up even though the latlongs are there. And it is proven out by the go to reverse it and look at one, they are all greyed out and nothing is selectable but the cancel button.
FH V.6.2.7 Win 10 64 bit

User avatar
tatewise
Megastar
Posts: 22393
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Map Life Facts - filtering?

Post by tatewise » 28 Aug 2021 15:27

Are the Place names really greyed-out so you cannot select them?
Unplotted Place names are displayed in grey to match the No Data grey Statistic.
However, they can still be selected and then plotted. So are you sure nothing is selectable?

If you change the Location Plot Options to save Database in: Source Records, etc, then the MapLifeFacts.loc file still holds the LatLong values but are no longer active because the data is in the GEDCOM records.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

User avatar
Ron Melby
Megastar
Posts: 790
Joined: 15 Nov 2016 15:40
Family Historian: V6.2

Re: Map Life Facts - filtering?

Post by Ron Melby » 28 Aug 2021 19:47

Mike I sent you my .loc offsite, all the data appears to be in it. And they are selectable, but as you say match the greyscale. When they show in the window, all grey and latlong not there as you are saying. But they exist. I was going to see if I could find in your code where you load it, and see if I could tell why it isnt loading correctly.
FH V.6.2.7 Win 10 64 bit

User avatar
Ron Melby
Megastar
Posts: 790
Joined: 15 Nov 2016 15:40
Family Historian: V6.2

Re: Map Life Facts - filtering?

Post by Ron Melby » 29 Aug 2021 12:59

Mike, some interesting results.

I have rerun place fields only and address fields only there are some consequences I do not understand and some, how did that ever happen then.

for my 1139 addresses 2 were unplotted, this is as it should be. They are cemeteries on private land with no address. The question arises, is there a place to stuff the lat long in that file that should ADDR be mashed again, I can recover the home made lat long? if the answer is keep themn in a text doc, or a source or a note, thats ok.

place only at 2354 has 23 no plots.
The very interesting and serindipitous part of PLAC is that I have some number of them incorrect.

Waubeek, , Dunn, WI, USA
should be:
Waubeek, , Pepin, WI, USA

this is brill with reservations,
I see no way to get a report of no plot, invalid, no data locations which would force me to find one, get out of MLF, go fix place record, (or write a list, fell swoop and such) go back in, get the next by paging thru old business, and so on.
it would be handy to

Additionally, I have some of them that are in the form:

,,,DE, DEU
...FR, FRA

and they dont plot either, however, I have no recollection of every looking up and forcing so many plots.
I can find a way to standardize or force plots, back to where to stuff them permanantly.

I mean I understand that you don't plot what is not available, or correct, and you have enough going on now, but I do not feel comfortable at this stage, loading and riffling thru your file with my program. LOL. Ideas?
FH V.6.2.7 Win 10 64 bit

User avatar
tatewise
Megastar
Posts: 22393
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Map Life Facts - filtering?

Post by tatewise » 01 Sep 2021 09:38

Ron, sorry for the delay in replying. I did not see your posting until today.

Protect the Lat/Long for No Plot Locations
To protect against 'mashed' data loss use backup, backup, backup, i.e. FH Project Backup.
Also, keep those manual Lat/Long in some other document or a Note within FH.

List the No Plot, Invalid, No Data Locations
The 'Next Unplotted' button will step through those unplotted Locations one by one.
You could investigate the problem and try Standardized alternatives until a solution is found.
Make a note of the Location and Standardized values to update the FH Address or Place after closing MLF.
i.e. Find the Address or Place that matches the Location value and change it to the Standardized value.
Then later in MLF, those Standardized values can be deleted for those Locations to check it still plots correctly.

I could see if a 'List Uplotted' button can produce a Result Set of unplotted Locations.
However, I don't see how you can fix the FH Address or Place in FH without using MLF to check it will plot.
So I think the procedure described above is more efficient, i.e. use MLF to find what does plot and then update FH.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

User avatar
tatewise
Megastar
Posts: 22393
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Map Life Facts - filtering?

Post by tatewise » 07 Sep 2021 15:08

Following on from discussions in this thread and offsite Email discussions with Ron I've updated the Map Life Facts plugin.
The changes focus mostly on improving the Geocode Plotting of Locations instigated by plot failures reported by Ron:
  1. Purge blank comma-separated parts that upset the Google Geocoder;
    #
  2. Improve the Smart Geocoding techniques that cater for Country Codes, Chapman Codes, etc:
    • The Region Bias Code or the Country in the Location is used to select relevant Smart Geocoding Codes;
      e.g. British Chapman Codes and County abbreviations are only applied to British locations;
    • The Country can be either the right-most or left-most comma-separated Location part;
      i.e. Both left-to-right and right-to-left Location parts are supported;
    • Location comma-separated parts can be in any combination of upper & lower case;
  3. If a plot is Not Found, the left-most comma-separated Location part is successively removed for each auto-geocode;
    #
  4. JavaScript try{ }, catch{ } & finally{ } are now used to avoid script error messages where possible;
My next development is to see if the Bing Maps API can be a beneficial alternative to the Google Maps API.

Try the attached Map Life Facts plugin Version 4.9.2 Date 07 Sep 2021.
Attachments
Map Life Facts.fh_lua
Version 4.9.2 Date 07 Sep 2021
(387.98 KiB) Downloaded 10 times
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

User avatar
Ron Melby
Megastar
Posts: 790
Joined: 15 Nov 2016 15:40
Family Historian: V6.2

Re: Map Life Facts - filtering?

Post by Ron Melby » 08 Sep 2021 16:10

sounds like its pretty snazzy, I have loaded it, and am thinking about renamin my loc file and running this version from scratch.

two questions arise, maybe 2.5.

assume that my bias is us
1) then; even though you know I dont say my PLAC is GBR, , Bethersden, , Kent, UK.
when you start whittling each successive split, is GBR anchored so that, if you dont find it in GBR or UK as the case may be in you 'normal' peoples encodings, that it is not found in GBR, instead of anywhere in the world.
2) does reversing PLAC on the window affect this or is that just a presentation not associated with the loc you run into the api?
2.5 then of course which takes precedence bias or loc(l or r)?
FH V.6.2.7 Win 10 64 bit

User avatar
tatewise
Megastar
Posts: 22393
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Map Life Facts - filtering?

Post by tatewise » 08 Sep 2021 18:13

1) I don't understand the question, but if the Location has GBR or UK in the first part or last part, then the British Smart Codes are applied.
2) The list Reverse Display Order has no effect on geocoding.
2.5) The Region Bias Code is the default, but if the Location first or last part identifies a Country that has Smart Codes then those take precedence for just that Location. The only Countries that have Smart Codea are USA, Canada, Australia and Britain (UK and Ireland).
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

User avatar
Ron Melby
Megastar
Posts: 790
Joined: 15 Nov 2016 15:40
Family Historian: V6.2

Re: Map Life Facts - filtering?

Post by Ron Melby » 08 Sep 2021 18:42

3. If a plot is Not Found, the left-most comma-separated Location part is successively removed for each auto-geocode;
1) say my PLAC is GBR, , Bethersden, , Kent,
I would suppose you would not tear off GBR, and go down the line, so if it is not found in GBR by reduction you wouldnt code the first Bethersden that google comes up with bad example I know.... ontario or london or some other PLAC would be a better example.

now that I have the right MLF.loc, I tried a save restore using my savrst plugin (with requires) faster save and restore, smaller file, and slightly prettier. it is an absolutely equivalent table. That is just a sidenote. There would have to be a one time conversion of tables that are saved using chillcode so....

I am going to geocode all addr and plac records all over again, see how it goes.
FH V.6.2.7 Win 10 64 bit

User avatar
tatewise
Megastar
Posts: 22393
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Map Life Facts - filtering?

Post by tatewise » 08 Sep 2021 19:53

The plot Not Found cases are relatively rare, and any improvement is a bonus. Nothing is perfect.

The plugin assumes the most common left-right Location strategy of 'Town, County, State, Country'.
So the Location would normally be , Bethersden, , Kent, GBR just like all your other Locations.
( Why would you put GBR first? )
If the plot is Not Found, first Bethersden would be removed, and then Kent removed, to hopefully get a successful plot.

This is not perfect and won't work for people who use a right-left strategy of 'Country, State, County, Town'.
If the Location is GBR, , Kent, Bethersden then first GBR and next Kent will get removed.
They may not plot, so Not Found will still apply, but that is no worse than if the removals had not been tried.

It works well for your Locations that were a Not Found plot such as:
Delta, , Keokuk, IA, USA
Dupois, , , SK, CAN
, , DE, DEU
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

User avatar
Ron Melby
Megastar
Posts: 790
Joined: 15 Nov 2016 15:40
Family Historian: V6.2

Re: Map Life Facts - filtering?

Post by Ron Melby » 08 Sep 2021 23:40

Mike, not sure how your new one does it.

I did not remove the loc file, simply said yes, geocode all for addys-source, and geocode all for place-records.
prior to running I had one address that was hand coded.

it reports all coded for both.

would it be different if I renamed the loc and started from scratch or not necessary for test?
FH V.6.2.7 Win 10 64 bit

User avatar
tatewise
Megastar
Posts: 22393
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Map Life Facts - filtering?

Post by tatewise » 09 Sep 2021 10:18

Ron, please click the plugin Help & Advice button on the Geocode Location Plots tab:
https://pluginstore.family-historian.co ... -plots-tab
The first few paragraphs explain the behaviour and options of the three geocode plot buttons.

Geocode Plot All Locations submits ALL Locations to the Google Geocoding Service at several plots per second, but Plotted locations can be excluded.

Geocode Plot Some Locations lets you choose which Locations to geocode.

There is no need to rename the loc file.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

Post Reply