* Challenges with Functions

For users to report plugin bugs and request plugin enhancements; and for authors to test new/new versions of plugins, and to discuss plugin development (in the Programming Technicalities sub-forum). If you want advice on choosing or using a plugin, please ask in General Usage or an appropriate sub-forum.
Post Reply
User avatar
Jane
Site Admin
Posts: 8514
Joined: 01 Nov 2002 15:00
Family Historian: V7
Location: Somerset, England
Contact:

Challenges with Functions

Post by Jane »

This is a continuation from the discussions in the Problem with Lookup Missing Census Facts (15699) thread related to slow IsAncestorOf() and IsDescendantOf() functions, and others, reported 2 years ago.
such as the Select Records dialogue Add Relatives button, run very fast.
It's worth remembering those only have to do one record "walk", on Bill's file is has to be done 19000 times, as it's processed from the line rather than the other way around.

Personally I have columns based on flags which I set from a Query to flag any line(2) I am currently working on. If anyone is having problems with slow running switching over to a "2 step" process, eg set flag and use that to condition will help the speed.
Jane
My Family History : My Photography "Knowledge is knowing that a tomato is a fruit. Wisdom is not putting it in a fruit salad."
User avatar
tatewise
Megastar
Posts: 28410
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Problem with Lookup Missing Census Facts

Post by tatewise »

Nice try Jane, but my Lookup Missing... Plugins use fhCallBuiltInFunction("IsAncestorOf",ptrIndi,ptrRoot) for every ptrIndi Individual against whichever ptrRoot Individual the user chooses, to find the desired Ancestors, and can be different every time, so Flags won't help. There are similar options involving IsDescendantOf and IsRelativeOf.

In an Individual Query on the Rows tab try Add if =IsAncestorOf(%INDI%,Record(53,"I")) is true.
Where the Record id is not the File Root.
Can you explain why it runs on Bill's 19000 Project in less than one minute?
It is exactly the same as the Records Window Column function that takes 30 minutes.
At that speed each "walk" would have to take 0.1 second x 19,000 = 30 minutes.
Likewise the Query Rows tab Relations tab options run very fast.

The File Root cache is built very quickly. There is no noticeable delay when you change File Root.
So FH could build a similar cache for any root person, such that subsequent calls were fast.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
User avatar
Jane
Site Admin
Posts: 8514
Joined: 01 Nov 2002 15:00
Family Historian: V7
Location: Somerset, England
Contact:

Re: Problem with Lookup Missing Census Facts

Post by Jane »

Mike you could always skip using the function and do the logic in Lua. If you are only interested in Ancestors of one person, during the run just build an table of the Ancestors for the person you are working on and use that as a look up. There is ancestor walking logic in the Count Ancestors plugin.

There are two different problems here, and as you have logged the problem on the record window with Calico I would leave it at that. When you did the test did you remove all other other computed columns from the Record window?
Jane
My Family History : My Photography "Knowledge is knowing that a tomato is a fruit. Wisdom is not putting it in a fruit salad."
User avatar
tatewise
Megastar
Posts: 28410
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Problem with Lookup Missing Census Facts

Post by tatewise »

@Jane ~ Yes, could "walk" the tree in Lua logic, but the Plugin options offer Ancestors, Descendants, Ancestors & Descendants, and All Relations, so there are several combinations of algorithm, and I had assumed the built in functions would be reasonably quick. But I already feel I spend too much time working around FH deficiencies.

I have retested the Individual Records Window Column for =IsAncestorOf(%INDI%,Record(53,"I")) with the only other columns being Individual Records and Record Id, and it still took 30 minutes for 19,000 records.
I used the Save to Query button to create an identical Custom Query for both the original Columns and the three final Columns. They both sort their Result Set on the IsAncestorOf column in only 5 to 8 minutes.

BTW: In this case the problem only affects large Projects, and there is the partial workaround of using the Select Records dialogue and the Add Relatives button to choose Ancestors, Descendants, or Ancestors & Descendants very quickly.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
User avatar
tatewise
Megastar
Posts: 28410
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Challenges with Functions

Post by tatewise »

Other functions that have needed a workaround or are poorly documented include:
plugins:code_snippets:estimated_birth_dates|> Estimated Birth Dates
plugins:code_snippets:estimated_death_dates|> Estimated Death Dates
plugins:code_snippets:get_day_number|> Get Day Number and similarly for =DayOfWeek() function
plugins:code_snippets:last_updated_date_and_time|> Last Updated Date and Time
Additionally, the Date:Compare(Date) method does not work as described (it treats 1900, May 1900 & 9 May 1900 as equal).
Whereas the DatePt:Compare(DatePt) method works correctly and also handles all DatePt not handled by =DayNumber().
However, it does that by only considering the Day, Month, Year and disregarding everything else such as the BC, Julian, Hebrew, French settings.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
User avatar
tatewise
Megastar
Posts: 28410
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Challenges with Functions

Post by tatewise »

I have updated plugins:code_snippets:ancestor_and_decendant_list_creation|> Ancestor and Descendant List Creation to include recursive functions to "walk" the tree and list Ancestors, Descendants, and All Relatives in same Pool.

It also seems that the functions RelationCode() and Relationship() are also slow if neither Individual is the File Root.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
Post Reply