* FH -> RootsMagic -> Ancestry (Work-in-Progress)

Importing from or exporting to another genealogy program. This is the place to ask.
avatar
Mark1834
Superstar
Posts: 403
Joined: 27 Oct 2017 19:33
Family Historian: V7
Location: South Cheshire

Re: FH -> RootsMagic -> Ancestry (Work-in-Progress)

Post by Mark1834 » 12 Jan 2021 23:06

As you are discovering David, the RM file comparison is probably the weak link in this process. There are two problems that you are likely to encounter. The first is what you are seeing, that there are a large number of records shown as not matching that are in fact identical.

The pragmatic fix for this is to use the second feature of my plugin, the "List Individuals" button. That gives you a file selection window entitled "Identify Sync File". Use that to select the .LST file in the same folder as you stored the RM database. This is the log file for the import. The plugin doesn't actually open the file, but reads its date/time stamp and gives you an ordered list of all individuals in your database changed since that moment (just one in your case). Use that as you crib-sheet to identify the record to check in the RM file comparison, and hopefully you will be able to see which changes have been made so you can copy them across to the master file.

The second problem is one you may come across later, where RM fails to recognise that two records relate to the same person, usually because either there have been a lot of changes, or alternatively there is very little information available, so it may confuse individuals of similar name. This is where the _UID tag comes in. This is not part of the core GEDCOM set (hence the underscore), but its intention is to act as a unique identifier for an individual across multiple applications. RM generates a _UID field for every person in its database, and uses it as the main identification parameter when comparing individuals. Although many programs support the _UID, FH does not. It neither generates _UIDs or stores those imported from other programs.

Shosh's idea was to read the _UID value after the GEDCOM is imported into RM, and create a custom attribute in FH to store it for future export. To do this effectively, it needs direct access to the RM database, otherwise it becomes another clunky manual step. That is possible in FH6, where it is possible to use a database library tool to interrogate (and even modify) the RM database from within FH via a dedicated plugin. Unfortunately, that library has not been made available for FH7, and it is not worth investing the considerable time required to implement it if it does not work in the current version. Until CP make the library available, this improved approach is on the back burner, so we are stuck with the present methods.

There's a lot of detail in this and other threads as we have fine-tuned how this process works. I am in the process now of summarising the key points for a page in the new KB, so hopefully that will be ready in the next few days.
Mark Draper

User avatar
tatewise
Megastar
Posts: 20104
Joined: 25 May 2010 11:00
Family Historian: V6.2
Location: Torbay, Devon, UK
Contact:

Re: FH -> RootsMagic -> Ancestry (Work-in-Progress)

Post by tatewise » 12 Jan 2021 23:29

FH does store _UID imported from other programs as UDF and remain forever unless deleted.
They are mentioned in Importing to Family Historian for Import from RootsMagic (RM) and Legacy and MyHeritage.
They can be managed by any Plugin and included in any exported GEDCOM.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

avatar
David Potter
Megastar
Posts: 787
Joined: 22 Jun 2016 15:54
Family Historian: V7
Location: United Kingdom

Re: FH -> RootsMagic -> Ancestry (Work-in-Progress)

Post by David Potter » 13 Jan 2021 08:03

Thank you Mark for taking time to explain things. That helped a lot. I'll have another go at this today and see how it comes together.

Thank you too Shosh.

David

avatar
Mark1834
Superstar
Posts: 403
Joined: 27 Oct 2017 19:33
Family Historian: V7
Location: South Cheshire

Re: FH -> RootsMagic -> Ancestry (Work-in-Progress)

Post by Mark1834 » 13 Jan 2021 08:58

Poor wording - I know that they are retained as undefined. What FH doesn’t do is generate its own or have a specific field for them. The KB article seems to just refer to them as local to particular apps, and makes no mention of their potential to unambiguously identify individuals across different apps.
Mark Draper

User avatar
tatewise
Megastar
Posts: 20104
Joined: 25 May 2010 11:00
Family Historian: V6.2
Location: Torbay, Devon, UK
Contact:

Re: FH -> RootsMagic -> Ancestry (Work-in-Progress)

Post by tatewise » 13 Jan 2021 09:43

The reason the KB suggests _UID are unwanted UDF is that nobody has ever shown any interest in them, until now.
Probably that is because most products don't have a merge sync function, or use other criteria to merge records, or users have no interest in merging. It is only RM that seems to be poor at merging and _UID may be a workaround.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

avatar
shoshk
Famous
Posts: 225
Joined: 13 May 2015 16:28
Family Historian: V7
Location: Mitzpe Jericho, Israel

Re: FH -> RootsMagic -> Ancestry (Work-in-Progress)

Post by shoshk » 13 Jan 2021 09:51

Mike, _UID is definitely not a workaround in RM.

It is a feature that allows you to differentiate between all those women named 'Mary' with no surname and no other details. :D
Shosh Kalson

avatar
Mark1834
Superstar
Posts: 403
Joined: 27 Oct 2017 19:33
Family Historian: V7
Location: South Cheshire

Re: FH -> RootsMagic -> Ancestry (Work-in-Progress)

Post by Mark1834 » 13 Jan 2021 10:12

The danger here is that we end up muddling different issues - RM manages merging an import into a database perfectly if _UID is available for the import (but merging is only additive, not update), but doesn't handle the comparison between databases correctly, even if _UID is consistent across them.
Mark Draper

User avatar
tatewise
Megastar
Posts: 20104
Joined: 25 May 2010 11:00
Family Historian: V6.2
Location: Torbay, Devon, UK
Contact:

Re: FH -> RootsMagic -> Ancestry (Work-in-Progress)

Post by tatewise » 13 Jan 2021 10:15

Most reasonable sync algorithms would use close relatives as well as personal data.
So you are saying 'Mary' has no close relatives and is completely bare of details.
That begs the question of why is she in your Project?
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

avatar
shoshk
Famous
Posts: 225
Joined: 13 May 2015 16:28
Family Historian: V7
Location: Mitzpe Jericho, Israel

Re: FH -> RootsMagic -> Ancestry (Work-in-Progress)

Post by shoshk » 13 Jan 2021 10:16

Yes, that is correct.

That is why I see direct access to the RM database as the only viable option to creating a process which will work reliably and with a minimum of effort on the part of the user.

So, I'm waiting for CP to add LUASQL back in.
Shosh Kalson

avatar
shoshk
Famous
Posts: 225
Joined: 13 May 2015 16:28
Family Historian: V7
Location: Mitzpe Jericho, Israel

Re: FH -> RootsMagic -> Ancestry (Work-in-Progress)

Post by shoshk » 13 Jan 2021 10:24

You are correct. Mary has parents, or a husband, or children. They could certainly be included in the sync algorithm. But RM doesn't seem to. In fact, this isn't a failing of just RM. Merge routines are not so great in other programs as well.

I believe that FH has one of the best because you have complete control over the process.

Anyway, _UID in RM is the solution; you can call it a workaround if you like. ;)
Shosh Kalson

avatar
Mark1834
Superstar
Posts: 403
Joined: 27 Oct 2017 19:33
Family Historian: V7
Location: South Cheshire

Re: FH -> RootsMagic -> Ancestry (Work-in-Progress)

Post by Mark1834 » 13 Jan 2021 10:35

In an ideal world, every app would support _UID as a unique personal identifier (so essentially a universal primary key for the database of individuals), so all these imperfect merges and syncs would be consigned to history. One parameter does it all. Meanwhile, back in the real world, we have to make the best we can of what the various competing software houses give us...

I should know better by now - if I had simply said in my explanation to David that "FH does not support use of _UID as a unique personal identifier", that would have avoided another 9 contributions to the thread... ;)
Mark Draper

User avatar
tatewise
Megastar
Posts: 20104
Joined: 25 May 2010 11:00
Family Historian: V6.2
Location: Torbay, Devon, UK
Contact:

Re: FH -> RootsMagic -> Ancestry (Work-in-Progress)

Post by tatewise » 13 Jan 2021 10:45

FamilySearch started the International Genealogical Index (IGI) in 1973 and maintained it up until 2008.
Strangely, the IGI was not incorporated into their GEDCOM specifications, nor taken up by any products.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

avatar
David Potter
Megastar
Posts: 787
Joined: 22 Jun 2016 15:54
Family Historian: V7
Location: United Kingdom

Re: FH -> RootsMagic -> Ancestry (Work-in-Progress)

Post by David Potter » 14 Jan 2021 19:18

Hi Both
I ran a second test this time using that second feature you mentioned that checks the LST file. It found no changed Individuals at all and there is for sure one. Question: is the Plug-In checking the Windows Date/Time modified or should it look for Date/Time info from inside the File (to which I can't see any info in the file that would help with this)?

If you look at the attached image the LST files are the same date but 3 with minutes difference between the Time value. I assume this should be enough of a difference.

Opening one of the files does not show any other Date/Time info? So very confused just now.

Cheers

David
Attachments
Capture.jpg
Capture.jpg (393.59 KiB) Viewed 325 times

avatar
Mark1834
Superstar
Posts: 403
Joined: 27 Oct 2017 19:33
Family Historian: V7
Location: South Cheshire

Re: FH -> RootsMagic -> Ancestry (Work-in-Progress)

Post by Mark1834 » 14 Jan 2021 20:40

David, I read those timestamps as you recreated your master file and imported into RM. Then changed a record, created an update file, and imported that.

All the routine does is read the date/time stamp of the file you select, and show records updated since then. So selecting your master should show your change, but not if you select the update, as the RM import was obviously later than the record update.

My usual workflow is
1. Generate the update gedcom.
2. List changes since last update by selecting update.LST file
3. Import update to RM and process there.

Does that make sense in terms of what you see?
Mark Draper

avatar
David Potter
Megastar
Posts: 787
Joined: 22 Jun 2016 15:54
Family Historian: V7
Location: United Kingdom

Re: FH -> RootsMagic -> Ancestry (Work-in-Progress)

Post by David Potter » 14 Jan 2021 21:01

Hi Mark. It didnt make any difference to the List count. Both files gave a zero Individual count?

I'll test again tomorrow, using that approach given in your last post.

Thanks for the help.

David

avatar
Mark1834
Superstar
Posts: 403
Joined: 27 Oct 2017 19:33
Family Historian: V7
Location: South Cheshire

Re: FH -> RootsMagic -> Ancestry (Work-in-Progress)

Post by Mark1834 » 14 Jan 2021 22:02

Just a thought - remember that individuals where either the Living or Private flags are set are neither exported nor listed.
Mark Draper

avatar
David Potter
Megastar
Posts: 787
Joined: 22 Jun 2016 15:54
Family Historian: V7
Location: United Kingdom

Re: FH -> RootsMagic -> Ancestry (Work-in-Progress)

Post by David Potter » 14 Jan 2021 23:30

Yes understood. The individual is myself and I get exported okay as I cleared out those flags. Also the update is to add my birth fact that gets picked in the update export. It is the List count that is not working for some reason.

Cheers

avatar
Mark1834
Superstar
Posts: 403
Joined: 27 Oct 2017 19:33
Family Historian: V7
Location: South Cheshire

Re: FH -> RootsMagic -> Ancestry (Work-in-Progress)

Post by Mark1834 » 15 Jan 2021 12:33

Sorry about that - fixed now. There is a subtle typo in line 344 of the plugin script, where "Min = M" should be "min = M". This resulted in it ignoring the minutes part of the update time, so recent changes were not listed. The short term workaround is to just select any other slightly older file to give a bigger time difference. It can be any file, as the plugin doesn't open the file or do anything with it other than read its timestamp. I'll post a corrected copy once we've shaken out any other early bugs, rather than have a plethora of different versions posted in different places.

Intriguingly, I was testing this with the Sample Project to make it easier to reproduce and share screenshots, and when I imported the original and modified versions into RM, its File Compare routine worked perfectly, and just showed the one record I had changed! That's the first database that I've ever seen that's worked as intended. Shosh has also seen the problems, and they are well discussed on RM forums, so it's not just me! All I need to do now is work out why... :)
Mark Draper

avatar
David Potter
Megastar
Posts: 787
Joined: 22 Jun 2016 15:54
Family Historian: V7
Location: United Kingdom

Re: FH -> RootsMagic -> Ancestry (Work-in-Progress)

Post by David Potter » 15 Jan 2021 13:58

Thank you Mark. Nice catch. I'll try the Work around and take another look at the RM Compare feature.

Thank you.

David

avatar
David Potter
Megastar
Posts: 787
Joined: 22 Jun 2016 15:54
Family Historian: V7
Location: United Kingdom

Re: FH -> RootsMagic -> Ancestry (Work-in-Progress)

Post by David Potter » 15 Jan 2021 14:05

Hi Mark
I couldn't get that Work Around to work so just changed the Min to min in the Plug-In and it correctly picked up the right Individual that had been updated.

Thank you once again.

David

avatar
Mark1834
Superstar
Posts: 403
Joined: 27 Oct 2017 19:33
Family Historian: V7
Location: South Cheshire

Re: FH -> RootsMagic -> Ancestry (Work-in-Progress)

Post by Mark1834 » 17 Jan 2021 09:36

Version 1.01 with a corrected selection routine is attached.
RootsMagic GEDCOM Export for Ancestry.fh_lua
(11.38 KiB) Downloaded 14 times
An overview of Ancestry synchronization via RootsMagic has now been published on the Knowledge Base, and this will always link to the latest version of the plugin.
Mark Draper

Post Reply