* [Wish List Item 626] Automatic generation of UniqueID

[jelv]

I'm a touch concerned that we are discussing settings for when Auto-Generation of UniqueId should / could be switched on, without any definition of what this UnqueId is, or how it behaves (or might behave) - so far as I can see and please correct me if I'm wrong (and yes, I have read that part of the GEDCOM 7 spec'n).

We know a lot about how FH behaves with UIDs because some of us have been using them for a considerable time - but we have to remember to manually generate them using the existing menu option whenever we have added new individuals and we are going to do a process that relies on them. All Mark is asking for is an option to bring FH in line with the likes of RootsMagic which automatically generates them for new individuals.

[AdrianBruce]

...
We know a lot about how FH behaves with UIDs because some of us have been using them for a considerable time - but we have to remember to manually generate them using the existing menu option whenever we have added new individuals and we are going to do a process that relies on them. All Mark is asking for is an option to bring FH in line with the likes of RootsMagic which automatically generates them for new individuals.

Sorry - I now understand the detail of the request - I'd missed that "an additional step of Tools > Record Identifiers > Create UniqueID where missing is required" is actually an existing menu option applicable across all Individuals.

[Vyger]

I just wanted to add a bit of additional fact to the UniqueID discussion as reference.

Round trip from Rootsmagic or Rootsmagic originated import via Family Historian to My Heritage preserves the UniqueID in any future downloaded Gedcom so Unique Person Identification is intact.

Disappointingly any round trip to Ancestry does not preserve the UniqueID generated by Rootsmagic. At present I don't know if Rootsmagic transfer the UniqueID to Ancestry via their Treeshare app or if Ancestry drop it. Either way a Gedcom downloaded from Ancestry does not contain any _UID line.

Individuals in a NEW file download via Rootsmagic Treeshare get a new UniqueID presumably assigned by Rootsmagic, so in my case;

1 _UID C7B607C34EF94BAFBEDDA07611E176680917
has subsequently become;
1 _UID AAE10A4C1EEF43ACAE97AEB529661A3967FF
albeit still with the same RIN as the original upload.

For any doubters of 36 bit hex UID possibilities I lifted the below from a web source;

Combinatorics
Possible 36 hex digit combinations
22,300,745,198,530,623,000,000,000,000,000,000,000,000,000 (~ NaNt)

Thats a LOT of zeros and more than I will ever need.

[Mark1834]

UniqueID is preserved by Ancestry, but is exported as a UID tag, which is the GEDCOM 7 designation. It’s what enables the Ancestry Sync plugin to compare the Ancestry GEDCOM export with the RM original (TreeSync can cause significant changes in data).

I have a revised wording for the Wish List that I haven’t got round to posting yet. I’ll do that later today.

[jelv]

I just set a UniqueId for an individual in a test project and then cloned said individual. The clone included cloning the exact value of the UniqueId. Now, of course, that's a PlugIn rather than native FH logic but the same thing happens when copying and pasting the items from one individual to an empty one.

That's a fault in the plugin! It should NOT copy the UniqueId. The additional option Mark proposed would allow the plugin to allocate the cloned copy a new UniqueId.

A related but independent option would be to provide an fhCalculateUniqueId(ptrI) function for plugin authors to create values directly.

[tatewise]

UniqueID is preserved by Ancestry, but is exported as a UID tag, which is the GEDCOM 7 designation.

When I export an Ancestry GEDCOM its header says it is 5.5.1 and not 7.0:
1 GEDC
2 VERS 5.5.1
IMO it should NOT be using the UID tag as it is invalid in 5.5.1!

[jelv]

Is that the only aspect of the stated GEDCOM spec. that the Ancestry export doesn't conform to?

[Vyger]

UniqueID is preserved by Ancestry, but is exported as a UID tag, which is the GEDCOM 7 designation. It’s what enables the Ancestry Sync plugin to compare the Ancestry GEDCOM export with the RM original (TreeSync can cause significant changes in data).

I do hope you are correct in some way Mark.

In my example I downloaded my Ancestry tree to a new empty Rootsmagic database and it would appear Rootsmagic assigned a NEW UID to all the individuals and did not preserve the UID of those in the original Treeshare upload.

[jelv]

That's RootsMagic TreeShare not retrieving the Unique Id from Ancestry.

If you export your Ancestry tree to a GEDCOM file you'll find they are all there.

[ColeValleyGirl]

I suggest this Wish should not depend on what other programmes do or don't do, but rather on whether it makes sense in the FH context to some people (who will presumably later vote for it) and there are no objections that can't be addressed.

Edit: This is a gentle nudge to stay on topic

[jelv]

It all makes sense in the FH context and much of the discussion has been due to lack of awareness of the existing handling of UniqueIds by FH.

The proposal is not based on what other software does, it is simply correcting what is an annoyance with the current situation. Pointing out that incidently, what is requested is the a feature of a competitors software, may make CP more inclined to add the feature requested.

Edit: I've seen (and contributed) to Mark's revised wording which clarifies it a bit. As it's his baby I'm leaving it to him to post.

[Mark1834]

To bring this thread back to its original purpose, my revised proposal is

Proposal: A Project-level option (probably under Tools > Preferences > General > Advanced, and off by default for consistency with previous practice) be provided that when set automatically generates a UniqueID value for new Individual records added either directly from the user interface or by merging another FH project or external GEDCOM file where this field is missing.

This limited change will be a significant improvement for users who routinely generate UniqueIDs in their projects, without affecting the current format of the UniqueID field or its behaviour on merging records.

I think this makes it clear that all I am requesting is a small addition to what FH does already. Specifically out of scope is changing the way CP have decided to implement this field in FH7.

I suggest a new thread be started if anybody wants to broaden the discussion into what UniqueID does and doesn't currently do, as far as it affects FH users.

[ColeValleyGirl]

I have no objections to the reworded proposal.

[jbtapscott]

Not something I am particularly interested in as I don't (currently) move data to other genealogy software, but if I did, the proposal looks spot on to me.

[NickWalker]

Sounds good to me although not something I need to use. I'll add this feature as an option in AS too at some point so that new individuals created in AS can be given an unique id too.

[jelv]

Sounds good to me although not something I need to use. I'll add this feature as an option in AS too at some point so that new individuals created in AS can be given an unique id too.

A good proportion of the new individuals I add are created using AS so if Nick is prepared to do this it would need the extra bit Mark suggested.

A related but independent option would be to provide an fhCalculateUniqueId(ptrI) function for plugin authors to create values directly.

[NickWalker]

Sounds good to me although not something I need to use. I'll add this feature as an option in AS too at some point so that new individuals created in AS can be given an unique id too.

A good proportion of the new individuals I add are created using AS so if Nick is prepared to do this it would need the extra bit Mark suggested.

A related but independent option would be to provide an fhCalculateUniqueId(ptrI) function for plugin authors to create values directly.

AS can create its own Unique ID, it doesn't need to use the method in FH.

[Mark1834]

In principle, I think UniqueID is calculable from a plugin, but the official method needs access to parameters such as precise system time and network card MAC address. They’re beyond my pay grade to do with plugin libraries, but if Nick and CP can do it more easily with their professional expertise and tools, that’s great.

[NickWalker]

In principle, I think UniqueID is calculable from a plugin, but the official method needs access to parameters such as precise system time and network card MAC address. They’re beyond my pay grade to do with plugin libraries, but if Nick and CP can do it more easily with their professional expertise and tools, that’s great.

As I understand it, I just need to use: Guid.NewGuid() in .Net to generate a unique id using the RFC 4122 standard so , at least on this occasion, it doesn't need a lot of expertise for me to add this to AS!

[Vyger]

UniqueID is preserved by Ancestry, but is exported as a UID tag, which is the GEDCOM 7 designation. It’s what enables the Ancestry Sync plugin to compare the Ancestry GEDCOM export with the RM original (TreeSync can cause significant changes in data).

Mark can you expand on this assertion a little more?. My RM originated 36 bit hex UID becomes a truncated 32 bit hex on Gedcom download from Ancestry, perhaps it is different using Treeshare? see below;

1 _UID 5A4718305B124219B6275DEFAC1472434FB5 (Rootsmagic originated UID)
1 UID 5A4718305B124219B6275DEFAC147243 (Ancestry Gedcom download truncated UID)

Both 32 and 36 bit hex are more than ample to uniquely identify individuals and help in the fight and time wasted reconciling duplication and perhaps there is a spec in the Gedcom 7 brief?

Again I can confirm that My Heritage do preserve this UID in the originating format on a round trip and Gedcom download via their Family Tree Builder software although do not preserve RIN's, see format below;

Code: Select all

0 @I1@ INDI
1 RIN MH:I1
1 _UID 5A4718305B124219B6275DEFAC1472434FB5

I really look forward to the day all genealogy software providers embrace the unique identification of individuals using a common format so transferring of data between platforms can be less of a duplication nightmare.

I also took the time to email Rootsmagic support and received the the reply below this morning;

"Jackson,

Ancestry has their own identification system that they use, so they do not use or store the RM UID number. That is why the Ancestry GEDCOM and Ancestry TreeShare both have the old UID stripped out when importing into the RM database. The RM RIN number is also not preserved by Ancestry.

RootsMagic, Inc."

[Mark1834]

Brief reply only as I won’t be back at the desk until this evening - the last four characters are an optional checksum. FH UniqueID values don’t use it either, they are also 32 chars, but hyphenated, which is the preferred (but not mandated) format in GEDCOM 7.

As you have shown, Ancestry does store the RM UniqueID, but presumably just as a dumb field that is not displayed in the UI. It is lost on GEDCOM import to RM, as it doesn’t recognise UID. I haven’t tried editing the tags to _UID before input, but it is what it is, so that’s what we have to work with.

To be fair, FH didn’t import UID via GEDCOM originally, but CP added it in a recent update at my request.

[tatewise]

Both 32 and 36 bit hex are more than ample to uniquely identify individuals and help in the fight and time wasted reconciling duplication and perhaps there is a spec in the Gedcom 7 brief?

As intimated by Nick yesterday, GEDCOM 7 specifies RFC 4122 which defines a 128-bit, i.e. 16-byte or 32 Hex character UUID.
GEDCOM 7 also says:

Note — Some systems used a 16-byte UUID with a custom 2-byte checksum for a total of 18 bytes:
• checksum byte 1 = (sum of (byte i) for i 1 through 16) mod 256
• checksum byte 2 = (sum of ((16 − i) × (byte i)) for i 1 through 16) mod 256
Use of checksums for UIDs is discouraged except in cases where error-prone input is expected and an appropriate action to take in case of an error is known.

[Mark1834]

I thought I had prepared an outline list of apps that use UniqueID, and it's here in the help page for the Format UniqueID plugin.

For some reason, it is not shown on the main plugin page, so that could be the Plugin Store WordPress bug that CP were referring to their 'WordPress expert' some time ago but apparently not yet fixed.

[tatewise]

For some reason, it is not shown on the main plugin page...

Mark, in that Help page in the WordPress editor, check the More Fields section at the bottom.
Have you ticked the Main Page Yes option as below? If not then it won't be included in the Plugin home page.

MainPage.png (3.33 KiB) Viewed 696 times

[Mark1834]

Yes, the plugin is correctly cited, and that option ticked:

Capture.PNG (5.73 KiB) Viewed 678 times

The help is listed in Plugin Help, and the link on the Help page back to the plugin works, but it's just the actual plugin page that says there is no help available.

What's usually worked in the past for clearing this type of fault is to copy the help text, permanently delete the help file, then copy the contents back into a new file. That hasn't made an immediate difference, but I've noticed other things in the Store like download counts don't update immediately, so perhaps there is a batch process that runs in the background and it will be back working tomorrow...