Page 1 of 1

error in import from Rootsmagic

Posted: 13 Jan 2021 19:11
by TMG_refugee
Sorry about the long post.
There has been some discussion on this forum about the GEDCOM standard and how FH interprets and implements that standard. In FH version 6 I was having difficulty importing from RootsMagic since they have implemented their interpretation of GEDCOM 5.5.1 to allow an EVEN tag in the GEDCOM file to have a descriptor on the same line. FH when importing implemented their version of GEDCOM 5.5 to disallow the descriptor on the same line as the EVEN tag resulting in thousands of error messages for me.

When FH version 7 came out I was happy to see that at least FH allowed the descriptor on a FACT tag in the GEDCOM file. I was thus able to edit my GEDCOM file and change all 1 EVEN to 1 FACT. This resulted in a clean import into FH.

Since facts or events are not transferrable from RM to FH I set about creating all my added facts (about 20) into FH and was able to import my file without errors. I tried this again a couple of days ago and again found all my FACT tags rejected and uncategorized and not matching up to my already created fact types. They were even reported as EVEN errors. In addition I am now getting many errors (info only) about image files:

l.477 - INFO ONLY: Loaded uncategorised data (non-GEDCOM): "2 FORM jpg"
l.483 - INFO ONLY: Loaded uncategorised data (non-GEDCOM): "2 FORM jpg"
l.493 - INFO ONLY: Loaded uncategorised data (non-GEDCOM): "2 FORM tif"
l.503 - INFO ONLY: Loaded uncategorised data (non-GEDCOM): "2 FORM jpg"
l.513 - INFO ONLY: Loaded uncategorised data (non-GEDCOM): "2 FORM jpg"
l.523 - INFO ONLY: Loaded uncategorised data (non-GEDCOM): "2 FORM jpg"

I have included references to the GEDCOM standards and the various corrections and interpretations by more knowledgeable GEDCOM people than I am. I would like to know if people think this is an error on FH’s part or that their implementation is correct. If an error should I report this to CP?

Addendums:
Gedcom 5.5.1 update released 11/15/2019 LDS
GEDCOM - FamilySearch Developer Center — FamilySearch.org
• Page 21 and 35 shows:
o 1 EVEN
• Page 48 clearly shows
o 1 EVEN Description
Another take on EVEN and FACT and the descriptor
Events and attributes · GenealogyJ (sourceforge.net)

“The GEDCOM standard is inconsistent on this fact. It mentions two variants:
Example 1: For a person that signed a lease for land dated October 2, 1837 would be recorded in
GEDCOM as:
1 EVEN
Events and attributes ꞏ GenealogyJ http://genj.sourceforge.net/wiki/en/manual/events
1 of 3 6/12/2020, 4:01 PM
2 TYPE Land lease
2 DATE 2 OCT 1837
Example 2:
1 EVEN Appointed Zoning Committee Chairperson
2 TYPE Civic Appointments
2 DATE FROM JAN 1952 TO JAN 1956
2 PLAC Cove, Cache, Utah
2 AGNC Cove City Redevelopment

FACT (Attribute)
The FACT tag is intended for a noteworthy attribute or fact concerning an individual, a group, or an
organization that cannot be expressed by predefined attribute types. In 5.5.1 it was introduced
additionally to the EVEN tag, to express the semantic difference between events and attributes.
A FACT must be qualified or classified by a subordinate TYPE tag (see below).
For example, the attribute could describe a person's skills, such as woodworking,:
1 FACT Woodworking
2 TYPE Skills
This groups the fact into a generic skills attribute, and in particular this entry records the fact that this
individual possessed the skill of woodworking.”


GEDCOM 5.5.1 Annotated version by Tamura Jones
www.tamurajones.net/DownloadGEDCOM551An ... tion.xhtml
The EVEN with descriptor is shown on page 35.

GEDCOM 5.5.5 released 10/2/2019 This is an update to correct errors only and not to provide additional functions.
GEDCOM Specifications www.gedcom.org/gedcom.html

GEDCOM 5.5.5: Just a Revision (tamurajones.net) www.tamurajones.net/GEDCOM555JustARevision.xhtml
Keith Riggle posted this about the EVEN tag at
genealogytools.com/the-event-structure-in-gedcom-files/
The partially expanded structure for a Family Event is as follows (you can find the full definition on pages 32 and 48 of the GEDCOM 5.5.1 standard):
n EVEN [<EVENT_DESCRIPTOR> | <NULL>]
+1 HUSB
+2 AGE <AGE_AT_EVENT>
+1 WIFE
+2 AGE <AGE_AT_EVENT>
+1 TYPE <EVENT_OR_FACT_CLASSIFICATION>
+1 DATE <DATE_VALUE>
+1 <<PLACE_STRUCTURE>>
You would think that, with the exception of the HUSB and WIFE elements, the Individual Event structure would be the same, but from the partially expanded structure that follows, we can see one crucial difference:
n EVEN
+1 TYPE <EVENT_OR_FACT_CLASSIFICATION>
+1 DATE <DATE_VALUE>
+1 <<PLACE_STRUCTURE>>
The crucial difference is that the Individual Event structure lacks the EVENT_DESCRIPTOR. Now why would this be? Clearly it was an oversight on the part of the authors and editors of the standard, because on page 48 of the standard, the following example is provided for an EVENT_DESCRIPTOR on an Individual Event:
1 EVEN Appointed Zoning Committee Chairperson
2 TYPE Civic Appointments
2 DATE FROM JAN 1952 TO JAN 1956
2 PLAC Cove, Cache, Utah
2 AGNC Cove City Redevelopment
The definition of EVENT_DESCRIPTOR even states that it is “Text describing a particular event pertaining to the individual or family” (emphasis added). So it should be obvious that omitting “[<EVENT_DESCRIPTOR> | <NULL>]” from the Individual Event structure was an oversight. Which brings me back to my original question, what should apps do about this mistake?



GEDCOM release schedule
date version status brief notes
1984 1.0 proposed standard First Version
1985-12 2.0 standard PAF 2.0
1987-02 2.1 standard PAF 2.1
1987-10-09 3.0 standard lineage-linked form
1989-08-04 4.0 standard
1991-09-25 5.0 standard lineage-linked structures
1992-09-18 5.1 draft
1993-11-04 5.3 draft Unicode, multimedia, name parts, schema (abandoned)
1995-08-21 5.4 draft citation-source-repository model, more name parts, submission record (obsolete)
1995-12-11 5.5 standard structured addresses, more name parts
1996-01-02 5.5 standard GEDCOM 5.5 with errata
1996-01-10 5.5 errata GEDCOM 5.5 errata sheet
1999-10-02 5.5.1 standard UTF-8, email, URLs, geographic coordinates
2000-12-18 5.5.2 unpublished draft no more escape sequences, GEDXML
2018-05-10 5.5.1 AE special release Annotated Edition; annotations & corrections.
2019-10-02 5.5.5 standard Maintenance release. Quality. Simpler & Stricter.

Re: error in import from Rootsmagic

Posted: 13 Jan 2021 19:49
by PeterR
The FH Sample Project has several records like this:
0 @O1@ OBJE
1 FILE Media\O1_The_Munros.jpg
2 FORM jpg
2 TITL The Munros
1 _DATE APR 1949
1 _KEYS Picture, Monochrome
1 CHAN
2 DATE 26 SEP 2014
3 TIME 15:16:31
What is the context of your rejected "2 FORM jpg"?

Re: error in import from Rootsmagic

Posted: 13 Jan 2021 20:07
by Peter_H_Williams
May I add my problem which appears similar in nature or should I raise it as new issue?

It concerns moving from 6.2 to 7.0 with a Gedcom that started life in DOS with Brothers Keeper, was transferred to TMG, then transferred to RootsMagic before moving to FH 5
I have 1,768 Individual Records but after upgrading to 7.0 I have only 41 Individual Records. The first time I did this last October FH7 it produced a log of 4,395 lines. I have done what I thought might be sufficient correction and tried again yesterday. The log is now only 411 lines but I still end up with only 41 Individual records.

Before going further should I begin a new thread or stay here?

Peter

Re: error in import from Rootsmagic

Posted: 13 Jan 2021 20:18
by tatewise
IMO the GEDCOM formal syntax rules carry more weight than the examples which know to have mistakes.
Events, both standard such as BIRT & DEAT and custom EVEN, have never had a descriptor value.
Attributes, both standard such as OCCU & EDUC and custom FACT, demand a descriptor value.

GEDCOM 5.5 had no FACT tag so did not allow custom Attributes, and some products chose workarounds.
FH added the _ATTR tag equivalent to FACT but many broke the rule and allowed EVEN to have a descriptor value.
GEDCOM 5.5.1 compliance should mean EVEN without descriptor values, but I suspect many products that broke the EVEN rule are continuing to do so for legacy convenience.

FH V7 should import both GEDCOM 5.5 and GEDCOM 5.5.1 compliant GEDCOM.
However, during beta testing, some GEDCOM 5.5 formats were not handled correctly and fixed.
Ideally, I think FH should be more tolerant and allow EVEN with descriptor values to be imported as FACT tags.

The FORM tag looks OK for GEDCOM 5.5.1

Does RM declare it is using GEDCOM 5.5.1 in the HEAD record?
Or is it mistakenly saying it is GECOM 5.5?

Peter H, if FH V6 was happily managing those Individual records they should all migrate to FH V7.
So that is a different problem altogether and needs its own thread, or simply report it directly to CP.

Re: error in import from Rootsmagic

Posted: 13 Jan 2021 21:25
by TMG_refugee
Mike,

I greatly value your expertise and knowledge and I am far from an expert on GEDCOM but from all that I have read it seems that the EVEN and FACT tag should be allowed to have a descriptor on the same line like:

1 EVEN some pertinent info on this fact

The info in the descriptor should be placed in the "{value}" field. FH did this when I imported FACT but seems to have now gone back to the EVEN model. I am not sure how to classify what FH has done with a fact that is uncategorized but is not connected to al the custom facts that I have created in FH which when FH does not reject the 1Fact with a descriptor it was linked to the custom fact that I had created. I cannot change Rootsmagic so I have to work with what I have so I hope FH can go back at least to the FACT tag allowing descriptor information. Therefore I can edit the GEDCOM file and change the 1 EVEN to 1 FACT and have the import work without errors.

When a fact or even is imported as uncategorized it appears to have been imported correctly but there is no connection to an actual fact type.

I have an update. I just experimented with a new small project to see if the EVEN or the FACT could be imported with a descriptor value. This test went back to the EVEN being rejected and the FACT being accepted and tied to my facts that I had created. I will see if I can reproduce the FACT being rejected but at least if the FACT with a descriptor is allowed then I can work with that. Grabbing at straws maybe something else in the GEDCOM file is causing this behavior. I will continue to test.

Rootsmagic is using GEDCOM 5.5.1 since at lest 2014 and states 5.5.1 in the header of the GEDCOM file.
0 HEAD
1 SOUR RootsMagic
2 NAME RootsMagic
2 VERS 7.6.3.0
2 CORP RootsMagic, Inc.
3 ADDR PO Box 495
4 CONT Springville, UT 84663
4 CONT USA
3 PHON 1-800-ROOTSMAGIC
3 WWW www.RootsMagic.com
1 DEST RootsMagic
1 DATE 12 JAN 2021
1 SUBM @SUB1@
1 FILE XXXXXXXXXXXXXX
1 GEDC
2 VERS 5.5.1
2 FORM LINEAGE-LINKED
1 CHAR UTF-8

On the jpg error messages. I have not found any practical error yet. All images seem to have been imported but I wonder what the hidden consequences might be.

Re: error in import from Rootsmagic

Posted: 13 Jan 2021 23:32
by AdrianBruce
tatewise wrote: 13 Jan 2021 20:18 IMO the GEDCOM formal syntax rules carry more weight than the examples which know to have mistakes.
...
Totally agree with you. The idea that the examples were written first and the syntax followed but they made a mistake - is, IMHO, fantasy. I believe that the syntax should be considered to be correct, but that's a belief, not evidence. I fear that people have believed the example, rather than the syntax because most of them haven't read the syntax and have programmed the examples, not the syntax. There - I've said it! ;)

Re: error in import from Rootsmagic

Posted: 13 Jan 2021 23:44
by TMG_refugee
Whether the documentation is correct or in error is not the important issue right now. If FH is going to go with EVEN and FACT with no descriptor then I have a large problem because I don't have a mechanism to get my data from RM to FH.

Is there any way to get a direction from CP on what they plan to do so we can address our real problems based on their direction?

Re: error in import from Rootsmagic

Posted: 14 Jan 2021 00:09
by tatewise
The rules that FH V7 should follow are that EVEN is NOT allowed a descriptor value and FACT is allowed a descriptor value.
Why on earth would GEDCOM 5.5.1 define EVEN and FACT tags to both allow a descriptor value?
They would then be identical, which is pointless.
Events do NOT have a value. Attributes do have a value. It is as fundamental as that.

If FH V7 is not importing GEDCOM to the rules above, then it is a bug.
But the context of the EVEN and FACT tags must be examined to determine the full scenario.
There are rules about where EVEN and FACT tags are allowed and they must always be followed with a TYPE tag.

I would bet that CP would say that FH is fully compliant with GEDCOM 5.5.1 and that the rules for EVEN and FACT are as I have stated above.

Re: error in import from Rootsmagic

Posted: 14 Jan 2021 12:25
by Peter_H_Williams
Mike

Thanks for your advice. I have referred it to CP.

Peter

Re: error in import from Rootsmagic

Posted: 14 Jan 2021 13:36
by TMG_refugee
Thank you all for your responses.

Based on Mike's comments I will go with this:
The EVEN tag will not allow a descriptor. The FACT tag will.

Bottom line for me is if I change all the EVEN that have descriptors to FACT with a descriptor then that is an acceptable path to take. I would not want to start on a path that might be altered by CP because I did not know how CP interprets these items.

Re: error in import from Rootsmagic

Posted: 14 Jan 2021 14:16
by tatewise
I am interested in your observation that previously defined FH Fact Types for custom fact Attributes did not match later imports from RM for the 'same' FACT & TYPE custom fact Attributes.
As long as the TYPE value matches the Fact Type name then they should apply to imported data.
The only caveat is that EVEN & TYPE only match Event definitions while FACT & TYPE only match Attribute definitions.

Earlier you say that RM uses GEDCOM 5.5.1 but unfortunately RM does not honour that specification.
Yes, it says VER 5.5.1 in the HEADer of exported GEDCOM, however, that is misleading.
It does not use FACT tags for custom Attributes but continues to use the EVEN tag with values on the same line.
In local media OBJEcts all the FILE, FORM & TITL tags are on the same level which is GEDCOM 5.5 format.
On the other hand, it does allow EMAIL & WWW & FAX tags in some contexts that are only specified in GECOM 5.5.1

The GEDCOM Assessment website also shows that RM does not fully support GEDCOM 5.5.1
See https://www.gedcomassessment.com/en/ass ... agic-7.htm

Also, my Export Gedcom File tests come to similar conclusions.

Therefore, RM is a weird hybrid mix of GEDCOM 5.5 and GEDCOM 5.5.1 so it is not surprising that FH gets confused.