Handling Uncategorised Data Fields

 Introduction

Uncategorised Data Fields (UDF) are data fields that Family Historian does not recognise as its own GEDCOM Data References, but have been retained for review, typically following the load of a GEDCOM file created by another application. These are often fields that are user-defined by other products, such _AKA (Also Know As) in PAF, or _MILT (Military) and _FOOT (Footnote) in Ancestry and FTM, or completely invalid GEDCOM fields such as CE81 in Generations.

A UDF will have an Item value as well as the Data Reference Tag referred to above.  When a UDF is deleted, both the Data Reference Tag and UDF Item value will be deleted.

Details

ƒh Version 7 creates many fewer UDF fields than Version 6 due to its enhanced direct import from other applications such as RootsMagic and Legacy.  The description on this page applies equally to both versions.  Always bear in mind when reviewing UDFs that they all contain data that was relevant in the originating application.  It may just be formatting data that is not used in ƒh, but be careful that you do not accidentally discard important data.  ƒh not understanding a field doesn’t mean that it is not potentially valuable!

Use Tools > Preferences… and look at the File Loading Options under File Load/Save to see how UDFs are being handled on import. The options are documented at Preferences Dialog: File Load/Save. For all options except Exclude, ƒh will create UDFs if it finds data it does not recognise, and you will then need to make decisions about how to handle them.

There are too many products and too many UDF to list them all here, so some investigation is needed to establish what data values they hold, and whether that is important. If necessary ask for advice in the FHUG Forums on Importing and Exporting.

The first time a GEDCOM file is opened by ƒh, (including opening a file from a previous Version of ƒh), it will report all UDF and other issues in the Exception Report, but once Saved they will no longer be reported on load. The report can be viewed using Window > Log Files.

Some popular user-defined Data Reference Tags are converted automatically to ƒh compatible tags and reported in a Process imported GEDCOM extensions log.

Each Exception Report for a UDF typically has the format:

I.6789 - INFO ONLY: Loaded uncategorised data (non-GEDCOM)

above this will be a line with the format:

Record Type=Individual. Gedcom Id=P4321. Record Number=1234.

where the Record Type may also be Family, Source, etc; the Gedcom Id= gives the pre-import record id; and Record Number gives the post-import record id.

If you are happy there is no useful data mentioned in the Exception Report, then the quickest way to proceed is to use Tools > Preferences… and modify the File Loading Options under File Load/Save. Select the two exclude options. The next time the Project is opened, all UDF will be deleted, but you must use Save to commit the changes to the GEDCOM file. We advise then restoring the two options to their Installation Settings.

By default ƒh will load all UDF fields, and they will appear in the Property Box on the All tab with a * star bullet point next to them. Use Tools > Uncategorised Data > Find Uncategorised Data… command to obtain a Named List of affected records.

The UDF List Plugin

The easiest way to find all UDFs is to use the UDF List Plugin. It will list full UDF details, and you can selectively:

  • Convert some UDF using a custom Plugin for the other product (see Import from… advice for that product)
  • Convert some UDF to valid fields using Tools > Uncategorised Data > Recover Uncategorised Data
  • Convert some UDF to ƒh Standard/Custom Facts using the Change Any Fact Tag Plugin (see below)
  • Manually copy the values to valid fields, and delete the UDF
  • Delete the UDF if of no interest

To delete UDFs, select the desired fields and press the keyboard DELETE key. In the Result Set of the UDF List Plugin, select any from the first column UDF Item, and Delete. There will be a warning message before confirming with OK. If after checking the effects, you change your mind, use Edit > Undo Delete Selected Data Items before closing ƒh.

Alternatively, it may be possible to change settings in the other product and export the GEDCOM file again.

The Change Any Fact Tag Plugin

Many other products allow Facts to be added that have user-defined GEDCOM fields, which ƒh treats as UDF, but can be converted to conventional Standard/Custom Facts by following the steps below.

  1. The first task is to identify those candidate UDF field Tag names.
    • One method is to run the UDF List Plugin, and in its Result Set examine the Item Data Reference column. All the candidates will only have two Tag names separated by a dot such as INDI._ELEC or INDI._MILT or FAM._SEPR, where the first tag is either INDIor FAMand the second tag often begins with an underscore (_). Click on the Item Tag or Item Data Reference column heading to sort into order.
    • Another method is to run the Change Any Fact Tag Plugin, set the Source Tag Set: to <UDF Tags>, and the Source Tag Name: drop-list will identify all candidate INDI UDF tags. Open the Family Records tab and repeat to identify any candidate FAM UDF tags.
  2. The second task is to discover each Fact Name used in the other product, so it can be represented in ƒh. This may involve some detective work, but some popular tags and names are listed later. Where possible, consider using GEDCOM Standard Facts, or existing Fact Sets such as the Extended Set, instead of creating new Custom Facts.
    • Download & install any required Fact Sets to create sets of Custom Facts.
    • To create a brand new Custom Fact use Tools > Fact Types, click the New button, enter the Fact Name, choose the Fact Type & Record Type, then click Create, and in the Fact Definition dialogue click OK (the details can be adjusted later). Mistakes can be corrected by using the Delete button, and starting again with the New button.
  3. Finally, run the Change Any Fact Tag Plugin, and click the Obtain Help & Advice button for details.
    • Set the Source Tag Set: to <UDF Tags>.
    • Select a Source Tag Name: from its drop-list.
    • Set the Target Tag Set: to Standard for Standard GEDCOM Facts, or the name of a downloaded Fact Set, or Custom for brand new Custom Facts.
    • Select the desired Target Tag Name: from its drop-list.
    • To perform the change click the green button below and use the Confirm Edit dialogue.
    • Repeat the above steps for each UDF, and switch to the Family Records tab as necessary.

Unfortunately, fact UDF Tags will often have subsidiary UDF tags, which the Plugin does its best to correct, but if it cannot they become Source Note text, that will need manual edits or another custom Plugin to fix them.

UDF Tags & Fact Names

Since ƒh V6.1.4 a new Fact Set called Extended Set is automatically installed that supports Ancestry and FTM predefined facts, so many of the facts listed below are handled automatically.

For each UDF Tag this table lists its Fact Name and recommended Fact Type & Fact Record for the Tools » Fact Types New dialogue. The Possible Alternative may be a Standard Fact, or downloadable Custom Fact in a Fact Set, or another way of handling that UDF Tag.

This list is currently focussed on Ancestry and FTM, but others can be added.

UDF Tag Fact Name Fact Type Fact Record Possible Alternative
_ARRI Arrival Attribute Individual Standard Immigration
_CIRC Circumcision Event Individual Standard Physical Description
_DCAUSE Cause of Death Attribute Individual Migrate to Standard Death Event Cause using dedicated Plugin
_DEG Degree Attribute Individual Standard Graduation or Education
_DEP Departure Attribute Individual Standard Emigration
_DEST Destination Attribute Individual Standard Emigration
_DNA DNA markers Attribute Individual
_ELEC Elected Attribute Individual
_EMPLOY Employment Attribute Individual Standard Occupation
_EXCM Excommunication Event Individual
_FUN Funeral Event Individual Standard Burial or Cremation
_HEIG Height Attribute Individual Military History Height (Military) or Standard Physical Description
_INIT Initiatory (LDS) Event Individual
_KNOW Known As Attribute Individual
_MDCL Medical Condition Attribute Individual Military History Medical History
_MILTID Military ID Attribute Individual Military History Military Service No.
_MILT Military Service Attribute Individual Military History Military Service
_MISN Mission (LDS) Attribute Individual
_NAMS Namesake Attribute Individual
_ORDI Ordinance Attribute Individual
_ORIG Origin Attribute Individual Standard Emigration
_SEPR Separation Event Family Standard Annulment
_WEIG Weight Attribute Individual Military History Weight (Military) or Standard Physical Description
ADDR Web Address Attribute Individual Migrate ADDR, EMAIL, PHON data to a Contact Info Attribute and then delete these three UDF Tags.
EMAIL Email Attribute Individual
PHON Phone Number Attribute Individual
Last update: 22 Mar 2022