Flexible CSV Importer Plugin

Introduction

This plugin takes a CSV file with one line per record and adds people, facts and simple relationships to the fh Project. It provides an interface to map columns to fields.

It assumes that column headings follow the convention set out below. If a valid data reference is not found, the plugin will attempt to construct one. Simple terms like "DOB", "Place of Birth" or "Spouse's father" should be interpreted correctly.

CSV Conventions

Custom facts can be imported by entering the appropriate GEDCOM tag e.g. EVEN-CUSTOM.

Custom reference numbers (using the tag REFN) are best used to identify the same individual in several lines of data. For example, in a CSV of Christening and Burial records, or to identify the same father and mother across Christening records for each of their children.

The REFN field does not need to be numeric, alternatively, a scheme like I1, I2, I3 or F01, F02, M01, M02 or even "father", "mother", "child1" could be used.

Whilst the plugin is reasonably intelligent Field Mapping may optionally be set in the CSV file itself, using the convention (this is the approach in the example below):

IDENTIFIER.FIELD.SUBFIELD

Where

IDENTIFIER is the individual or family

P = Primary - the primary person of interest in the record
PF = Primary's father
PM = Primary's mother
S = Spouse - of the primary person
SF = Spouse's father
SM = Spouse's mother

Family identifiers are the two individuals separated by an underscore:

P_S = the primary person and their spouse
PF_PM = parent family of the primary person
SF_SM = parent family of spouse

FIELD is the GEDCOM tag for the fact. e.g.BIRT = Birth, DEAT = Death.

SUBFIELD is the GEDCOM tag for the detail of the fact.
e.g. DATE = Date, PLAC = Place, NOTE2 = Note.

For example, the primary person's birth place is: P.BIRT.PLAC

Marriage date of the primary person's parents: PF_PM.MARR.DATE, however, it is possible to specify this fact against the Parents’ own records (row) rather than at the child level.

Special Cases

Name fields are assumed to be full names ie. Given Surname, unless followed by a colon and followed by the name part GIVEN, SURNAME or SURNAME_FIRST. e.g. P.NAME:GIVEN.

Age at the time of a family event (e.g. marriage) is indicated by a colon and the individual's identifier. For example, P_S.MARR.Age:P and P_S.MARR.Age:S.

Multiple columns can be concatenated into the same field but the columns must be named in a logical manner and be unique e.g. TEXT1, TEXT2 or NOTE-A, NOTE-B. P.NOTE-A, P.NOTE-B would be combined and imported into P.NOTE using the concatenation format set in the selector on the first tab. This also works on the address fields but it may be preferable to format the address elements as one column with any required commas in the spreadsheet / CSV file as the use of 'newline' may have undesirable effects in the formatting.

Relationships

The marriage relationships are mapped in the column S.REFN.

A child is mapped to the parents using PF.REFN and PM.REFN.

Subsequent Marriages

When there is a second (or more) marriage the husband needs to be repeated and this repeated row holds the second marriage data. The husband is treated as the 'Primary' and the wife as 'Spouse'. The husband's data row needs to include the new marriage details before the new spouse. The import does not work correctly if you try and put the data on the spouse's row.

Sources

Separate source citation details can be added for each line. Column headings for citation elements are of the form:

SOUR>TITL = Source Title
SOUR.DATE = Entry Date
SOUR.QUAY = Assessment ~ 1, 2, 3, 4 or P, S, Q, U
SOUR.PAGE = Where within Source
SOUR.TEXT = Text From Source
SOUR.NOTE = Citation Note

Without a prefix as above, they are added to every new item. With an IDENTIFIER prefix, they add a whole record citation. With an IDENTIFIER.FIELD prefix they add Name & Fact citations.

If the SOUR>TITLE matches the chosen global Source record Title for every item imported then the Citation is applied to that link.

Sample Plugin Demonstration Data

You can download this sample data in CSV format to demonstrate some functions of the Plugin. It is strongly suggested that you run this plugin in a new, empty fh Project.

sample_import.csv Sample Import demonstration CSV download.

The data would normally be collated in a spreadsheet (Excel, LibreOffice etc.). It is important to save the output file in CSV format.

For clarity, the column headings in the sample data file are as follows:

REFN, P.NAME, P.SEX, P.CHR.DATE, P.CHR.PLAC, P.CHR.SOUR>TITL, P.BURI.DATE, P.BURI.PLAC, P.BURI.NOTE1, P.BURI.NOTE2, P.BURI.SOUR>TITL, PF.REFN, PM.REFN, S.REFN, P_S.MARR.DATE, P_S.MARR.PLAC, P_S.MARR.ADDR1, P_S.MARR.Address2, P_S.MARR.SOUR>TITL, P_S.MARR.AGE:P, P_S.MARR.AGE:S, P.OCCU, P.OCCU.DATE, P.OCCU.PLAC, P.OCCU.SOUR>TITL, P.OCCU.SOUR.PAGE, P.OCCU.SOUR.TEXT1, P.OCCU.SOUR.TEXT2, P.OCCU.SOUR.DATE, P.OCCU.SOUR.NOTE, P.NOTE1, P.NOTE2, P.NAME.SOUR>TITL, P.NAME.SOUR.PAGE, P.NAME.SOUR.TEXT, P.NAME.SOUR.DATE, P.NAME.SOUR.NOTE1, P.NAME.SOUR.NOTE2, P.SOUR>TITL, P.SOUR.PAGE, P.SOUR.TEXT, P.SOUR.DATE, P.SOUR.QUAY, P.SOUR.NOTE1, P.SOUR.NOTE2, P.CENS.DATE, P.CENS.PLAC, P.CENS.ADDR, P.CENS.SOUR>TITL

These are only examples of input data columns. In the sample data, note that individuals REFN = 1 and 2 are repeated to include additional facts.

Running the Plugin

Directions and Settings

Directions and Settings

When this dialog appears choose your import CSV file.

If no Source record is required (and no Citations) change the option to N.

There is a choice of separator for multiple note and text fields e.g. comma, semicolon, space, newline or two newlines.

Normally the REFN numbers in your import file will be retained in the Custom ID field. If this is not required change the setting to N.

Checking the Mapping

Field Interpretation

The second tab of the plugin is important. Check this before running the actual Import to your project (step 4).

If anything cannot be mapped in the process it appears red in the first column and can be corrected either in the input file and re-imported or modified in the mapping table.

The field interpretation can also be used to re-map specific columns to alternative tags e.g. birth to christening by selecting the dropdown arrows in the field label and detail label columns.

It is recommended that the data is "cleaned" ready for import in your spreadsheet before creating your CSV file rather than correcting mappings each time during import.

More Information

CC Attribution-Noncommercial-Share Alike 4.0 International
Runs using DokuWiki Recent changes RSS feed www.rjt.org.uk