* Families with Flexible CSV Importer

For users to report plugin bugs and request plugin enhancements; and for authors to test new/new versions of plugins, and to discuss plugin development (in the Programming Technicalities sub-forum). If you want advice on choosing or using a plugin, please ask in General Usage or an appropriate sub-forum.
Post Reply
avatar
gsward
Famous
Posts: 146
Joined: 11 Mar 2011 14:47
Family Historian: V7

Families with Flexible CSV Importer

Post by gsward »

I am trying to use the Flexible CSV Importer plugin. I am having difficulty getting the plugin to recognise my family links and marriage information. I have looked at the notes in the plugin itself and tried several variations without success.

These are my current CSV headings:

REFN,P.NAME.GIVEN,P.NAME.SURNAME,Sex,BirthDate,BirthPlace,DeathDate,DeathPlace,Father,Mother,Spouse,Marriage Date,Marriage Place,Occupation,Occupation Date,Occupation Place

I am putting the REFN number for the individual in the appropriate Spouse, Father and Mother fields but the plugin highlights these rows in red and does not import the links.

How should I set the data up to represent a second spouse when my first problem is sorted?

Thanks,

Graham
User avatar
Jane
Site Admin
Posts: 8507
Joined: 01 Nov 2002 15:00
Family Historian: V7
Location: Somerset, England
Contact:

Re: Families with Flexible CSV Importer

Post by Jane »

You do not appear to have read the help for the plugin which states
This plugin first assumes that column headings follow the convention set out below.
If a valid data reference is not found, the plugin will attempt to construct one.
Simple terms like "Place of Birth" or "Spouse's father" should be interpreted correctly.

Custom facts can be imported by entering the appropriate GEDCOM tag.

Custom reference numbers (using the tag REFN) can be used to identify the same individual in
several lines of data. For example, in a CSV of birth record transcipts to identify
the same father and mother across birth records for each of their children.

Field mapping may optionally be set in the CSV file itself, using the convention:
IDENTIFIER.FIELD.SUBFIELD

IDENTIFIER is the individual or family:
P = Primary - the primary person of interest in the record,
PF = Primary's father, PM = Primary's mother,
S = Spouse - of the primary person,
SF = Spouse's father, SM = Spouse's mother.

Family identifiers are the two individuals separated by an underscore:
P_S = the primary person and their spouse
PF_PM = parent family of primary person
SF_SM = parent family of spouse

FIELD is the GEDCOM tag for the fact. eg BIRT = Birth, DEAT = Death.

SUBFIELD is the GEDCOM tag for the detail of the fact.
eg DATE = Date, PLAC = Place, NOTE2 = Note.

For example, the primary person' birth place is: P.BIRT.PLAC
Marriage date of primary person's parents: PF_PM.MARR.DATE

SPECIAL CASES:
"Name" fields are assumed to be full names ie. Given Surname, unless followed by
a colon and followed by the name part GIVEN, SURNAME or SURNAME_FIRST. eg P.NAME:GIVEN.
Age at the time of a family event (eg marriage) is indicted by a colon and the individual's
identifier. For example, P_S.MARR.Age:P

Separate source citation details can be added for each line.
Column headings for citation elements are:
CITE.PAGE = where within source
CITE.TEXT = text from source
CITE.DATE = entry date
CITE.NOTE = note
At the minimum you need Primary's Father or PF.REFN etc, it's better to use the codes as defined to ensure the matching works, I have not used the plugin for a while, but when I have played codes avoid the program having to guess.
Jane
My Family History : My Photography "Knowledge is knowing that a tomato is a fruit. Wisdom is not putting it in a fruit salad."
avatar
gsward
Famous
Posts: 146
Joined: 11 Mar 2011 14:47
Family Historian: V7

Re: Families with Flexible CSV Importer

Post by gsward »

It was the syntax for the spouse that was missing

PF.REFN

If I want to add ages at marriage and use

P_S.MARR.Age:P

It creates duplicate marriage facts. How do I prevent that happening?

Thanks,

Graham
User avatar
tatewise
Megastar
Posts: 28333
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Families with Flexible CSV Importer

Post by tatewise »

I suspect you need to use:
P_S.MARR.HUSB.AGE and P_S.MARR.WIFE.AGE to match GEDCOM structure.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
avatar
gsward
Famous
Posts: 146
Joined: 11 Mar 2011 14:47
Family Historian: V7

Re: Families with Flexible CSV Importer

Post by gsward »

Worth a try, but unfortunately that did not work for me.

Graham
avatar
gsward
Famous
Posts: 146
Joined: 11 Mar 2011 14:47
Family Historian: V7

Re: Families with Flexible CSV Importer

Post by gsward »

After some more testing and trying different column headings and looking at the code, I have established a method that gets the marriage ages of both husband and wife attached to the event. I have also successfully entered a second marriage for one of the parties.

Here is a summary of my findings:

Age at marriage
Column headings are for the husband (P) and wife (S)
P_S.MARR.AGE:P
P_S.MARR.AGE:S


Relationships
Ther marriage relationships are mapped with the column
S.REFN
A child is mapped to the parents using
PF.REFN and PM.REFN

Subsequent marriages
When there is a second marriage the husband needs to be repeated and this repeated row holds the second marriage data. The husband is treated as the 'Primary' and the wife as 'Spouse'. Things do not work correctly on import if you try and put the data on the spouse's row. Also, the husband's data row needs to occur before his spouse.

Here is a subset of my data table (which is saved as a CSV file of course. I have omitted a few columns of additional data for clarity here e.g. Birth and Death dates and places etc.
Data.JPG
Data.JPG (74.24 KiB) Viewed 8716 times
Checking the mapping
The second tab of the plugin is useful, although I did need to expand the width each time to see the full extent of the mapping. Check this before running the Import step. I needed to pay particular attention to the rows P_S.MARR.AGE:P and
P_S.MARR.AGE:S to ensure they are mapped correctly to Marriage, Age: Primary and Age: Spouse. If anything is cannot be mapped in the process it appears red in the first column.
Mapper.JPG
Mapper.JPG (169.27 KiB) Viewed 8716 times
User avatar
tatewise
Megastar
Posts: 28333
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Families with Flexible CSV Importer

Post by tatewise »

Perhaps this Plugin would benefit from a guide in the [kb]|[/kb]?
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
avatar
gsward
Famous
Posts: 146
Joined: 11 Mar 2011 14:47
Family Historian: V7

Re: Families with Flexible CSV Importer

Post by gsward »

I have some more data to run through the plugin so will start a KB page as you suggest.

Graham
User avatar
tatewise
Megastar
Posts: 28333
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Families with Flexible CSV Importer

Post by tatewise »

May I suggest you add the help to plugins:help:index|> FH V5+ Plugins > Plugin Help and Advice for published Plugins.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
avatar
gsward
Famous
Posts: 146
Joined: 11 Mar 2011 14:47
Family Historian: V7

Re: Families with Flexible CSV Importer

Post by gsward »

I have now created a KB page for this plugin plugins:help:flexible_csv_importer|> Flexible CSV Importer Plugin.

Please take a look if you are interested, comments welcome and particularly if you find any additional usage tips.

Thanks,

Graham
avatar
gsward
Famous
Posts: 146
Joined: 11 Mar 2011 14:47
Family Historian: V7

Re: Families with Flexible CSV Importer

Post by gsward »

Mike and I are pleased to announce that a new version of the Flexible CSV Importer plugin is now available from the FH Plugin Store.

The latest version is now 1.5 which is probably an understatement compared with the previous version. All the dialogues have been re-written, there is new functionality and options for attaching sources and CSV syntax checking is improved.

The KB help page has been updated and includes a sample file and some suggested headings for your own CSV file to get you started.

An import tool is of course always depends on the CSV file you use and we cannot anticipate every scenario or inconsistency. If the plugin does not quite import the data as you expect you may, of course, be able to adjust your import file.
avatar
Mike
Silver
Posts: 8
Joined: 10 Jan 2019 16:46
Family Historian: V6

Re: Families with Flexible CSV Importer

Post by Mike »

Mike & Graham - this is what I’ve been seeking for some time - thank you and well done..........Mike
avatar
obstinatesnooper
Silver
Posts: 6
Joined: 12 Aug 2018 18:36
Family Historian: V6.2
Location: USA
Contact:

Re: Families with Flexible CSV Importer

Post by obstinatesnooper »

It would be very helpful if you could add to the Help file for the plugin that if the tags you are importing to are Attributes that the column heading should be P._ATTR-Name. (Or, whatever role instead of P the tag applies to.)

I gave up importing my group of nearly 1000 names until I accidentally learned that Attributes in FH take the _ATTR- label, and this information is probably useful for future users of the plugin (which is AWESOMESAUCE, btw).

Is there any way to import to flags directly? That can easily be accomplished after the fact using FH's tools, but to be able to skip a step would be useful.

Cheers,
Otis
User avatar
tatewise
Megastar
Posts: 28333
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Families with Flexible CSV Importer

Post by tatewise »

Thank you Otis.
The advice about using P._ATTR- for custom Attributes will be added to the Help soon.

To add Record Flags, or almost anything else not explicitly handled by the plugin, use valid GEDCOM tags.
So for Record Flags use P._FLGS.__PRIVATE or P._FLGS.__LIVING, etc, and put True in the column cell, or Private or Living as appropriate if you prefer, although any text will work.

I am actually surprised at the extent to which this plugin is being used. :D
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
avatar
obstinatesnooper
Silver
Posts: 6
Joined: 12 Aug 2018 18:36
Family Historian: V6.2
Location: USA
Contact:

Re: Families with Flexible CSV Importer

Post by obstinatesnooper »

There is a hard core group of users that see the value in organizing data with spreadsheets where all else fails, and when something pops up that allow that data to easily go into a database that can also export views of that data when needed, well... :idea: :D

Even TMG failed at being a good place to research this particular dataset ( :o ) when I was exploring how to track it way back when, but FH is spot-on. Now if I could only index the content of FH with DEVONthink.
Post Reply