* Importing from 'Generations'

Importing from another genealogy program? This is the place to ask. Questions about Exporting should go in the Exporting sub-forum of the General Usage forum.
Post Reply
User avatar
Elisa
Gold
Posts: 11
Joined: 19 Mar 2021 16:18
Family Historian: V6.2

Importing from 'Generations'

Post by Elisa »

I imported all my data into FH and found that it doesn't fit very well, so I am now considering a new import. I have found the guidelines Importing to Family Historian but need help please

I made up lots of custom Notes for example Cen61 - CN61, Cen71 - CN71, CenY - CENY etc
these have only imported under the ALL tab and are not in a format that makes for easy copy and pasting

The first step in the guidelines: Where possible before exporting your file, go through any custom tags and make sure any ones with standard GEDCOM tags are defined as such, for example, all Census Facts should have a tag of CENS. Don’t worry if you can’t do them all, as you can convert many of them later as required (see Keeping Required Custom Tags and Notes below).
so, do I convert all tags to CENS and how do I know now which other facts or notes ones are standard (apart from the Census ones which I know I made)?
I fear this is going to be a long slow process!

thanks
User avatar
tatewise
Megastar
Posts: 28341
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Importing from "Generations"

Post by tatewise »

I presume you are following the Import from Generations Family Tree sub-section.

There it mentions a Generations Pre-Load Source Clean plugin that is recommended to be run before importing the GEDCOM into FH, but the Knowledge Base has recently been migrated from an old format to a new format and that has gone missing in the process. So I am investigating.

However, I suspect what you have are Note records, which are probably listed in the Records Window in the Notes tab.
If so and they each have Links = 1 then the Clean Up Notes plugin from the Plugin Store should fix them.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
User avatar
Elisa
Gold
Posts: 11
Joined: 19 Mar 2021 16:18
Family Historian: V6.2

Re: Importing from "Generations"

Post by Elisa »

Thank you , Yes they are notes but they didn't appear under the notes tab
I will experiment with that then.

Before I get to that stage, I have noticed in the import error log, all of the EXCLUDED lines are level 3 notes. An example of a level 3 note is the "Detail" to the right of the source in the source citations. The log show:
EXCLUDED: Invalid line level. Branch ignored: "3 NOTE www.scienceandsociety.co.uk/index.asp"

These are not coming in. I have tried editing the GEDCOM file and replacing level 3 with level 2 and the information was then imported (there were no excluded lines in the log). However the text came in as a TEXT linked to the level 1 source, not as a detail of the source.

Is there a better way to deal with these rather than moving them to level 2?

Another way was to change "3 TEXT" to "3 NOTE". This then imports the information as a note under the source, form many records, but not sources of general notes. Any advice welcome!
User avatar
Elisa
Gold
Posts: 11
Joined: 19 Mar 2021 16:18
Family Historian: V6.2

Re: Importing from "Generations"

Post by Elisa »

Thank you
I was just trying to find the plug ins :)
User avatar
tatewise
Megastar
Posts: 28341
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Importing from "Generations"

Post by tatewise »

Most plugins are found in the Plugin Store as explained in the FHUG Knowledge Base Family Historian Plugins advice.

Could you attach some screenshots of the Notes so we can advise what to do.
See Forum Usage Tips under Attachments.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
User avatar
Elisa
Gold
Posts: 11
Joined: 19 Mar 2021 16:18
Family Historian: V6.2

Re: Importing from "Generations"

Post by Elisa »

Thanks again for your help

I still cannot find the "Generations Post Edit" plugin in the store or in the forum. Can you please direct me?

On the Notes, by changing all the TEXT tags to NOTE and raising all the level "3 NOTE"s to "2 NOTE"s I have managed to stop any EXCLUDED lines in the log, and the information comes in one level higher - but is there at least!
User avatar
tatewise
Megastar
Posts: 28341
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Importing from "Generations"

Post by tatewise »

Sorry, we are not doing very well :oops:

I think the 'Generations Pre-Load Source Clean' plugin is intended to fix the TEXT/NOTE problem but must be run on the Generations GEDCOM file prior to importing into FH.

The missing plugin, and another plugin I have discovered, are attached below.
[ EDIT Attachments deleted as no longer needed. ]
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
User avatar
Elisa
Gold
Posts: 11
Joined: 19 Mar 2021 16:18
Family Historian: V6.2

Re: Importing from "Generations"

Post by Elisa »

Thank you - that's very kind of you

I need to get out for some fresh air now!
:)
User avatar
ColeValleyGirl
Megastar
Posts: 5465
Joined: 28 Dec 2005 22:02
Family Historian: V7
Location: Cirencester, Gloucestershire
Contact:

Re: Importing from "Generations"

Post by ColeValleyGirl »

The two plugins "Generations pre-load source clean" and "Generations - Post Edit" are now linked from the KB article Importing to Family Historian and can be removed from this thread.

The third plugin is no longer required.
User avatar
tatewise
Megastar
Posts: 28341
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Importing from "Generations"

Post by tatewise »

Helen, is this a change in policy, because I thought Plugins were not going to be hosted in the FHUG KB?
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
User avatar
ColeValleyGirl
Megastar
Posts: 5465
Joined: 28 Dec 2005 22:02
Family Historian: V7
Location: Cirencester, Gloucestershire
Contact:

Re: Importing from "Generations"

Post by ColeValleyGirl »

They're not hosted on the KB; the KB just links to the location where they are hosted.
User avatar
tatewise
Megastar
Posts: 28341
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Importing from "Generations"

Post by tatewise »

Nevertheless, they are hosted in the FHUG web domain. So can other plugin authors use that? :D
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
User avatar
ColeValleyGirl
Megastar
Posts: 5465
Joined: 28 Dec 2005 22:02
Family Historian: V7
Location: Cirencester, Gloucestershire
Contact:

Re: Importing from "Generations"

Post by ColeValleyGirl »

They're Jane's plugins hosted in Jane's space at Jane's cost. What do you think?

They're not in the plugin store because they're specific to a single purpose, and could do damage if they were run on the wrong files by users who aren't importing from Generations.
User avatar
tatewise
Megastar
Posts: 28341
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Importing from "Generations"

Post by tatewise »

Just struck me as unusual rather than the Plugin Store.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
User avatar
Jane
Site Admin
Posts: 8508
Joined: 01 Nov 2002 15:00
Family Historian: V7
Location: Somerset, England
Contact:

Re: Importing from 'Generations'

Post by Jane »

As Helen said, I did not want them in the plugin store as they are specific to problems caused only when loading some old Generations files.
Jane
My Family History : My Photography "Knowledge is knowing that a tomato is a fruit. Wisdom is not putting it in a fruit salad."
User avatar
dichorionic
Gold
Posts: 25
Joined: 03 Jan 2016 17:57
Family Historian: V7
Location: Devon

Re: Importing from 'Generations'

Post by dichorionic »

Over the last decade I've tried several times to migrate my main file of ~2000 individuals and ~2000 sources to FH and give up after a few days. The main problem is the huge number of UDFs relating to inline Notes citations, the handling of the Details note, as noted above, but also the anomalous user-defined fields that Generations allowed. With each FH update I hope that it will improve the situation but, while I no longer get exceptions, the UDFs have actually increased. The user-defined fields I can manage, but I suspect the inline citations, especially those relating to multiple individuals may have to be tackled manually and that may take weeks.

Generations was excellent for its time but I am well aware that it ceased production nearly 20 years ago and few family historians will still have it. Does anyone have any further suggestions?
Peter
User avatar
Jane
Site Admin
Posts: 8508
Joined: 01 Nov 2002 15:00
Family Historian: V7
Location: Somerset, England
Contact:

Re: Importing from 'Generations'

Post by Jane »

It's likely the notes could be converted using a plugin, but over the years I have found different releases of generations have slightly different formats.

When the notes are exported to gedcom does it include markers in the notes for the location of the link?
Jane
My Family History : My Photography "Knowledge is knowing that a tomato is a fruit. Wisdom is not putting it in a fruit salad."
User avatar
tatewise
Megastar
Posts: 28341
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Importing from 'Generations'

Post by tatewise »

Peter, if you can post some representative fragments of GEDCOM or All tab UDF and describe what they represent and how you want them to appear in FH, then I might be able to construct a Plugin to convert them. No promises until I've seen the problem. :D
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
User avatar
dichorionic
Gold
Posts: 25
Joined: 03 Jan 2016 17:57
Family Historian: V7
Location: Devon

Re: Importing from 'Generations'

Post by dichorionic »

Jane: Thank you for your reply. Simple answer is 'No'. There are no markers within the gedcom Notes field (see below). Furthermore, I have been tying up some of the UDF List entries to the records in Generations & FH. The first Notes citation is included in FH but the remainder are starred as UDFs. However (and a big one), I have come across a couple of records where a further citation has been omitted in the gedcom!

Mike: Again, thank you.

0 @N64@ NOTE
1 CONC executor to father. 1854 admitted as pensioner to Queens' College, Cambridge University. MA from London University Subscribed to "The Organ, its History and Construction" in 1855
2 SOUR @S102@
2 SOUR @S246@
2 SOUR @S194@
2 SOUR @S624@

In FH only @S102@ (the first citation) is shown. The other three are UDFs. However, the gedcom omits a fifth source citation to @S1857@ completely.

Just to complicate matters further (gulp), The Generations Address field may contain several locations over a lifetime with, as a result, several source citations. I can change the Address field to Residence but I suspect I may have to replace the single Address field with a Residence field for each location.
User avatar
tatewise
Megastar
Posts: 28341
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Importing from 'Generations'

Post by tatewise »

Clearly, if Generations is omitting data from the exported GEDCOM file then there is nothing we can do about that.

I have been trying to repeat your symptoms for the Note records with UDF Source Citations.
The behaviour of FH V7.0.3 is different to FH V6.2.7 and both differ from your description.
The behaviour is similar whether New Project... Import GEDCOM file is used or the standalone GEDCOM file is opened.

FH V7.03 completely excludes those invalid 2 SOUR @S...@ lines so any fix must be applied directly to the GEDCOM file.

FH V6.2.7 reports all those invalid 2 SOUR @S...@ lines as UDF but as soon as a Save is performed they all get deleted!
A plugin could fix those providing it is run before a Save is performed.

I never get the first 2 SOUR @S...@ line accepted as a valid Source Citation.
Can you post an FH screenshot of the Note record All tab that shows those Source Citations & UDF.

The error is the leading digit 2 that should be 1 but a global edit of the GEDCOM file to change 2 SOUR to 1 SOUR will upset other valid 2 SOUR lines as the problems only applies to Note records.

Regarding Address fields, I think you are saying they are imported as a custom fact or perhaps a UDF.
Either way, I suspect the Change Any Fact Tag plugin can convert those to Residence facts.
However, I would need to see some sample GEDCOM or FH All tab screenshots to be certain.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
User avatar
dichorionic
Gold
Posts: 25
Joined: 03 Jan 2016 17:57
Family Historian: V7
Location: Devon

Re: Importing from 'Generations'

Post by dichorionic »

Hi Mike,

Many thanks indeed for your helpful post. I have spent many hours over the last few days experimenting and testing Change Any Fact.

Firstly, an admission. I have not lost any data. I had, erroneously, been comparing the migrated file with an earlier iteration. A detailed comparison of the files revealed the 'missing' data. Sorry.

A couple of UDF tags I do not need to migrate at all, so I've excluded those.
For my file HEAL, MILI, DIV, SEPA and DEGR in Generations are more easily handled on a case by case basis and excluding the UDFs afterwards.

I have Changed Any Fact Tag individually for ADDR in Generations to RESI without a problem. I mention this as Generations would include several addresses with several sources in one Tag, rather than multiple Tags. The migration proved to be similar to Occupation where, again, several occupations and sources were represented. In due course, I shall work through all these Residences and Occupations separating them into separate Tags.

Generations Notes remains a source of concern. I attach a screenshot of the Generations field.
Lewis James Ibbs Generations Note.jpg
Lewis James Ibbs Generations Note.jpg (57.6 KiB) Viewed 8139 times
Here is the gedcom code:

0 @N85@ NOTE Daughters Frances Mary Louisa and Eliza Edna with Lewis James snr in 1861 census. Other children with Edna, while he was visiting George Mavin. Owned or occupied 4 properties in 1862. Trustee Bedford Charity 1862. Died ?1883-84, (Decease
1 CONC d in WFG's marriage but not so stated for LJJ's and TC's) but Edna described as 'Head' in 1871 census. Not listed in 1881, 1891, 1901 census. Received Family Bible 25 Feb 1847.
1 SOUR @S1272@
2 SOUR @@S1273@@
2 SOUR @@S119@@
2 SOUR @@S1322@@
2 SOUR @@S39@@

Here is the FH screenshot:
Lewis James Ibbs FH Note.jpg
Lewis James Ibbs FH Note.jpg (52.31 KiB) Viewed 8139 times
and the All Tab:
Lewis James Ibbs FH Note All.jpg
Lewis James Ibbs FH Note All.jpg (54.78 KiB) Viewed 8139 times
You'll see that only the first Source is cited, the remaining are UDFs. However, the first source is level 1 while the remainder are, for some reason, level 2. This must have originated from Generations. However, I also note that these fields have double ampersand delimiters e.g. @@S119@@. I have seen this with other Tags but have no idea why.

I don't want to lose these source citations, obviously. Amending every Note Tag Source citation to level 1 in this gedcom file manually would be a huge task.

On a separate note, when the ADDR Tag was converted to RESI I saw that the address(es) appear as a Source Note quoting the ADDR UDF Tag rather than as text within the Residence field. Is there a reason for this?

Many thanks for your assistance so far. I have certainly had a steep learning curve these last few days!
Peter
User avatar
tatewise
Megastar
Posts: 28341
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Importing from 'Generations'

Post by tatewise »

ADDR UDF
The reason ADDR addresses become a Source Note is explained in the Change Any Fact Tag plugin Help & Advice.
Presumably, those addresses are on the same line as the ADDR tags, just as jobs are on the same line as OCCU tags.
The RESI tag doesn't allow addresses on the same line so the plugin puts any data that cannot be converted into a Source Note.

It may be better if you use Tools > Fact Types... to define a new custom Attribute (not Event) called Address and use Change Any Fact Tag to convert from the ADDR UDF to the Address Attribute and then the addresses will be kept with the Address fact. I can provide more precise details if required.

NOTE SOUR UDF
I must admit I am baffled by your descriptions of those Note record Sources.
Your latest posting of GEDCOM and screenshot are very different from your Thu 22nd Apr 2021 description.
Both still differ from my experiments of importing what you described into FH V6.2.7.
It would be useful to see the Generations GEDCOM before it is imported to FH so I can replicate your symptoms.
Exactly what FH version are you using? Use Help > About Family Historian... to check.

When a link such as 2 SOUR @S1273@ becomes a UDF, the link is invalid, so FH turns it into plain text.
To do that GEDCOM requires that each @ is replaced by @@ so it does not get mistaken as a link.

Once I can replicate your description of the symptoms then I can write a small plugin to correct those SOUR UDF.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
User avatar
dichorionic
Gold
Posts: 25
Joined: 03 Jan 2016 17:57
Family Historian: V7
Location: Devon

Re: Importing from 'Generations'

Post by dichorionic »

P.S. - Incidentally, there was another set of UDF Source Texts which related to Detail on a Source citation in Generations. An example is given below of a citation to an LDS FamilySearch film.
Detail.jpg
Detail.jpg (54.22 KiB) Viewed 8127 times
This seems to have been a Generations idiosyncrasy. Why they were created I'm not sure as the obvious place to put the Detail is within the citation itself. Fortunately, I only have 23 so they will also be enterred manually.
User avatar
dichorionic
Gold
Posts: 25
Joined: 03 Jan 2016 17:57
Family Historian: V7
Location: Devon

Re: Importing from 'Generations'

Post by dichorionic »

Thank you, again. I wasn't expecting such a rapid response.
I have checked FH and it is v.6.2.7

Mike,

Thank you. I understand about the ADDR > RESI transfer, and you're right, all the addresses in Generations were on a single line with the citations in-line.

My apologies for any inconsistency between my post of 22 April and today. As I mentioned, I had inadvertently used an earlier version of the Generations file. The example I gave appears as this in Generations:
Robert Giles Note.jpg
Robert Giles Note.jpg (59.3 KiB) Viewed 8123 times
and the Gedcom should, instead, have read:

0 @N64@ NOTE executor to father. 1854 admitted as pensioner to Queens' College, Cambridge University. MA from London University Subscribed to "The Organ, its History and Construction" in 1855 Declared bankrupt 16/11/1880 and annulled 07/01/1881
1 SOUR @S102@
2 SOUR @@S246@@
2 SOUR @@S194@@
2 SOUR @@S624@@
2 SOUR @@S1857@@

and on FH
Robert Giles FH.jpg
Robert Giles FH.jpg (50.79 KiB) Viewed 8123 times
You'll see that the first source is level 1, while the remainder are level 2

I was going to attach a 'clean' copy of the pre - processed gedcom, without the practice edits and replacements I've been doing in recent days. However, I received an error message saying the file was too large.
Peter
User avatar
tatewise
Megastar
Posts: 28341
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Importing from 'Generations'

Post by tatewise »

Peter, just fragments of the Generations GEDCOM for the two Notes & Sources will do.

The HEAD record at the top of the Generations GEDCOM would be useful too.

Also, try the attached Plugin Generations - Post Edit that is a variant of the one in Importing to Family Historian.
I have adjusted it to work for your form of SOUR UDF.
On about line 70 the ref '~.TEXT.SOUR' is replaced with '~.SOUR.SOUR'
Maybe some Generations versions produce the invalid NOTE.TEXT.SOUR but this one is producing NOTE.SOUR.SOUR
Last edited by tatewise on 01 Feb 2024 18:07, edited 1 time in total.
Reason: Attachment deleted - contact Mike Tate if needed
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
Post Reply