* FTM to FH - Citation to Source

Questions regarding use of any Version of Family Historian. Please ensure you have set your Version of Family Historian in your Profile
Post Reply
avatar
quarlton
Diamond
Posts: 84
Joined: 26 Feb 2004 13:07
Family Historian: V7
Location: Lincolnshire
Contact:

FTM to FH - Citation to Source

Post by quarlton » 30 Jan 2019 09:13

Hi

I've been given a gedcom from FTM 2012 which loads OK after following the guides on this forum.
However, the originator has used a blanket source for events such as 1881 Census.
They have then added a citation to each person's event, with attached media.

Is there a way, either simply or using code, to convert the citations to a source.

Many thanks

Example as below:

1 CENS
2 DATE 31 MAR 1901
2 PLAC 3 Wilmslow St, Stockport, Cheshire
2 SOUR @S6@
3 PAGE Family of James Herbert MARSDEN & Eliza DAVIES in 1901
3 DATA
4 TEXT PLAC: 3 Wilmslow St, Stockport, Cheshire
5 CONT PAGE: RG13, piece 3288, folio 31, page 19
5 CONT NAME: James H. MARSDEN, Head, M, age 24, Railway Platelayer, Stockport, Cheshire
5 CONT NAME: Eliza MARSDEN, Wife, M, age 25, Cotton Mill Weaver, Matlock, Derbyshire
5 CONT NAME: William MARSDEN, Son, age 4, Heaton Norris, Lancashire
5 CONT NAME: Louisa MARSDEN, Daur, age 2, Heaton Norris, Lancashire
3 OBJE @O57@
Dave Simpson ~ Boulton, Braham, Carney, Simpson and Jacobs

User avatar
tatewise
Megastar
Posts: 21534
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: FTM to FH - Citation to Source

Post by tatewise » 30 Jan 2019 10:57

Yes, it should be possible with a Plugin.
The assumption is that the Census fields including all the Citation fields are identical for all members of the same household.
In your example that would be James H. MARSDEN, Eliza MARSDEN, William MARSDEN & Louisa MARSDEN.
(Although their Individual record Names may be slightly different.)

This was discussed at length in Converting Sources: Method 2 to Method 1 (16509).

It is tricky to come up with a generic Plugin that works for all cases because it depends on exactly which fields have been replicated in each Citation.

In your example the Where within Source (PAGE) field could become the Method 1 Source record Title.
The Text From Source (TEXT) would migrate to the Source record Text From Source.
The Media link (OBJE) would migrate to the Source record Media tab.
The Plugin would keep track of the Census & Citation field values that spawned a Source record.
Thus if those same field values were encountered again, the same Source record Citation would be applied.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

avatar
quarlton
Diamond
Posts: 84
Joined: 26 Feb 2004 13:07
Family Historian: V7
Location: Lincolnshire
Contact:

Re: FTM to FH - Citation to Source

Post by quarlton » 30 Jan 2019 12:20

Many thanks Mike.

I'll have a look at the reference (16509) you mentioned and have a bash.
Dave Simpson ~ Boulton, Braham, Carney, Simpson and Jacobs

avatar
Mark1834
Megastar
Posts: 707
Joined: 27 Oct 2017 19:33
Family Historian: V7
Location: South Cheshire

Re: FTM to FH - Citation to Source

Post by Mark1834 » 31 Jan 2019 14:56

Happy to assist if you want to take the Python route, as this is an easy modification of the general script described in the cited thread. You would need to install the free software, modify a script with my guidance and run it from the command line, so does require some familiarity with more general computer use. The processing is exactly as Mike described for a plug-in.
Mark Draper

avatar
Mark1834
Megastar
Posts: 707
Joined: 27 Oct 2017 19:33
Family Historian: V7
Location: South Cheshire

Re: FTM to FH - Citation to Source

Post by Mark1834 » 03 Feb 2019 12:52

Dave has tested a simplified Python script that does what he has described, and it can be posted here for others to use and develop as appropriate.

As posted here, it takes a single named source and splits it up such that each lumped citation becomes a new source with a name of the pattern Original Source Name: Lumped Citation (e.g. UK Census 1881: RG11/1234 folio 56 p7). It needs intermediate levels of PC literacy to run, as you have to be comfortable downloading and installing Python (free from http://www.python.org) and modifying the attached script to reflect your own source name and file paths/names.

Processing is exactly as Mike described above, with no duplication of sources and citation media automatically becoming source media.

Relatively minor changes to the script can be made to give additional or modified functionality, such as changing the template source name or adding the source type tag. Slightly more complex changes can be made to refer to the location or date of the cited event in the template name if required.

As written, it is designed for a GEDCOM file exported by another application, so is ideal for newcomers to FH. However, it could also be tweaked to work directly with the FH GEDCOM file if required. A proper plug-in would be preferred in the latter case due to ease of use, but as discussed above it is virtually impossible to generalise to a single plug-in given the wide variety of ways users have defined lumped sources.

Feel free to copy and play - it is a rather niche application, but could be useful for some people. The forum does not permit attaching Python scripts directly, so you need to change the filename from SourceSplit.txt to SourceSplit.py.
Attachments
SourceSplit.txt
(4.04 KiB) Downloaded 74 times
Mark Draper

avatar
quarlton
Diamond
Posts: 84
Joined: 26 Feb 2004 13:07
Family Historian: V7
Location: Lincolnshire
Contact:

Re: FTM to FH - Citation to Source

Post by quarlton » 03 Feb 2019 16:07

Thanks Mark

Your script did exactly what I needed to do, and was relatively straightforward (thanks to your notes) to tweak it for the specific sources.
Dave Simpson ~ Boulton, Braham, Carney, Simpson and Jacobs

avatar
cschroeder
Gold
Posts: 11
Joined: 14 May 2016 20:44
Family Historian: V6.2

Re: FTM to FH - Citation to Source

Post by cschroeder » 04 May 2021 18:39

Hi Mark,

the script works well for the first time, i have to modify it slightly (specify encoding for files and declare F0s as bool).
All Level 2 SOUR are splittet perfect, but the Level 1 SOUR are not changed.

I tried to modify the script but python is not my scripting language, can't get it working. Can you help me?

avatar
Mark1834
Megastar
Posts: 707
Joined: 27 Oct 2017 19:33
Family Historian: V7
Location: South Cheshire

Re: FTM to FH - Citation to Source

Post by Mark1834 » 04 May 2021 18:59

Ok - I’ll read your posts again for the details and suggest modifications. It may be the end of the week before I get back to you.
Mark Draper

avatar
cschroeder
Gold
Posts: 11
Joined: 14 May 2016 20:44
Family Historian: V6.2

Re: FTM to FH - Citation to Source

Post by cschroeder » 04 May 2021 19:32

that would be great

avatar
Mark1834
Megastar
Posts: 707
Joined: 27 Oct 2017 19:33
Family Historian: V7
Location: South Cheshire

Re: FTM to FH - Citation to Source

Post by Mark1834 » 09 May 2021 09:58

The Python script has now been modified to include sources linked to Records rather than just Facts. As before, it has been changed to a text file to allow attaching here, so if you want to use it, rename it to .py file.

Later versions of Python can be slightly fussy about the encoding of text files. FH7 saves its GEDCOM file as UTF-8 BOM, rather than standard UTF-8. If that causes a error, convert (a copy of it) to standard UTF-8 using a suitable text editor such as Notepad++.

Quite a few people have downloaded the original script considering how niche it is. It may be appropriate to convert it to a standard FH plugin to make it easier to use. The only reason this is in Python is that when I needed it, I could write Python and not plugins and Python is ideally suited to manipulating GEDCOM files.

I can see a use for a core plugin that makes the same conversion - all uses of a lumped generic source get converted to new sources based on Name:Where Within, with all attached media and notes moving into the new source. This could then be modified as necessary to include other details in the source title, such as the place and date of the supported fact.

It wouldn't work very well for RootsMagic sources, as they are all based on templates, but could be ideal for projects imported from Family Tree Maker. That uses generic sources, but they are all lumped, a format that FH doesn't handle very well.

Is anybody else working on this? If not, I'll put it on my to-do list. Not at the top, so it won't be released this week.... ;)
Attachments
SourceSplit (1.01).txt
(4.18 KiB) Downloaded 2 times
Mark Draper

User avatar
ColeValleyGirl
Megastar
Posts: 2774
Joined: 28 Dec 2005 22:02
Family Historian: V7
Location: Cirencester, Gloucestershire
Contact:

Re: FTM to FH - Citation to Source

Post by ColeValleyGirl » 09 May 2021 10:12

Mark, I'm (unsurprisingly) not working on it but I can definitely see a use (not mine - I'm a died in the wool splitter). It might be nice to go the other way as well -- I recently had cause to help somebody unsplit their GRO index sources -- luckily there were not so many that it couldn't be done manually, but could have been a nightmare. Identifying the 'pattern' to use for un-splitting might be more difficult though.

avatar
Mark1834
Megastar
Posts: 707
Joined: 27 Oct 2017 19:33
Family Historian: V7
Location: South Cheshire

Re: FTM to FH - Citation to Source

Post by Mark1834 » 09 May 2021 10:33

Interesting - I’ve not come across anyone wanting to convert that way around. It feels like a later development, so version 1 just does lumped to split. Perhaps set it up with a series of standard templates for the new source names - source/citation, source/citation/place/date, source/citation/name/date/place, etc. We don’t have to go overboard on complexity to cope with every conceivable configuration, but a few standard options should suit most users.
Mark Draper

User avatar
ColeValleyGirl
Megastar
Posts: 2774
Joined: 28 Dec 2005 22:02
Family Historian: V7
Location: Cirencester, Gloucestershire
Contact:

Re: FTM to FH - Citation to Source

Post by ColeValleyGirl » 09 May 2021 10:42

Mark, this came about as a result of somebody using the Civil Registration Index template and not understanding that particular template was designed for lumpers... I agree it might be a later development, and only if the demand is there.

avatar
cschroeder
Gold
Posts: 11
Joined: 14 May 2016 20:44
Family Historian: V6.2

Re: FTM to FH - Citation to Source

Post by cschroeder » 10 May 2021 17:27

thank you Mark,

the only problem i found is that some sources are created twice (INDI or FAM <-> facts), but merging these
sources was done within an hour...

you saved me dozens of hours of work.. really really nice

Post Reply