* FTM to FH - Citation to Source

Questions regarding use of any Version of Family Historian. Please ensure you have set your Version of Family Historian in your Profile. If your question fits in one of these subject-specific sub-forums, please ask it there.
Post Reply
avatar
quarlton
Famous
Posts: 166
Joined: 26 Feb 2004 13:07
Family Historian: V7
Location: Lincolnshire
Contact:

FTM to FH - Citation to Source

Post by quarlton »

Hi

I've been given a gedcom from FTM 2012 which loads OK after following the guides on this forum.
However, the originator has used a blanket source for events such as 1881 Census.
They have then added a citation to each person's event, with attached media.

Is there a way, either simply or using code, to convert the citations to a source.

Many thanks

Example as below:

1 CENS
2 DATE 31 MAR 1901
2 PLAC 3 Wilmslow St, Stockport, Cheshire
2 SOUR @S6@
3 PAGE Family of James Herbert MARSDEN & Eliza DAVIES in 1901
3 DATA
4 TEXT PLAC: 3 Wilmslow St, Stockport, Cheshire
5 CONT PAGE: RG13, piece 3288, folio 31, page 19
5 CONT NAME: James H. MARSDEN, Head, M, age 24, Railway Platelayer, Stockport, Cheshire
5 CONT NAME: Eliza MARSDEN, Wife, M, age 25, Cotton Mill Weaver, Matlock, Derbyshire
5 CONT NAME: William MARSDEN, Son, age 4, Heaton Norris, Lancashire
5 CONT NAME: Louisa MARSDEN, Daur, age 2, Heaton Norris, Lancashire
3 OBJE @O57@
Dave Simpson ~ Boulton, Braham, Carney, Simpson and Jacobs
User avatar
tatewise
Megastar
Posts: 28333
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: FTM to FH - Citation to Source

Post by tatewise »

Yes, it should be possible with a Plugin.
The assumption is that the Census fields including all the Citation fields are identical for all members of the same household.
In your example that would be James H. MARSDEN, Eliza MARSDEN, William MARSDEN & Louisa MARSDEN.
(Although their Individual record Names may be slightly different.)

This was discussed at length in Converting Sources: Method 2 to Method 1 (16509).

It is tricky to come up with a generic Plugin that works for all cases because it depends on exactly which fields have been replicated in each Citation.

In your example the Where within Source (PAGE) field could become the Method 1 Source record Title.
The Text From Source (TEXT) would migrate to the Source record Text From Source.
The Media link (OBJE) would migrate to the Source record Media tab.
The Plugin would keep track of the Census & Citation field values that spawned a Source record.
Thus if those same field values were encountered again, the same Source record Citation would be applied.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
avatar
quarlton
Famous
Posts: 166
Joined: 26 Feb 2004 13:07
Family Historian: V7
Location: Lincolnshire
Contact:

Re: FTM to FH - Citation to Source

Post by quarlton »

Many thanks Mike.

I'll have a look at the reference (16509) you mentioned and have a bash.
Dave Simpson ~ Boulton, Braham, Carney, Simpson and Jacobs
User avatar
Mark1834
Megastar
Posts: 2458
Joined: 27 Oct 2017 19:33
Family Historian: V7
Location: South Cheshire, UK

Re: FTM to FH - Citation to Source

Post by Mark1834 »

Happy to assist if you want to take the Python route, as this is an easy modification of the general script described in the cited thread. You would need to install the free software, modify a script with my guidance and run it from the command line, so does require some familiarity with more general computer use. The processing is exactly as Mike described for a plug-in.
Mark Draper
User avatar
Mark1834
Megastar
Posts: 2458
Joined: 27 Oct 2017 19:33
Family Historian: V7
Location: South Cheshire, UK

Re: FTM to FH - Citation to Source

Post by Mark1834 »

Dave has tested a simplified Python script that does what he has described, and it can be posted here for others to use and develop as appropriate.

As posted here, it takes a single named source and splits it up such that each lumped citation becomes a new source with a name of the pattern Original Source Name: Lumped Citation (e.g. UK Census 1881: RG11/1234 folio 56 p7). It needs intermediate levels of PC literacy to run, as you have to be comfortable downloading and installing Python (free from http://www.python.org) and modifying the attached script to reflect your own source name and file paths/names.

Processing is exactly as Mike described above, with no duplication of sources and citation media automatically becoming source media.

Relatively minor changes to the script can be made to give additional or modified functionality, such as changing the template source name or adding the source type tag. Slightly more complex changes can be made to refer to the location or date of the cited event in the template name if required.

As written, it is designed for a GEDCOM file exported by another application, so is ideal for newcomers to FH. However, it could also be tweaked to work directly with the FH GEDCOM file if required. A proper plug-in would be preferred in the latter case due to ease of use, but as discussed above it is virtually impossible to generalise to a single plug-in given the wide variety of ways users have defined lumped sources.

Feel free to copy and play - it is a rather niche application, but could be useful for some people. The forum does not permit attaching Python scripts directly, so you need to change the filename from SourceSplit.txt to SourceSplit.py.
Attachments
SourceSplit.txt
(4.04 KiB) Downloaded 185 times
Mark Draper
avatar
quarlton
Famous
Posts: 166
Joined: 26 Feb 2004 13:07
Family Historian: V7
Location: Lincolnshire
Contact:

Re: FTM to FH - Citation to Source

Post by quarlton »

Thanks Mark

Your script did exactly what I needed to do, and was relatively straightforward (thanks to your notes) to tweak it for the specific sources.
Dave Simpson ~ Boulton, Braham, Carney, Simpson and Jacobs
avatar
cschroeder
Gold
Posts: 22
Joined: 14 May 2016 20:44
Family Historian: V6.2

Re: FTM to FH - Citation to Source

Post by cschroeder »

Hi Mark,

the script works well for the first time, i have to modify it slightly (specify encoding for files and declare F0s as bool).
All Level 2 SOUR are splittet perfect, but the Level 1 SOUR are not changed.

I tried to modify the script but python is not my scripting language, can't get it working. Can you help me?
User avatar
Mark1834
Megastar
Posts: 2458
Joined: 27 Oct 2017 19:33
Family Historian: V7
Location: South Cheshire, UK

Re: FTM to FH - Citation to Source

Post by Mark1834 »

Ok - I’ll read your posts again for the details and suggest modifications. It may be the end of the week before I get back to you.
Mark Draper
avatar
cschroeder
Gold
Posts: 22
Joined: 14 May 2016 20:44
Family Historian: V6.2

Re: FTM to FH - Citation to Source

Post by cschroeder »

that would be great
User avatar
Mark1834
Megastar
Posts: 2458
Joined: 27 Oct 2017 19:33
Family Historian: V7
Location: South Cheshire, UK

Re: FTM to FH - Citation to Source

Post by Mark1834 »

The Python script has now been modified to include sources linked to Records rather than just Facts. As before, it has been changed to a text file to allow attaching here, so if you want to use it, rename it to .py file.

Later versions of Python can be slightly fussy about the encoding of text files. FH7 saves its GEDCOM file as UTF-8 BOM, rather than standard UTF-8. If that causes a error, convert (a copy of it) to standard UTF-8 using a suitable text editor such as Notepad++.

Quite a few people have downloaded the original script considering how niche it is. It may be appropriate to convert it to a standard FH plugin to make it easier to use. The only reason this is in Python is that when I needed it, I could write Python and not plugins and Python is ideally suited to manipulating GEDCOM files.

I can see a use for a core plugin that makes the same conversion - all uses of a lumped generic source get converted to new sources based on Name:Where Within, with all attached media and notes moving into the new source. This could then be modified as necessary to include other details in the source title, such as the place and date of the supported fact.

It wouldn't work very well for RootsMagic sources, as they are all based on templates, but could be ideal for projects imported from Family Tree Maker. That uses generic sources, but they are all lumped, a format that FH doesn't handle very well.

Is anybody else working on this? If not, I'll put it on my to-do list. Not at the top, so it won't be released this week.... ;)
Attachments
SourceSplit (1.01).txt
(4.18 KiB) Downloaded 104 times
Mark Draper
User avatar
ColeValleyGirl
Megastar
Posts: 5464
Joined: 28 Dec 2005 22:02
Family Historian: V7
Location: Cirencester, Gloucestershire
Contact:

Re: FTM to FH - Citation to Source

Post by ColeValleyGirl »

Mark, I'm (unsurprisingly) not working on it but I can definitely see a use (not mine - I'm a died in the wool splitter). It might be nice to go the other way as well -- I recently had cause to help somebody unsplit their GRO index sources -- luckily there were not so many that it couldn't be done manually, but could have been a nightmare. Identifying the 'pattern' to use for un-splitting might be more difficult though.
User avatar
Mark1834
Megastar
Posts: 2458
Joined: 27 Oct 2017 19:33
Family Historian: V7
Location: South Cheshire, UK

Re: FTM to FH - Citation to Source

Post by Mark1834 »

Interesting - I’ve not come across anyone wanting to convert that way around. It feels like a later development, so version 1 just does lumped to split. Perhaps set it up with a series of standard templates for the new source names - source/citation, source/citation/place/date, source/citation/name/date/place, etc. We don’t have to go overboard on complexity to cope with every conceivable configuration, but a few standard options should suit most users.
Mark Draper
User avatar
ColeValleyGirl
Megastar
Posts: 5464
Joined: 28 Dec 2005 22:02
Family Historian: V7
Location: Cirencester, Gloucestershire
Contact:

Re: FTM to FH - Citation to Source

Post by ColeValleyGirl »

Mark, this came about as a result of somebody using the Civil Registration Index template and not understanding that particular template was designed for lumpers... I agree it might be a later development, and only if the demand is there.
avatar
cschroeder
Gold
Posts: 22
Joined: 14 May 2016 20:44
Family Historian: V6.2

Re: FTM to FH - Citation to Source

Post by cschroeder »

thank you Mark,

the only problem i found is that some sources are created twice (INDI or FAM <-> facts), but merging these
sources was done within an hour...

you saved me dozens of hours of work.. really really nice
Post Reply