*Converting Sources: Method 2 to Method 1

Questions regarding use of any Version of Family Historian. Please ensure you have set your Version of Family Historian in your Profile
gsward
Diamond
Posts: 67
Joined: 11 Mar 2011 14:47
Family Historian: V6

Converting Sources: Method 2 to Method 1

Postby gsward » 12 Jan 2019 17:20

My specific case is that I have some census sources that have been created using Method 2 ("lumped") that I had created in a previous program that I used (Genbox), and before I knew better. I have ignored them for a long time, but would now really like to tidy them up. The original sources do not have any attached media.

This question has come up from time to time and there are a number of threads that offer partial solutions.

Using Excel VBA to convert from lumped to split sources (15390)
Sorting Source Citations (15939)

Has anyone come up with a useful strategy for this apart from creating new Method 1 sources in AS and finally deleting the original source?

Graham

User avatar
tatewise
Megastar
Posts: 15904
Joined: 25 May 2010 11:00
Family Historian: V6.2
Location: Torbay, Devon, UK
Contact:

Re: Converting Sources: Method 2 to Method 1

Postby tatewise » 12 Jan 2019 18:42

There is not much point in converting from Method 2 to Method 1 unless in doing so you add transcript Text From Source and a Media census image in the process. Also each Method 1 source must be cited by the Individual records of the people in the same Census household, and that cannot be automated from the Method 2 sources.

So the only worthwhile technique is to use AS and finally delete the Method 2 source.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

Mark1834
Diamond
Posts: 91
Joined: 27 Oct 2017 19:33
Family Historian: V6.2
Location: South Cheshire

Re: Converting Sources: Method 2 to Method 1

Postby Mark1834 » 13 Jan 2019 10:41

The Excel routines were useful for tidying and converting my own tree, which was originally developed in Family Tree Maker, which has a more traditional "database" approach that doesn't work well in FH due to fundamental limitations of GEDCOM. It was based on older routines that I used with FTM to produce more detailed custom reports that were not available in that package.

I am now about to start a similar process for my wife's tree, and I will use the experience gained in processing FH data for Ancestry import to replace the Excel routines with similar ones written in Python. They should have the advantage of being much simpler to code, and avoid the occasional problems within Excel of corruption between what is text and what is number format. They also have another significant advantage of being more portable, working on any system without needing additional commercial software.

Regarding Ancestral Sources, I realise this is probably a minority view on this Forum, but I prefer to enter data manually and copy events and sources using the routines within FH rather than going through AS. I limit the events I create from each source, for example capturing residence and occupation from Census returns or Parish Registers in the transcript rather than as separate events. If you prefer to record everything as an event, AS would be the way to go. As ever, there is no "right" or "wrong" way to do things - it is what works for you in terms of what you want to achieve as your end result.
Mark Draper

User avatar
tatewise
Megastar
Posts: 15904
Joined: 25 May 2010 11:00
Family Historian: V6.2
Location: Torbay, Devon, UK
Contact:

Re: Converting Sources: Method 2 to Method 1

Postby tatewise » 13 Jan 2019 14:18

On a point of clarity, as a general rule AS does not create Residence facts, at least not from Census records.
AS also does not necessary create Occupation facts.
Yes, the default configuration does encourage that, but it is perfectly feasible to adjust AS so it does not.
The easiest method is to simply not fill in the Occupation (or Residence) fields in the green forms, but just add those details to the Auto Text pane below, which ends up in the Text From Source field of the Source record.
I also recognise that some users prefer to use just the features of FH.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

Mark1834
Diamond
Posts: 91
Joined: 27 Oct 2017 19:33
Family Historian: V6.2
Location: South Cheshire

Re: Converting Sources: Method 2 to Method 1

Postby Mark1834 » 22 Jan 2019 22:59

These conversion routines are now done, and I have two Python scripts exactly analogous to the previous Excel/VBA versions but vastly superior (more compact, easier to follow, much quicker, and above all, free!).

The first formats the Family Tree Maker export GEDCOM to address various issues raised in the direct import (most commonly removing the extra illegal detail with the CENS, DEAT and BURI tags and convert it to notes). FH does this as well, but even with a fairly modest tree it gave me an 8000 line log file. Pre-processing the GEDCOM file removed all of this and gave me a clean import without any warnings apart from the generic one about mishandling of CONC/CONT continuation lines that is an Ancestry/FTM trait. It also converted media records to FH format, adding titles and dates from parsing the filename.

The second takes this formatted output and splits all my lumped Census, Parish Register, Probate, MIs and various Quaker databases to individual sources with meaningful names derived from the lumped name, citation, and party names/event locations/event date. All citations are redirected to the appropriate split source, and all citation images automatically become source images. The modified GEDCOM this produces also imports cleanly to FH. Naming is not perfect, as it uses the birth name in the database, but it will be a simple matter to work through the split sources and add married names as appropriate (e.g. to burials and probate records).

Each script is only around 200 lines or so of code, which sounds a lot but is tiny compared to some of the more complex plug-ins, which can run to 10,000 lines or even more. They are far too customised to my way of naming and lumping to be worth uploading directly, but if there is anybody else who needs to split sources and is happy to dive into a bit of Python (which is free, and cross-platform), I'd be happy to share my approach. These were written for a FTM export file, but they would also work with the FH GEDCOM database file directly with appropriate minor syntax changes.
Mark Draper

User avatar
tatewise
Megastar
Posts: 15904
Joined: 25 May 2010 11:00
Family Historian: V6.2
Location: Torbay, Devon, UK
Contact:

Re: Converting Sources: Method 2 to Method 1

Postby tatewise » 23 Jan 2019 09:04

Mark, that sounds like some of those features may be very useful to Ancestry/FTM migrants.

I thought the CONC/CONT issue had been resolved in FH V6.2.6, because Ancestry/FTM now follows the standard, so FH no longer needs the special workaround.

When your script splits the lumped Method 2 Sources to split Method 1 Sources I suspect that for say Census Events it produces a separate Citation and Source record for each member of the same household.
If so, then it makes finding those household member's Individual records rather more difficult.
The Source records for each such household need to be merged to achieve the recommended structure.

Similar scenarios may apply where say a Birth Certificate is cited not only by the Birth Event of the child, but also by the Name fields, and perhaps other facts for the parents. I realise that you may not use such facts, but I mention it for the benefits of those who do.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

Mark1834
Diamond
Posts: 91
Joined: 27 Oct 2017 19:33
Family Historian: V6.2
Location: South Cheshire

Re: Converting Sources: Method 2 to Method 1

Postby Mark1834 » 23 Jan 2019 22:49

Mike,

Splitting can be customised according to what information is available. For Census entries, I already have the PRO reference as the detailed citation (the citation Where within Source), so this readily creates sufficiently distinct split source names, one per household. I add the location for convenience but not the name of the individual for just the reason you describe above - it would introduce unnecessary fragmentation. A typical Census source name would therefore be something like 1881 UK Census, location (PRO reference).
Similarly for Probate entries, where I already have the name and date as the Where within Source, so the separated source title would be England & Wales Probate Register, 1892, John Smith.
Parish Registers are more difficult, as I have not recorded anything in the Where within Source field, so use the individual name and event details to derive a name such as Baptism Register, Chelsea, Middlesex, 1838, Jane Smith. For most entries this is fine, but it does create problems when a definite birth date in a different year is quoted in the register. A second source is created referring to the birth year, so would need editing manually afterwards. However, this is a very small minority of entries in my experience. It also doesn't combine "batch" family baptisms to one source, but it is personal preference as to whether they are one source or multiples ones. A more significant issue may be the one you allude to, where for example, a baptism register entry is also cited as evidence of father's occupation or family residence. In this case it would create artefact entries referring to the parents, which is clearly incorrect. However, it is important to note that if the register entries are already sufficiently identified in the Where within Source (e.g. by register page number or event date), this can be used directly without the need to create artificial names. There are probably other options as well - Python list structures fit the design of GEDCOM files very well, so manipulation is relatively straight forward.

I think the reality is that there is no realistic way of creating a generalised splitting tool that works "out of the box" under all scenarios - as evidenced by all the discussion on this forum over the years without such a tool appearing!
Mark Draper

User avatar
tatewise
Megastar
Posts: 15904
Joined: 25 May 2010 11:00
Family Historian: V6.2
Location: Torbay, Devon, UK
Contact:

Re: Converting Sources: Method 2 to Method 1

Postby tatewise » 24 Jan 2019 00:27

Thank you for that excellent explanation of how various details are used to collate associated groups of citations to one Source.
As you say, it won't necessarily work for everyone, but does provide anyone following this thread with some useful pointers.
If Method 2 Source users have consistently used Where within Source in the Citations, then it does suggest that a Plugin might be feasible to convert to Method 1 Sources.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry


Return to “General Usage”

Who is online

Users browsing this forum: No registered users and 11 guests