* FH TMG Direct import issues following V6.0.1

[mitcheip]

Hi, I have been having a look at the TMG import and checked my finding in V5 with what has been delivered in V6.0.1. To be be honest I wasn't expecting them to be fixed for this release but I checked them anyway. Most are have not been fixed, one is a great improvement, another has a partial fix and there is two introduced bug when compared to 5.0.11. These have all been reported to Calico.

In a previous life I managed IT software implementations and data conversion/mapping was always a big issue, usually requiring a team and a lot of end users to make it happen. So I'm not expecting perfection and know there will be some "manual" work required but the first 4 or 5 of the issues below prevent "Go Live" with FH for me.

1. To make sure that notes were recorded as Facts I changed the “Notes” Tag in TMH to MiscNotes and changed the GEDCOM Tag to Misc, (Which is a standard GEDCOM Tag). These notes are then imported into FH as “Note” Facts. This worked in FHV5. V6.0 has introduced a bug where the fact is created but the actual note is not imported. A major issue

2. Another bug introduced in FH V6 is that the Memo field of an event in TMG is not converted to a note in FH. This did work in V5. Strangely there are a few cases where the Memo field is brought across. For example I use the Memo Field in TMG to record the occupation of a person in a tag type of Occupation. This note appears in FH but its the only one I have found to date. A major issue

3. In TMG each exhibit has a Caption and description held on the "Description" tab of the exhibits Properties dialogue, neither of these are imported into FH. Perhaps they should both be imported into the Picture Notes field in the "Edit Media Item" dialogue with the prefix "Caption:" and/or "Description:" (only if they are populated). Not Fixed. Interestingly the GEDCOM import does put the TMG Caption field into the untitled “Notes” field in the “Edit Media” dialogue which, if the “Use Note as Caption” box is clicked gives me what I want. In fact I wrote my first Plug-in to set the “Use Note as Caption” if there was a Note populated.

4. In TMG there is the ability to link a single exhibit to many events, (Tags). The Same applies to FH where the same Multimedia Object can be referenced by many Facts/Individuals/Family/Sources. During the direct Import each time an Exhibit is encountered a new multimedia record is created, if its the same file name as one that already exists (1), (2), (3) etc is appended to the end of the Title and the file name. The worst case I have is where the same image is repeated 6 times in FH and I have at least 70+ duplicates. If the file name already exists in FH perhaps a Link should be created rather than a copy of the image. Not fixed

5. There is a memo field in TMG attached to the Father_Bio and Mother_Bio Tags, this hasn't been brought across. Not fixed

6. Places are recorded as a comma separated sentence in FH. TMG has 10 fixed fields for the place record. I suppose in many cases this is not an issue although I can't see how, in FH, to run a query "Which place records do NOT have a county", for example. Another issue is that the coordinate field in the TMG Place record is just added to the end of comma separated sentence in FH. - This issue has been partly addressed with the new place structure, but the Lat/Long from TMG is in a “Part” field rather than in its correct fields.

7. The date on one occupation event in TMG was "Sep 1968 to Jul 1978", this imported into FH as "Sep 1968". A second date "Before 1915" date in TMG has come across to FH as "1915". Not Fixed
8. Flags with a value of “?” or “N” in TMG are not imported. Calico has explained their logic and I can understand it given the number of flags that are defaulted to “N” or “?”. I have some “real” “N” and “?” flags, eg “Will found?” which quite rightly has a “N” in many cases. I some how need to identify and change the real “N” and “?” flags in TMG

9. The "Witness" and "Role" concept in TMG is not handled at all well in FH. Placing the info in the notes field of the principle doesn't help give an overall picture of the persons life. Major improvement!!! I have check quite a number of examples and they all seem to be working correctly. I suppose the only issues I have is that Roles that are imported into a particular Fact Type are not added to the available Roles when entering another Fact of the same type. For example if there is a role of “Daugther-in-law” imported for a census, should this be in the pick list for the next census? The other issue is that the “Role” of Principles in a Fact are not brought across. For example “Head of household” or “Wife”. I think this is more of an enhancement to FH given that the functionality doesn't appear to be there.

10. Exhibits attached to Sources in TMG are not coming across to FH. Not fixed

11. Haven't yet come across how/if the TMG concept of sort date is implemented in FH. In TMG you can put an event in approximately the right position in someone’s time line by using the "sort date" without having to enter a "real" date. Not fixed

[tatewise]

In general I agree that most of those issues ideally should be fixed.
However, I have a few comments:

1. There is no MISC GEDCOM tag in the GEDCOM 5.5 Specification as supported by FH. (It is not even in Draft Release 5.5.1) Which GEDCOM standard are you referring to?

6. In FH, to run a query "Which place records do NOT have a county" would rely on having assigned meanings to each Place Part Column. Then the =TextPart() function could be used to filter/test the contents of the particular column assigned to County.
It would be easy with a Plugin to move Coordinates to the LAT/LONG fields

7. Strictly speaking Sep 1968 to Jul 1978 should be FROM SEP 1968 TO JUL 1978 in GEDCOM format, although case should not matter.

8. Probably you need a Will not found? flag set to Y.

[jimlad68]

Having moved from TMG via a GEDCOM file I'm afraid I can't help with most of your points.

Point 6. " Places are recorded as a comma separated sentence in FH."
Whilst experimenting I noticed this, however via a GEDCOM file all the commas were kept (if you set up the export correctly) and I did like TMGs use of field 10 for the lat/long. But not much help if you need the direct import.

However, in the light of potential geocoding and existing "usage standards" (see various forum topics)
you might feel you need to revamp your PLACes anyway.

Best format of PLACe and ADDRess for Map Geocoding (11937)
a bit old now PLACe, ADDRess structure, level use-Lat/ Long from TMG (11246)

Point 11. " sort date", yes I missed this too. I can only refer you too an earlier topic for a partial workaround:
" - There are no SORT DATES in FH, but there is another feature that can be used that will set a tag/fact type for a time period (e.g. before birth). If you wish to add your own dates beforehand (e.g. date 150 for research notes etc) you can do this up front with TMGUtility. However these may appear in some reports etc. In FH you could also enter your date as a date phrase with as interpreted as value (standard Gedcom)."
N.B. using early dates may upset FHs LUA life date estimates, but I manage quite happily without it.
from The Master Genealogist to be Discontinued (11350)

[mitcheip]

@Jim, Thanks for the pointer about sort date, I'll have a look at that. I have used the TMG Utility many times.

With regard to the Place issue I did mention to Calcio when I reported this issue that if the fields are to be imported as a comma separated string then then my personal preference would be to import them in "Full" format rather than "Tidy". I am assuming that if there were spaces for blank fields things like the place list would have a "Part" that would be, for example, equivalent to County, even though it might be empty.

@Mike,

Re my point 1, the issue of missing Note data. It was back in October 2014 when I came across this issue and all I know is that I changed the TMG database as I described, (in fact I have just found a email from Calcio telling me to create MISC events for the same purpose that I used Notes for in TMG.). The actual note, something like "He left 21 houses", was contained in the "memo" field of the NOTE tag in TMG and before V6 this created the correct NOTE Fact in FH with the Note field of that Fact having the same text, ie "He left 21 houses". Now a NOTE fact created but the the Note Field within that fact is empty. I feel that this is the same issue as my point 2. I think the GEDCOM mention in the TMG sense is a red herring given that TMG doesn't store its data in a GEDCOM format and its the FH direct import that's converting from the TMG database to FH GEDCOM

Re my point 7, incorrect date conversion I have done some more digging. It seems that any date in TMG with a “to” in it doesn't convert correctly during a direct import to FH V6. The examples I have found converting to a single date figure are :-
1. _Sep 1968 to _Jul 1978 converts to Sp 1968
2. _ _ 2000 to _ Dec 2003 converts to 2000
3. From 1996 to 1997 converts to 1996
Also I have found a couple examples of dates with the Bef prefix converting incorrectly
1. Bef 15 Mar 1950 converts to 15 mar 1950

I'll report this finding to support. Given the, literally, dozens of allowable date formats in TMG and a couple of dozen modifiers, (Circa, Cir, C, about, abt are all allowable forms for + or - 20 years), I'm not surprised Calcio haven't got the conversion 100% correct yet. By the way, for example "btw 1987 and 1991", "aft Oct 1960" and many other formats import correctly.

See my comment to Jim about Place and I'll try my very very limited Plug-in skills on moving the Lat/Long to the correct field.

[DonF]

I'd agree with mitchep's analysis -I reported most of the same problems to Calico yesterday.
There are also other issues, like:

1. FH doesn't carry over TMG's 'last edited' date

2. FH doesn't honour TMG 'Primary image' annotation on media

3. witness info dropped from some events

4. Individual Census report doesn't picks up census entries if person is a witness

5. chart L-R/top-down options are inconsistently interpreted across various charts (actually a v5 bug)

and his point on event order/sorting is actually quite complicated - the lack of a Sortdate function plays havoc with the order of Facts

[Coolgarra]

Mike - you mention in #6 that it would be easy to create a Plugin to move coordinates to the LAT/LONG fields...how much effort would this be? I am having this issue with my coordinates, and, although I 'could' just manually re-enter them all (a couple of hundred), I would prefer not to, if it didn't take too much effort to write the plug-in.

Is there anyone who might be willing to give it a go? I am happy to test it.

Thanks
Chris

In general I agree that most of those issues ideally should be fixed.
However, I have a few comments:

1. There is no MISC GEDCOM tag in the GEDCOM 5.5 Specification as supported by FH. (It is not even in Draft Release 5.5.1) Which GEDCOM standard are you referring to?

6. In FH, to run a query "Which place records do NOT have a county" would rely on having assigned meanings to each Place Part Column. Then the =TextPart() function could be used to filter/test the contents of the particular column assigned to County.
It would be easy with a Plugin to move Coordinates to the LAT/LONG fields

7. Strictly speaking Sep 1968 to Jul 1978 should be FROM SEP 1968 TO JUL 1978 in GEDCOM format, although case should not matter.

8. Probably you need a Will not found? flag set to Y.

[tatewise]

Download & install Copy LatLong in Name to LatLong Plugin Version 1.2.

It expects LatLong values in two comma separated Place Parts in the form , 99.99 , 99.99 with any number of digits and an optional leading minus character, plus optional white space characters before or after the numbers.

It tries to merge any Place records that have the same names after removing the LatLong parts.

Use File > Backup/Restore > Small Backup before running Plugin just in case!!!

Use Edit > Undo Plugin Updates after running Plugin but BEFORE closing FH if the results are not OK.

[Coolgarra]

Thanks Mike! That worked a treat!
Cheers
Chris