* Ancestry, FTM, FH and workflow

Importing from or exporting to another genealogy program. This is the place to ask.
Post Reply
avatar
Nick-V
Superstar
Posts: 268
Joined: 18 Nov 2009 17:50
Family Historian: V6
Location: London, England

Ancestry, FTM, FH and workflow

Post by Nick-V » 28 Feb 2015 15:02

EDITS
This thread became long and complex. The current outcome is now an updated Knowledge Base article.

This is an on-going topic. If you wish to add, correct or discuss anything please post below.

This thread has now been split into different discussions focussed on specific aspects of the concept:

Ancestry, FTM, FH and workflow ~ Media (12429)
Ancestry, FTM, FH and workflow ~ Sources (12430)
Ancestry, FTM, FH and workflow ~ Data retention (12431)

ORIGINAL POST
I have read some topics here about FH and Ancestry working together and would like to invite further debate albeit starting with my own current observations and limited knowledge. I feel we would benefit significantly from more work-arounds and proper longer term solutions.

Having adequate integration between software such as FH and sites such as Ancestry is becoming essential - both have a vital role. Perhaps spending a few tens of pounds on software such as FH and FTM is irrelevant. Annual data subscriptions at Ancestry approach £200 so a charge of £200+ for appropriate software might also be acceptable. It is the data in my existing tree and my time that has the most value.

Like other people, my requirements include:

1) Using Ancestry to match with vital records - to be able to update my data easily (Ancestry/FTM allows you to update automatically from a match) - to avoid hints having to be reviewed more than once. ON a tree of just 3100 people I currently have 3700 hints (920 people) to review - last week I had it down to 1200 hints (250 people). After a merge and Ancestry learning a little more, it increased - this increase is what nervous breakdowns are made of!

2) Using various other sites to match with members trees - most (not Geni) accept Gedcom uploads (but not the attached pictures) so this already works adequately.

3) Publishing my own web site including tree, primarily to make navigation more relevant and easier for family members but also to include other media. Using Ancestry's charts and search features is OK, however, a simple hierarchical (not many-to-many) approach is missing such as the descendant (top down) and ancestor (bottom-up) outline reports produced from FH (once simplified and included on an FH web site).

4) Outputting well presented and customised charts and reports to PDF. FH does this very well...FTM is inadequate and Ancestry provides nothing.

5) Entry and quality assuring data. Entry of data in all three products is adequate. Only FH enables such data to be selected, sorted, adjusted and generally played with to improve consistency and accuracy.

6) Entering and distributing pictures. In FH we can select faces from pictures and both faces and full pictures can be available on FH generated web sites. Using plugins we can export Gedcoms with "face only" pictures (not BOTH face and full although this enhancement could be made - refer Mike Tate below) for use elsewhere (e.g. Ancestry, PHPGEDVIEW, etc.). There are also pictures sources in Ancestry that can be saved but even with use of FTM in the middle they cannot be transferred back to FH automatically as they are not in the Gedcom - volumes currently appear low so this can be handled manually at present.

7) Census records affect the data for multiple people. Using the FH Ancestral Sources plugin is the only way I've found to enter data in one place and have multiple records updated (could be made even quicker, the number of necessary actions/clicks could be reduced especially in the attaching of images). Conversely, in Ancestry it appears that a Census with 9 entries generates 9 hints to review and accept! Indeed important clues could be missed if the original document is not reviewed.

It seems to me that Ancestry is arguably the best site for matching of vital records. It's FTM software is generally too basic and inflexible compared with FH, however, it is increasingly capable of synchronising with Ancestry and merging Gedcoms which FH cannot do. There are issues in the way fields are used in the various software despite the Gedcom "standard". Perhaps I need to compromise my use of FH to avoid some of these data exchange problems. FH is much preferred and indeed necessary for data management, charts and reports, web creation, photo faces, census entry.

So the struggle for me - should Ancestry/FTM be the master database or FH ? There is no simple answer.

1) Should data be edited (sometimes automatically from hints) in Ancestry/FTM then exported to FH for some of the other tasks (this would make picture publication and face editing impossible).

2) Or should data (and pictures) be edited in FH then exported (using a plugin) to FTM for merging then merged with an existing tree in Ancestry to preserve old hints. Note that accepting hints results in automatic data changes and addition of people that would have to be done manually in FH. The two would get somewhat out of sync.

In the interim, one step in the right direction must surely be to do whatever we can both in software and in our use of these software tools to make data and picture exchange (both ways) more straightforward even if compromises have to be made.

In the long term it would be good if FH mimicked FTM in its synchronisation with Ancestry, it will likely remain a more comprehensive (if not so simple to use) tool than FTM. It's recent effort to link with alternative subscription site MyHeritage might be more about marketing and the same or worse functional issues remain.

If direct integration is not feasible then finding a way to "two-way" the data, pics and hint markers simply between FH and FTM and to prevent FH users from using non-Ancestry fields etc. may be the way to go. Without FH synchronising directly with Ancestry perhaps a solution might be a circular trip from FH 1) to FTM 2) to Ancestry 3) to FTM and 4) back to FH. Merges (synchronisations) require discrepancies to be handled. At 1 we might say FH is always correct but at 4 this is not necessarily so.

It ain't easy...but maybe we can evolve something together...

Please excuse any errors and confusions in my lengthy attempt to document this...I'll edit this posting a little with things I learn from any responses.
Last edited by Nick-V on 13 Mar 2015 13:26, edited 16 times in total.

User avatar
tatewise
Megastar
Posts: 16643
Joined: 25 May 2010 11:00
Family Historian: V6.2
Location: Torbay, Devon, UK
Contact:

Re: Ancestry, FTM, FH and workflow

Post by tatewise » 28 Feb 2015 15:46

Just a few comments on some details.

6) You correctly say: "Using plugins we can export Gedcoms with "face only" pictures (not BOTH face and full it seems) for use elsewhere." It would be fairly easy to add a BOTH option to my Export Gedcom File Plugin, which would be a combination of the PART and AFILE options. In the meantime, adding a link somewhere in FH to the whole image will include an exported whole image.

The difficulty with synchronising to any other product is that their data formats are rarely publicised. This goes for both online products like Ancestry & FMP, and offline products like FTM, TMG & Legacy.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

avatar
Nick-V
Superstar
Posts: 268
Joined: 18 Nov 2009 17:50
Family Historian: V6
Location: London, England

Exporting Media

Post by Nick-V » 28 Feb 2015 16:02

Thanks Mike. Prompt as usual :)

6) Yes adding a link to both the face and full image in FH is a good workaround before exporting to Ancestry. However, I suspect a consequence is that the image would be published twice in reports and the FH web site (unless there is some way to privatise individual images). Life would be easier if the export plugin could do both or if we didn't in fact have (or chose to use) this useful feature of FH! The fact that Ancestry is not currently showing many of my images (greyed box only) is the first priority and data the second!

Integration: yes agreed, and worse, such formats change - there also comes a maintenance burden for developers as such formats evolve, and changes can occur without warning (e.g. Facebook plugins). As Ancestry and FTM are compelled to work together I envisage simply using FTM as a staging post, a tool that translates between FH's output and Ancestry - both ways. I suspect the most elegant solution will lie in FH's exports and imports...the picture issue, hint flags and discrepancy management on re-import (merging back in).
Last edited by Nick-V on 08 Mar 2015 10:25, edited 1 time in total.

User avatar
tatewise
Megastar
Posts: 16643
Joined: 25 May 2010 11:00
Family Historian: V6.2
Location: Torbay, Devon, UK
Contact:

Re: Ancestry, FTM, FH and workflow

Post by tatewise » 28 Feb 2015 16:22

You can tick Exclude from Reports in any Media link to hide that image in Reports and therefore in website pages.

Regarding discrepancy management, have you investigated the FH File > Merge/Compare File and Edit > Merge/Compare Records?
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

User avatar
tatewise
Megastar
Posts: 16643
Joined: 25 May 2010 11:00
Family Historian: V6.2
Location: Torbay, Devon, UK
Contact:

Re: Ancestry, FTM, FH and workflow

Post by tatewise » 28 Feb 2015 16:56

I've had a quick look at my Export Gedcom File Plugin and adding a BOTH option is very easy.

This relies on FTM/Ancestry accepting both the PART mode and the AFILE mode of Media export. i.e. Local Media Objects used in Multimedia tabs as for PART frames, plus Multimedia Records with absolute FILE links as for AFILE full frames.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

avatar
Nick-V
Superstar
Posts: 268
Joined: 18 Nov 2009 17:50
Family Historian: V6
Location: London, England

Exporting Media

Post by Nick-V » 28 Feb 2015 17:38

Mike again thanks.

I now see how to hide media from reports using the link. Clearly it would be easier if I didn't have to link pictures twice and hide one.

The better solution may come from your export idea. I don't know the Gedcom spec or what FTM can handle. Excuse my guess but FTM seems to import happily whatever tag your export routine puts in the "for Ancestry" export file. And if you put one twice (i.e. for face and full), and create two image files, that should work. But if we are considering data going in and back out of Ancestry we need to consider retaining the PART coordinates...without investigating I don't know how destructive Ancestry and FTM (which could be different) are with Gedcom tags that they doesn't use. This needs considering further if we progress the data in and out method...not just for images.

I have yet to investigate the merge/compare facility...it may help towards an answer...but clearly we need to think about minimising the risk of differences occurring to make this batch process sensible operationally. I'm flooded with stuff but will get to it...!
Last edited by Nick-V on 08 Mar 2015 10:24, edited 1 time in total.

User avatar
tatewise
Megastar
Posts: 16643
Joined: 25 May 2010 11:00
Family Historian: V6.2
Location: Torbay, Devon, UK
Contact:

Re: Ancestry, FTM, FH and workflow

Post by tatewise » 28 Feb 2015 18:19

If you can get the Merge/Compare Files to work satisfactorily then the Face Frame PART co-ordinates can be retained. FYI: They are held in the Media Records but rely on a special ID in the Multimedia/Fact links.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

avatar
Nick-V
Superstar
Posts: 268
Joined: 18 Nov 2009 17:50
Family Historian: V6
Location: London, England

Exporting Media

Post by Nick-V » 28 Feb 2015 20:21

OK thanks...perhaps _AREA {134,240,282,402} ? What I hope is that the ID will be retained thru the whole trip via FTM to Ancestry then back again to FTM and FH (with the future in mind).

BTW, another issue in this is that whilst Ancestry can import the pics it does not appear to retain the order and worse, it doesn't set the profile picture (not even assuming the first one).

The road ahead will be paved with many such obstacles !
Last edited by Nick-V on 08 Mar 2015 10:24, edited 3 times in total.

User avatar
tatewise
Megastar
Posts: 16643
Joined: 25 May 2010 11:00
Family Historian: V6.2
Location: Torbay, Devon, UK
Contact:

Re: Ancestry, FTM, FH and workflow

Post by tatewise » 28 Feb 2015 21:24

Yes, the co-ordinates are in the _AREA tag and the special ID is in the _ASID tag.
Because they are unique FH Custom tags, unrecognised by any other program, Export Gedcom File removes them, so they won't even start the round trip.

There is a lot more about all FH Custom tags and Gedcom supported by other programs in Knowledge Base > GEDCOM Extension List and the Plugin Knowledge Base > Export Gedcom File ~ Output Formats.

BUT, when performing Merge/Compare Files the _AREA and _ASID tags at the start of the trip will still be in FH when the returning Gedcom is merged, so they can be retained.

The exported Gedcom has the Media linked in FH order, so what FTM/Ancestry does with them is up to its interpretation of the Gedcom.

I suspect every genealogy package uses different methods for assigning Individual Profile pictures.
e.g. How does TMG do it?

Personally, I would not try to include Media in the round trip, and just use FTM/Ancestry for match hints.

If you want to trial the revised Export Gedcom File Plugin with a BOTH option for Media then try the attachment.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

avatar
Nick-V
Superstar
Posts: 268
Joined: 18 Nov 2009 17:50
Family Historian: V6
Location: London, England

Exporting Media and "Full Replacement"Round Trip processing

Post by Nick-V » 28 Feb 2015 22:30

Thanks Mike

I will trial the export with "both" option...appreciated! While on that topic, is there a new version of the Improve Website plugin yet for that small change in _nameindex.js that defaults the search button or should I keep replacing that file after each run for now?

I am learning a little about these IDs from simply sending data between FH and FTM in various ways. I have already noted that FTM (which is out of our control) does several things with tags that make a "full replacement" round trip (the safest first choice) unlikely nee impossible!. For example it moves ADDR into a note on import (I suppose I could use the note field in FH instead but...) and loses NICK data. I bet there is a lot more...

Without looking much further I agree that a round trip involving "merging back" to the original data seems a more likely option to investigate next including considering the existing merge routine you mentioned. With images, which are not necessary for matching, they could be sent to Ancestry and abandoned. If Ancestry does not order them as provided or assign a profile image then our choices are moan at them for a modification, do it manually or forget it.

Thanks for the links about tags - I'll refer as appropriate.
Last edited by Nick-V on 08 Mar 2015 10:27, edited 1 time in total.

avatar
Nick-V
Superstar
Posts: 268
Joined: 18 Nov 2009 17:50
Family Historian: V6
Location: London, England

Exporting Media

Post by Nick-V » 28 Feb 2015 23:47

There might be a little confusion over the Export Gedcom BOTH option.

On a FH website (given the option I chose when creating it) when you click on a face image it takes you to a page with the whole image. In this way viewers get the face but also have the opportunity to see the family picture it was cropped from.

What I had in mind for Ancestry or wherever is that again, when viewing an individual, access to both the face and the family image would be available. Well, the face is available as I'd expect and I can see you have attached a FILE to each INDividual.

I can see that the family pic has also been created and it seems you have attached a _FILE to an OBJEct.

When I open the Gedcom in FH it seems:
1) the face images are "hard" attached to the individual - you cannot click over to the media list - well that's fine
2) the family image is "hard" attached to the media list - BUT it does not appear linked to the individuals appearing in it. In FTM this image does not display (File not found).

Sorry, I don't yet understand the different terms PART AFILE, etc. so can't explain in technical language. And I can't confirm what FTM accepts...what can I do to test or clarify? If we can't link invdividuals to a common family photo perhaps we need to output the original photo multiple times...once for each face attached to it (messy).
Last edited by Nick-V on 08 Mar 2015 10:27, edited 1 time in total.

User avatar
tatewise
Megastar
Posts: 16643
Joined: 25 May 2010 11:00
Family Historian: V6.2
Location: Torbay, Devon, UK
Contact:

Re: Ancestry, FTM, FH and workflow

Post by tatewise » 01 Mar 2015 00:47

The problem with exporting Gedcom is there are often several interpretations of a natural English requirement. Now that you have mentioned the webpage full image popup scenario and wanting to have the full images linked alongside the part images it puts a different spin on the export option. I will have a look at that alternative.

It is also interesting that FTM ignores NICK tags and mishandles ADDR tags. That prompts some further export Gedcom amendments for FTM. I cannot afford to fully test exported Gedcom to every program, and rely on feedback from others to say what works and what does not, especially since so few program are truly Gedcom compliant.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

avatar
Nick-V
Superstar
Posts: 268
Joined: 18 Nov 2009 17:50
Family Historian: V6
Location: London, England

Exporting Media and thinking about other data

Post by Nick-V » 01 Mar 2015 01:07

Sorry for any confusion on images. It is probable that FTM doesn't accept _FILE tags anyway. I'm currently uncertain how individuals link to shared media in standard 5.5 Gedcoms (if at all). If they don't, the only potential and messy solution is to duplicate the fill image for every face and use FILE tags at individual level. Sorry...I don't know the available technical options...

Regarding treatment of fields I am uneasy as it would take considerable testing to determine what FTM (and all software as you mention) does with every tag. Indeed, the situation could well change with future versions.

My initial urge was to try to find out, but based on your "for matching only" point earlier it might be easier for me to make the shortest possible list of 1) what fields help Ancestry to match well and 2) what fields Ancestry might want to (usefully) update after a match. If it can be kept very simple I might just output the minimum info which might in turn simplify the return merge. Thinking aloud...!
Last edited by Nick-V on 08 Mar 2015 10:28, edited 1 time in total.

User avatar
AdrianBruce
Megastar
Posts: 851
Joined: 09 Aug 2003 21:02
Family Historian: V6.2
Location: South Cheshire
Contact:

Re: Ancestry, FTM, FH and workflow

Post by AdrianBruce » 01 Mar 2015 07:34

Do not even bother trying to directly download a GEDCOM from Ancestry, its format is so wrong it's a waste. The only viable way of getting stuff out of Ancestry on a GEDCOM is via a sync with FTM. I have no idea if anyone else syncs with Ancestry. Remember please that these APIs are generated for a reason - in the case of Ancestry it's to increase sales of FTM. In the case of FamilySearch it's to suck your data into the Mormon church's systems. To avoid any doubt on that last one, note that I can live with that - just remember it.

Going back to GEDCOM format issues, it is generally not possible to confine yourself to a subset of FH and achieve a successful export as the issue is not just missing bits but inability to program correctly the bits that are there.
Adrian

avatar
Nick-V
Superstar
Posts: 268
Joined: 18 Nov 2009 17:50
Family Historian: V6
Location: London, England

Thinking about other data

Post by Nick-V » 01 Mar 2015 10:53

Adrian thanks...I suspect you are not alone in thinking LDS has motives that are not so widely discussed, I thought the same when I encountered their site.

Regarding synchronising FH with Ancestry we are currently thinking that FTM must do the sync with Ancestry. The next bit is working out how and what should be:

1) exported from FH to be merged with FTM
2) exported from FTM to be merged with FH

There definitely are many issues, both FH and FTM/Ancestry use Gedcom files in non-similar ways (use of custom tags). And use users also use fields differently too, like addresses, or using "immigration" instead of "travelled". I have no idea if we can find a way forward today, but the importance of record matching and then automating the thousands of updates that arise makes the effort worth a try.

I've just looked briefly at the FH import/merge which can show clearly the differences between two data sources. I am thinking to list all differences and start to think about how each may be dealt with. It could be that:

1) some data is excluded from the exchange
2) I use some fields in FH differently to the way I do at present
3) some data is deleted before import
4) some new automation needs to be developed

We will see...
Last edited by Nick-V on 08 Mar 2015 10:29, edited 2 times in total.

User avatar
tatewise
Megastar
Posts: 16643
Joined: 25 May 2010 11:00
Family Historian: V6.2
Location: Torbay, Devon, UK
Contact:

Re: Ancestry, FTM, FH and workflow

Post by tatewise » 01 Mar 2015 13:07

To clarify one point. The FH use of GEDCOM is 100% Standard. FH does use Custom tags, but that is allowed in the Standard. Whereas, many other programs do not adhere to the Standard. However, the net result is they all use different dialects of GEDCOM that makes import & export a minefield. So if import & export is an important objective you just have to really understand the GEDCOM specification and how each program interprets it.

Regarding the Export Gedcom File Plugin, it is always refreshing for another user to bring a new perspective. On reflection, I can see no reason why both PART and FULL Frame images can't be exported all the time. The Plugin just inherited a method used in an earlier Plugin.

Media objects are linked to image files in two ways in Standard GEDCOM:
  1. Local Media Objects in Multimedia tabs, etc, use the FILE tag to directly link to each file. So to share the same file simply means the FILE tag has the same path in each Local Media Object.
  2. Media Record links in Multimedia tabs, etc, link to shared Media Records that have a FILE or _FILE tag to directly link to each file, and have other tags to record more details than possible with Local Media Objects. Strangely, the GEDCOM Standard does NOT define a FILE tag for Media Records, which is why FH uses the Custom _FILE tag. However, the use of the FILE tag has become a de facto standard supported by most programs (including FH).
So my plan is to modify the Plugin to always include both PART and FULL Frames, and offer either Local Media Objects or Media Records as alternative modes of linking the media files.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

avatar
Nick-V
Superstar
Posts: 268
Joined: 18 Nov 2009 17:50
Family Historian: V6
Location: London, England

Exporting Media

Post by Nick-V » 01 Mar 2015 13:56

Thanks Mike. I've edited my use of "standard".

I appreciate your openness to potential change, discussion and your useful explanations.

I have a little concern about the proposed specification for the export program that we might discuss and research:

1) I can imagine other users wanting to export both part (single face) and full (group of people) images. When creating a FH web you can chose to link from the part thumbnail to "display full picture on its own page" (I'd like to emulate this in FTM/Ancestry) or to display the part in various ways. Personally I think the thumbs are big enough and its nice click to see the family picture from which it was taken. However, I can also foresee users only wanting to export the part on the basis they cropped out other stuff quite deliberately and possibly for security reasons. I cannot imagine users wanting full pictures and no parts. I think an option is wise.

2) The way the receiving software deals with pictures should determine the linking method used so perhaps this should be an option? Thoughts include:

a) I "think" FTM is refusing _FILE tags - different software may react differently
b) some software may only use FILE links directly to the record
c) some software may be happy to use links to a common media record that also has the advantages of additional information (I would prefer this for both part and full)
d) some software may allow both
e) if exporting full images I wonder if a common media record link or common folder link will work or if (sadly) the full image would have to be duplicated (I really hope not!).

Clearly we want to keep things straightforward - do we need to test other software more or is it easy to provide options?
Last edited by Nick-V on 08 Mar 2015 10:29, edited 1 time in total.

User avatar
tatewise
Megastar
Posts: 16643
Joined: 25 May 2010 11:00
Family Historian: V6.2
Location: Torbay, Devon, UK
Contact:

Re: Ancestry, FTM, FH and workflow

Post by tatewise » 01 Mar 2015 14:42

1) Can't help you with displaying full images via part images in FTM/Ancestry. If there is a privacy issue with any full image, then perhaps the user should not being using Face Frames but should physically crop the image. Alternatively, the full image could be deleted from the exported project before publishing. The snag with an option such as Exclude Full Images is it applies globally not selectively per image.

2) Agreed. That is the concept of the Export Gedcom File Plugin, but to some extent I rely on testing and feedback from users to set the default options for each target program.
  1. Most programs won't accept the FH Custom _FILE tag. That is why my Plugin converts it to FILE that has become a de facto standard in Media Records.
  2. Agreed. That is why my Plugin offers Local Media Objects.
  3. Agreed. Although custom FH info must be converted to Notes.
  4. Agreed. Most do.
  5. Common links to a single full image file is the standard way.
As other software programs change over time, more testing will be an ongoing exercise. In my Plugin the Extra Options tab allows many combinations of the export rules to be chosen, and preserved against the (CEA/B) Custom Export Alpha/Bravo export modes. The other program related export modes (FTM, TNG, etc) are essentially predefined combinations of those same Extra Options rules. If necessary more rules can be added, and that has already happened. Checkout the Plugin Help & Advice for full details.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

avatar
Nick-V
Superstar
Posts: 268
Joined: 18 Nov 2009 17:50
Family Historian: V6
Location: London, England

Exporting Media

Post by Nick-V » 01 Mar 2015 15:06

OK thanks. I did a little more editing Gedcoms and importing to FTM. I "believe" the following is true to FTM:

1) It accepts both local media objects and media links and uses them appropriately
2) If using the _FILE tag it creates objects but directs them all to C:\File Not Found.jpg - FILE tags work fine
3) Creating a single full image as an media object with links to several individuals would work
4) It appears the date gets put into the notes field...not sure this is necessary as FTM appears to support a date for media

EDIT: the reason the images in Ancestry were greyed out is the upload from FTM can appear to have finished but then take a further day or two to complete.
Last edited by Nick-V on 08 Mar 2015 10:30, edited 1 time in total.

User avatar
tatewise
Megastar
Posts: 16643
Joined: 25 May 2010 11:00
Family Historian: V6.2
Location: Torbay, Devon, UK
Contact:

Re: Ancestry, FTM, FH and workflow

Post by tatewise » 01 Mar 2015 15:55

1)-3) OK, that is good.
4) GEDCOM 5.5 does NOT define a DATE tag for Media Records, so FH uses Custom _DATE tag and Plugin converts to a valid NOTE. What non-standard tag does FTM use for its Media Record Date? To answer, export a GEDCOM from FTM and examine Media (OBJE) Records, or edit an FH exported GEDCOM to include a DATE tag to see if FTM will import it.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

avatar
Nick-V
Superstar
Posts: 268
Joined: 18 Nov 2009 17:50
Family Historian: V6
Location: London, England

Exporting Media

Post by Nick-V » 01 Mar 2015 18:04

I entered a unique date but cannot find it in the GEDCOM exported from FTM. I presume it is only stored in their proprietary FTM file.

Regarding images...when I had my first quick look at FH's import/merge it showed my existing images and also wanted to import new images. Clearly image file names are changed on export in a structured manner therefore they do not match with the original image. Further the original image has a relative path to the file where the exported are explicit paths.

I will think further about the usefulness and feasibility of doing a conversion to change names and paths then reimporting this back into FH as the master so that images match on merge...probably a long shot as other details change on export but worth a few minutes thought.
Last edited by Nick-V on 08 Mar 2015 10:31, edited 1 time in total.

User avatar
tatewise
Megastar
Posts: 16643
Joined: 25 May 2010 11:00
Family Historian: V6.2
Location: Torbay, Devon, UK
Contact:

Re: Ancestry, FTM, FH and workflow

Post by tatewise » 01 Mar 2015 18:29

In order to create PART image files distinct from FULL image files, when there is actually only one original image file, requires filename changes (or a complex use of folders that would have a similar efffect).

My Plugin uses the Special ID (_ASID) mentioned above or "0" if none, and the Media Record ID, with a letter "O" in between, prefixed to the original filename.
e.g.
0O1 Filename.jpg is the full image from Media Record #1
1O1 Filename.jpg is 1st part image from same Record #1
2O1 Filename.jpg is 2nd part image from same Record #1
0O2 Imagefile.png is the full image from Media Record #2
etc...
Therefore the import merge/compare of Media is tricky, especially if PART image files are involved, because they do not exist in the original FH Project.

Relative file links can only work in the context of an FH Project, and to import/export Media requires absolute file links, otherwise the target program does not know where to find the files.

I suspect by now you are starting to appreciate the complexities involved, and to understand why many users choose one program for their master database, and only export the bare minimum to achieve specific objectives with other software. What may be possible in the longer term is an Import Gedcom File Plugin, but don't hold your breath.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

avatar
Nick-V
Superstar
Posts: 268
Joined: 18 Nov 2009 17:50
Family Historian: V6
Location: London, England

Exporting Media and thinking about other data

Post by Nick-V » 01 Mar 2015 19:56

Yes, I noticed how you created a unique prefix to each file name and it works well. What I am considering is exporting to create these file names then reimporting into FH (to rename FH's files to the same as the export). If FH can work with full paths then the issue of media matching would go away...what FH has and what FTM has would be the same. Of course the obvious problem (at present) with this approach is that the export also changes other fields!

Yes, I could foresee complexities but I still don't know how many...there are significant opportunities here. Is it impossible? Am I wasting my time? Matching properly when volumes of hints are involved is a "must have". I refer back to my recent increase in hints to a totally overwhelming 3,700 (920 people)! Seriously, am I supposed to carefully review all those hints and add Census information and other details including media manually. Not if there is another way !

Despite being basic, I foresee FTM taking users away from FH and other software (and so does Ancestry - and it locks people in). Conversely, there is a great gap that FH could fill if it can be made to link adequately with FTM (or whatever others produce in future) via the obligatory standard Gedcom. "An advanced family tree tool and the only one that can work properly with Ancestry" is quite a claim.

I don't want to have to choose FTM as my master database and simply output to FH for the few good things it does but I will resort to this if I have to. For now, and working together with you, we might create a better alternative. And if we had input from other users and the author we might crack this! If you think its a non-runner then I'm listening...
Last edited by Nick-V on 08 Mar 2015 10:31, edited 1 time in total.

avatar
Nick-V
Superstar
Posts: 268
Joined: 18 Nov 2009 17:50
Family Historian: V6
Location: London, England

Re: Ancestry, FTM, FH and workflow

Post by Nick-V » 01 Mar 2015 20:30

Unfortunately FH converts file links to relative on import and also creates relative links in the course of operation. Surely an option to always use full paths to a user specified folder could be implemented.

As an experiment I just replaced the project's gedcom with one exported using your plugin. Existing media appeared to work fine with full path links. I then did a merge/comparison against the same file exported previously. Well, media matched 100% as I expected.

The differences (you'll be pleased to know) were:
1) Header - it wanted to replace it - no idea why
2) Places, Media, Repositories, Sources, Notes, Families, Individuals - none

Note for media files: FH uses a sub folder of the project called "..\project name.fh_data\media\" and FTM uses a sub folder of the project storage folder called "..\project name Media\". Perhaps it would be easy for FH to use a full path to the FTM folder instead.

If your plugin can simply work its magic on media but not other fields, and FH can be persuaded to not use relative links surely media matching can be done?

It's time to look at the other data !

avatar
Nick-V
Superstar
Posts: 268
Joined: 18 Nov 2009 17:50
Family Historian: V6
Location: London, England

Potential enhancements to export plugin

Post by Nick-V » 06 Mar 2015 09:41

I hope Mike and others find useful the summary at the bottom of this post. I have done nothing else this week than learning about Gedcom files, investigating how FH and FTM/Ancestry use them differently and considering how to enable family data currently stored in FH to be enhanced significantly and easily by the thousands of valid useful "hints" from vital records waiting for me in Ancestry. Commercial point...I bet there are a lot of Ancestry users that would prefer something more comprehensive like FH if it were more compatible - an opportunity !

Despite FH and FTM's differences I remain convinced data can be translated, using a custom import/export routine and after more investigative work. After all, we are only talking about simple Gedcom text file and paths to images. The current plugin helps but image handling need a little need more work (links to the full images are missing and _PHOTO tag), other fields need to be translated some of which are below and there is not yet an import with translation (only an import and merge routine) back into FH.

FH remains my preferred software, however, considering the benefits and effort involved I've concluded that moving my data to Ancestry and updating there is far easier than moving the numerous hints info to FH manually.

It's disappointing indeed somewhat remiss of the Gedcom "specification" that data exchange is not straightforward. FH rejects certain data especially where Values appear against tags like "1 DIV Cruelty" and "1 CAUS Heart Failure". FTM uses data very simply, it's concept is that facts have a Value, Date, Place, Description and Note and these need to accommodate everything somehow, even the name suffix is crammed into the NAME field like "John /Smith/ MBE".

I now need to get on with the bulk matching of hints using Ancestry/FTM by exporting and otherwise manipulating my data there, and hope that one day I can manipulate my data back again into FH. I urge that routines be written to export and import with translation (perhaps with non FH/FTM Gedcoms in mind). The info below may be a little help...it is a summary of additional field translations, I would also fix the image issue and set different defaults to the existing translations. I would be happy to provide much more info including a spreadsheet documenting this week's work that includes a data dictionary, test data, observations and potential solutions should such detail be of interest.

I'm not an expert, I am quite happy to be wrong if it provokes someone else to be right, and I've made an effort...if there is any interest from others in developing something, researching further or resolving this important matter in any other way I remain interested in working together.

Mike, let me know if a telephone meeting would be helpful. I am concious of my long forum postings - perhaps pushing this forward requires something different - what is Simon's position in all this?

Simple summary of SOME findings:
EDITED 8 Mar 2015 10:

1) Fact Sentences (2 _SENT), Remove, One was created accidentally

2) Nickname (2 NICK), Promote to 1 ALIA, NICK imports to FTM but fails to appear in index of names (it seems because it is not marked as "Preferred"). FTM uses ALIA on export.

3) Name Prefix (2 NPFX), Promote to 1 TITL, NPX will import to FTM but FTM employs TITL on export (thinking ahead)

4) Name Suffix (2 NSFX), Append to NAME, NSFX imports OK in Ancestry but FTM causes another NAME record (bug). Append to NAME: Forename /Surname/ Jnr MBE and FTM will put into suffix field. Delete NPFX record.

5) Repository Email (1 _EMAIL), Rename to 1 EMAIL, Perhaps FH should use this.

6) Cause of death or divorce (2 CAUS), Promote DIV, CAUS to 1 DIV Value (not DEAT, CAUS), CAUS in FTM is an event called Cause of Death "1 CAUS" (auto-update reasons) whereas with DIV the Value field is used. Note FTM imports 2 CAUS from FH but exports 1 CAUS.

7) To/From Place (2 _PLAC), Move to 1 [event], Used when an event has 2 places i.e. immigrated so for these evens provide choice to move to Value field or Notes.

8) Address (2 ADDR), Move to 1 [event] Value for events but move to Fact Note for attributes - discuss, Some facts like Immigration and Emigration have both To/From and Address in which case move Address to Fact Note (see 7)? Cannot move address to Value on RESI event as Value is ujpdated by Ancestry i.e. Marital Status: Married; Relation to Head of House: Head. This may have to remain in NOTE.

9) Profile photo (1 _PHOTO), Create 1 _PHOTO @M???@, Create 1 _PHOTO @M???@ record to point to first OBJE for a person if there is no PHOTO record already (I don't know if FH can keep the record for round trip).

10) Output ROOT person first to make them home person. Delete ROOT record.
Last edited by Nick-V on 08 Mar 2015 12:30, edited 5 times in total.

Post Reply