* Export Gedcom File plugin new features

For users to report plugin bugs and request plugin enhancements; and for authors to test new/new versions of plugins, and to discuss plugin development (in the Programming Technicalities sub-forum). If you want advice on choosing or using a plugin, please ask in General Usage or an appropriate sub-forum.
User avatar
tatewise
Megastar
Posts: 28341
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Export Gedcom File plugin new features

Post by tatewise »

The Updating existing FH records with data from CSV (19765) thread discusses new Export Gedcom File features:
  1. Synthesise missing Birth events from 1st fact with Normal Time Frame = Soon After Birth and with a Date.
    e.g. Baptism or Christening events or any custom Soon After Birth facts.
  2. Synthesise missing Death events from 1st fact with Normal Time Frame = Post-Death and with a Date.
    e.g. Burial, Cremation or Probate events or any custom Post-Death facts.
  3. Allow all the plugin settings to be saved in the global Program Data or local to the Project.
    A new administration tab will offer that choice and the ability to copy global settings to/from local settings.
  4. Allow the plugin settings for a target product to be copied to/from the (CEA/B) Custom Export Alpha/Bravo settings.
    A new administration tab will allow the target product to be chosen and provide the copy to/from operations.
The attached Export Gedcom File plugin Version 4.6.1 Date 12 Sep 2021 ZIP file implements the first two above.
Download then right-click and Extract All... before installing into FH.

On the Other Options tab, there is a tick option Make missing Birth/Death events for DNA products at the bottom.
The synthesised events use only the Year from the Date and add the approx/circa qualifier, e.g. 2 DATE ABT 1900
These events set the Place from the last comma-separated part of the dated fact if it exists, e.g. 2 PLAC England

The GEDCOM Export Mode: (FTD) Family Tree DNA now defaults to using this feature instead of replacing BAPM, CHR, BURI, CREM events.

The next development will be the latter two settings features above.
Last edited by tatewise on 19 Feb 2022 16:52, edited 1 time in total.
Reason: Attachment deleted as later version is in the Plugin Store.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
avatar
quarlton
Famous
Posts: 166
Joined: 26 Feb 2004 13:07
Family Historian: V7
Location: Lincolnshire
Contact:

Re: Export Gedcom File plugin new features

Post by quarlton »

Hi Mike

I've just had to use the plugin with the 'Keep Media folders' option for the first time.

My first thought was that it wasn't working as the progress bar wasn't showing, so I aborted it.

Eventually I discovered that what was actually going on was that it was busy recreating the Media folder structure in the Export folder.

Now on the project I was using, the folder Media is quite deep as far as folders and sub-folders are concerned.
It took about 5 minutes to create the structure, during which time there was no on-screen indication.

Would it be possible at the next update to add a message to say what is happening?
I appreciate that this might be of limited interest - but if you don't ask you don't get 😀

Many thanks

Dave
Dave Simpson ~ Boulton, Braham, Carney, Simpson and Jacobs
User avatar
tatewise
Megastar
Posts: 28341
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Export Gedcom File plugin new features

Post by tatewise »

Thank you for that suggestion. It will be incorporated into the next version.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
User avatar
tatewise
Megastar
Posts: 28341
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Export Gedcom File plugin new features

Post by tatewise »

Sorry for the delay in developing items 3. & 4. for settings management listed in my first posting earlier.
I have been distracted by issues with the utf8 and fhFileUtils library modules.

See the attached Export Gedcom File plugin Version 4.6.3 Date 22 Oct 2021 ZIP file.
Download then right-click and Extract All... before installing into FH.

The progress bar now covers creating the Media folder structure in the Export folder as requested by Dave.

There is a new Manage Settings tab as illustrated below.
The upper section manages whether all the plugin settings are held local to the Project or in the global ProgramData.
The lower section allows the plugin settings for a GEDCOM Export Mode target product to be copied to/from the (CEA/B) Custom Export Alpha/Bravo settings.

3. Choose where settings are held for the current Project
The radio buttons Project local data and Global ProgramData choose where all plugin settings are held and automatically load the chosen set.
The two large buttons below, allow the settings to be copied FROM one location TO the other.
This feature affects all the settings on all the tabs.
In Global ProgramData mode the Export folder needs some consideration such that is valid for all Projects.
If it is located anywhere within the Project's Public folder, then it automatically adapts to each Project.
If it is located elsewhere, then all Projects will share exactly the same Export folder.

4. Copy settings for current GEDCOM Export Mode to/from (CEA/B) Custom Export Alpha/Bravo
The GEDCOM Export Mode product to be managed must first be selected on the Basic Options tab.
The Manage Settings tab allows the product settings to be copied to/from (CEA) or (CEB).
The name of the product is listed against (CEA) or (CEB) in the GEDCOM Export Mode droplist.
The settings can be separately adjusted to provide alternative export modes for the same product.
In this state, the buttons are only operative when the matching product is selected.
That state is cancelled when the Reset settings for (CEA/B) to default button is used.
This feature only affects the settings on the Basic/Extra/Other Options tabs and NOT the Labels or the Shared Settings on the Basic Options tab.

Do the features offer what was expected and do the messages and button names make sense?

ExportGedcomFileManageSettings.png
ExportGedcomFileManageSettings.png (41.5 KiB) Viewed 4485 times
Last edited by tatewise on 19 Feb 2022 16:55, edited 1 time in total.
Reason: Attachment deleted as later version is in the Plugin Store.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
avatar
quarlton
Famous
Posts: 166
Joined: 26 Feb 2004 13:07
Family Historian: V7
Location: Lincolnshire
Contact:

Re: Export Gedcom File plugin new features

Post by quarlton »

Thanks for the update Mike.
Just done a test run and it worked fine.
Dave Simpson ~ Boulton, Braham, Carney, Simpson and Jacobs
User avatar
tatewise
Megastar
Posts: 28341
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Export Gedcom File plugin new features

Post by tatewise »

Thank you Dave.

I have performed more thorough testing of various GEDCOM Export Mode products to check that all the export rules are correctly applied when settings are copied to (CEA/B) Custom Export Alpha/Bravo, i.e. when say TNG is copied to CEA that all the TNG rules are applied to CEA exported GEDCOM.
That testing has revealed that for some products some relatively minor rules were not always applied correctly.
The original product export modes are OK. It is only when exporting with CEA or CEB that there may be issues.

The attached Export Gedcom File plugin Version 4.6.4 Date 25 Oct 2021 ZIP file should fix those rules.
Download then right-click and Extract All... before installing into FH.
Last edited by tatewise on 19 Feb 2022 17:09, edited 1 time in total.
Reason: Attachment deleted as later version is in the Plugin Store.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
User avatar
tatewise
Megastar
Posts: 28341
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Export Gedcom File plugin new features

Post by tatewise »

Helen, in Updating existing FH records with data from CSV (19765) you requested:
Saving settings globally would be a great help -- starting from scratch in a new project is a pain at present. However, the ability to have 'named sets' of customised settings would also be good -- for example, if you're experimenting with settings for one of the export modes, but want to remember 'a set that works' that you can revert to if things go wrong... Or you want different settings for different instances of the same destination.
The attached plugin offers those features on the Manage Settings tab and has been available for about a month.
Before I publish it in the Plugin Store could you please comment on whether it satisfies your request?
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
User avatar
ColeValleyGirl
Megastar
Posts: 5465
Joined: 28 Dec 2005 22:02
Family Historian: V7
Location: Cirencester, Gloucestershire
Contact:

Re: Export Gedcom File plugin new features

Post by ColeValleyGirl »

Mike, I am up to my neck in work at the moment, and will not have a chance to do any testing.

However, I've installed it and opened it, and believe the options on the Manage Settings tab will let me do what I asked for.

Thanks.
User avatar
Mark1834
Megastar
Posts: 2458
Joined: 27 Oct 2017 19:33
Family Historian: V7
Location: South Cheshire, UK

Re: Export Gedcom File plugin new features

Post by Mark1834 »

I haven't had any need to run the Export Gedcom File plugin up until now, but a couple of comments from taking this version for a test drive:

The Unique ID field appears to be exported as it stands without converting the format to what the target app expects and supports. Most apps discard the FH version (hyphenated, no checksum), but would recognise the more common format of unhyphenated with a checksum (I have tested RM, Legacy, My Family Tree in the past). A workaround is to run the Format Unique ID plugin first, but it depends on the user knowing what the target app supports.

A common working mode with multiple apps is to have media in a separate location, so that one app doesn't have to refer to files in another app's folder. I couldn't find any way to include the file path of my media files in the export GEDCOM without unnecessary copying of gigabytes of data. It must be there, so which setting did I miss?
Mark Draper
User avatar
tatewise
Megastar
Posts: 28341
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Export Gedcom File plugin new features

Post by tatewise »

Thank you for the feedback.

I freely admit I'm a novice when it comes to Unique ID.
May I borrow the code from your plugin to export the more common format of unhyphenated with a checksum?
That should work with the products you mentioned and we'll have to wait and see about others.

The plugin deliberately puts all exported files in the same folder as explained in the Help & Advice FAQ:-
https://pluginstore.family-historian.co ... m-file-faq
That also provides the answer to your question about using separate export locations.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
User avatar
Mark1834
Megastar
Posts: 2458
Joined: 27 Oct 2017 19:33
Family Historian: V7
Location: South Cheshire, UK

Re: Export Gedcom File plugin new features

Post by Mark1834 »

Of course - plugins are there for cribbing ;). There is a more formal definition of the checksum calculation method in Gedcom L, but the method I use of extracting characters from the hex string is easier to code (IMO of course) and does the same thing. Unhyphenated with checksum is fine for all versions of RM and Legacy, but My Family Tree claims to be fully GEDCOM 7 compliant so expects to see it in that form (hyphenated no checksum as per FH, but with a UID tag, not _UID).

Not sure the FAQs actually address my issue. My media are external to the project with absolute file paths in the GEDCOM file, so I would want the export GEDCOM to refer to exactly the same locations so I can carry on using the same files without any copying. It's what the standard FH GEDCOM export does, but I could not replicate that in the plugin.
Mark Draper
User avatar
tatewise
Megastar
Posts: 28341
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Export Gedcom File plugin new features

Post by tatewise »

Sorry, I misunderstood what you wanted to achieve.

Many target products cannot use the FH Media as they stand because of Face Frames & Detail Frames.
So the plugin has to crop those Frames out of the Media images and create new Media records for them.
Also, the Basic Options tab, Shared Settings on the right, change the Media files in various ways.
That all leads to needing edited variants of most if not all the Media in the Export folder.

However, if the target product can work with unadulterated FH Media, c.f. (GST) GedSite, then use the Multimedia Conversion option (FILE~ABS) Multimedia via Absolute Link Records. But few products allow that mode.

What target product(s) are you exporting for?
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
User avatar
Mark1834
Megastar
Posts: 2458
Joined: 27 Oct 2017 19:33
Family Historian: V7
Location: South Cheshire, UK

Re: Export Gedcom File plugin new features

Post by Mark1834 »

I was thinking about the way I migrated to FH in the first place. I always used external media in FTM as well, and it was very easy to set up such that both apps used the same files during the transition. I'm sure I shared media with Gramps as well while working out which app to use.

I rarely use frames, and most of my media are plain vanilla image files. Out of curiosity, I set up a very simple test project with one person with one fact. I attached an image file to the person, and also an image of the source document supporting the fact.

After a standard GEDCOM export from FH, the resultant file imported ok into just about every product I tried, with the right media in the right place. Sometimes, right first time (RM, Legacy), sometimes after a bit of fiddling (FTM, My Family Tree).

Perhaps that's out of scope for the plugin, and it only deals with complex cases (so its complementary to the FH version, not replacing it), but it did strike me as odd that it can't seem to produce a basic bog-standard GEDCOM file. My main driver was to get a fuller project into RM, but it was for experimental purposes rather than an urgent need to migrate a full and accurate project copy.

That's probably all the time I'll spend on it for the moment.
Mark Draper
User avatar
tatewise
Megastar
Posts: 28341
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Export Gedcom File plugin new features

Post by tatewise »

For some cases where few FH specific features are used then the plugin may not be required.
The core features of GEDCOM are sufficient to get away with the FH built-in Export > GEDCOM File... command.
If you know enough about those features &/or not bothered if some data does not survive the migration then that works.

You were effectively using the plugin's Multimedia Conversion option (FILE~ABS) Multimedia via Absolute Link Records, but that only supports a subset of features.

The more FH features that are used, plus subtle differences in GEDCOM 5.5 & 5.5.1, and depending on which FH version and which target product, then the plugin becomes essential.
Not all products support UTF-8, or support same-sex partnerships differently from FH. Some don't even support multiple instances of facts. Some identify preferred Media images by different custom tags. Some don't support Media at all. Products support 'rich text' to different degrees. They support Fact Witnesses differently or not at all. The list goes on... Users are regularly requesting new features to migrate FH structures into other products. You have just joined them regarding UID formats. You only need to review the plugin Help & Advice pages for each product to see how their details vary from FH.

So if you are lucky and the products are reasonably compatible and product-specific features are not significant, etc, etc, then migration between products only needs a basic GEDCOM.
You have invested some effort in importing RM specific features into FH. Well, the reverse is often true for exporting and that is what the plugin tries to resolve.

It is not easy for users who are less well versed in the subtleties of GEDCOM to know whether they can get away with the Export > GEDCOM File... command or not. They are more secure in using the plugin.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
User avatar
Mark1834
Megastar
Posts: 2458
Joined: 27 Oct 2017 19:33
Family Historian: V7
Location: South Cheshire, UK

Re: Export Gedcom File plugin new features

Post by Mark1834 »

Err, where exactly did I say that the plugin was not required...? Quite the opposite, it's a popular and powerful utility that provides valuable extra functionality. I was just expressing mild surprise that it can't write a basic GEDCOM file with external links if that is all that is required (particularly if you are advising the inexperienced user to always use the plugin).

But it's your plugin, so you define the scope.
Mark Draper
User avatar
tatewise
Megastar
Posts: 28341
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Export Gedcom File plugin new features

Post by tatewise »

For the third time of saying:
Use the Multimedia Conversion option (FILE~ABS) Multimedia via Absolute Link Records

The concept of "a basic GEDCOM file" depends on a great many factors and whether that setting will work is indeterminate.

But in your simple example cases it should work.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
User avatar
Mark1834
Megastar
Posts: 2458
Joined: 27 Oct 2017 19:33
Family Historian: V7
Location: South Cheshire, UK

Re: Export Gedcom File plugin new features

Post by Mark1834 »

Ah, the penny has dropped. I'll explain why I was confused, not to perpetuate a discussion that has run its course, but hopefully to prevent others falling down the same rabbit hole...

I didn't recognise (FILE~ABS) Multimedia via Absolute Link Records in a menu entitled Multimedia Conversion: as equivalent to the CP setting of Use Full File Paths for External File Links as an option for Media Records. IMO, the CP version is much clearer.

I did check the Help & Advice page before posting, but there it talks about keeping the original media format, whereas what I wanted to do was keep the original file link, which isn't the same thing.

Finally, I was thrown off the trail by misinterpreting your comment that few products allow the FILE~ABS mode, while it's actually the standard GEDCOM mode supported by virtually all desktop products (but not of course more recent mobile and online ones).
Mark Draper
User avatar
tatewise
Megastar
Posts: 28341
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Export Gedcom File plugin new features

Post by tatewise »

Mark, the Media record link options are even more confusing than you suggest.

In your scenario, where Media files are OUTSIDE the Project, the CP options 'Use Existing File Paths for External File Links' and 'Use Full File Paths for External File Links' both have the same effect because your links are already full absolute paths.
Those two options only have a different effect for Media INSIDE the Project Media folder when relative links are used.

The CP Help page, under Exporting Media, explains relative and absolute file links, so the plugin uses that more precise terminology.

I will review the Plugin Multimedia Conversion options and Help & Advice in the light of your comments and see if they can be made clearer.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
User avatar
Mark1834
Megastar
Posts: 2458
Joined: 27 Oct 2017 19:33
Family Historian: V7
Location: South Cheshire, UK

Re: Export Gedcom File plugin new features

Post by Mark1834 »

Thanks. As a purely qualitative comment, I generally find UIs and help files easier if they are structured around the outcome I want to achieve (create new files for frame images, copy existing files, etc), rather than the process of how it is achieved (Part Frames via Local Media Objects, etc)...
Mark Draper
User avatar
tatewise
Megastar
Posts: 28341
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Export Gedcom File plugin new features

Post by tatewise »

I'm not sure I understand.
From the Help file https://pluginstore.family-historian.co ... ion-advice:
(PART~LMO) Part Frames via Local Media Objects
"Exports Face/Detail part-frames as cropped files for LMO and often will not include full-frame file versions."

That is intended to define the outcome not the process :-
Face/Detail part-fame media become cropped files and are linked to Local Media Objects (i.e. not Media Records).
The full-frame media versions will often be omitted.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
User avatar
Mark1834
Megastar
Posts: 2458
Joined: 27 Oct 2017 19:33
Family Historian: V7
Location: South Cheshire, UK

Re: Export Gedcom File plugin new features

Post by Mark1834 »

Or to put it another way - the UI is in plain English and speaks to the user in terms they understand, and the documentation gives the nitty gritty technical details for those that are interested. The other extreme is a UI that uses precise technical language, but most users need to go to the manual to work out what it means.

My guiding principle is that the typical target user shouldn't need to go to the documentation for basic operations that should be intuitive. For example, if I'm trying out a new family history app and it's not obvious how to add individuals, families, facts and sources, etc, it gets deleted.
Mark Draper
User avatar
tatewise
Megastar
Posts: 28341
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Export Gedcom File plugin new features

Post by tatewise »

OK, to help me get a handle on improving the UI, what would you suggest instead of Part Frames via Local Media Objects and perhaps some of the other options that need improving.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
User avatar
Mark1834
Megastar
Posts: 2458
Joined: 27 Oct 2017 19:33
Family Historian: V7
Location: South Cheshire, UK

Re: Export Gedcom File plugin new features

Post by Mark1834 »

I'll have a doodle in a day or two - other stuff I want to finish first...
Mark Draper
User avatar
Mark1834
Megastar
Posts: 2458
Joined: 27 Oct 2017 19:33
Family Historian: V7
Location: South Cheshire, UK

Re: Export Gedcom File plugin new features

Post by Mark1834 »

IMO, there is limited benefit in tweaking the odd word here or there, as I would make more radical changes. I think some of the language is unnecessarily complex. It takes too long to do a mock-up, but hopefully where I am coming from is clear from a description.

I think the area to look at first is Media records. Delete all reference to Local Media Objects. I tried looking them up, and there is nothing at all in the KB, and the first Forum reference basically said "FH doesn't use them". That's all I need to know. It's an unfamiliar term to FH users, so get rid of it. Turn it around to familiar language - "Use Media Records" Y/N tick-box , and the help can describe how they are not supported by all products and the alternative is associating media directly with facts etc.

Media links have 4 basic options - keep existing absolute file path, copy to new files in the same folder/subfolders as the GEDCOM file, copy somewhere else, or omit. Present those options explicitly in plain English.

Similarly for frame options - create new image of same size, create new image upscaled to roughly match the original, or omit.

Get rid of all options around rescaling or converting media format. That's scope creep and isn't needed for GEDCOM export.

Character Set - simplify to ISO/ANSI/UTF-8. Why do we need all the options? Other apps generally limit it to these.

Export Mode/Text Options/RT Formatting broadly ok.

There is one author and hundreds of users. Why should they all have to learn the author's language rather than the author using their language? An imperfect analogy might be Joop van Beek’s plugins - they’re in English, not Dutch with a translation in the Help pages.

These are just suggestions from seeing the plugin as it is now without all the history of why things are as they are, so take or discard as you see fit.
Mark Draper
User avatar
tatewise
Megastar
Posts: 28341
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Export Gedcom File plugin new features

Post by tatewise »

Thank you for your thoughts. May I provide a little background and my own thoughts?

I'm not sure what Forum Search you used but the first entry I found is Media Records and Local Media Objects (16733).
That gives a summary and explains where the LMO term is used in FH, which I copied years ago. It is not my invention.
An important point is that most products that support Media only use LMO and don't allow Media Records.
That is because GEDCOM 5.5 allowed File Links in LMO but not in Media Records.
FH v2 - v6 had to extend the GEDCOM with a custom _FILE tag to use Media Records with File Links.

There is another basic Media link option: 'keep existing relative file path'.

There are various frame options: create only a new cropped image; keep only the original full image; produce both forms;
Depending on what the user wants, any of those may be needed to replicate what is available in FH where 'cropped' part frames and full images are both provided.

However, those two settings together with "Use Media Records" Y/N tick-box, replace one drop-list with three controls in what is already a busy dialogue. Nevertheless, I will look at amending the wording, while recognising that handling Media migrating between products is more complicated than it appears at first sight.

The other Shared Settings are either crucial for handling some file formats (PNG, GIF, TIF) that do not crop/resize well, or have been specifically requested by users. I dare not remove them completely but they could move to another tab.
Some are not so much to do with GEDCOM export, as to improve the media presentation in other products, which is the overall objective.

The character set options could probably be slimmed down.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
Post Reply