* Export Public GEDCOM File - new plugin draft

[Tony Jones]

Just exported my tree to Mac Family Tree (no media), all facts, geocoding — nice and nippy and did exactly what I needed to create a portable tree I can use on my iPad when I go to a records office.

Thanks very much

[Mark1834]

Thanks for the feedback, Tony. I’m not familiar with Mac Family Tree, but its companion iPad app (MobileFamilyTree) looks very good from the slick promo video. It’s only £15, so I’m tempted to buy it anyway for the extra bells and whistles over the perfectly adequate but more limited GedView.

I’ll add it to the list of supported destinations in the plugin help.

[Tony Jones]

It’s on sale at the moment. Visually it really is impressive, good maps and event timeline features

[mjashby]

For me it's an OK software with very clever visuals. But it seems that the focus of development has been more on ensuring full advantage is taken of Apple's reputation for the quality of it's graphics support, rather than easing the, often repetitive, input of user data. The one 'unique' feature advantage it does have for a desktop family history software is internet networking capability (using iCloud) i.e. multi-device and multi-user data maintenance is possible.

However, for a European produced genealogy software that been around since 1998 and, presumably, wants to be taken seriously, I'm always surprised at the continuing absence of support for anything other than the Gregorian Calendar; and years that only commence on 1 January.

Mervyn

[Tony Jones]

Yes, I don’t see me using it for data capture anytime soon!

[mjashby]

Tony,

I'm sure the iOS version could be an effective way of highlighting your research priorities and capturing information when you are working remotely, or just for interacting with the online data providers on the iPad/iPhone. When I did try the iOS version, some time ago now, it was definitely useful in that way if only to capture the base results from Ancestry/FindMyPast searches; and it also provided a direct link to FamilySearch data etc., so at £15 it's certainly not a bad investment. And, if you ever get bored, it can be quite entertaining to play with the 3D Tree views for a few minutes.

Though Family Historian remains my preferred application, I do like to have a secondary application running alongside and, as a Mac User, I do keep looking at MacFamilyTree from time to time but it never quite convinces me, as it appears to be best suited to presenting the results of relatively small and highly detailed research projects (preferably with lots of graphical content to make use of many of its features).

Mervyn

[Mark1834]

The draft plugin and associated help file have now been updated to version 0.3 (download link).

Main menu.png (22.13 KiB) Viewed 779 times

The most significant change is the addition of support for part-frame images cropped from a larger file (adapted from Adrian Cook’s kindly provided template). A new menu option allows the user to toggle between full-frame and part-frame export. When selecting part-frame, these are always created in the export folder, irrespective of other options, as they don’t exist in FH as separate files so have to be generated on the fly.

There is also a new option appended to the Sources menu, enabling export of source images to be controlled independently of individual/family images. This can be useful for saving space when uploading to your tablet of choice for use off-line, or simply for users who want to avoid breaching copyright requirements for source images while still including family photos. By design, this simple plugin does not include options for resizing full images, but greater flexibility is available from apps such as IrfanView.

Two new feedback messages, below, give the user the option to cancel the actual copying of media files to their new location once the GEDCOM file has been generated (but not yet written to disk), plus an enhanced status report when the export is completed.

Copy confirmatin.png (10.35 KiB) Viewed 779 times

End message.png (6.54 KiB) Viewed 779 times

I played with adding FH6 compatibility as well, but haven’t pursued it. Reading fact sets was much easier than I expected (literally just one line of code to change from FH7), and while none of the other differences were insurmountable, there were too many to justify the extra coding complexity for what is now likely to be a very small user base.

This is probably about as far as we can go for new features and options while still adhering to the design objective of being both easy to use and maintain or adapt, while still offering comprehensive compatibility with current apps and websites, but I’ll leave the draft here for another couple of weeks or so for any further suggestions or bug fixing before it goes to the store.

[tatewise]

Mark, that download link demands I'm signed into my Microsoft account, which I was, but fails after 3 attempts.
The earlier link to the help file works fine.

[Woodg]

Yes, I have a similar issue. I need to login to my OneDrive but then there is nothing to download.

[Mark1834]

Oops, apologies - all I thought I did was overwrite the two shared files, but I seemed to have created a new link somehow. I've deleted the superfluous links, which unfortunately kills the earlier version, but I have successfully edited the link in my recent posting. It is too late for a non-admin to change the one at the top of the thread (but interested readers will probably scroll down to here anyway).

[tatewise]

The new link works but the plugin fails when a Media record has no FILE link:
[string "C:\ProgramData\Calico Pie\Family Historian\Pl..."]:510: attempt to perform arithmetic on a nil value (local 'j')

After fixing that by adding a FILE link, many Media Records had FILE links that included such as {154,46,243,127} in the end of the filename, e.g. C:\Users\Mike\OneDrive\Desktop\O1_The_Munros_{154,46,243,127}.jpg
Such media files did not exist so displayed the missing file link X.
What did exist were uncropped media files with similar names, e.g. ...\O1_The_Munros.jpg
They displayed correctly.

So the problem seems to arise when the same media file is linked as both a full image and a part frame.
The 'Family Historian Sample Project' has many examples of that scenario and exhibits the problem extensively.

[Mark1834]

Good catch on the first point - the plugin checks for unlinked files but not Media Records with no file reference at all. I suspect it won’t actually arise very much in practice, but it’s a simple fix to trap the error.

I must admit, I can’t duplicate the second scenario. I tested with the Sample Project of course, and it works as expected for me. I suspect it may arise from the interlock between the tick box and radio buttons, as I can see that it’s not quite right.

On reflection, it would be better to simplify the Media options, as I’ve probably fallen into the common trap of bolting on the crop facility to the existing options, rather than considering what the best overall solution would be.

I think we need just three options - no media, minimum copying with just the new cropped files created in the export folder, or copy all used media (existing full-frame file or new cropped file) to the export folder. One or other should fit any destination or use scenario, it’s easy to describe in everyday non-technical language, and easier for me to code and maintain.

I’ll update the draft later in the week.

[tatewise]

I'm surprised you cannot repeat the symptoms I described.
I've just reset the Family Historian Sample Project to be sure we would both have the same dataset.
Then run the Export Public GEDCOM File - V0-3 plugin with the settings shown below:

image.png (32.33 KiB) Viewed 667 times

Open the exported GEDCOM standalone in FH and the Media Window has broken links with odd FILE paths:

image.png (48.56 KiB) Viewed 667 times

[Mark1834]

Nope - still can't replicate it - works perfectly me with a reset Sample Project, which I even deleted prior to loading the GEDCOM file to ensure it was seeing the correct files. The {nnn,nnn,nnn,nnn} suffixes are just the _AREA tags, which I left in that form just in case I needed the numbers, say for a batch job in IrfanView.

File Manager shows the files have all been copied ok (into a previously empty folder), with the cropped files as new images and the full frame as simply copies of the original file.

Your output suggests that the cropped files have not been created correctly. Have you checked they are in the export folder? If not, do you have something unusual in your FH setup that inhibits the copy operation? If you step through the CopyCroppedFile function that does the actual cropping and saving, does anything odd show up? Could it be a OneDrive sync issue, as you commented that you don't leave it logged on permanently?

Clipboard01.png (477.26 KiB) Viewed 651 times

Files.png (138.56 KiB) Viewed 651 times

[tatewise]

CopyCroppedFile(...) fails at the very first im.FileImageLoad(source) but the if Error then return does not report the 'Image load error ' .. Error. IMO such error reporting needs to be fixed.

The failure is due to an accented UTF-8 character in my usual Project Location filepath.
However, any Project Name with UTF-8 accents/symbols would similarly fail.
I deliberately use that Project Location path to ensure my plugins handle such characters.

If I use an ANSI compatible filepath the plugin correctly creates cropped media files.

My Export Gedcom File plugin employs the usual trick of copying files with UTF-8 chars in the path to an ANSI only file path before invoking the im library functions and if necessary renaming them afterwards.
i.e. Follow your own International Character Handling for Plugin Authors advice regarding File Access and Manipulation for File Handling Within IUP and Associated Libraries CD and IM

BTW: I earlier said I was signed into my MS OneDrive account.

[Mark1834]

Ah - a key piece of information not mentioned before. I'm slightly surprised that you didn't recognise the difference straight away and just assumed that I had not done basic testing with the Sample Project, but so be it. Now the issue has been diagnosed it's a simple fix.

The rather childish sign-off jibe doesn't work, by the way. The KB is a Wiki, and while I have contributed to that page, it's not my page, and I'm certainly not the originator or lead author.

[tatewise]

I'm sorry I did not recognise the cause straight away. I had lot going on yesterday.
If the error reporting had been working I'd have probably identified the problem sooner.
I assumed that any plugin involving file handling would be tested with files using non-ANSI character paths.

[Mark1834]

I have now updated the plugin draft and help file to version 0.4, which fixes the issues raised above and in addition greatly simplifies the media export options, as per the screen grab below.

Media.png (3.58 KiB) Viewed 505 times

This provides an easy to use "what you see is what you get" approach to media export, and avoids the complex and rather technical options required for comprehensive export that seem to cause so much traffic on this site asking for help or advice.

The first option represents the minimum amount of copying required for export, while avoiding unnecessary copying of potentially thousands of files and gigabytes of disk space. The second option copies all media to the export folder, which is the preferred option if you want to resize images prior to upload or combine manually into a single GEDZIP file using an app such as 7-Zip.

This time, I've managed to do the update without messing up the downlink link (a few posts back)!

[tatewise]

I have the following comments on this latest version...

If the Media record FILE link does not include a backslash \ then the following error is produced:
[string "C:\ProgramData\Calico Pie\Family Historian..."]:544: attempt to perform arithmetic on a nil value (local 'j')
That can occur if forward slash / is used instead or a URL is the media link.

The Help for Media Options says: 'Create new “Face Link” files in export folder’ creates new image files for any Media viewed in Family Historian as a “Face Link” close-up, which are written to the same folder as the exported GEDCOM file. Full-frame images are not copied, ...
However, full-frame images that don't have any cut-outs are copied.

Under Source Options the Help for “Include Source images” does not mention that the option to 'Create new “Face Link” files in export folder’ applies here too although the FH setting is Detail Link not Face Link.

Under Selecting Individuals and Families for Export the Help says: All Individuals marked as either Living or Private are excluded completely from the export, along with their corresponding family records, either as a spouse, parent or child.
However, an Individual with the Living Flag is excluded but his wife and children are included and the Marriage Event has Source details that clearly identfy the husband.
That can be illustrated in the FH Sample Project by setting the Living Flag on Anthony Edward MUNRO.
His spouse Susan Isabel DOWLING and their children are included in the export with the Family record and her Marriage Event sources reveal his details.
His other spouse Juia Amanda FISH correctly does exclude the Family record so the Marriage Source details are also excluded.

[Mark1834]

Thanks Mike, some interesting points there -

That can occur if forward slash / is used instead or a URL is the media link. - How did you add the media to the project please? I must admit, I didn't know you could add an external url instead of a file. I tried it using one of the images on this site, and it showed correctly in FH, but also exported correctly in the plugin. Checking the GEDCOM file, it was showing a file link to my local internet cache, not the original url. If I changed that manually to the url, it did indeed generate the error you described, but it didn't display in FH either.

full-frame images that don't have any cut-outs are copied. - Can you confirm that, please? I've just tried it exporting into an empty folder, and it works as predicted. There is deliberately no option to clear the destination folder, as I think it would be too error-prone, but version 0.4 does highlight any existing files before proceeding. How is the "Empty Export Folder" button supposed to work in the Export Gedcom File plugin? I tried it, and it doesn't seem to do anything (it's a button rather than an option, so I would expect any action to be immediate).

the FH setting is Detail Link not Face Link - I must admit, I probably do have inconsistent terms in the plugin, as FH doesn't seem consistent either. Personally, I dislike "Link to Face" description used in the main program, as I use it for highlighting a detail in a source image much more frequently than linking to photographs. I'm trying to use the same terminology as FH, but it's a moving target!

the Marriage Event has Source details that clearly identfy the husband - Good catch. The selection routine is a straight copy-paste from the Ancestry plugin, which does not include sources. I'll think about the best way of dealing with that before releasing the store version.

[tatewise]

That can occur if forward slash / is used instead or a URL is the media link. - Just to be thorough I manually edited the FILE link to replace a \ back slash with a / forward slash to see what happened. Media record URL can be opened the same as any non-image media by double-clicking its icon or clicking the Open in Editor/Player button.

full-frame images that don't have any cut-outs are copied. - On closer inspection the media files are not copied into the export folder but the exported GEDCOM includes Media records that link to the original full image held in the original Project. So they either get included in the data imported to the destination or the destination gets a Media record with a broken media link.
e.g.

Code: Select all

0 @O33@ OBJE
1 FILE C:\Users\Mike\OneDrive\Documents\Family Historian Projects\Family Historian Sample Project\Family Historian Sample Project.fh_data\Media\Places\O33 Midsomer Norton Church.jpg
2 TITL  Midsomer Norton parish church
2 FORM jpg
0 @O66@ OBJE
1 FILE C:\Users\Mike\OneDrive\Documents\Family Historian Projects\Family Historian Sample Project\Family Historian Sample Project.fh_data\Media\O12_Ian_and_Charlotte_and_family.JPG
2 TITL Ian and Charlotte and family
2 FORM jpg

the FH setting is Detail Link not Face Link - In FH for highlighting a detail frame in a source image linked to a Source Citation the option is called Link to Detail and not Link to Face, which is why I raised the issue.

the Marriage Event has Source details that clearly identify the husband - OK.

[Mark1834]

the exported GEDCOM includes Media records that link to the original full image held in the original Project

That's exactly what is meant to happen . If all I want the export for is a one-off transfer to RM or FTM for uploading to Ancestry (for example), copying the file into the export folder is unnecessary. Both RM and FTM are perfectly happy with linked media (it's the only option in RM), so as long as I convert the Media folder path reference to an absolute path, the recipient app finds it, as illustrated in this screen grab, where the frame-free birth certificate is not copied, but imports into RM without any issues.

Media.png (96.61 KiB) Viewed 413 times

The disadvantage of doing this is that the export GEDCOM is still linked to the original Project, which is what you were alluding to. If the original project is amended, the link can get broken. It was precisely this scenario that I was trying to avoid with the now-deleted option to recreate the Media folder in the export folder, but I was never happy with being able to describe this in clear terms in either the UI or the help file.

So the plugin is doing exactly what I intend it to do, but it may need some tweaking of the help file to emphasize this continued link and dependency. First option for a one-off transfer, second option for longer term use in-situ on the user's PC.

In a way, this illustrates my essential design philosophy for the plugin - to provide workable solutions with a minimum of complexity. If the user wants to control the export and configuration in more detail, they use the Export Gedcom file plugin. They are not competing solutions, but complimentary ones that sit at different points on the functionality - simplicity line.

[tatewise]

Ah! Now I understand but the Help did not convey that process to me.
Perhaps the Help needs to make it clearer that the only difference between the two options is that in the first case the non-cut-out Media is kept in the original Project and in the second case it is copied to the export folder.
The cut-out Media is handled the same way in both cases.
In both cases all the Media is linked to the exported GEDCOM so from the UI they look identical.

[Mark1834]

I have now updated the draft plugin to version 0.5 and updated the link in the first post in this thread.

Export Public GEDCOM File.png (22.28 KiB) Viewed 207 times

• The automatic removal of private individuals and related information has been tweaked to make it more thorough.
• This automatic editing of a tree on the fly is one of the key features of this plugin compared with other exports, but earlier versions didn’t report exactly who they had removed and why, either before or after the export. A new Private Records button on the main menu tabulates this, and can be called Independently of the actual export.
• More detailed investigation of compatibility with RootsMagic showed that while all versions import the standard Media export, they do not assign the profile picture, so do not transfer this to Ancestry. As this is likely to be a common way of using the plugin, I have added an “optimise for RM” option to the main menu (similar in concept to Census/Residence for Ancestry). With this selected, the first Media record, which is the one also displayed in FH), is assigned as the profile image and also displayed in Ancestry trees.

I haven’t updated the help file with these changes, but they will of course be included in the Store version when it’s submitted.

[Mark1834]

No further issues have been raised, so the plugin is now available in the Plugin Store, and includes an updated help file.

The links earlier in this thread will be deleted in due course.