For users to report plugin bugs and request plugin enhancements; and for authors to test new/new versions of plugins, and to discuss plugin development (in the Programming Technicalities sub-forum). If you want advice on choosing or using a plugin, please ask in General Usage or an appropriate sub-forum.
Mark1834 wrote: ↑26 Apr 2021 17:59
@Teresa, the finished product (hopefully by end of this week) will import all your custom RM templates directly, along with re-creating all the source data granularity that is lost on the normal GEDCOM import. Going forward, you can create new templates directly in FH. It does everything that RM does, and some more!
Thanks, Mark - that's great...I'll definitely have a boo when it's ready.
And yes, I have also created templates in FH. I love each program for various reasons, so maintain trees in each. Lots of work, but I also am all about not having all my eggs in one basket.
If I were just starting out, doubtless I would choose FH as my primary program.
Teresa
---
Teresa Basińska Eckford
Librarian & family historian http://writingmypast.wordpress.com Researching: Spong, Ferdinando, Taylor, Lawley, Sinkins, Montgomery; Basiński, Hilferding, Ratowski, Paszkiewicz
tatewise wrote: ↑26 Apr 2021 19:15
So that offers little hope of assistance with FM imports of RM Templates, which if FH offered Direct Import from RM then CP would have to consider how to handle RM Citation metafields in the RM Bibliography. The only chink is they say they will investigate and check it out.
I'm a "bad" genealogist, and rarely use that Bibliography field as I don't cite by EE, more a hybrid of Strathclyde and EE, favouring Strathclyde. I'll be curious to see how this all works.
---
Teresa Basińska Eckford
Librarian & family historian http://writingmypast.wordpress.com Researching: Spong, Ferdinando, Taylor, Lawley, Sinkins, Montgomery; Basiński, Hilferding, Ratowski, Paszkiewicz
Teresa, for most scenarios it will work well, especially if you make little use of the Bibliography.
However, the publically published plugins needed to work for as many scenarios as there are users.
So any limitations need to be made clear to users of the Bibliography that include Citation-specific Metafields.
Source record Metafields are perfectly OK.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
tatewise wrote: ↑28 Apr 2021 09:32
Teresa, for most scenarios it will work well, especially if you make little use of the Bibliography.
However, the publically published plugins needed to work for as many scenarios as there are users.
So any limitations need to be made clear to users of the Bibliography that include Citation-specific Metafields.
Source record Metafields are perfectly OK.
Yes, I understand that. It's always up to the user to decide if the plug-in offers the usability they require and to take appropriate cautions when using it. If I didn't work full-time, I'd be building way more templates in FH than I already do. Just not enough days in the week or hours in the day...plus, my husband does like to see me from time to time when I'm NOT doing genealogy
---
Teresa Basińska Eckford
Librarian & family historian http://writingmypast.wordpress.com Researching: Spong, Ferdinando, Taylor, Lawley, Sinkins, Montgomery; Basiński, Hilferding, Ratowski, Paszkiewicz
The plugin is now ready for more general user testing. I have developed it using a tiny database that I constructed in RM, plus the FH Sample Project exported to RM, then imported back again. Everything seems to work as intended, but how well does it cope with real RM projects?
To recap, all sources in RM are based on templates, either one of the 400+ supplied with the program, or those customised by the user. Unfortunately, when FH imports a GEDCOM file originating in RM, everything is converted to what FH7 calls generic sources. The custom template definitions are discarded completely, and while all the source data are imported, the structured version is hidden in Undefined Fields, and only mashups are visible.
This plugin does three things:
1. By reading the GEDCOM file that you used to transfer your database to FH, it extracts all the user-defined template definitions, and re-creates them as FH Source Templates. These templates should be virtually identical to the RM originals, as the basic structure of templated sources is very similar in the two apps. Footnote and bibliography definitions are copied across verbatim, with only minor translation (replacing [] with {}).
2. RM does not export its built-in template definitions, so we have to reconstruct them from the data included in the database. Every field used is included, along with an approximate label. No type definitions are included, so everything defaults to plain text (other RM types are date, place and name), and there are no data prompts. These should be reasonably clear, but you can always tidy them up in FH afterwards if you wish.
3. Finally, all sources are linked to their original templates, and the source data moved from Undefined Fields back into its correct place in the individual source fields. All other project data are left unchanged. This means that it doesn't matter if you have updated your project since your original import. Nothing entered by you is overwritten or deleted. The UDF fields are then deleted, but only after successful copying. Source level fields (yellow background in RM) are added back into their original sources, and citation level fields (green background in RM) are added to the appropriate source citation for the referenced fact. I have chosen not to update source titles, but that can be changed if necessary.
This plugin does not
A. Translate Bibliographies and Footnotes completely from RM to FH format. Hopefully the basic changes made create at least useable versions, but how good are they in the real world? I am not intending to create a "universal translator" (we'll leave that to Dr Who), but if there are any simple changes we can make that would be of use to the general user, I'm happy to add them.
B. Restructure the splitting/lumping of your data. This is intended as a general plugin, not tailored to specific users. If you use a bibliography layout that is not permitted by FH (e.g. including citation fields), you will have choose between modifying your data structure or changing the bibliography format. That is your choice, and the plugin doesn't make it for you.
Feedback welcome - what I'm aiming for is something that does most of the formatting for most of the users, leaving you to fine tune details manually. Realistically, it's a plugin that will be run once only, and only by a limited number of people.
As ever, it is best to try it out on a copy of your project, not the original. You can always select Edit > Undo Plugin Updates from the FH menu if you don't like the results, but better safe than sorry!
(Attachment deleted - replaced by version 2, below)
Last edited by Mark1834 on 29 Apr 2021 19:54, edited 1 time in total.
Thanks, Mark - I'll have a go this weekend and see what happens. Sounds very interesting and I'm curious to see the results
Teresa
Mark1834 wrote: ↑28 Apr 2021 22:15
The plugin is now ready for more general user testing. I have developed it using a tiny database that I constructed in RM, plus the FH Sample Project exported to RM, then imported back again. Everything seems to work as intended, but how well does it cope with real RM projects?
<SNIP>
Feedback welcome - what I'm aiming for is something that does most of the formatting for most of the users, leaving you to fine tune details manually. Realistically, it's a plugin that will be run once only, and only by a limited number of people.
As ever, it is best to try it out on a copy of your project, not the original. You can always select Edit > Undo Plugin Updates from the FH menu if you don't like the results, but better safe than sorry!
---
Teresa Basińska Eckford
Librarian & family historian http://writingmypast.wordpress.com Researching: Spong, Ferdinando, Taylor, Lawley, Sinkins, Montgomery; Basiński, Hilferding, Ratowski, Paszkiewicz
if Field.HINT then
p = fhCreateItem('PROM', pF)
fhSetValueAsText(p, Field.HINT)
end
if Field.LONGHINT then
if fhGetTag(p) == 'PROM' then
fhSetValueAsText(p, Field.HINT .. ' ' .. Field.LONGHINT)
else
p = fhCreateItem('PROM', pF)
fhSetValueAsText(p, Field.LONGHINT)
end
end
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
Sorted. If both HINT and LONGHINT are present, it concatenates them for the FH hint in the form "hint, longhint", which I think looks neater in typical examples than just a space. If only one is present, it uses that one as the FH hint, whichever way around it is.
Progress! I look some more later, but it looks like it brought in all my custom templates. I'll play with them a bit later. However, it didn't do well on the RM Templates. It brought in #139 and #172, but they were incorrect. The fields were brought in, but the templates had data. I'll post the error, and then what one of the templates look like. After those two it gave the error message below.
Screenshot 2021-04-29 155644.png (41.86 KiB) Viewed 5046 times
Screenshot 2021-04-29 155705.png (61.64 KiB) Viewed 5046 times
Thanks Jackie. There are two separate issues here. Firstly, the Bibliography and Footnote definitions for built-in templates are not included in the GEDCOM export, so I have used an associated source to create an example. On reflection, it might be better to omit them altogether. I anticipate that the typical RM user would have used only a small number of the hundreds of examples provided, so while they are copying the original template title across, they might as well copy the Bibliography and Footnotes as well if they want to keep them.
The error you reported comes at the final stage, as the UDF fields are copied back to the sources. I suspect it may be related to blank fields or missing data, so I probably need to build a bit more checking into the code. I'll look at it this evening, and hopefully third time lucky!
Thanks for looking. As far as me personally, I almost never use the standard templates, so I'm happy with it just bringing in the custom templates. But if you are going to publish it in the store, others may make much more use of the standard templates.
I have not tried to recreate Bibliography and Footnote entries for built-in RM Templates. Any that are included in project Source Records are now left as Undefined Fields, so you can copy them into your new templates and modify as required if you want to keep them.
There is now more extensive checking for error conditions, which should help prevent unexpected crashes. It also detects if it is being run for a second time on the same project, and avoids creating duplicate templates.
There was also a mistake in one of the template selecting routines that meant templates were not always being selected correctly. RM template identifiers are all numbers (<10000 for built-in, >= 10000 for user-defined), but numeric data is generally stored in FH as text. This doesn't usually matter, but when checking equality, both sides must the same variable type. At one point in the script this was not the case, so templates were not being identified correctly, resulting in the error reported above. Hopefully, now fixed.
Last edited by Mark1834 on 01 May 2021 09:31, edited 1 time in total.
Welcome to FHUG Ruth. I assume from your profile that you looked at FH6 a while ago, but have now made the move. FH7 is a much better fit for RM users than FH6, and you might find the related thread on importing the RM To-Do List useful as well. Neither templated sources nor research notes are available in FH6.
Thanks Mark. Yes I had previously reviewed vs 6, but found it a bit daunting. I decided to give vs 7 another look, and really liked what I saw. I am slowly learning the different features, and am really looking forward to using this software.
Ruth
Ruth, could you please update your Profile to the right of your posting to say Family Historian: V7
Click your username, chose Edit profile, select FH V7, then scroll down to Submit button.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
Sorry about this Jackie. I can see what has happened here, but I’m not clear why. The script is reading in your sources, but it can’t find an identifier for the template. The code looks ok, so my assumption about how RM structures its GEDCOM must be incorrect. I don’t have a real database to test on, but I’ll experiment tomorrow.
Which built-in sources do you use please, so I can try to replicate your sources as closely as possible?
Which built-in sources do you use please, so I can try to replicate your sources as closely as possible?
I don't think this will help much. I switched to entirely custom templates a long time ago, because I always want to change something later. The only built in source I've actually used is Newspaper, Online Archives. And only for source.
Thanks Jackie. The only way I can reproduce the error is by modifying the GEDCOM file before running the plugin to force an error condition, so can I ask for your help to diagnose what is happening please? I don't want to post multiple test copies of the plugin, but if you could add one additional line of code, it would tell us which source is causing the problem.
Starting from a fresh test database, could you open the plugin window, and instead of running the import plugin, click on More>> at the bottom of the window, and Edit... to open the plugin in the debugger.
Scroll down to the line where the problem occurs, 345, and add the following line of code between lines 344 and 345 (Lua doesn't worry about whitespace, so blank lines and indentations have no significance, they are just there to help readability).
if not tonumber(TID) then fhMessageBox(fhGetDisplayText(pS) .. ' - ' .. fhGetDisplayText(pT) ) end
It should then look something like this
Capture.PNG (11.87 KiB) Viewed 4766 times
Run the modified plugin by clicking on the Go button at the top of the window (you do not have to save it first).
The error is caused by the template identifier (TID) not being a pure number (either missing or containing non-numeric characters), so the tonumber() conversion returns a nil result. The new line of code will display the name of the errant source and the TID, as in this example where I created a new Newspaper Source and manually deleted the TID prior to running the plugin to force the error. When you click on ok, the script will resume and crash.
Capture1.PNG (2.75 KiB) Viewed 4766 times
What message do you get please? Once we know what is causing the problem, we can move on to why and fix it.
You could check if pT:IsNull() that would indicate there is no TID tag at all.
I noticed that there is a lot of management of table values to check for uniqueness or existence in another table.
Have you considered using the value as the table key instead of the table value?
i.e. Instead of table.insert(tblA,Value) use tblA[Value] = true
That ensures there is only one entry for each unique Value and avoids the need for function UniqueValues(tblA) and probably function InTable(val, tblA).
That is the conventional Lua technique for building a dictionary of unique table entries to avoid searching tables for values.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry