* Ability to Change Type of Fact

For existing requests please see The Wish List
User avatar
tatewise
Megastar
Posts: 22116
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Ability to Change Type of Fact

Post by tatewise » 18 Jul 2021 10:26

Mark, I noticed in your Plugin that it determines whether a Fact is an Event or an Attribute by checking the Fact Type definition for -IE or -IA, or -FE or -FA.
Did you know there are fhIsEvent(ptr) and fhIsAttribute(ptr) API functions that perform that test more effectively?
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

avatar
Mark1834
Megastar
Posts: 857
Joined: 27 Oct 2017 19:33
Family Historian: V7
Location: South Cheshire, UK

Re: Ability to Change Type of Fact

Post by Mark1834 » 18 Jul 2021 14:01

Let me test my understanding before I write any new code -

“My objective is to present the user with a complete listing of available (i.e. not hidden) facts from all Fact Sets (Global and Project). Where the same Fact Name exists in multiple Fact Sets, the Fact Label is taken from the Fact Set with the highest priority.”

Given that I’m not considering any other property of facts, I think that is a complete definition of the task in hand.

This does not preclude the possibility of having two distinct facts with the same label (e.g., creating a custom Christening fact, but labelling it as a Baptism). It’s weird and perverse, but not prohibited as far as I can tell.
Mark Draper

User avatar
ColeValleyGirl
Megastar
Posts: 3018
Joined: 28 Dec 2005 22:02
Family Historian: V7
Location: Cirencester, Gloucestershire
Contact:

Re: Ability to Change Type of Fact

Post by ColeValleyGirl » 18 Jul 2021 14:13

Sounds about right to me, Mark.

Re Mike's suggestion, I'll say that it doesn't help to be able to derive something from a ptr when you don't have a ptr (i.e. you're parsing fact set files to derive facts that might not be in use yet).

avatar
Mark1834
Megastar
Posts: 857
Joined: 27 Oct 2017 19:33
Family Historian: V7
Location: South Cheshire, UK

Re: Ability to Change Type of Fact

Post by Mark1834 » 18 Jul 2021 14:20

Thanks Helen - agree not useful for reading fact sets, but might be for record facts. On its own, it just replaces one line with a different line, but it might help to generalise that chunk without repeating for Individual and Family.
Mark Draper

User avatar
tatewise
Megastar
Posts: 22116
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Ability to Change Type of Fact

Post by tatewise » 19 Jul 2021 09:12

Mark, your understanding of available facts from multiple Fact Sets is correct, including the possibility of distinct facts with the same label.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

avatar
Mark1834
Megastar
Posts: 857
Joined: 27 Oct 2017 19:33
Family Historian: V7
Location: South Cheshire, UK

Re: Ability to Change Type of Fact

Post by Mark1834 » 20 Jul 2021 13:23

Version 2 is attached. As well as the enhancements and corrections described earlier, this version now supports all Fact Sets and eclipsed Facts, including those saved at the Project level. Similar existing facts are now sorted chronologically, as I figured out a way of doing that with only a handful of lines of code.

Inevitably, the extra functionality has fattened it out a little, but it is still only a 4 page code listing, so hopefully free of bloat and reasonably accessible to other users for copy and adaptation if required.
Last edited by Mark1834 on 20 Jul 2021 14:12, edited 1 time in total.
Mark Draper

User avatar
tatewise
Megastar
Posts: 22116
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Ability to Change Type of Fact

Post by tatewise » 20 Jul 2021 13:55

(1) Unfortunately, this version raises an error:
An error has occurred - plugin failed to complete
[string "C:\Users\Mike\AppData\Local\Temp\~fh532D.tmp"]:68: attempt to concatenate a nil value (local 'FactName')
No changes have been made to data records.
The only Tools > Fact Types... definition is:
Name: Death Informant
Label: Death Informed
Abbreviation: Death Informed

But there is also an <undefined> attribute with a Name and Label of Death Informed.

The plugin is looking for _ATTR-DEATH_INFORMED but the tblAllFactHeadings only holds Death Informant => "[FCT-_ATTR-DEATH_INFORMANT-IA]"

The GEDCOM contains:
1 FACT
2 TYPE Death Informed

So the problem is caused by a custom fact that has no Tools > Fact Types definition.
That happens quite often when a GEDCOM is imported from another product.
It is such Facts that often need changing to an existing FH Fact.
It should be unnecessary to have to create a definition for such Facts just to change them and then have to delete the definition.

(2) After getting over that problem it also reports:
An error has occurred - plugin failed to complete
[string "C:\Users\Mike\AppData\Local\Temp\~fhCB91.tmp"]:137: attempt to index a boolean value (local 'FileContents')
No changes have been made to data records.
That happens when a standalone GEDCOM file is open in FH instead of a Project.

(3) It is still not including custom facts that have non-alphanumeric characters in their Name or Label.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

avatar
Mark1834
Megastar
Posts: 857
Joined: 27 Oct 2017 19:33
Family Historian: V7
Location: South Cheshire, UK

Re: Ability to Change Type of Fact

Post by Mark1834 » 20 Jul 2021 14:16

Thanks Mike - good to test with other Fact Sets, as I only have fully formed and well-behaved ones!

I've deleted the plugin file to stop anybody else downloading it, and I'll post an update when it's fixed. It sounds like I just need to go through the import routines to ensure that they correctly handle missing data. I'll look at it later today.

The second error was easy to spot. I made sure that I had project level facts to test the ranking with, but omitted to cater for projects that don't actually have a fact set definition file! Easy fix.

OK - we're both editing our posts as we gather more information! I'm off to do something else for a couple of hours, and I'll come back and look at it later tonight or tomorrow morning.
Last edited by Mark1834 on 20 Jul 2021 14:26, edited 1 time in total.
Mark Draper

User avatar
tatewise
Megastar
Posts: 22116
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Ability to Change Type of Fact

Post by tatewise » 20 Jul 2021 14:25

OK, but not expecting to find many problems I started posting and then found more. Sorry!
So have you seen all three problems that I've posted?
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

avatar
Mark1834
Megastar
Posts: 857
Joined: 27 Oct 2017 19:33
Family Historian: V7
Location: South Cheshire, UK

Re: Ability to Change Type of Fact

Post by Mark1834 » 20 Jul 2021 14:30

Neither was I, but that's life! Please find as many problems as you can. You've probably got the most diverse collection of data available to anybody, so if you can't break it, it is probably safe.

I'll settle down later and work through them - they don't sound serious, but my takeaway is that I probably need to test missing data more thoroughly.
Mark Draper

User avatar
tatewise
Megastar
Posts: 22116
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Ability to Change Type of Fact

Post by tatewise » 20 Jul 2021 15:27

The reason why some Facts are missing from the New Event and Attribute lists is that the Plugin assumes that Fact Set files hold the Fact definitions in Item order. i.e. Item1 is 1st, Item2 is 2nd, and so on...

Therefore, as soon as an out of order Fact definition is found at plugin line 196 then the rest of the Fact Set is skipped.

An example is the ...\Custom\Military2.fhf Fact Set from Military History, but there are others.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

avatar
Mark1834
Megastar
Posts: 857
Joined: 27 Oct 2017 19:33
Family Historian: V7
Location: South Cheshire, UK

Re: Ability to Change Type of Fact

Post by Mark1834 » 20 Jul 2021 22:14

I'll look at the other issues tomorrow, but there is definitely something odd about the Military Fact Set on the KB :?.

All of the FH7 Fact Sets that I used for development had fact detail sections listed in exactly the same order as the summary list at the top of the file. I checked that point specifically so I could make one run through the file for all facts, not a separate parse for each one. I've now checked the other Fact Sets on the KB, and the same applies to all of them.

Not only does Military2 have one Fact out of order (Service is listed as Item1, but is third in the detailed listing), there are two other Facts (Army Reserve and Awarded Military Pension) with detailed definitions but they are missing completely from the summary listing!

If I manually edit the Fact Set file to add the two missing facts to the summary list and increase the total count to 36, they still don't get included when the Fact Set is imported into FH. There must be something wrong with the definitions, but I can't spot anything unusual.

The Military Fact Set file looks quite old. Did it come from an earlier version of FH that had a bug in managing Fact Sets? Has it been corrupted by manual editing at some point? If we can't resolve how this arose, I think the best route for the plugin will be to ignore detailed descriptions that are not in the summary listing, and assume that detailed listings can be in any order. If it is a bug, it may exist in other custom Fact Sets as well.
Mark Draper

User avatar
tatewise
Megastar
Posts: 22116
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Ability to Change Type of Fact

Post by tatewise » 21 Jul 2021 10:04

The Military Fact Set download page says it was originally created by annettelindahutton (16/10/10), and updated by tatewise (27/09/11), which puts in the FH v4.1 era, and most of the others are of that era or even earlier.
I know the Military Fact Set is derived from a much earlier version.

How the anomalies arise I cannot possibly guess, but as you say, any private custom Fact Sets could be similar.

Those two 'missing' Event definitions are faulty. The FCT code does not match the Name.
i.e. TO_ARMY_RESERVE does not match Army Reserve, which if changed to To Army Reserve gets it working.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

avatar
Mark1834
Megastar
Posts: 857
Joined: 27 Oct 2017 19:33
Family Historian: V7
Location: South Cheshire, UK

Re: Ability to Change Type of Fact

Post by Mark1834 » 21 Jul 2021 13:35

Mostly sorted. Import is independent of sequencing, so it is perfectly happy with the current Military fact set. All facts are now displayed correctly (they were all imported, but not always displayed - not due to non-alphanumeric characters, but where the Name and Label differed). File import is fully encapsulated in another function that checks for the file’s existence first and converts the contents to a table rather than a single string.

I’ll also update the Military fact set on the KB later. We might as well restore the missing facts, even if we’ve gone over 10 years without anybody noticing :D.
Mark Draper

avatar
Mark1834
Megastar
Posts: 857
Joined: 27 Oct 2017 19:33
Family Historian: V7
Location: South Cheshire, UK

Re: Ability to Change Type of Fact

Post by Mark1834 » 21 Jul 2021 17:06

Hopefully now fully sorted. I was able to reproduce the problem with undefined facts by creating a user-defined Fact in RootsMagic, name Vaccinated, abbreviation (label) Jabbed. When importing to FH, the label came across but the name did not, as RM does not include it in the export GEDCOM.

This undefined fact crashed plugin 0.2, as it used the fact name as part of chronological sorting. Now fixed to use label instead, which every fact has.

Everything seems to be working as intended now, but I’ll do further testing before posting the update (not tomorrow, as no power all day due to network maintenance).
Mark Draper

avatar
RJDoggett
Silver
Posts: 8
Joined: 16 Dec 2020 21:22
Family Historian: V7

Re: Ability to Change Type of Fact

Post by RJDoggett » 21 Jul 2021 22:09

As a former TMG user, the ability to change a Fact type being added to FH would be useful - TMG allowed Tag types to be changed. TMG also allowed the individuals linked to a Tag to changed. These features were particularly useful when correcting errors in the database.

From the threads above, it is accepted that there will be limitations, probably in relation to family versus personal events. However, the ability to change the Fact type to another compatible one without deleting and re-entering data will be beneficial.

From the threads above is seems as if an add-in will be quite complex. The point made previously that the capability to change Fact types should be implemented by CP will, in the longer term, provide a better solution. CP will be able to determine the Fact type groups that can be readily changed and those that cannot.

avatar
Martin Tolley
Platinum
Posts: 44
Joined: 02 Aug 2015 10:48
Family Historian: V6

Re: Ability to Change Type of Fact

Post by Martin Tolley » 22 Jul 2021 14:05

RJDoggett - agree.

User avatar
AdrianBruce
Megastar
Posts: 1288
Joined: 09 Aug 2003 21:02
Family Historian: V7
Location: South Cheshire
Contact:

Re: Ability to Change Type of Fact

Post by AdrianBruce » 22 Jul 2021 14:20

RJDoggett wrote:
21 Jul 2021 22:09
... CP will be able to determine the Fact type groups that can be readily changed and those that cannot.
Am I missing something about what you mean by "Fact type groups"? Given that we can create our own Fact Types, I don't see how CP can determine Fact to Fact change compatibility any better than the next person?
Adrian

User avatar
tatewise
Megastar
Posts: 22116
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Ability to Change Type of Fact

Post by tatewise » 22 Jul 2021 14:55

I suspect the primary groups are Individual versus Family and Event versus Attribute.
You might also include the Normal Time Frame values as distinct groups.
However, Events can usually be changed to Attributes, but the reverse may not be easy if it has a value.

So if the fact being changed is an Individual Event with Normal Time Frame = Life, then the initial shortlist of candidates would be all the other similar facts, but perhaps with the option to relax the Event/Attribute &/or the Normal Time Frame filters.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

avatar
Mark1834
Megastar
Posts: 857
Joined: 27 Oct 2017 19:33
Family Historian: V7
Location: South Cheshire, UK

Re: Ability to Change Type of Fact

Post by Mark1834 » 22 Jul 2021 15:22

That’s an interesting possibility. I’m not going down the time frame route, as I’d rather leave that to users, as I do with output formatting and witnesses. However, it would be a trivial change to the plugin to permit conversion between attributes and events if that is considered worthwhile. I would flag a warning first, and save any attribute value as a local note if converting to an event.
Mark Draper

User avatar
tatewise
Megastar
Posts: 22116
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Ability to Change Type of Fact

Post by tatewise » 22 Jul 2021 15:41

We are perhaps in danger of mixing up the ideal CP solution Wish List request, with what Marks' plugin might offer.

The latest postings referred to RJDoggett's comment that "CP will be able to determine the Fact Type groups".
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

avatar
Mark1834
Megastar
Posts: 857
Joined: 27 Oct 2017 19:33
Family Historian: V7
Location: South Cheshire, UK

Re: Ability to Change Type of Fact

Post by Mark1834 » 22 Jul 2021 16:23

No confusion with me - one is a practical discussion of what we can do for ourselves as users now, and the other a hypothetical discussion about what CP might (or might not) do at some point in the future.

To address RJDoggett's point directly, this plugin is actually fairly simple, both in terms of structure and scope. Deliberately so. The development process is typical of what usually happens with such things. The author gets it working with their own data, and posts a first draft here in the expectation of other users finding issues, such is the diversity of data available to us. After one or two further iterations, a stable version emerges. Plugins that prove useful to other users generally (as opposed to a specific one or two person niche use) get submitted to the CP Plugin Store for greater visibility, and that may well happen with this one in due course.

As was pointed out earlier, there is a fundamental contradiction in CP being less likely to fix issues that plugins have addressed already. Personally, I don't have a problem with that. CP are a small company with limited resources and a niche product (as are RM and many of the other players in this space - the big corporations are not interested in desktop software any more). I would rather they concentrated on getting the core application right, and adding features that plugins couldn't address.

I see no reason not to permit interconversion between Events and Attributes (the boundary is a bit grey anyway), so I will make the executive decision. The plugin will offer that facility. New version to be posted over the weekend.
Mark Draper

avatar
Mark1834
Megastar
Posts: 857
Joined: 27 Oct 2017 19:33
Family Historian: V7
Location: South Cheshire, UK

Re: Ability to Change Type of Fact

Post by Mark1834 » 25 Jul 2021 10:14

Ready to go with the next version now. There are quite a few enhancements from the first prototype -
  1. The plugin works on the Individual displayed in the Property Box. If the box is closed or contains a different record type, it says why it cannot proceed, rather than just exiting silently.
  2. Both Individual and Family Facts are listed together in the Original drop-downs. I found this more intuitive than separating Individual and Family Facts, as it is closer to the way the FH Property Box works.
  3. Similar original Facts are listed in chronological order.
  4. All Facts from all Fact Sets are included in the New dropdowns. Previously, Project Facts were not included, and some others were missing due to a combination of a code error and inconsistent structuring of Fact Set files.
  5. All details of the original Fact are copied. Previously, some were omitted.
  6. Interconversion between Events and Attributes is permitted after a warning. Converting an Attribute to an Event, or another Attribute to Residence, can result in the Fact value being lost, but this is saved as part of the new Fact as a local Note.
  7. I have rethought how to handle witnesses. If the old Fact has witnesses, and that role does not exist in the new Fact, the Fact is not changed. Where all the witness roles exist in the new Fact, they are copied with all other Fact properties.
  8. Custom Facts imported from other applications can be processed in the same way as any other Fact.
It works as intended with everything I can test it on, but over to users with their greater variety of data to try to break it :).
Attachments
Change Specific Fact (0.3).fh_lua
(15.17 KiB) Downloaded 6 times
Mark Draper

User avatar
tatewise
Megastar
Posts: 22116
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Ability to Change Type of Fact

Post by tatewise » 25 Jul 2021 15:08

I have run some quick checks and found the following issues:
  1. If an Attribute with a Value and a local Note is changed to an Event (or Residence), the Value is moved to a 2nd local Note that is only visible on the All tab, e.g. %INDI.FACT.NOTE2[2]%. Should it be prefixed or suffixed to the existing Note?
  2. The Original facts are not always in alphabetical order. It seems that Facts <undefined> in Fact Types are listed first.
  3. The Original Event list shows Family 'Census (family)' events as just 'Census'. They are shown OK in the New Event list.
  4. The changed fact is moved after all the other facts when viewed on the All tab and would also appear last in Reports.
    It would be better if the position were preserved or at least the user warned about the change so they are prompted to restore its original position. Don't be fooled by the Facts tab that automatically sorts by Date and Normal Time Frame.
    e.g.
    ptrNew = fhCreateItem(strNew,ptrRecord) -- Create the New Fact
    fhMoveItemBefore(ptrNew,ptrOld) -- Move the New Fact before Old Fact
    doCopyChildrenItems(ptrOld,ptrNew)
    doDeleteItem(ptrOld)
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

avatar
Mark1834
Megastar
Posts: 857
Joined: 27 Oct 2017 19:33
Family Historian: V7
Location: South Cheshire, UK

Re: Ability to Change Type of Fact

Post by Mark1834 » 25 Jul 2021 19:40

Ok, thanks. Good that they are just cosmetic issues rather than affecting data integrity.
  1. Deliberate choice to add a second note, as we have no way of knowing what’s in the first note, and adding the attribute value may not be appropriate.
  2. Yes, it sorts on Fact Label, then date. It could be tweaked if necessary to use the description if label is blank.
  3. That’s how FH displays the data. I could synthesise my own description, but I think it’s only a potential issue for the family Census, where the same type of event can be Individual or Family. In practice, I don’t think it’s a problem.
  4. Interesting, I’ve never needed to reorder the All tab, as the Reports that I use sort automatically. I can see by experiment that not all do, though. Let’s see if it’s an issue for real users, as I suspect it might be a niche requirement.
Mark Draper

Post Reply