* Search and Replace Plugin

Writing and using plugins for Version 5 and above.
Post Reply
avatar
David Potter
Megastar
Posts: 839
Joined: 22 Jun 2016 15:54
Family Historian: V7
Location: United Kingdom

Search and Replace Plugin

Post by David Potter » 05 Apr 2021 20:11

@ Mike Tate
Mike I ran this Plugin this evening to change some rich text within various Source records.
Search = "The Mark of"
Replace = "X The mark of"

The result was that the linked Individual within the text box that the change occurred was set to an unknown state. IE, underlined with dashes. See attached image. I have seen this issue before and reported it to CP but was never able to pin point the 'potential' root cause behind it. And CP were not able to find the cause either. This evening every record updated by the Search and Replace plugin given those update criteria shown above has the issue where the Individual Link has been lost.

Could you please check the Plugin to see if there is something that might cause this and let me know, otherwise I'll need to report another occurrence to CP with no conclusive root cause.

Thank you

David
Attachments
Capture.jpg
Capture.jpg (199.2 KiB) Viewed 745 times

User avatar
tatewise
Megastar
Posts: 21305
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Search and Replace Plugin

Post by tatewise » 05 Apr 2021 21:06

I have run some quick tests replacing text just prior to a link, including links within a table cell, and all is OK.

If you use Edit > Undo Plugin Updates before closing FH do the links return to normal?

With the links upset can you open the GEDCOM file in a plain text editor like Window Notepad and search for that X The mark of text and the cell contents should look similar to:
| X The mark of <rec=2,"John Potter"> Father Fiddington, Ashchurch |

Presumably, something will not be quite like that to cause the upset link.

I have experimented with upsetting the GEDCOM to see what produces the dotted underlines.
It is when the digit after rec= is too large.
It should be the same as the <rec=2,"John Potter"> link a few cells to the left.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

avatar
David Potter
Megastar
Posts: 839
Joined: 22 Jun 2016 15:54
Family Historian: V7
Location: United Kingdom

Re: Search and Replace Plugin

Post by David Potter » 05 Apr 2021 21:26

Hi Mike
Sorry I have shut down for the evening and not that easy to restart due to remote garden office. I'll check out what you have suggested tomorrow evening due to work hours. Thank you for looking into this.

David

User avatar
tatewise
Megastar
Posts: 21305
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Search and Replace Plugin

Post by tatewise » 05 Apr 2021 21:41

That looks like it may be an AS generated Text From Source table.
Are the upset links always the 2nd of two links to the same record?
If so, I wonder if AS is getting the rec= number wrong?
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

avatar
David Potter
Megastar
Posts: 839
Joined: 22 Jun 2016 15:54
Family Historian: V7
Location: United Kingdom

Re: Search and Replace Plugin

Post by David Potter » 05 Apr 2021 21:50

Hi Mike yes I'm using AS7 for everything just now. And no I have seen upset links for different Source Types. EG Census records where only a single link would be present.

David

User avatar
tatewise
Megastar
Posts: 21305
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Search and Replace Plugin

Post by tatewise » 05 Apr 2021 22:05

I have tried various ways to 'upset' the links and only a rec= number that is too large does it.
Theoretically, the 1st link should use <rec=1,"..."> the 2nd uses <rec=2,"..."> and so on, unless the same record link occurs again, when the same <rec=1,"..."> should be used as the 1st occurrence. The numbers are nothing to do with Record Id.
The numbers refer to the GEDCOM lines 3 _LKID numbers after the Rich Text and the 2 _LINK_I Record Id links:
2 _LINK_I @I3@
3 _LKID 1
2 _LINK_I @I6@
3 _LKID 2
2 _LINK_I @I2@
3 _LKID 3
If in the above example any Rich Text link was <rec=5,"..."> then there is no matching 3 _LKID 5 so it gets dotted underlines.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

avatar
David Potter
Megastar
Posts: 839
Joined: 22 Jun 2016 15:54
Family Historian: V7
Location: United Kingdom

Re: Search and Replace Plugin

Post by David Potter » 06 Apr 2021 06:05

Hi Mike. Thanks for looking into this further. I can't do much until this evening. But I always check what AS7 has created when it arrives in FH7 and have never seen missing links immediately after creation. It occurs sometime after by some yet to be discovered task. I did ask CP could it be found some way by query and they said no. I'm thinking if it is the Recid being too large that could be found by query right? That might help to find the pattern of events causing this to happen.

Thanks

David

User avatar
NickWalker
Megastar
Posts: 1490
Joined: 02 Jan 2004 17:39
Family Historian: V7
Location: Lancashire, UK
Contact:

Re: Search and Replace Plugin

Post by NickWalker » 06 Apr 2021 10:31

Hi Mike

I'm fully willing to accept AS might be doing something wrong here because this is a relatively new feature, but I've mimicked David's birth source and it seems to work correctly. Things that AS does that might not match what you've seen in FH:

The IDs might not be sequential (e.g. there could be record numbers 1, 2 , 3 and 7) - this can certainly happen in FH when links are deleted.
The order of the IDs in _LKID numbers does not necessarily match the order they appear in the source text - this can be seen in FH when links are copied and pasted in the text.
Someone who is linked more than once gets two separate record ids - I accept that FH doesn't do that but I don't think this should matter?
The _LKID numbers may not necessarily be in ascending order.

Cheers

Nick
Nick Walker
Ancestral Sources Developer

http://www.ancestralsources.com

User avatar
tatewise
Megastar
Posts: 21305
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Search and Replace Plugin

Post by tatewise » 06 Apr 2021 10:55

Thank you Nick, that gives more insight into the various possible link parameters.

With that information, I can reproduce the fault. If the <rec= numbers are not sequential and the Search and Replace plugin is used as David described earlier, then the non-sequential _LINK_I and _LKID tags get resequenced so the <rec= numbers may no longer refer to the correct link or any link at all.
I'd found that simply reading, changing, and saving the Rich Text field did not preserve the links, so had to rebuild them, but did not realise they might not be ordered or sequential.

I had not realised the link numbers could be so haphazard, so will update the Search and Replace plugin. :oops:
These new features are driving me up the wall :roll:

Try the attached Search and Replace plugin Version 3.2.1 Date 06 Apr 2021
Attachments
Search and Replace.fh_lua
Version 3.2.1 Date 06 Apr 2021
(253.79 KiB) Downloaded 15 times
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

User avatar
NickWalker
Megastar
Posts: 1490
Joined: 02 Jan 2004 17:39
Family Historian: V7
Location: Lancashire, UK
Contact:

Re: Search and Replace Plugin

Post by NickWalker » 06 Apr 2021 11:53

Yes I found this LinkID stuff quite difficult to deal with when I first started to implement this feature. But this thread has encouraged me to go back and look at it. Although AS is technically correctly recording the data (just to reassure David and others!), in the next version it will not create a new ID if an individual is linked more than once and will ensure the numbers are recorded in sequential order with no gaps in the numbering.
Nick Walker
Ancestral Sources Developer

http://www.ancestralsources.com

avatar
David Potter
Megastar
Posts: 839
Joined: 22 Jun 2016 15:54
Family Historian: V7
Location: United Kingdom

Re: Search and Replace Plugin

Post by David Potter » 06 Apr 2021 16:22

Thank you both very much. It would appear you have found and fixed this issue. I'll try the new Plugin this evening and report back.

Well done indeed.

Cheers

David.

avatar
David Potter
Megastar
Posts: 839
Joined: 22 Jun 2016 15:54
Family Historian: V7
Location: United Kingdom

Re: Search and Replace Plugin

Post by David Potter » 06 Apr 2021 18:36

Hi Both
I have tested the new Plugin in my Test DB and the issue did not repeat. 85 records updated and no links lost.

Thank you very much indeed. I just need to work through the list of missing links in my 'live' database and correct and hopefully the problem will be history.

One thought though - I was seeing this problem occur with Census records long before Nick had built the new Link Feature. And of course Census records only have the one Individual Link. But that scenario I'm sure was not linked to the Search and Replace Plugin as it was very sporadic with only a few instances seen whereas this issue resulted in a mass of lost links. So there may still be something odd going on although far less aggressive.

Thank you once again.

David

User avatar
tatewise
Megastar
Posts: 21305
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Search and Replace Plugin

Post by tatewise » 06 Apr 2021 18:58

It is certainly something that plugin authors will have to bear in mind when manipulating Rich Text Links.
It is possible that some other process was similarly upsetting those Census links.

I find your comment interesting that "of course Census records only have the one Individual Link" as my Census Source record transcripts would have a link for every member of the household.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

avatar
David Potter
Megastar
Posts: 839
Joined: 22 Jun 2016 15:54
Family Historian: V7
Location: United Kingdom

Re: Search and Replace Plugin

Post by David Potter » 06 Apr 2021 19:04

Hi Mike sorry I should clarify. I meant one link per person given that the whole household is listed. Whereas If I understand the problem you have solved it is to do with a person being recorded twice in the Rich text.

Thanks once again.

User avatar
tatewise
Megastar
Posts: 21305
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Search and Replace Plugin

Post by tatewise » 06 Apr 2021 19:17

No, I thought it might be the twin links, but it is nothing to do with that and can arise with any combination of links or possibly even with a single link. As Nick said, if there were two links and the first one gets deleted, then the surviving one has <rec=2, which would have been upset by the 'Search and Replace' plugin because it is not numbered sequentially from 1.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

avatar
David Potter
Megastar
Posts: 839
Joined: 22 Jun 2016 15:54
Family Historian: V7
Location: United Kingdom

Re: Search and Replace Plugin

Post by David Potter » 06 Apr 2021 19:23

I see, okay Mike I'll keep an eye out for similar issues going forward. Is there a simple query to find these issues? At the moment I have to scan All sources text from source and view all links.

Cheers

User avatar
tatewise
Megastar
Posts: 21305
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Search and Replace Plugin

Post by tatewise » 06 Apr 2021 19:30

Bother! I had quickly composed a Plugin to do just that before we found the cause and I fixed the Search and Replace plugin, but I deleted it. :o Let me see if I can reconstruct it.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

avatar
David Potter
Megastar
Posts: 839
Joined: 22 Jun 2016 15:54
Family Historian: V7
Location: United Kingdom

Re: Search and Replace Plugin

Post by David Potter » 06 Apr 2021 19:38

You're a star Mike. I can then hopefully discover any other patterns causing this issue. By running the query or plugin after any updates.

User avatar
tatewise
Megastar
Posts: 21305
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Search and Replace Plugin

Post by tatewise » 06 Apr 2021 19:42

OneDrive Recycle Bin comes up trumps :D

Try the attached Find Broken Rich Text Links plugin Version 0.1 Date 06 Apr 2021.
Attachments
Find Broken Rich Text Links.fh_lua
Version 0.1 Date 06 Apr 2021
(2.48 KiB) Downloaded 15 times
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

avatar
David Potter
Megastar
Posts: 839
Joined: 22 Jun 2016 15:54
Family Historian: V7
Location: United Kingdom

Re: Search and Replace Plugin

Post by David Potter » 06 Apr 2021 19:57

Super I'll give it a whirl. Thanks Mike. Probably be tomorrow morning now as in need of an early night.

avatar
David Potter
Megastar
Posts: 839
Joined: 22 Jun 2016 15:54
Family Historian: V7
Location: United Kingdom

Re: Search and Replace Plugin

Post by David Potter » 07 Apr 2021 06:31

Hi Mike
The Find Broken Links Plugin find 83 records in my live DB. A quick look shows a few records with no apparent broken link. I'll check this out further later this evening and advise.

Thank you very much.

User avatar
tatewise
Megastar
Posts: 21305
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Search and Replace Plugin

Post by tatewise » 07 Apr 2021 09:10

Yes, that Plugin was written before I fully understood the haphazard nature of the link numbering, so I'm not surprised it has listed a few false positives, but it should catch all the broken links. If you can manage with those 83 without an update to the Plugin that would help me.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

avatar
David Potter
Megastar
Posts: 839
Joined: 22 Jun 2016 15:54
Family Historian: V7
Location: United Kingdom

Re: Search and Replace Plugin

Post by David Potter » 07 Apr 2021 11:22

Yes of course Mike. No Problem at all.

Thank you.

Post Reply