* Trying to import Census Reference info for use in FTAnalyzer

[David Potter]

Sorry Mike I had to pop out for a bit.

The desired format is: RG11, Piece: 2332, Folio: 85, Page: 24


I may have a solution but it requires a lookup into the Source > Note tab, select the note labelled #ORA which may not be the first Note and then make use of the last few lines that provide the source reference we need.

Here's an example
Looking at the attached image you will see a record in the Source > Notes tab.
The entry beginning #ORA contains all the source reference data.
The last few lines contain the Source Reference info we are looking for.
We can see the Class, Piece, Folio and Page in the correct sequence.
So we need to extract those four values and restructure them to meet the desired format; RG11, Piece: 2332, Folio: 85, Page: 24

If this is too much of a task just to help me. I'm quite happy to format the Templated Source > Reference field in order that we just then need a copy routine.
However, If this solution works for you, I will double check that all Sources have the #ORA Note. Which I believe they do.

Thank you so much for your support.

[tatewise]

I've assumed that both the Reference metafield and Publication Info field need to be updated to the format:
RG11, Piece: 2332, Folio: 85, Page: 24

Try the attached Copy Census Source Reference to Publication Info plugin Version 0.1 Date 11 Jan 2024.

If it can't find a matching ORA Note it copies the Reference metafield to the Publication Info field instead.

If the Publication Info field already had a value then it is not overwritten.

If you want any other plugin behaviour then let me know rather than run the plugin.

The Result Set lists the new Publication Info field values for you to review.
If any look wrong then use Edit > Undo Plugin Updates immediately before closing FH.
Let me know how it goes.

[David Potter]

Thank you so much Mike very much appreciated. I'll get this tested this morning. It seems you have covered everything.

[David Potter]

Hi Mike, initial testing has thrown up some unexplained issues.

Test prep
Copy FH Database
For All 1881 Census Source records delete the contents of the Templated Source Reference field, and delete using Query > Field Select Publication Information > Clear Contents

Double Check all #ORA Notes contain the info we wish to extract and reformat.

Run Plug for UK 1881
Viewing your result query we can see some success and some missing. The pattern behind the missing Publication info appears to be that the Templated Source > Reference field was not itself populated which then appears to prevent the Publication Info field from becoming populated.
Well that's how it looks at least.
See attached.

Thank you.

[David Potter]

One Request Mike, could the result query also include what is now populated within the Templated Source Reference field please. This makes it easier to check.

Thank you.

[tatewise]

The clue is in my plugin specification:
If it can't find a matching ORA Note it copies the Reference metafield to the Publication Info field instead.

However, you erased all the Reference metafields, so when the ORA Note does not match, the Publication Info field remains empty.
So the root cause is that the ORA Note does not match the plugin criteria which are:
RG11...digits...digits...digits
where ... can be any characters including newline and digits must be one or more digits 1 - 9.

I did not feel the need to include the Reference field in a column because the Publication Info field is always the same value.

What might be more useful is the ORA Note text in a column so it can be reviewed to see why it does not match.

[David Potter]

Okay thanks Mike,

What exactly does If it can't find a matching ORA Note it copies the Reference metafield to the Publication Info field instead., mean? What is it trying to match with? I was hoping it would refresh both Reference and Publication Info so they are the same after the update.

I have checked all the #ORA Notes and they all look okay! I'll double check now.

[David Potter]

Okay I think I have found the problem, the successes we see all have the #ORA Note at the top of the Note window. In some sources there are more than just the #ORA Note. So where we see a missing result it is because the #ORA Note is not at the Top. See Attached.

That's the pattern I see...

Thanks Mike

[tatewise]

The plugin is only looking in the 1st local Note instance.
It never occurred to me that there might be multiple local Notes.

Is it always the case that the ORA Note contains #ORA near the beginning?
If so then the plugin can use that to discover the appropriate local Note to search.

[David Potter]

Yes Mike I use a FH 7 Hashtag = #ORA to create the start of the Note and add the ORA extracted data to.

So all will begin with #ORA but may not be the first Note record.
I have tried to columnize the ORA data but it will not do that without further programming.

[tatewise]

Try the attached Copy Census Source Reference to Publication Info plugin Version 0.2 Date 12 Jan 2024.

Where are you trying to columnize the ORA data?

[David Potter]

ORA allows the option to extract in a simple horizontal way but it stops short of columnizing the field data.

Thank you Mike, I'll make a new test.

[David Potter]

Hi Mike, generally a better result, but for some reason no Templated Source > Reference field was populated and very oddly the last 6 Source records, the Publication Info is incorrect.

See attached

[David Potter]

Mike, could be pure coincidence but those last 6 Source have a much higher REC ID value.

[tatewise]

I don't think the Record Id have anything to do with it. You presumably added those much later than the others.

I can only assume that the text RG11 occurs earlier in the #ORA Note than the Source.Class line in those cases.
So I have made the plugin only search from the Source.Class text onwards.

Sorry, about not setting the Reference value. It was due to the Reference field being deleted completely and not just deleting its value, which leaves an empty field behind.

I've improved the way it includes the #ORA Note text in the Result Set.

Try the attached Copy Census Source Reference to Publication Info plugin Version 0.3 Date 12 Jan 2024.

[David Potter]

Thank you very much Mike. I'll retest this when I get home this evening.

[David Potter]

Hi Mike
Tested thoroughly and all working okay. Super work and thank you so much, this saved me a lot of time correcting data fields manually.

Can I please ask for one more adjustment: If the Plug-In is solely for my benefit can you please expand the list of selectable Census Years to be 1841 - 1921, this is to allow me to keep consistency in my database for all 'currently' available Census years.

Thank you - very much appreciate for you help in this matter.

[tatewise]

To cater for all those census years I need to know the formatting involved for each year.
i.e. The ORA format and the required Reference / Publication Info format.

I know many years will be the same (1861 to 1901) but 1841, 1951, 1911 & 1921 are different.

The formatting for 1841 and 1911 must be FTAnalyzer compatible.

[David Potter]

Good Morning Mike

The 1841 UK Census #ORA data looks like the attached image. So uses the HO107 designator rather than RG##

Followed by:
Folio
Page
Line

Then the 1911 is simplified to:
Class
Piece
Schedule Number

Where is Class is RG14

These are the only England and Wales Census records currently supported by Lost Cousins, I'll take a look at the remaining England and Wales Census data later today if that's okay. I have things to do this morning.

Thanks

[tatewise]

IMO you need to investigate a bit further.

I think 1841 also requires a Piece and Book number.
In addition, the plugin needs the FTAnalyzer reference format for Piece, Book, Folio, Page, Line, etc.

I think 1911 may need more than Piece and Schedule numbers.
Likewise, the plugin needs the FTAnalyzer reference format for Piece, Schedule, etc.

[David Potter]

I'll make some further testing to be sure FTA accepts the format for 1841 and 1911 Census. Or work out what it should be.

[jelv]

I think 1911 may need more than Piece and Schedule numbers.

LostCousins only uses piece and schedule number for the 1911 census.

[David Potter]

Thank you John. Any chance you could post a screen shot of the format used.

[Ruth]

Mike, there is no line number used in the reference for 1841 census. Lost cousins requires Piece, book, folio, page.
eg HO 107 Piece 749, Book 11, Folio 7, Page 9

[David Potter]

Hi Mike

I can confirm the following format for the 1841 Census works okay with Lost Cousins. See Attached
I'll take a look at the 1911 Census format needs tomorrow.

Have a good evening.