* Trying to import Census Reference info for use in FTAnalyzer

[David Potter]

Hi Mike

The 1911 Census Reference accepted by Lost Cousins is as per the attached image.

There are some challenges when extracting from the #ORA notes files for both 1841 and 1911 in that the data we need to extract is spread around within the #ORA note.

I have attached images of those two files showing where and how that data is made available. A further problem with the 1911 Census is there is no RG14 mentioned in the #ORA data so that will need to be 'forced' into the Plug-In routine.

Thank you.

[tatewise]

The 1841 details are absolutely fine.

The 1911 details have got me confused.
Your ORA screenshot today is dramatically different from the one you posted Sat 13th Jan 2024 08:56.
Today's version not only has no RG14, but also has no Source. prefixes and no Schedule Number.
I can easily synthesise RG14 but the plugin cannot magic a Schedule number out of thin air!

My guess is that today's ORA example is faulty in some way.

If the genuine example is the Sat 13th Jan 2024 08:56 version, then the details are absolutely fine.

[David Potter]

Let me check a few more 1911 to check the pattern. This Census is for an Institution so that may explain why.
Back soon.

[tatewise]

Ah! That might be the explanation.
In the earlier posting the Source. values are after the Household Members list which is very short.
In the latest posting the Household Members list will be very long and you have not scrolled down far enough to show the Source. values, which should include the Source.Class RG14, etc, etc.

[David Potter]

I have checked quite a few now and honestly the information is not consistently placed or is missing. Attached are two examples, the first being the screen shot I sent before but definitely scrolled to the bottom of what is available, there is no Schedule number. Note: a lot of the #ORA data capture omits the Class reference RG14, most have both Schedule and ED, institution, or vessel, some one or the other.

Let me test with both Schedule and ED, institution, or vessel in Lost Cousins, if it accepts it we could pass both or whichever one is available.

Phew...

[David Potter]

JELV is correct, for the 1911 Census Lost Cousins only requires Piece and Schedule Number. What it does when loading the GEDCOM is strip out unnecessary information. However, I still need a useful reference within FH. So I would say we provide the following:

RG14, Piece: 32044, ED, Institution or Vessel: 08, Schedule Number: 94 (if both ED... and Schedule exist)[/b

Or just RG14, Piece: 32044, Schedule Number: 94 from #ORA (if it exists without ED, Institution or Vessel),
Or just RG14, Piece: 32044, ED, Institution or Vessel: 08 from #ORA (if it exists without Schedule Number)

Is that possible?

Sorry to mess you about...

[tatewise]

It would be a lot easier if there were no colon (:) and comma (,) punctuation in the Reference format:
e.g.
RG14 Piece 32044 ED, Institution or Vessel 08 Schedule Number 94
RG14 Piece 32044 ED, Institution or Vessel
RG14 Piece 32044 Schedule Number 94

For consistency, the 1841 and 1881 Reference format would also omit that punctuation.
e.g.
HO107 Piece 1234 Book 99 Folio 12 Page 4
RG11 Piece 1224 Folio 64 Page 12

Are you, FTAnalyzer, and Lost Cousins happy with that?

[David Potter]

Let me test that out Mike. Thank you.

[tatewise]

I've actually found a fairly simple way of producing the original format you preferred with colons & commas.
Sorry if I have put you to extra work.

[David Potter]

Hi Mike I made the test without the colons and commas and all looks good in both FTA and Lost Cousins. Strangely FTA re formats the Census Reference and replaces the Colons and Comma's. So good either way it would seem.

I'll go for what ever is easiest for you. My slight preference would be with the Colons and Comma's

Thanks again.

[tatewise]

OK, try the attached Copy Census Source Reference to Publication Info plugin Version 0.4 Date 15 Jan 2024.

It should produce your preferred formats with colons & commas for 1841, 1881 & 1911.

If that works satisfactorily it should be easy to extend it for the other years from 1851 to 1921.

It may report the following errors in the Result Set:
No local Notes found! if the Source record has no local Notes.
No #ORA local Note found! if none of the Source record local Notes contain '#ORA'.
Existing Publication Info value! if the Publication Info field is not blank.
If the Result Set has a blank Publication Info entry then the #ORA note did not match the plugin criteria.

[David Potter]

Thank you Mike.
I'll put this through testing and report back, probably tomorrow.

[David Potter]

Hi Mike, just a quick thank you, 1841 is looking very good in FH, but I need to investigate some anomalies in FTA before loading to Lost Cousins. I'll carry of tomorrow now...

Thank you very much indeed.

[brianlummis]

I hope that I am not going to put a spanner in the works but I seem to recall that there are some "Gotchas" when using FTA to upload to Lost Cousins.

Lost Cousins relies upon a true transcript of the Census to be uploaded in order that a match can be made, but by using GEDCOM the Name loaded may not be what is on the Census. For example my family name is Lummis but is often recorded in the Census as Lummas, Lummes, Lumas, Lomas etc. By using FTA they will all be loaded as Lummis and, unless another cousin has also used FTA, a match is unlikely to be found. The other common problem is that Lost Cousins upload calculates the age based on the birth date in the GEDCOM. The age given on the Census often differs from the "true" age thus making another possible mismatch.

So whilst using FTA might save a lot of time entering the data, it only becomes useful if it is checked against the original Census entry to confirm that all is well. I suppose that it might be possible to devise a Plugin to check to see if there were differences between GEDCOM and the Text from Source to identify any mismatches but that may be a step too far.

I am quite happy to be corrected if my recollections are incorrect.

Brian

[Ruth]

Lost Cousins has a checking facility, whereby you can double check all the references that have been entered/uploaded, including age. These can easily be corrected if required. There are instructions with every census, which you need to follow. There is also the additional option to make corrections if needed regarding misspelt names. and correct birth dates.

If you have not already joined Lost Cousins, I suggest you do, and read all the information there. It is free to join.

[David Potter]

Hi Mike, I can confirm all looks good in FH for the three census you have coded. As expected 1911 needs a few tweaks due to missing ORA data to around 5 records. I can handle that manually.

Many thanks for your nice work. I need a few days to get this through the FTA > Lost Cousins route but as far as FH is concerned looks very nice indeed. Once I done this task I'll focus on the remaining census years we talked about.

[David Potter]

Thank you Brian for your input, yes I'm aware of the FTA behaviour but to be fair it has helped me more than hindered.

As Ruth says there are correction opportunities at every stage of the transfer.

[David Potter]

Hi Mike
All three census years 1841, 1881 and 1911 now exist in the Lost Cousins Website. Out of 598 individuals only 27 were rejected due to 'Cannot Determine Age'. I need to find out how to correct that issue.

I'll come back in a few days to discuss those remaining Census years.

Thanks once again. Great success.

[David Potter]

Hi Mike, sorry for the delay...
Here are the remaining Census formats, if you would be so kind as to add these to Plug-In when you have a moment. Thank you very much.

1851
HO107, Piece: 1952, Folio: 336, Page: 47

1861
RG9, Piece: 534, Folio: 63, Page: 6

1871
RG10, Piece: 1004, Folio: 68, Page: 40

1891
RG12, Piece: 2049, Folio: 161, Page: 12

1901
RG13, Piece: 826, Folio: 23, Page: 6

1921
RG15, Piece: 4488, Schedule Number: 11

[tatewise]

Please post the ORA format for 1851 which I suspect differs from 1841.
I assume the ORA format for 1861 to 1901 is the same as for 1881 unless you tell me differently.
I assume the ORA format for 1921 is the same as 1911 unless you tell me differently.

[David Potter]

Hi Mike, sincere apologies for the delay in replying , been a bit poorly lately. Here is the additional information you requested. I have included the 1939 Register so although not a Census if you can do anything with this method of ORA coding that would be very welcome.

I have provided all the Census years not yet coded just for clarity and completeness.

Thank you - please see attached.

[David Potter]

Continued...

[David Potter]

Continued...

[tatewise]

Thank you for those ORA listings which mostly seem OK. However, I have some comments.

1. 1861 ORA lists Source.Class Rg 9 whereas I was expecting Source.Class RG9 in upper case with no space.
   Does the plugin need to cope with such Class abnormalities, not just in 1861 ORA, but every other year too?
   Do they need to be adjusted so the Publication Info always uses upper case RG with no space?
   #
2. 1921 ORA has no Source... items at all so which items must the plugin use to extract Class, Piece, Schedule Number, and maybe ED, Institution or Vessel?
   What variants are possible, similar to the 1911 ORA that had various combinations of those items?
   What format is required in the Publication Info?
   #
3. 1939 ORA ~ what Source... items must the plugin extract?
   What format is required in the Publication Info?

[David Potter]

Hi Mike, thank you for this.

Re the 1861 Census, I would suggest only correcting this one anomaly so rg 9 becomes RG9 so caps and no space, unless you intend to make wider use of the final version Plug-In for a wider user base; then others might want it selectable, I don't know, but I prefer the Caps format; so consistent with other Census years. It is evident this ORA capture of the Source info is quite crudely performed. I have checked a sample of 1861 census records I have, and they switch between upper case and lower case, no consistency that I can see.

For 1921, the notable difference is that these records are for now only coming from Findmypast instead of Ancestry my preference. So Class becomes Archive series. Again looking at what I have in my collection the format would be:

Archive series: RG15, Piece: 26361, Schedule number: 267, Schedule type: W, Page 12 (IF AVAILABLE). Only institution's appear with a Page number.

For 1939 this is going to need formatting as follows:
Source Reference: RG101/5088B, Line Number: 31, Schedule Number: 38, Sub Schedule Number: 1, Enumeration District: ODBF
Note: No period between Source and Reference

Those are the terms consistently applied. Example attached.

Sorry this has turned out to be such a challenge and thank you very much for your patience and skills.