* Problem with Lookup Missing Census Facts

For users to report plugin bugs and request plugin enhancements; and for authors to test new/new versions of plugins, and to discuss plugin development (in the Programming Technicalities sub-forum). If you want advice on choosing or using a plugin, please ask in General Usage or an appropriate sub-forum.
User avatar
ColeValleyGirl
Megastar
Posts: 5465
Joined: 28 Dec 2005 22:02
Family Historian: V7
Location: Cirencester, Gloucestershire
Contact:

Problem with Lookup Missing Census Facts

Post by ColeValleyGirl »

Is this expected behaviour when dealing with a polygamist, or is it a bug? Or am I making a mistake in the settings?

I have a person in my tree (David John 1833-?) who emigrated with his first wife Mary (1831-?) to Utah in 1861 and subsequently acquired a second wife Jane (1844-?). All of them were born in the UK.

David has census entries in Wales for 1841-1861; in 1861 his wife Mary is with him, so she has a census entry for that year as well.

David and Mary have census entries in the USA (1870, 1880, 1900) and Jane (who was in the same household) has a census entry in 1870. I've yet to track Jane down in subsequent years, but he remained married to both women until his death.

I run Lookup Missing Census Facts and select options as follows:
Screenshot 2018-02-15 15.22.15.png
Screenshot 2018-02-15 15.22.15.png (38.67 KiB) Viewed 12180 times
Jane is shown as having missing census entries in the UK (as part of a spouse family with David John) from 1871 onward, in spite of being recorded in Utah in 1870.

David is shown as having missing census entries in the UK (as part of a spouse family with Jane) from 1871 on, in spite of having census entries in the US over that period.

Mary is shown as having missing census entries in the UK (not as part of a spouse family) from 1871 onwards, in spite of having census entries in the US over that period.
User avatar
tatewise
Megastar
Posts: 28341
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Problem with Lookup Missing Census Facts

Post by tatewise »

The Plugin is checking U.K. Census Records, so despite having Census Events recorded in USA for 1870, 1880, 1900, etc, they could theoretically have visited the UK in 1871, 1881, etc.

The converse will apply if you alter the Plugin to check U.S.A. Census Records.

See plugins:help:lookup_missing_census:frequently_asked_questions#can_a_person_be_recorded_as_abroad_on_a_census_date|> Can a person be recorded as abroad on a Census date?
So either ignore the listing, or add any Fact for each explicit Census Date in both UK and USA.
They can be either Individual or Family facts.
If you don't want them to appear in Reports, then use a unique (perhaps custom) Fact, and Exclude it via Report Options.

The Plugin assumes monogamy, so 'wives' will only be paired with one 'husband' at a time, based on Marriage, Divorce, Birth & Death events, etc.

The only other possibility is for the Plugin to be modified to detect Facts with a Date Period and Place that excludes them from a country and thus its Census events for that period.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
User avatar
ColeValleyGirl
Megastar
Posts: 5465
Joined: 28 Dec 2005 22:02
Family Historian: V7
Location: Cirencester, Gloucestershire
Contact:

Re: Problem with Lookup Missing Census Facts

Post by ColeValleyGirl »

As you know, I have a strong aversion to inventing data just to make software work :cry: -- even if it wasn't going to be a huge amount of work.

I had hoped "Tick Use their Facts to decide if each person was at home or abroad on missing Census dates if you want the Plugin to recognise any Fact on a Census Date for people that were abroad" would be more flexible in is application, especially as the theoretical case you suggest is totally unrealistic. However, I see the difficulty in modifying the plugin -- even if somebody was in every US census from 1870 onwards -- because the census dates don't align. And you couldn't easily determine a 'margin' to check, and having the user set it as a global option (rather than per person) would be equally error prone.
The Plugin assumes monogamy, so 'wives' will only be paired with one 'husband' at a time, based on Marriage, Divorce, Birth & Death events, etc.
Understood -- looks like I will have to abandon use of the plugin for this projects, where they're all polygamists who emigrated from England and Wales to Utah.
User avatar
tatewise
Megastar
Posts: 28341
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Problem with Lookup Missing Census Facts

Post by tatewise »

I knew about your aversion to inventing data, which is why I suggested how the facts could be excluded, but I fully understand.

I wonder if the Plugin could use Emigration Events "to decide if each person was at home or abroad on missing Census dates"?
i.e. After an Emigration to another Place, all Census Dates after that Emigration Date are not checked, providing the Use their Facts to decide if each person was at home or abroad on missing Census dates is ticked.
Would that work with your data?


The association of 'wives' with spouse families is important, as it affects their Surname in Census searches, and may add the husband as an additional filter.

I could look into the Plugin to see if polygamy can be catered for.
It should be OK for a husband with many wives, as each wife will just adopt his Surname.
But what about a wife with many husbands? Which Surname should she adopt?
Also which 'partners' should be included in the search filters for other family members?
Should the husband search include all his wives and all her husbands?
Should the wife search include all her husbands and all his wives?
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
User avatar
Jane
Site Admin
Posts: 8508
Joined: 01 Nov 2002 15:00
Family Historian: V7
Location: Somerset, England
Contact:

Re: Problem with Lookup Missing Census Facts

Post by Jane »

One other option occurred to me, what about looking at the fact immediately prior to the Census date if that's in the US then look in the US other wise look in the country for the last fact?

One problem would be someone in the 1870 census in the US could conceivably also be in the 1871 UK census if they came home between the two.
Jane
My Family History : My Photography "Knowledge is knowing that a tomato is a fruit. Wisdom is not putting it in a fruit salad."
User avatar
tatewise
Megastar
Posts: 28341
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Problem with Lookup Missing Census Facts

Post by tatewise »

Yes, that is the problem, and the strategy does not work in general for every combination of Census countries handled by the Plugin. It also depends on how far before the Census Date the immediately prior Fact Date happens to be, and whether that allows enough time to travel home. There are further complications depending on the format of that prior Date. What if it is just a year, or approximate, or a range?
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
User avatar
ColeValleyGirl
Megastar
Posts: 5465
Joined: 28 Dec 2005 22:02
Family Historian: V7
Location: Cirencester, Gloucestershire
Contact:

Re: Problem with Lookup Missing Census Facts

Post by ColeValleyGirl »

I wonder if the Plugin could use Emigration Events "to decide if each person was at home or abroad on missing Census dates"?
That would work for me -- when I don't have a definite Emigration date, I could at least justify entering an estimated range. Jane's suggestion would also work and not require any additional data entry -- but it does have the drawback she suggests.
I could look into the Plugin to see if polygamy can be catered for.
As most forms of polygamy involve one man many wives (odd, isn't it?) you should be able to assume the wife's surname is adopted from the man's. The other assumptions are harder -- I have examples where all the wives and all the children are enumerated in the same household in some years, and in separate households in others... I'd be inclined to keep it simple and search for the husband and each wife as a separate couple with their associated children rather than assuming the households are combined.

Others may want to weigh in on the discussion -- thys is the first time I've been researching a large tree of polygamists so they may have experience and knowledge that I lack.
User avatar
tatewise
Megastar
Posts: 28341
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Problem with Lookup Missing Census Facts

Post by tatewise »

Try the attached Lookup Missing Census Facts Plugin Version 4.2.1 Date 17 Feb 2018.
[ Attachment deleted as superseded by Plugin Store version. ]

If the Use their Facts to decide if each person was at home or abroad on the missing Census dates is ticked, then the Plugin now detects all standard Emigration & Immigration events, and both their To/From Place fields, and uses them to make its home or abroad decision. That is in addition to the earlier feature using facts with same date as the Census.
e.g.
If the person Emigrates from England to USA in 1885 then only a subset of Census Record are considered.
For U.K. Census Records only 1790 to 1881 will be considered.
For U.S.A. Census Records only 1890 to 1940 will be considered.
For other countries, all years will be excluded.

All Date formats are allowed in the Emigration & Immigration events, but currently qualifiers such as Approximate are ignored, and the first Date in a Date Range is taken at face value, and the implications of Before or After are ignored.

Multiple Emigration & Immigration events are handled, so the person can be considered at home in multiple countries.

Only specific place names are recognised for determining the home country.
i.e.
1841 - 1901 England, Wales, Scotland, UK, U.K.
1911 - 1939 England, Wales
1790 - 1940 America, USA, U.S.A.
1749 - 1911 Ireland, Eire
1825 - 1945 Canada
1828 - 1921 Australia
plus a few others...

BTW: I have done nothing about polygamy yet, but it is on my to do list.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
User avatar
ColeValleyGirl
Megastar
Posts: 5465
Joined: 28 Dec 2005 22:02
Family Historian: V7
Location: Cirencester, Gloucestershire
Contact:

Re: Problem with Lookup Missing Census Facts

Post by ColeValleyGirl »

Hmm -- still not working as I expect. I'm testing it with a single individual who appears in the 1840, 1850 and 1860 censuses in Wales, then has an emigration event in 1863 (to USA from England) and then has entries in the US census for 1870, 1880 and 1900. He dies in 1908.

I'm running the plugin to look for missing US census (although I know there are none). The result set includes 1840, 1850 and 1860 censuses in the US.

If I run it for UK census records, it correctly tells me there are no missing census entries in the UK.
User avatar
LornaCraig
Megastar
Posts: 3190
Joined: 11 Jan 2005 17:36
Family Historian: V7
Location: Oxfordshire, UK

Re: Problem with Lookup Missing Census Facts

Post by LornaCraig »

I have done nothing about polygamy yet, but it is on my to do list.
I hope you don't mean you are going to try polygamy yourself! :lol:
Lorna
User avatar
tatewise
Megastar
Posts: 28341
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Problem with Lookup Missing Census Facts

Post by tatewise »

What exactly are your Emigration fact details?
Date, Place & To fields?
Does the To field match America, or USA, or U.S.A. ?

Chance would be a fine thing ;) You haven't met my wife :D
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
User avatar
ColeValleyGirl
Megastar
Posts: 5465
Joined: 28 Dec 2005 22:02
Family Historian: V7
Location: Cirencester, Gloucestershire
Contact:

Re: Problem with Lookup Missing Census Facts

Post by ColeValleyGirl »

Immigration
Date: 1863
Place: USA
Place (from): Wales
User avatar
johnmorrisoniom
Megastar
Posts: 901
Joined: 18 Dec 2008 07:40
Family Historian: V7
Location: Isle of Man

Re: Problem with Lookup Missing Census Facts

Post by johnmorrisoniom »

I have just tested and get the same sort of results
I already have UK census entries for 81,91 and 01, person emigrated from Liverpool to New York in 1909
UK census search only shows one son missing 1911 and 1939 (Although he was born USA in 1910)

US census search shows missing in 1880 and 1900 (As well as those after 1910, the only entry I have so far found)
User avatar
tatewise
Megastar
Posts: 28341
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Problem with Lookup Missing Census Facts

Post by tatewise »

The son born USA in 1910 does not exclude him from appearing in UK 1911 & 1939 records.

I have been considering some sort of timeline based on all facts to determine whether home or abroad, but is very complex.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
User avatar
tatewise
Megastar
Posts: 28341
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Problem with Lookup Missing Census Facts

Post by tatewise »

As you were, I think I have found the problem. :oops: Silly coding bug!

Try the attached Lookup Missing Census Facts Plugin Version 4.2.2 Date 18 Feb 2018.
[ Attachment deleted as superseded by Plugin Store version. ]
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
User avatar
BillH
Megastar
Posts: 2245
Joined: 31 May 2010 03:40
Family Historian: V7
Location: Washington State, USA

Re: Problem with Lookup Missing Census Facts

Post by BillH »

Mike,

Three quarters of my family come from either Norway or Germany. Could the plugin add those countries as ones that are checked for this new processing?

Thanks
User avatar
tatewise
Megastar
Posts: 28341
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Problem with Lookup Missing Census Facts

Post by tatewise »

Bill,
Are those Norwegian and German census records online somewhere?
The strategy of the Plugin is to offer browser online searches for missing Census records at specific websites.

However, even if it just said the missing census is not available online for the census date, then that would be helpful.
The same could be done for all the existing countries.
There might need to be an option to exclude any census where no searchable online records exist.

Some of the Norwegian censuses were taken over two days, so the Plugin would also need adjusting to cater for Census Events with either Date.

We would need to agree which census dates to include, as many of them seem to be incomplete.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
User avatar
ColeValleyGirl
Megastar
Posts: 5465
Joined: 28 Dec 2005 22:02
Family Historian: V7
Location: Cirencester, Gloucestershire
Contact:

Re: Problem with Lookup Missing Census Facts

Post by ColeValleyGirl »

Mike that new version works as I expect -- thanks.
User avatar
BillH
Megastar
Posts: 2245
Joined: 31 May 2010 03:40
Family Historian: V7
Location: Washington State, USA

Re: Problem with Lookup Missing Census Facts

Post by BillH »

Mike,

I should have been more specific in my request. What I was seeing is that if someone emigrated from Norway to the USA in say 1873, the plugin was still showing them as being missing from the USA census in 1850, 1860, and 1870 when they were not here yet. In those cases where I have either an emigration or immigration even, I was hoping to have these excluded from the report. The same is happening for my folks from Germany.

The Norway censuses are available on the Norwegian Digital Archives site, https://digitalarkivet.no/en/search/persons/advanced. The advanced person search allows you pick which census to search or to search all censuses. There is a better search for each census individually at https://digitalarkivet.no/en/content/censuses. I wasn't really thinking of having the plugin changed to do these searches, but if it is easy enough to do you could take a look at it. The only national searches were for 1801, 1865, 1875, 1891, 1900, and 1910 that are available so far. Some of them were over several days, but isn't that like the USA where some enumerators took weeks to finish.

I do not know of any place where German censuses are online or even if there are any censuses at all.

Thanks
Last edited by BillH on 19 Feb 2018 18:04, edited 2 times in total.
User avatar
BillH
Megastar
Posts: 2245
Joined: 31 May 2010 03:40
Family Historian: V7
Location: Washington State, USA

Re: Problem with Lookup Missing Census Facts

Post by BillH »

Mike,

I should say that searching in Norway censuses is really tricky. They rarely have the entire name as we know it. For example. My gg grandfather is typically known as Niels Olsen Vatndal. Olsen is the patronymic and Vatndal is the farm name. This is a common way to reference folks in Norway and that is how I have them entered into FH. When searching in the archives, use of this style name rarely works very well if at all. I usually have to search for something like Niels Ols* or even N*ls Ols* to find him. Often the ending of the patronymic is either ...son or ...sson instead of ...sen.

In any case, I guess what I'm saying is the plugin really doesn't need to be changed to search the archives, but it would be nice if the emigration/immigration date could exclude censuses for the country they emigrated to if the census date is before they emigrated. Does that make sense?

Thanks
User avatar
tatewise
Megastar
Posts: 28341
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Problem with Lookup Missing Census Facts

Post by tatewise »

Yes, the Plugin Version 4.2.2 Date 18 Feb 2018 posted on Sun Feb 18, 2018 10:26 pm should do that.

See the rules posted on Sun Feb 18, 2018 4:04 pm with Version 4.2.1 Date 17 Feb 2018 that had a bug.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
User avatar
BillH
Megastar
Posts: 2245
Joined: 31 May 2010 03:40
Family Historian: V7
Location: Washington State, USA

Re: Problem with Lookup Missing Census Facts

Post by BillH »

Mike,

I was confused in two ways.

First I was confused by your comment in an earlier post in this thread that indicated that "Only specific place names are recognized for determining the home country" and Norway and Germany are not in that list.

Then I was confused because the two people I checked using the latest version of the plugin actually did not have immigration events for the USA. The last immigration event I had was for them coming into Canada. I thought I had the one for the USA, but I've not found a record to show that leg of their journey so I don't have an event for it (I don't create the event until I have some type of source for it even though logically I know it had to have happened).

When I checked a few folks where I did have the event for the immigration into the USA, it did work fine.

Sorry about the confusion. And thanks for adding this functionality to the plugin.
User avatar
BillH
Megastar
Posts: 2245
Joined: 31 May 2010 03:40
Family Historian: V7
Location: Washington State, USA

Re: Problem with Lookup Missing Census Facts

Post by BillH »

Mike,

Spoke too soon.

I do have a gentleman that I just checked that immigrated into the USA on 26 Jun 1873. Jon Gabrielsen Hauge. The immigration "To" field has: New York, New York County, New York, USA. There is nothing in the "From" place field only the "To" place field. I don't use both place fields. I only use the "From" date on emigration facts and the "To" date on immigration facts. The other info is in the note field. Even though he didn't immigrate until 1873 he is shown as missing the 1820, 1830, 1840, 1850, and 1860 USA censuses.
User avatar
tatewise
Megastar
Posts: 28341
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Problem with Lookup Missing Census Facts

Post by tatewise »

Bill, I presume it also lists the 1870, 1880, 1890, etc, USA Census dates ?

You are using Plugin Version 4.2.2 ( not Version 4.2.1 ) and double-checked the ticked options ?

Could you post a screen-shot of his Facts tab.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
User avatar
BillH
Megastar
Posts: 2245
Joined: 31 May 2010 03:40
Family Historian: V7
Location: Washington State, USA

Re: Problem with Lookup Missing Census Facts

Post by BillH »

Mike,

I was confused again. :oops:

I am really having a "senior moment" day. I got confused by two similar names, Jon Gabrielsen Hauge and Jen Gabrielsen Hauge. For Jon Gabrielsen Hauge, it is correctly showing just the 1890 census which he doesn't have. He has censuses for 1880, 1900, and 1910 so they aren't showing and he died in 1917. The ones before he immigrated are also not showing. Looks like the plugin is working fine. Maybe I need to go in for some new glasses. :)

I have what is a related question. I have folks that were born in Norway and who died in Norway and never emigrated to the USA, but they show up on the report as missing all the USA censuses up until they died. Could the plugin handle the situation where someone was born in another country and never emigrated to another country and exclude these from the report? I can think of a couple potential problems. One would be where the name of the country changed but it really is the same country. Another would be where they did emigrate but there just isn't a emigration or immigration fact to document it. Not sure if this situation really could be handled or not. I thought you might be able to determine if it could be done or not.

Thanks
Post Reply