Family Historian User Group

Hi Mike,

I was just wondering if it would make sense to add an option to the plugin so that we could set an age threshold. If the person is under this age then a missing census would not be displayed in the output.

It seems like I spend a lot of time wading through suggestions for possible census records that would never exist because the person is so young. Until 1850 in the US, no census contained children's names, only the head of household. If a person doesn't want to use a threshold, then maybe they could code 0 or 999 or leave the threshold blank or something like that.

Bill

Yes, that could be added to the next version Bill.

But should it be only applied to nominated Census Countries/Years?

You say in USA it is only relevant before 1850, so if you set threshold to say 20 Years, then everyone under 20 would be omitted from the later Census Years too!

One possible solution is for the Plugin to tag each Census Year with an Age Threshold (which would mostly be 0 except for USA pre-1850).
Then the user interface could have an Age Threshold On/Off toggle option.

A variant is for the Plugin to tag each Census Year with an Age Threshold applicable flag (which would mostly be Off except for USA pre-1850).
Then the user interface could have adjustable Minimum Age value that is only applied to those Years with flag = On.

Mike,

I was thinking that even in later US censuses where children are shown, I would probably exclude young children from the plugin output anyway since the head of household would already be on the output. I don't really need a separate entry on the output for each child.

Either option would work fine. I would probably use the same age for all censuses so for me maybe just having one age field and an on/off flag for each census would be great. Whatever you think would work best for everyone is good for me.

Thanks!

Bill

I have implemented a revised version 3.5 of this Plugin available from my OneDrive at Lookup Missing Census Facts.
If there are any problems then the Plugin Store version 3.4 can be reloaded.

It adds the Minimum Age threshold option for certain Census Records because they only list adults:
UK 1821 Dartford, Kent
USA 1790-1840 & 1890 Civil War
Ireland 1749, 1766, 1831-1851
Canada 1825, 1842

It also adds some new Census Records:
UK 1801/1821 Dartford, Kent
USA 1890 Civil War
Ireland 1766
Canada 1825, 1842
Canada 1906, 1916 now restricted to Manitoba, Saskatchewan, and Alberta.

Any feedback will be welcome, but note that the Help & Advice has not been updated and still relevant to V3.4.

Mike,

Thanks for the quick update to the plugin. As far as I can tell it seems to be working fine for the given censuses. I have a couple of thoughts though.

As I mentioned earlier, it would really be nice to be able to exclude people from all censuses based on age if I could. Now, if I have a family with 10 children, all 10 of them end up showing on the report along with the mother and father. For me at least, this really isn't necessary. I would like to eliminate all the children from the report because if I find the census for the father or mother, I'll find it for everyone else as well. One line on the report is all that is really necessary. Could there be an option to allow the date limit to apply to all censuses?

As for the 1890 US census that you added. I think it is a bit confusing to call it the "USA 1890 Civil War". It really isn't a Civil War census, it is a census of survivors of the Civil War still living in 1890 and is more commonly referred to as the "1890 Census Survivors Schedule" or "1890 Census Veterans Schedule". It isn't a separate census from the 1890 Census, it is just an additional schedule for that census similar to some of the agricultural schedules on earlier censuses.

Thanks,
Bill

Bill requested:

It would really be nice to be able to exclude people from all censuses based on age if I could.

What I am not sure about is how widely that would be desirable.
Are there other Plugin options that might achieve your goal?
For example, if you really are just investigating one person, or one couple, then use the choose Anyone via the Select Records dialogue and select the one or two people you are researching.
Alternatively, review the options in the Lookup Web Page, as all columns can now be sorted, and may be sorting on Family Date will tend to bring adults to the top and relegate youngsters to the bottom.

Bill said:

As for the 1890 US census that you added. I think it is a bit confusing to call it the "USA 1890 Civil War".

I agree the USA 1890 Veterans Census needs more accurate description.

Mike,

No... I was reporting all descendants of a chosen person. The report had about 1000 entries, but there were many that were for young children that I really don't need to see since their parents are also listed on the report for the same census. As soon as I get the census entered for the parent I will automatically get it for the children as well.

I'm not sure how widely used the option would be, but since I am the only one that requested the age limit anyway, maybe it would be OK to add it?

I just noticed a possible problem with the age limit.
I used an age limit of 18 for this run of the report. Mary D Henshaw was born in 1852. Even given the rage of 1852+/-5 years, the earliest she could have been born would be 1847. Yet, she shows up on the report as missing her 1860 census.

Thanks,
Bill

That is because the Minimum Age only currently applies to pre-1850 Census years and 1890.

Mike,

Oops... just as I posted my message I figured out my error. Senior moment.

Bill

Don't worry, we all get them.

I have updated the Plugin with an extra option as you requested - available from same OneDrive link above.

Mike,

That was quick. Using the same criteria with the additional option selected, the report is much smaller. That will really help.

Thanks much!

Bill

Mike,

Another question on the minimum age threshold. I thought it applied to pre-1850 censuses as you mentioned above. I set a threshold for age 21, but Levi Henshaw is appearing on the report for 1820 when he would have been 5 and 1830 when he would have been 15.
Am I confused again? Shouldn't these two entries be excluded from the results base on the age limit? There are quite a few others like this as well.

Thanks,
Bill

That is odd, and I cannot repeat it, so perhaps there are some unusual circumstances.

You have ticked Minimum Age applies to every Census? so whether the Census is pre-1850 is irrelevant.
Therefore, the Minimum Age at Census dates of 21 should be applied universally.

The test threshold is based on the greatest possible Age allowing for the tolerance on the Birth Year.
So for Levi Henshaw in 1820 that would be 1820 - 1815 + 5 = 15 but that is still less than 21.

The Plugin does not discard web search pages it produced from its earlier runs.
So can you double-check the page you posted was produced after the Minimum Age settings were adjusted.

If you untick Use their Facts to decide if each person was at home or abroad... does it make any difference?

Mike,

Where are the old web search pages kept? The report I'm looking at is the one thrown up in the browser as soon as the plugin finishes running.

I ran the plugin today and it seems to have produced 4 files in the Plugin Data folder, but they are all dated today. I don't see any old ones.

Un-ticking the home and abroad option makes no difference.

Thanks

When I run the Plugin multiple times, each run creates a new tab in the Firefox browser.

There should only be one Lookup Missing Census Facts.dat file in the Project's Plugin Data folder.

In the C:\ProgramData\Calico Pie\Family Historian\Plugin Data\ folder there will be multiple files all with today's date.
Lookup Missing Census Facts.dat
lookupmissingcensusfacts.css
lookupmissingcensusfacts.js
lookupmissingcensusfacts-sdt8.7.html or similar with a variant per run associated with each browser tab.

OK... this is very strange.

I had all the files as you specify with only one .html file.

I closed FH and went back in and tried the plugin again. Now I am not getting these erroneous entries. I didn't change anything else. I guess I'll have to wait to see if it happens again sometime.

Another question on the date/age processing. If I have a specific birth date for an individual, it would be nice if the plugin didn't use the +/- 5 years date range as this is quite a large span. Would it be possible to use a range of 0 years if the entire birth date is known? Alternatively, would it be possible to add a parameter to the plugin so that we could specify the +/- amount to use?

Thanks!

The Plugin uses that tolerance as a minimum because I believe Ancestry and FMP both use that tolerance on the Birth Date when searching for matching Census entries. Also it caters for dates with estimated/calculated/approximate qualifiers that the plugin does not specifically detect.

Just because people are listed in the search web page does not mean you have to use them to invoke a search.
If their age is marginal just ignore those entries. The list is not definitive, only advisory.
Who knows what DoB people put in the Census, so just because you think it is not worth a search does not mean they don't exist in a Census. If they are near the Age threshold, they may still have a Census entry. That is why the Plugin uses the -5 offset to err on the side of caution.

I agree that I can just ignore them, but there do tend to be quite a few entries where subtracting -5 from their birth date causes them to appear in the list even though they would be younger than the specified threshold. It makes the report bigger and a bit harder to wade through.

It was just an idea.

Ancestry actually lets me specify how many years to allow using this popup:

As I said, just because the Birth Date you have for them suggests they are below the threshold does not mean they won't appear in the Census as people often misrepresent their age.

The tolerance adjustment is always at least -5 years, so you can simply set the threshold 5 years higher to counteract its effect.

I can live with the way it is and I see that there may be workarounds, but I do disagree a bit.

I was not referring to someone who misstates their age on a census or somewhere else which I realize happens frequently. I was referrning to the situation where I have the full birth date. This usually comes from a birth record or baptism record which is not often wrong and if it is it is usually only by a day or two. It was in those cases where I was hoping to be able to avoid the +/- 5 years. If I know the actual date then I don't need any +/- date.

Several points.

When searching for Census records, the Birth Year in the online data is derived from the Age entered in the Census.
So if someone says they are 20 in the 1850 Census then the derived Birth Year is 1830 in the online data.
But they may have misrepresented their Age and they were actually 22 so the actual Birth Date in your FH data is in 1828.
If the search is for a Birth Year based on your known Birth Date of 1828+/-0, then it won't match that Census where the Birth Year is 1830.
That is why there is a tolerance. It is to allow for people falsifying their age in the Census. Otherwise you won't find them.

Yes, Ancestry & BMD allow a smaller tolerance but I think they both default to +/-5 years.

The Plugin web page search links with the preset search criteria are just an aid.
As it says in the Plugin Help, you will often have to Edit the Search criteria to resolve a match.
That can be increasing or decreasing the DoB tolerance, adding or removing relatives, making names more or less variable, etc.
Whether to tighter or relax those criteria depends on how many matches are returned initially.

I guess I was confused about how it worked.

I didn't realize you were looking at the age specified in the actual census. I thought you only cared about their birth date and the date of the census. How does that even work on censuses in the USA prior to 1850. They didn't even have an age specified for the person being enumerated.

I was thinking that if a person was known to have been born on say 1 Mar 1820 and the census was for 1 Jun 1840, then if the threshold was age 21 then the 1840 census would not need to be shown as a possible one to need looking at. No need to use +/- 5 years.

OK, the Plugin calculates and employs the Date of Birth (DoB) and Date of Death (DoD) in a variety of ways and until now I have not discussed all the processes.

1) The DoB is derived in many ways but in general it is an Estimated DoB as summarised in the plugins:help:lookup_missing_census:frequently_asked_questions#how_is_the_actual_or_estimated_birth_year_determined|> How is the actual or estimated Birth Year determined? I believe the UK & USA did not have official Birth Certificates much before 1840 which probably explains why early USA Census records record no age. So I question whether it could be known for certain that somebody was born on say 1 Mar 1820.

2) The earliest DoB and the latest DoD are used to determine if the person could be alive at the Date of each Census. At this point if the DoB and DoD are derived from the Individual record's explicit Dates in primary Events such as Birth, Baptism, Christening, Death, Burial, Cremation, then the earliest, mid & latest Date will be the same, i.e. zero tolerance. But if Date Ranges or estimations from other Facts or relatives Facts are involved then the tolerance may be greater. Note that here we are talking about full Dates not just a year.

3) Thereafter, the Age filter compares the Census Year and Birth Year with a minimum tolerance of +/- 5 years. To offset that you can add 5 years to the Minimum Age at Census dates setting. But remember that just because you believe somebody was younger than 21 from your research does not guarantee that the person had the same belief at the time of the Census. They may have thought or wished to be considered that they were older than 21, and so were included in the Census. If you offset the tolerance you may never find such Census entries.

4) The Plugin web page search criteria always include the Birth Year and tolerance, which can be adjusted in the Edit box of both Ancestry and FMP. I accept that in some early Census records that do not record Age then those criteria are irrelevant. Remember it is the web site, under your direction, that is applying these criteria, not the Plugin. The Plugin does not look at the Census records at all, but simply provides a set of reasonable search criteria as a starting point. It is up to you to use fuzzy logic and intuition to adjust the criteria as necessary, which includes Sex, Birth Place, and relatives. For example the Plugin assumes young people will be recorded in a household with their parents, while married couples will be recorded with their spouses and not their parents. But none of those criteria may be correct because men in particular may be working away from home. The Plugin is simply offering some rational search criteria that are often useful in filtering the records, and avoid silly mistakes about dates, places, and names that we are prone to make when researching. The clever search tactics are your task.

Thanks for the explanations. It is a pretty complex process.

As far as birth dates, many of those prior to 1840 come from church baptism records. I have baptism records going back as early as the 1600's in some cases. I also get them from death records which of course are not always as accurate.

#3 is a good point and something I need to keep in mind.

Thanks