Page 1 of 1

Messed up dates

Posted: 08 Jul 2018 20:56
by Kim Travis
Six months ago I started porting my data from several Peditree databases into a single FH database. Having thought I had nearly finished working through all the very complex steps, I spotted a problem. When I imported my Gedcom files into FH all that time ago some date formats have been misinterpretted, without any error or warning being generated. For example 1808/9 has been interpretted as Sep 1808; also Mar 1808/9 has been interpretted as 9 Mar 1808. However, 13 Mar 1808/9 is fine as it was interpretted as a data phrase and was correctly fixed by the useful Fix Date Fields plugin. There are many thousands of these problem dates, so I reckon I now have to turn back the clock 6 months and start from scratch....... But can someone explain what has happened? Presumably Peditree has specified dual dates in the Gedcom files using a non-standard format, so what is the correct format?
I also noticed that all date ranges in which one of the dates was aproximate, indicated by "ABT", have had this approximate indication removed on import into FH - unlike the dual dates problem, I can live with this one.

I wonder how many other sly and unwanted changes have ocurred, and whether I can ever get this database porting done without a great loss of data quality. I have checked the data at every step, but I have not checked all the data at every step. Right now I wish I hadn't started.

Kim

Re: Messed up dates

Posted: 08 Jul 2018 21:55
by LornaCraig
The explanation is that double dating should use two characters after the slash. So for example Mar 1808/9 should have been entered as Mar 1808/09. However this format is normally used only for dates prior to 1752 when the Gregorian calendar was introduced, so it is not clear why you would have a date in this form for 1808/09.

Here is the relevant paragraph from the FH help files:
With the Gregorian calendar only you can optionally specify a year modifier - a 2-digit value in the second year field, following the ‘/’ character. Where a date is shown in this format (e.g. 1639/40), the year after the slash indicates the year as we think of it now (1640 in this case), and the year before the slash is the year as it would have been recorded at the time (1639 in this case). This way of writing dates is called double dating.

The year modifier, if supplied, should always be 2 digits, and one greater than the 2 digits before the slash character, or 00 if the last 2 digits before the slash character are 99 (e.g. 15 April 1699/00). For English and American dates, year modifiers are only applicable to years prior to 1752, and even then are only used for dates between 1 January and 24 March.
Regarding the date ranges, as you have discovered these cannot start or end with an approximate date, although a single date can be qualified as approximate.

Re: Messed up dates

Posted: 09 Jul 2018 10:22
by tatewise
The fundamental problem is that Peditree (like many other genealogy products) is not correctly implementing the GEDCOM specified Date formats.

However, if your examples are representative, the problem is not too bad.
e.g. Sep 1808 is not dramatically different from 1808/09.
As Lorna says, what is more problematic is what is meant by the double date of 1808/09.

As you discover each such date just manually correct it.
If there are many similar incorrect dates then the Search and Replace Plugin can fix them all in one go.
The snag is knowing which Sep 1808 is genuinely correct and which is a corruption of 1808/9.

Re: Messed up dates

Posted: 09 Jul 2018 18:42
by Kim Travis
Thanks both. I suspected that Peditree was the main problem. But I do object to FH interpretting 1634/5 as May 1634 - I suppose that nowhere in the Gedcom standard is 1634/5 described as a valid format, let alone as representing May 1634. No error message or warning was created. So in my view FH is at fault by accepting these incorrect dates without throwing an error or warning.

You are quite right that my choice of example year was poor. And yes Mike, I have no way to distinguish correct and incorrect dates within FH. I'm considering some sort of semi-automatic solution using Notepad++ to view and search the original Gedcom files from Peditree and view and edit the Gedcom file underlying FH at the same time. But there is steep learning curve to be climbed in order to work out if this is feasible.

Re: Messed up dates

Posted: 09 Jul 2018 19:42
by tatewise
I sympathise with your view. You are correct that 1634/5 is not a valid GEDCOM date, and it is surprising that FH does not report any date warnings. Even worse, there is a Tools > Preferences > File Load/Save > Accept Dates in Numeric Format option, but regardless of its setting (even No) those Peditree dates are converted just the same.

I realise it won't help you, but please report the problem to Calico Pie using how_to:about#problem_reporting|> Problem Reporting.

That you have the original Petigree GEDCOM files does offer the possibility of an automated solution.
However, it is complicated by there being multiple such files merged into one FH Project.
So associating each Fact and its Date from Peditree to FH is non-trivial.
If there really are thousands of malformed dates then automation (perhaps with a Plugin) may be a solution.
For that to be feasible, it would help if all your Individual records had unique Names and Birth fact details.
Then a Plugin should be able to read through each Peditree GEDCOM file and match Individual records.
The Facts for each record should then match up fairly easily.
If you fail to find any other workable solution, I might be persuaded to write the Plugin.

Re: Messed up dates

Posted: 09 Jul 2018 20:20
by Kim Travis
Thanks Mike. I will report it to Calico Pie. I shall persist a while with creating a semiautomated solution before considering taking you up on you kind potential offer.

Re: Messed up dates

Posted: 18 May 2020 15:06
by RodC
Kim,

Did you get anywhere with this problem?
Perhaps a mod in FH v.7?

Best

Rod C

Re: Messed up dates

Posted: 24 May 2020 07:55
by Kim Travis
I think I reported it (long time ago now so can't be certain) - no idea if it has been fixed. But for my one-off import from Peditree I manually fixed the offending dates. Sorry this doesn't help you much,

Kim

Re: Messed up dates

Posted: 24 May 2020 11:02
by tatewise
Rod, what is your interest?

The crux of Kim's problem was that the "messed up dates" were not spotted until many imported GEDCOM had been merged into one master Project.
It is important to import and review any new GEDCOM in a Project of its own before merging.
However, this dual year date treated as month & year is unusual as the conversion is performed silently.

The problem cannot have been fixed, because FH V6.2.7 has not been updated since May 2018.