* Larger projects

Questions regarding use of any Version of Family Historian. Please ensure you have set your Version of Family Historian in your Profile
Post Reply
avatar
E Wilcock
Megastar
Posts: 1009
Joined: 11 Oct 2014 07:59
Family Historian: V6.1
Location: London
Contact:

Larger projects

Post by E Wilcock » 07 Jan 2020 13:51

I now have a research project the fh folder for which is 238 Megabytes.
It lists 5086 people
I tend to put anyone there who doesnt fit on my three more specific Place study files. So it is growing all the time.

I had assumned from users doing one name or place studies that large files were no problem in fh.
However, on Sunday, I had some difficulty moving this project to my laptop via drop box.

Why is it so big? Could one just move the gedcom? And, if so, where exactly is the gedcom? I assume in the data file.
And should I avoid adding to this project?

User avatar
Jane
Site Admin
Posts: 8016
Joined: 01 Nov 2002 15:00
Family Historian: V6.2
Location: Somerset, England
Contact:

Re: Larger projects

Post by Jane » 07 Jan 2020 16:09

As projects go that is still pretty small, it's worth remembering when using Dropbox it's generally doing updates via the internet connection, so you need to make sure it's finished sending up before you turn off your main machine.

My project is still fairly small, just under 2Gb for the folder and works just fine with Dropbox.

What is the exact error were you getting on Dropbox?
Jane
My Family History : My Photography "Knowledge is knowing that a tomato is a fruit. Wisdom is not putting it in a fruit salad."

User avatar
tatewise
Megastar
Posts: 18837
Joined: 25 May 2010 11:00
Family Historian: V6.2
Location: Torbay, Devon, UK
Contact:

Re: Larger projects

Post by tatewise » 07 Jan 2020 19:48

As Jane says, yours is quite a small Project. My quite modest main Project with 677 Individuals is about 1GB.

You ask "Why is it so big?"
To answer that needs a little understanding of the Project folder structure.
So let's say the Project name is SwissFamily and use Windows File Explorer to take a look.
( However, beforehand, create a File > Backup/Restore > Full Backup just in case. )

Inside the Family Historian Projects folder will be the SwissFamily Project folder.
That folder should only contain 3 items: Public folder, SwissFamily.fh_data folder, and SwissFamily.fh_proj file.

The Public folder holds whatever Diagrams, Reports, etc, you have saved from FH.
It may have an Export folder holding Export Gedcom File Plugin exported GEDCOM and Media files.
However, none of it is essential to the operation of FH or the Project, but might hold some large files.
So, its contents can mostly be deleted without much harm, and should be reviewed from time to time.
That is especially so for your research project, as its contents is in flux, and probably leaves redundant files behind.

The SwissFamily.fh_data folder is more critical to the operation of FH and the Project.
It will have about 6 folders and the SwissFamily.ged GEDCOM file that will be quite small.
Review those folders as follows and checkout their sizes:
  • Cache folder has a ThumbCache.dat file of image thumbnails that can be very large, and if so, needs deleting.
  • Charts folder holds your FH View > Saved Charts and in that command redundant ones can be deleted.
  • Media folder obviously holds your media files, but run the Check for Unlinked Media Plugin to delete unused files.
  • Plugin Data, Settings & Snapshots folders ~ leave well alone.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

User avatar
Valkrider
Megastar
Posts: 1285
Joined: 04 Jun 2012 19:03
Family Historian: V6.2
Location: Spain
Contact:

Re: Larger projects

Post by Valkrider » 07 Jan 2020 20:53

My personal project is 3.74gb and the largest of my Surname Studies is 2.02gb. I have a lot of images and personal photos in my personal project so to me your project is not very large at all. Dropbox and a slow internet connection are likely to be your problems. That is why I use a local NAS and backup to the cloud overnight along with an auto incremental backup every 2 hours.

avatar
E Wilcock
Megastar
Posts: 1009
Joined: 11 Oct 2014 07:59
Family Historian: V6.1
Location: London
Contact:

Re: Larger projects

Post by E Wilcock » 08 Jan 2020 09:32

Thank you all. I am out today (not taking laptop) but will look at all this. I am grateful for your help.
Yes, giving time is essential and at one point I didnt realise that.
Thank you all for reassuring me the size of the Project is fine.

I conclude that my problem lay in drop box. I use drop box only for transfering files. But kept getting notices to upgrade as it was so full.
The reason it was full was that each time I tried to delete there was a warning from Dropbox that the file (Project) would be deleted everywhere. On all my devices.
I do not sync to drop box, and after searchng for solutions on line, I calculated that the threat to delete from all devices, referred only to the drop box folder on each device. Not to the family historian project folder on my C:/ drive.
I backed up fh to a memory stick. disconnected the stick and risked deleting almost everything from the remote drop box site.

I am posting this because drop box has now become intertwined with Google and at the time it completely defeated me.
Although I took the Project to the archive with me, I didnt actually edit it so my fh data survived on my home computer.

avatar
mezentia
Famous
Posts: 161
Joined: 12 Jan 2007 21:14
Family Historian: V6.2
Location: Stourbridge

Re: Larger projects

Post by mezentia » 10 Jan 2020 23:02

Projects can get an awful lot bigger. I've been researching now for over 15 years, and as Mike tends to recommend, I have one very large project that covers biological families for me, my wife, and my son-in-law, along with adoptive families, step-families and their biological relations too. As a result, the project contains information on about 16500 people in 5800 families. My project file now has reached some 34.5Gb, of which the vast majority is made up of media files. I keep all my projects on a NAS box (Raid 1), that is backed up in its entirety to a separate external drive every night on a weekly cycle of a full backup followed by 6 daily incremental backups. I currently have at least three complete cycles backed up. In addition, I use AlwaySync to synchonise the NAS files to my laptop on a regular basis, and take a full copy to an SD card from time to time. In addition, I do a small backup every time I quit FH, and that folder is also backed up on a regular basis. Even so, as all the equipment tends to be in my study, I am concerned that it is still vulnerable to, say, fire or theft. Cloud storage is an option as I have quite a fast internet connection through Virgin Media, but I have not looked into the possibility yet in any detail. I use TP-Link Homeplug devices to connect my PC, CCTV and NAS to my router as this provides a more reliable and much faster internet connection, and given that it only takes a few minutes at the most to synchronise my laptop and PC, I would assume that a similar synchronistaion to cloud storage wouldn't take a dissimilar amout of time once the bulk of the files have been uploaded. The main point is, however, that I an fortunate to live where I can have a 200Mbps internet connection (and could easily double that speed if I was prepared to pay more) which obviously makes cloud storage a viable option.

User avatar
tatewise
Megastar
Posts: 18837
Joined: 25 May 2010 11:00
Family Historian: V6.2
Location: Torbay, Devon, UK
Contact:

Re: Larger projects

Post by tatewise » 11 Jan 2020 10:58

mezentia has identified a crucially important aspect of backups is to keep a copy on media remote from your home.
Otherwise, fire, theft, flood, power surge, lightening strike, etc, that affects your whole home will destroy everything.

BTW: Although you may have a 200Mbps Internet download speed, the upload speed is likely to be about a tenth of that.
So the initial upload to cloud storage may take longer than you realise. But once there, the incremental updates are quick.
Also there are size limits, e.g. OneDrive offers only 5GB free storage, and maximum 10GB file size.
So your 34.5GB Project Full Backup ZIP files probably exceed that 10GB file size limit.
Google Drive and Dropbox have different limits.
See Knowledge Base > Downloads and Links ~ Synchronise and Backup Tools.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

Post Reply