* FH7 Full backup very slow
FH7 Full backup very slow
Hi all,
I would be very interested to know if other FH7 users are experiencing slower backups. A 464 file full backup (FH7) just took 5 minutes 49 seconds. As a comparson I took a copy of the entire project folder using windows explorer (464 files 264MB) which took only 2 minutes 59 seconds. In both cases the files went to the same external drive.
I will need to check my network performance in case some other gremlin has crept in, but I find it hard to see why FH7 should take so much longer even given that it is compressing the files. I don't want to raise a bug report if it's just my system.
Simon
I would be very interested to know if other FH7 users are experiencing slower backups. A 464 file full backup (FH7) just took 5 minutes 49 seconds. As a comparson I took a copy of the entire project folder using windows explorer (464 files 264MB) which took only 2 minutes 59 seconds. In both cases the files went to the same external drive.
I will need to check my network performance in case some other gremlin has crept in, but I find it hard to see why FH7 should take so much longer even given that it is compressing the files. I don't want to raise a bug report if it's just my system.
Simon
- PeterR
- Megastar
- Posts: 1135
- Joined: 10 Jul 2006 16:55
- Family Historian: V7
- Location: Northumberland, UK
Re: FH7 Full backup very slow
2,084 files, 767 MB — Full backup (to local hard drive on 9-year-old laptop) in 3 min 12 sec.
Peter Richmond (researching Richmond, Bulman, Martin, Driscoll, Baxter, Hall, Dales, Tyrer)
- Jane
- Site Admin
- Posts: 8514
- Joined: 01 Nov 2002 15:00
- Family Historian: V7
- Location: Somerset, England
- Contact:
Re: FH7 Full backup very slow
Remember when using backup option a Zip file is being created. You could try backing up locally and then copy the zip file as appending to zip files is likely what is slowing it down.
Jane
My Family History : My Photography "Knowledge is knowing that a tomato is a fruit. Wisdom is not putting it in a fruit salad."
My Family History : My Photography "Knowledge is knowing that a tomato is a fruit. Wisdom is not putting it in a fruit salad."
Re: FH7 Full backup very slow
Hi Jane,
I shall give that a try at some point and see if compression is the issue.
After my first post I ran a LAN speed test and the 6 Mbps upload speed it quoted looks suspiciously slow. I think I will have to do a thorough network check to see if there are any bottlenecks or faulty settings. Drive, switch, router and windows to fight. I may be busy this weekend!
Thanks PeterR for your experience.
I will still be interested in other views/experiences.
I shall give that a try at some point and see if compression is the issue.
After my first post I ran a LAN speed test and the 6 Mbps upload speed it quoted looks suspiciously slow. I think I will have to do a thorough network check to see if there are any bottlenecks or faulty settings. Drive, switch, router and windows to fight. I may be busy this weekend!
Thanks PeterR for your experience.
I will still be interested in other views/experiences.
- tatewise
- Megastar
- Posts: 28413
- Joined: 25 May 2010 11:00
- Family Historian: V7
- Location: Torbay, Devon, UK
- Contact:
Re: FH7 Full backup very slow
I don't understand why Jane's explanation only affects FH and not File Explorer appending files to the ZIP file.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
- Mark1834
- Megastar
- Posts: 2511
- Joined: 27 Oct 2017 19:33
- Family Historian: V7
- Location: South Cheshire, UK
Re: FH7 Full backup very slow
I was assuming that Simon ran his benchmark without compressing. Even so, that's fairly pedestrian network performance, so presumably a wi-fi connection? If you can run it wired, it could be a factor of 10 quicker if it is gigabit all the way through. I've just tested with a similar size folder (total size and number of files), and it went across in less than 20 seconds.
Mark Draper
- tatewise
- Megastar
- Posts: 28413
- Joined: 25 May 2010 11:00
- Family Historian: V7
- Location: Torbay, Devon, UK
- Contact:
Re: FH7 Full backup very slow
Yes, perhaps Simon can confirm what he was comparing.
The FH File > Backup/Restore commands only ever produce standard Windows Compressed ZIP files.
So the comparison would be a Windows File Explorer Send to > Compressed (zipped) folder on the Project folder.
Just copying the raw folder & files does not have the compression processing overhead.
The FH File > Backup/Restore commands only ever produce standard Windows Compressed ZIP files.
So the comparison would be a Windows File Explorer Send to > Compressed (zipped) folder on the Project folder.
Just copying the raw folder & files does not have the compression processing overhead.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
-
- Superstar
- Posts: 251
- Joined: 05 Nov 2020 12:16
- Family Historian: V7
- Location: Hertfordshire, UK
Re: FH7 Full backup very slow
I can't say that I noticed any appreciable difference between FH6.2 and FH7 backup times, although I didn't actually time them. I've just done a full FH7 project backup to local disk at 47 seconds for 912MB/910 files. It's quite an old PC with two WD SATA drives that I copied across to.
Re: FH7 Full backup very slow
Right, some more information:
1. My network is wired and if working properly should be a 1 gigabit network. But is clearly not operating at that kind of speed. Something I will need to investigate.
2. My test copy was simply that with no compression.
3. I have just done what Mike suggested and sent the folder to a zipped folder and then copied that. The compression process took less than one minute for the file compression (using 7 zip) and then a further 1 minute 38 seconds to copy the compressed folder to another location on the same drive.
4. So:
a) Straight copy of raw project folder no compression 2 minutes 59 seconds.
b)Compression followed by copy of zip folder 2 mins 38 seconds.
c) FH full backup 5 minutes 49.
So whatever the poor working of my ethernet network FH7 full backup does seem - to me - to be introducing further time.
1. My network is wired and if working properly should be a 1 gigabit network. But is clearly not operating at that kind of speed. Something I will need to investigate.
2. My test copy was simply that with no compression.
3. I have just done what Mike suggested and sent the folder to a zipped folder and then copied that. The compression process took less than one minute for the file compression (using 7 zip) and then a further 1 minute 38 seconds to copy the compressed folder to another location on the same drive.
4. So:
a) Straight copy of raw project folder no compression 2 minutes 59 seconds.
b)Compression followed by copy of zip folder 2 mins 38 seconds.
c) FH full backup 5 minutes 49.
So whatever the poor working of my ethernet network FH7 full backup does seem - to me - to be introducing further time.
- Mark1834
- Megastar
- Posts: 2511
- Joined: 27 Oct 2017 19:33
- Family Historian: V7
- Location: South Cheshire, UK
Re: FH7 Full backup very slow
If you can set up two separate computers (which can be any OS) on your network, iPerf3 is an excellent tool for benchmarking the network speed. Just run it on each end, one in client mode, the other as server. The figures it gives you are well above real world file transfer speeds, but represent the fastest that the connection can achieve (e.g. ~ 950 Mbs wired gigabit, ~ 100 Mbs very close range 2.4 GHz wireless N, ~ 290 Mbs Ubiquity PtP wireless to outbuilding).
Mark Draper
Re: FH7 Full backup very slow
Thanks for the tips Mark. I reckon I can do that - I've got the boxes and cable. I'll have a go this weekend.
- tatewise
- Megastar
- Posts: 28413
- Joined: 25 May 2010 11:00
- Family Historian: V7
- Location: Torbay, Devon, UK
- Contact:
Re: FH7 Full backup very slow
You are still not comparing like for like.
FH uses the standard Windows compression not 7zip.
As I said, use Windows File Explorer right-click the Project folder and Send to > Compressed (zipped) folder
You then get a ZIP folder very similar to the FH Full Backup ZIP folder. (FH actually omits a few unnecessary files.)
FH uses the standard Windows compression not 7zip.
As I said, use Windows File Explorer right-click the Project folder and Send to > Compressed (zipped) folder
You then get a ZIP folder very similar to the FH Full Backup ZIP folder. (FH actually omits a few unnecessary files.)
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
- Mark1834
- Megastar
- Posts: 2511
- Joined: 27 Oct 2017 19:33
- Family Historian: V7
- Location: South Cheshire, UK
Re: FH7 Full backup very slow
I used a Linux box plugged into gigabit ethernet as my common server, then just went around every system I could (Windows box, ancient laptop, NUC, various generation of Raspberry Pi), wired, wireless next to access point, wireless other end of house, etc. It gave an excellent overview of how everything was performing.
Makes sense to sort the plumbing out before benchmarking FH routines.
Makes sense to sort the plumbing out before benchmarking FH routines.
Mark Draper
Re: FH7 Full backup very slow
I may have found the problem. Changed windows 10 network adapter speed/duplex setting from auto-negotiation to 100Mb full duplex. Full backup just took 1minute 11 which feels about right. Almost 5 minutes faster than it has been running.
I wonder if one of this year's windows updates played with or reset network settings.
Thanks all.
I wonder if one of this year's windows updates played with or reset network settings.
Thanks all.