* organizing files on hard drive

Homeless Posts from the old forum system
Locked
User avatar
jmurphy
Megastar
Posts: 715
Joined: 05 Jun 2007 23:33
Family Historian: V6.2
Location: California, USA
Contact:

organizing files on hard drive

Post by jmurphy »

You all know what happens. You see something, like the free download for a sample issue of a magazine, or a record you want to save for later study. You save it to your hard drive. Somewhere.

A couple of months later, you see it again. And you save another copy. Somewhere else.

Pretty soon, you end up with duplicate files all over the place.

Can anyone recommend a good program to search for these duplicates and/or to reorganize files and folders?

And what are your techniques to keep yourself organized so you don't end up with all those duplicates in the first place?

For instance, I switched over to Nick Walker's system of having a folder for each census year when I installed the latest Gedcom Census. It seemed like overkill at first, but having used it for a while, I see the wisdom of it. Much better than trying to keep stuff organized by surname.

Thanks, Nick.

Jan

ID:2873
User avatar
Jane
Site Admin
Posts: 8508
Joined: 01 Nov 2002 15:00
Family Historian: V7
Location: Somerset, England
Contact:

organizing files on hard drive

Post by Jane »

What I do is have a downloads directory, this contains all my downloads split by type so

Utilities
Family History
Graphics

I don't have too many categories. I use DownloadThemAll add on for firefox to download and it asks my where I would like to put each downloaded file and then tells me if I already have one with the same name.

I will keep an eye out for a dup file checker.
avatar
KSS
Diamond
Posts: 76
Joined: 15 Nov 2007 20:45
Family Historian: V6
Location: Derbyshire

organizing files on hard drive

Post by KSS »

I can't reccomend anything but when I entered 'Find duplicate files' in Google I got 335,000 hits. There must be something useful in that lot!
avatar
KSS
Diamond
Posts: 76
Joined: 15 Nov 2007 20:45
Family Historian: V6
Location: Derbyshire

organizing files on hard drive

Post by KSS »

Why do I make spelling mistakes when using a keyboard? 'recommend'
avatar
rclrocco
Gold
Posts: 27
Joined: 14 Feb 2008 19:11
Family Historian: None

organizing files on hard drive

Post by rclrocco »

KSS,

They're called senier momunts. your spolling goes to pot after so muny years, a bit like your hard drive and its duplicate files :-)

My real problem is filing a record somewhere like a marriage certificate; a few years pass; I get inspired and come back to that family branch and think I must get that Marriage certificate; i spend a few pounds ordering it; it arrives and I think, hey this looks familiar. :-(

Senior moment or deja vu, take your pick[confused]
User avatar
jmurphy
Megastar
Posts: 715
Joined: 05 Jun 2007 23:33
Family Historian: V6.2
Location: California, USA
Contact:

organizing files on hard drive

Post by jmurphy »

KSS said:
Why do I make spelling mistakes when using a keyboard? 'recommend'
The Firefox browser now includes a spell checker so it will catch those typos for you.

Jan
User avatar
Jane
Site Admin
Posts: 8508
Joined: 01 Nov 2002 15:00
Family Historian: V7
Location: Somerset, England
Contact:

organizing files on hard drive

Post by Jane »

rclrocco said:
My real problem is filing a record somewhere like a marriage certificate; a few years pass;
That's why you should attach the certificate/source to your FH data, that way when you think I need that Cert there is a quick check to see if you already have it.
User avatar
gerrynuk
Megastar
Posts: 565
Joined: 25 Apr 2007 09:21
Family Historian: V6
Location: Welwyn Garden City
Contact:

organizing files on hard drive

Post by gerrynuk »

For what its worth, I add a flag to everyone for whom I have a Birth/Marriage/Death Certificate or a copy of a baptism/marriage/burial entry in a parish register. I have added a condition in the Boxes tab of the Diagram Options so that these flags display and it very quickly shows which certificates or register entries I have or still need.

Gerry
avatar
RichardDBlake
Newbie
Posts: 1
Joined: 23 Oct 2006 15:40
Family Historian: None

organizing files on hard drive

Post by RichardDBlake »

I use 'Duplicate File Finder'. You can find it at http://www.brooksyounce.com/. It will find identical files even with different names. And the best thing is...it's free.

Richard
User avatar
jmurphy
Megastar
Posts: 715
Joined: 05 Jun 2007 23:33
Family Historian: V6.2
Location: California, USA
Contact:

organizing files on hard drive

Post by jmurphy »

Jane said:
What I do is have a downloads directory, this contains all my downloads split by type so

Utilities
Family History
Graphics

I don't have too many categories. I use DownloadThemAll add on for firefox to download and it asks my where I would like to put each downloaded file and then tells me if I already have one with the same name.

I will keep an eye out for a dup file checker.
Just wanted to get back to this point -- part of my problem stems from the fact that one of the magazines I subscribe to has a downloadable file every issue of the weblinks contained inside the issue -- and the filename is always the same. They can have files called 'weblinks.pdf ' all in the proper subdirectory for each issue, and not have to change the links each time the new issue comes out.

But once I download them, I want each weblinks file to be named for the issue it belongs to, so I change the name. If I can't find where I put the folder for those files, I may start a new folder, and eventually I have little clumps of these files all over the place, with overlapping subsets of files.

The same thing happens with Excel spreadsheets of extracted search results from websites -- if I can find my previous worksheet, it's easy to open it up, insert a fresh worksheet, and add to it, but if not, I start a new file.

So my stuff is a mess. I not only need to find duplicate files which do not have the same name, but also find duplicate file names that refer to two completely different files.

Lately I've been using the Firefox extension Scrapbook to save screen captures, so I generally only scrape-off search results and put them into Excel if I have to manipulate the results (sorting, etc.) -- but there I have the same problem, saving the same search multiple times.

I'm also testing the Firefox extension zotero which also keeps a record of when and where you captured something, and generates citations for you. So pretty soon I will have captures BOTH in Scrapbook and in Zotero to deal with!

One of the projects I'd like to do is simply sit down, start over, and make a proper record of everything I already have. It's one of many reasons I wish more genealogy programs included a proper research log.

Thanks to Richard for the link -- I'll check it out!

Jan
User avatar
Jane
Site Admin
Posts: 8508
Joined: 01 Nov 2002 15:00
Family Historian: V7
Location: Somerset, England
Contact:

organizing files on hard drive

Post by Jane »

Jan, why not just copy them over and rename the files, just right click and change the name to include the issue number. You can just right click on the file, select Rename and change the file name, just make sure if when you click rename if the file extension shows that you leave it on the end.

I always put stuff in big buckets. You can then use search routines to find what you are looking for. I tend to have a very structured approach to disk as I have 5 320Gb hard disks in my main computer so I need to be structured.

I would suggest approaching this like any research project filing. Make a plan, build a new directory structure based on what you need to store and stick to it. Move all those scattered files to the new structure and think carefully before storing anything downloaded outside of that area.
User avatar
jmurphy
Megastar
Posts: 715
Joined: 05 Jun 2007 23:33
Family Historian: V6.2
Location: California, USA
Contact:

organizing files on hard drive

Post by jmurphy »

You're quite right, Jane.

That's what I've done with the Census images and it has worked out well. The only difficulty is making the time to sit down and do it properly -- and creating a sensible set of folders.

Jan
avatar
MCGaylard
Silver
Posts: 7
Joined: 27 Sep 2007 10:16
Family Historian: None

organizing files on hard drive

Post by MCGaylard »

CNET.COM is a good source of downloadable software

I use WinMerge as a utility for comparing project folders. It will highlight the differences between two folders and their subfolders. It will then allow you to copy from left to right or right to left to synchronise the folders without having to leave the utility.

When I want to compare the contents of two files that I have highlighted as different I use WinDiff. This will allow me to compare the contents of the file with the differences highlighted. It works on most file types including binary.

I also have a hex editor should I need one.

All of these tools were free from CNET and I swear by them, especially the WinMerge one. This has saved me days of work when comparing the FH generated web content and my own to detect changed pages.
User avatar
jmurphy
Megastar
Posts: 715
Joined: 05 Jun 2007 23:33
Family Historian: V6.2
Location: California, USA
Contact:

organizing files on hard drive

Post by jmurphy »

Thanks for the tip. I'll check them out.

I've found some good stuff on CNet and on ZDNet also.

Jan
User avatar
jmurphy
Megastar
Posts: 715
Joined: 05 Jun 2007 23:33
Family Historian: V6.2
Location: California, USA
Contact:

organizing files on hard drive

Post by jmurphy »

Tried to download WinMerge, but the download wouldn't start on CNet or ZDNet. I'll cruise by tucows next and see what I can find there.

Jan
User avatar
stephenjones
Gold
Posts: 21
Joined: 25 Aug 2006 11:07
Family Historian: V7
Location: Yorkshire

organizing files on hard drive

Post by stephenjones »

You could of course always try the website - http://winmerge.org/.

Just a thought...
User avatar
jmurphy
Megastar
Posts: 715
Joined: 05 Jun 2007 23:33
Family Historian: V6.2
Location: California, USA
Contact:

organizing files on hard drive

Post by jmurphy »

Next step: clearing enough space to install a dupe checker.

I need a portable app.

I suppose I could just toss my entire Genealogy folder onto my external hard drive.

That would give me the room to zip up some of the non-genealogy gunk.

Jan
avatar
RalfofAmber
Famous
Posts: 173
Joined: 25 Nov 2006 19:34
Family Historian: None

organizing files on hard drive

Post by RalfofAmber »

I use a piece of freeware called SyncBack SE from 2BrightSparks which allows me to synchronise a memory stick with a hard drive structure on my desktop. This allows me to work from the stick on a laptop and then recombine my work without needing to worry about copying file structures.
User avatar
jmurphy
Megastar
Posts: 715
Joined: 05 Jun 2007 23:33
Family Historian: V6.2
Location: California, USA
Contact:

organizing files on hard drive

Post by jmurphy »

I've now discovered that there is a portable app flavor of WinMerge, so perhaps I can install it on my USB stick and run it on the hard drive -- thus avoiding the logjam of not being able to install WinMerge on my drive because I don't have enough room because I need to run WinMerge. [grin]

In the meantime, I have been making a small amount of room by tossing out great handfuls of messages from RootsWeb mailing list digests.

It is nice to be able to read the new messages in digest form offline, but if I really need to find one of them again, I can search the archive on RootsWeb itself.

Jan
User avatar
jmurphy
Megastar
Posts: 715
Joined: 05 Jun 2007 23:33
Family Historian: V6.2
Location: California, USA
Contact:

organizing files on hard drive

Post by jmurphy »

As I said before, I use an extension on my web browser called Scrapbook which allows me to do easy screen captures. Thus I can save search results from sites without having to print out the page.

I've now reached the point where I have over 1000 pages, and have decided to make a change in the way I keep them organized.

Based on the example of how Nick keeps Census pages organized, and Jane's 'many files in one large bucket' approach, I decided to re-arrange all my Scrapbook pages so they would be organized by source. That way if I am online searching a particular record set, I can look through that source's
'folder' to see which searches I have already done. If multiple surnames appear in a saved page, I no longer need to save multiple copies in various surname folders.

Since I save the 'see all persons on page' report as well the page which shows a household census record, this should make it far easier to pull up the 'transcription' of a particular census page when I am working with Gedcom Census.

There is a search function built into Scrapbook, so if I need to run a global search on my own data for all records in which a particular surname or person appears, I can make the computer do that for me.

Scrapbook allows you to make 'sticky notes' so one of the things I plan to do as I go through and re-analyze my data this time is to note the maiden names of married women. As I understand it, the search should also include the text in those notes.

Scrapbook also allows you to create your own note files, so it seems that it could be pressed into service as a journaling device -- if you are searching the 1891 census, for example, you could open up your note page and make a note of the date, what your search was, and what you found.

A feature I really like with Scrapbook is 'open source URL' which sends you back to the page you captured as a live page.

One of my Scrapbook folders is called 'where I left off' -- if I have a set of tabs open, I can save all the tabs to that folder and return to it later.

There is a similar extension called 'zotero' which I have also installed but haven't used much. If it turns out that zotero is more useful, I could (theoretically) transfer my work from Scrapbook to zotero by re-visiting each page/search with Scrapbook's 'open source URL' function and then re-capturing with zotero. We'll see.

At any rate, I finally figured out how the 'multiple Scrapbooks' work, so I can make a copy on my USB stick, put all the non-Genealogy captures into their own Scrapbooks, etc.

I've also started to weed out the 'sound-alike' records. When you have data which belongs to your file, you can use 'text from source' and notes to put the 'transcript' into Family Historian. But what about all the people who turn out to be different people than the ones you were looking for?

As I re-visit these pages, I am transferring those 'former relatives' page captures to a 'sound-alikes' folder and making brief notes to remind myself why I now think those records do not belong to the people in my file. I can still get at them, but they don't clutter up my main data files.

(The other exception to the 'everything by source' rule are searches I have made for other researchers -- if I am corresponding by email with someone and want to pull up her search results quickly, the number of saved pages is small enough that everything fits nicely in a small folder of its own.)

It seems to be working so far -- so we'll see what happens as time goes on.

Jan
avatar
RalfofAmber
Famous
Posts: 173
Joined: 25 Nov 2006 19:34
Family Historian: None

organizing files on hard drive

Post by RalfofAmber »

On the subject of winmerge, I am using it to work out how exactly making a project in FH4 shuffles my media around from where I have it in FH3. I discover it has basically moved a few leaf nodes around I had in parallel and placed them all in a media node (which I also had for general media). Winmerge is great, I am less keen on the reorganisation of my files as it means the media now forever would live in the one family tree, I can't think through how to share it from a subfolder (which I could of windows was more like Unix!)
User avatar
Jane
Site Admin
Posts: 8508
Joined: 01 Nov 2002 15:00
Family Historian: V7
Location: Somerset, England
Contact:

organizing files on hard drive

Post by Jane »

Tony you did not have to move the media in if you did not want to. Just attach it remotely.
avatar
RalfofAmber
Famous
Posts: 173
Joined: 25 Nov 2006 19:34
Family Historian: None

organizing files on hard drive

Post by RalfofAmber »

I may yet do that - I just feel that I want the relative addressing and everything else to work when I sync my structure to a USB drive and work from a laptop.

What I want is relative addressing to a master root (or a /media concept in UNIX).

Will keep playing until I get the effect I want!
Locked