* Import gedcom with photos from online url

[Beerbadger]

In the successful run this is what I did :
Using Firefox I logged into Ancestry, created a new tree, added one person, then ran the Download plugin.
I then did this (but I missed the download Gedcom step to get the new person into an FH project)
After that I went to my main working tree in Ancestry, navigated to a person related to my first person in the test tree and used the Ancestry functionality to 'Save to Tree' and saved him as the father to my original person in my test tree. I then upload a random photo from my hard drive to this new person in Ancestry.
I download the Gedcom (with just the one person), create a new Project in FH by uploading the Gedcom and then run the Download plugin in FH and get the no URL error message.

So thanks a lot for asking me to list the exact steps, it made me realise I had missed one. It worked with the added person and photo.

Now I am wondereing if I have missed that download step with my main tree that I am being unsuccessful with.

I already have the tree in FH and have added local media to it that I don't want in Ancestry. So I am trying to run the download plugin on an FH Project that has been downloaded via Gedcom from Ancestry a couple of weeks ago and had local media added to it.
Does the plugin need a fresh Gedcom ? Don't know why it should but I will try that now.
Download my full tree from Ancestry to a new FH project and try the download plugin.
Cheers
Daryl

[Beerbadger]

Hi Mike,
Yes I know that my loal files are not known to the plugin, I just meant the files stored in the FH Media directory when I said local.

I have re-done my main tree download from Ancestry and get the same failed majority, oddly a different quantity.

I am now wondering whether the download plugin is failing on media that I have copied from other people's trees; but it also fails on the census and BMD records from Ancestry repositories.

You are right, of course, the 7 files that were successful were on Ancestry, but had been uploaded by me, as a result they were in the FH Media directory before I ran the download plugin.
That seems to be the common denominator - the media I don't own doesn't download. This is odd since I can download it manually one at a time.
Is there something about a copied media URL that stops the automatic download.
Cheers
Daryl

[Beerbadger]

Hello Mike,
Its something to do with the URLs being different.
I used a photo from someone else's tree to attach to one of the people in my small test tree, that has previously been working and it failed with the new photo.
The URL that the download plugin is looking for (ie says it failed on) is in the attached screenshot. The report the download plugin failures produce doesn't have copyable text as it is a Read Only Query. Side question - is there some way of changing that to copy the text from it?
The same photo URL from Media Property box for the individual in FH:
http://trees.ancestry.com/rd?f=image&gu ... 1686&pid=5
the same photo from right click/Copied Image Location from the View Image (no top bar, no side bar) in Ancestry:
https://mediasvc.ancestry.co.uk/v2/imag ... 9534883721
Different media- a 1939 Register image from and Ancestry repository:
blob:null/7fab3f5a-f2a3-4180-8e78-3303cdd96c8e
The download plugin does not even attempt to download this type of image, so there is no failure notification. I did think that I had more images than the number of failures the Download plugin was reporting.
Cheers
Daryl

[tatewise]

The Plugin is downloading the Media you own and overwriting the copies you already had in the Project.
In File Explorer, check the Date modified for those successfully downloaded images and they will have today's date.

The Plugin currently assumes the Ancestry GEDCOM is always imported as a New Project... and expects Media Records.
Other methods of importing the GEDCOM require the use of File > Validate... to convert the LMO to Media Records.
This is a prototype Plugin exploring the feasibility of bulk downloads. I've tried before with no success but now partial success.

Presumably, you can only manually download the media you don't own when signed in to Ancestry.
If you copy or bookmark the URL of such a media image and sign out, then you cannot download it.

Whereas if you repeat that exercise for the media you do own then they will download when signed out.

That is the problem facing the Plugin. It needs your account authority to download media you don't own.
That is the brick-wall I've faced in the past and again now. How to give the Plugin that authority?
It may need the user to enter their Ancestry Username & Password but in the past even that would not work.

The necessary URL and authority to download 'private' Ancestry media is a bit of a mystery.

Interestingly, the URL you posted lets me display the image:
https: //mediasvc.ancestry.co.uk/v2/image/namespaces/1093/media/7879023e-...-645ace931639.jpg?client=trees
That is the standard URL template, but with an id between media/ and ?client that is not in the GEDCOM download.
So how do we get that id into the Plugin?

[Beerbadger]

quick response -
I was able to view the image I had taken a copy of (in clipboard) the URL for.
I logged out of Ancestry, I even closed Firefox and used Internet Explorer and I was able to see the image for

https://mediasvc.ancestry.co.uk/v2/imag ... 9534883721

[tatewise]

If I plug the id of 7879023e-d8eb-4e6d-8d65-645ace931639 into the URL guid= in the FH Media File box:
e.g.
http: //trees.ancestry.com/rd?f=image&guid=7879023e-d8eb-4e6d-8d65-645ace931639&tid=78194667&pid=520
Then the Plugin downloads it successfully:

Download.jpg (9.86 KiB) Viewed 4812 times

So how do we get the id of 7879023e-d8eb-4e6d-8d65-645ace931639 into the Ancestry GEDCOM?
I feel we are getting close. If only we could sort out that last step.

BTW: blob:null/7fab3f5a-f2a3-4180-8e78-3303cdd96c8e is not a http URL so the Plugin can't use it.
I tried plugging id 7fab3f5a-f2a3-4180-8e78-3303cdd96c8e into the guid= as above but it still failed.

[Beerbadger]

Yes that is the photo.
However, I've just tried copying the id from your post and it doesn't work.
I tried copying the id from my data directly and it doesn't work. So I'm not fully understanding what you are transferring to where.

In the GEDCOM file I find there is an extra URL before the one for the non owned photo

DATA
4 TEXT http://trees.ancestry.com/pt/AMTCitatio ... id=0&pid=5
1 OBJE
2 FILE http://trees.ancestry.com/rd?f=image&gu ... 1686&pid=5

Also why does it say 520 at the ned of the one you used and only 5 at the end of mine ?
So even if I get the plugin to work it will not download census/birth/marriage/death indexes etc images ?

[tatewise]

It has got pid=520 at the end because I used the URL you posted yesterday Wed 17th Mar 2021 10:46
I simply replaced the guid= value. However, I don't think the pid= value is significant.
BTW: The URL in my last post has a space after http: to prevent it from becoming a hyperlink in these postings such that the entire URL can be more easily seen. That space must be removed for the URL to work. In future, I'll use CODE as below.

I'm editing the File box in the Media Record that we have discussed before so it contains:

Code: Select all

http://trees.ancestry.com/rd?f=image&guid=7879023e-d8eb-4e6d-8d65-645ace931639&tid=78194667&pid=520

Darryl, are you familiar with GEDCOM structure and format? If not then I'll need to give a short tutorial.

The 4 TEXT line is the transcript subsidiary part of a Citation and entirely separate from the following lines.
The 1 OBJE 2 FILE lines are a Local Media Object that the Plugin ignores.
Please use the File > Validate... command to convert it to a Media Record.

That LMO problem is recurring so often that I've modified the Plugin to detect and report them.
Please download & install the Download Online Media plugin Version 0.7.3 Date 25 Mar 2021.

[Beerbadger]

Hi Mike,
I'm about overloaded with this topic. Being new to FH I am not sufficiently familiar with its functionality.

I am not sure what the significance of the Local Media Object is and the need to use File/Validate. When do I do that ? What does it do ? What should I expect to see ?

I am not yet familiar with the GEDCOM format. I ahve downloaded the technical spec and I have the plugin for Notepad++ that can present them with highlighted keywords. I want to get more familiar with it, so your mini explanation helps.

I have just discovered that if I go to a media record and press the elipsis at the end of the URL box I get offered a place to put a file on my hard drive. I believe that having saved it there it should show up in the Property Box bottom pane of the Media Record. It doesn't and sometimes says 'Bad jpg'

If the 2 FILE lines are not where the plugin is getting its URLs from then I don't know where it gets them from. This is a small test file and apart from the 4 TEXT lines, which I now see are not relevant the only other lines with URLs are the 2 FILE lines.
To clarify:
Is the download plugin only intended to obtain photographs ?
If so, how do I get the images of the records census/birth/marriage/death indexes etc into FH ? Is this what the File/Validate should do to convert them to LMOs ?
Sorry to have taken up so much of your time, but I hope it has helped you in your thinking about the Download plugin.

[tatewise]

Daryl, sorry for the delayed response but I shall try and address your questions.

If you need a short tutorial on GEDCOM format and structure then please start a new posting on that topic.

Hopefully, you understand the difference between a Local Note such as on each Fact, each Citation, etc, and a shared linked Note Record that can be attached to many of those simultaneously.
Each Local Note is unique, whereas one Note Record can be linked to many records, etc.
Similarly, a Local Media Object (LMO) is uniquely attached to one item, and one Media Record can be linked to many records.
If the same Media file needs to be linked to several items a separate LMO is needed for each one, whereas only one Media Record is needed and is linked to all those items.
Some products prefer LMO and others like FH prefer Media Records.
Usually, such products do not offer much support for the type they don't prefer.

When you use File > Project Window > New Project to import a GEDCOM file all LMO are silently converted to Media Records.
That is what my Download Online Media Plugin assumes is the normal process so it only handles Media Records.
If you use other methods such as File > Merge Compare/File the LMO are NOT converted, so you need to immediately use the File > Validate command with its default settings to perform that conversion. It also checks for any other GEDCOM abnormalities but rarely finds anything wrong.
Unfortunately, by any of the above processes, each LMO is converted to a separate Media Record.
So if the same Media file is attached by multiple LMO there will be multiple copies of identical Media Records and files.

FH does not understand the URL in the File link box as it only expects Windows file paths.
So FH thinks the URL is a malformed file path. Clicking the ellipsis is intended to let you choose the correct file path.
If you browse to any image file on your PC and select it, then the File link box will record its file path and the image will be shown in the bottom pane of the Media Record and wherever that Media item is used.
The objective of my Download Online Media Plugin is to use the URL to locate the Media image online, download it to the PC, and change the File link box from the URL to the file path of that downloaded file.

Tomorrow I hope to summarise the progress of my more detailed investigation of what downloads are feasible from Ancestry, FindMyPast, et al...

[tatewise]

Following further investigations and trials here is a progress report on my Download Online Media plugin.

(1) Public URL
The plugin works for any Media Record URL that is publicly accessible so needs no username/password authorisation.
This is the case for the OP by Colin (Valkrider).

(2) Ancestry Family Trees
The plugin needs a Media Link URL template that forms the basis for direct Media downloads.
The plugin obtains the 'guid' from each Media Record URL and inserts it into that template.
That works for publicly accessible media that the user previously uploaded to the Family Tree from PC files.
Media URL from other user's Family Trees or Ancestry's own archive of records usually do not work with that method.

I've created a 14-day free trial account in order to investigate Media from Ancestry's own archive of records.
However, the Family Tree downloaded GEDCOM contains absolutely no URL details for such BMD/Census records.
So there appears to be no way of automatically downloading such Media. Is this a free trial account 'feature'?

Does anyone with a subscription Ancestry account have a downloaded GEDCOM with BMD/Census Media URL?
If so, would you be prepared to share a small subset of such a GEDCOM with those URL for my experiments?

However, Ancestry Family Trees can migrate to FH with Media via FTM or RootsMagic so my Plugin is not essential.

(3) FindMyPast Family Trees
I have a subscription FindMyPast account which seems to be more fruitful.
There is enough information in the downloaded GEDCOM to compose URL for all forms of Media.
Media that the user previously uploaded to the Family Tree from PC files does automatically download as above.

The BMD/Census Media from FindMyPast's own archive of records need a different technique.
The user must sign in to FindMyPast in their default browser so the Plugin has authorised access.
The Plugin invokes the default browser with a URL that displays a file download dialogue popup.
The user must click Save, save the file to the Project Media folder, close the browser tab, and tell the Plugin to proceed.
Those four clicks must be repeated for every such Media image file to be downloaded. That is extremely tedious!
If there were a way to download the Media by invoking the browser in a modal window and automatically 'clicking' the Save button then it would be much more user friendly. Any suggestions would be welcome.

Some of those BMD/Census Media have no image so the URL just shows a transcript page that can't be downloaded.

A snag is that all the Media Records are linked directly to the Individual record Media tab instead of Source Citations.
However, most Source Citations have a REF tag with enough information to identify the Media Record.
So the Plugin could move most such links from the Individual Media tab to the Source Citations.

(4) MyHeritage Website
I do not have a subscription MyHeritage account which limits my experiments.
Media that the user previously uploaded to the website from PC files does automatically download as above.

However, MyHeritage Trees can migrate to FH with Media via their Family Tree Builder so my Plugin is not essential.

[Beerbadger]

Hi Mike, Thanks for your detailed response to my new user questions.