* Problem with code snippet

Writing and using plugins for Version 5 and above.
Post Reply
User avatar
ColeValleyGirl
Megastar
Posts: 1367
Joined: 28 Dec 2005 22:02
Family Historian: V6.2
Location: Cirencester, Gloucestershire
Contact:

Problem with code snippet

Post by ColeValleyGirl » 08 Nov 2019 11:29

This code snippet (taken from Knowledge Base > Zip File Create & Extract (code snippet)) causes a plugin to hang:

Code: Select all

function extractZip(zipFile,folder)
    --  Create com object to work with Files and Folders
    local shell = luacom.CreateObject("Shell.Application")
    local source = shell:NameSpace(zipFile)  -- Get the zip file
    local items = source:items()
    local dest = shell:NameSpace(folder)   -- Get the destination folder file
    dest:CopyHere(items)    --  Copy the Zip to the Folder
    repeat                  --  Wait for copy to Zip to complete
    fhSleep(1000,500)
    until items.count == dest:items().count
end
The zip file is extracted correctly (with the normal Windows progress but the plugin hangs tereafter.

The problem goes away if I remove this code:

Code: Select all

    repeat                  --  Wait for copy to Zip to complete
    fhSleep(1000,500)
    until items.count == dest:items().count

User avatar
tatewise
Megastar
Posts: 16613
Joined: 25 May 2010 11:00
Family Historian: V6.2
Location: Torbay, Devon, UK
Contact:

Re: Problem with code snippet

Post by tatewise » 08 Nov 2019 12:39

I think Jane wrote that snippet, and I've experimented with it.

It seems that items.count is the number of files within the ZIP file, and that dest:items().count is the number of files in the target folder. So if you are extracting into a folder that is not initially empty then the counts never match.

By counting number of files already in folder beforehand and including that in repeat until loop it works OK.

Code: Select all

function extractZip(zipFile,folder)
    --  Create com object to work with Files and Folders
    local shell = luacom.CreateObject("Shell.Application")
    local source = shell:NameSpace(zipFile)  -- Get the zip file
    local items = source:items()
    local dest = shell:NameSpace(folder)   -- Get the destination folder file
    local files = dest:items().count       -- Count of files already in folder
    dest:CopyHere(items)    --  Copy the Zip to the Folder
    repeat                  --  Wait for copy to Zip to complete
    fhSleep(1000,500)
    until files + items.count >= dest:items().count
end
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

User avatar
ColeValleyGirl
Megastar
Posts: 1367
Joined: 28 Dec 2005 22:02
Family Historian: V6.2
Location: Cirencester, Gloucestershire
Contact:

Re: Problem with code snippet

Post by ColeValleyGirl » 08 Nov 2019 12:49

That works, and makes sense. Thanks, Mike.

User avatar
tatewise
Megastar
Posts: 16613
Joined: 25 May 2010 11:00
Family Historian: V6.2
Location: Torbay, Devon, UK
Contact:

Re: Problem with code snippet

Post by tatewise » 08 Nov 2019 13:03

The only slight snag is when the files in the target folder are essentially the same as in the ZIP file.
i.e. You are extracting subsequent versions of similar ZIP files into the same folder and overwriting the earlier files.
Then the counts test will succeed almost immediately.
The only absolutely satisfactory solution is to always use an empty target folder.
So I am not sure how best to update the Code Snippet?
Perhaps it needs to advise that it only works for empty target folders, and discuss the workaround I posted here?
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

User avatar
ColeValleyGirl
Megastar
Posts: 1367
Joined: 28 Dec 2005 22:02
Family Historian: V6.2
Location: Cirencester, Gloucestershire
Contact:

Re: Problem with code snippet

Post by ColeValleyGirl » 08 Nov 2019 13:14

To be honest, the sleep statement is somewhat redundant... as far as I can tell, isn't invoked until after the extract completes -- unless you can see otherwise.

User avatar
tatewise
Megastar
Posts: 16613
Joined: 25 May 2010 11:00
Family Historian: V6.2
Location: Torbay, Devon, UK
Contact:

Re: Problem with code snippet

Post by tatewise » 08 Nov 2019 13:38

It is needed to check the extract actually worked, because there appears to be no error response from extraction process.
c.f. You Research Planner 2 which fails to extract without any error message.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

User avatar
ColeValleyGirl
Megastar
Posts: 1367
Joined: 28 Dec 2005 22:02
Family Historian: V6.2
Location: Cirencester, Gloucestershire
Contact:

Re: Problem with code snippet

Post by ColeValleyGirl » 08 Nov 2019 13:42

Best publish it with a warning then... However, the snippet doesn't report an error in any circumstances, so I'm not convinced it's useful.

User avatar
tatewise
Megastar
Posts: 16613
Joined: 25 May 2010 11:00
Family Historian: V6.2
Location: Torbay, Devon, UK
Contact:

Re: Problem with code snippet

Post by tatewise » 08 Nov 2019 13:57

You are correct, the snippet is flawed, because its repeat loops never terminate if there is an error.
So it needs 'improving' further to explain that, or include an error message escape from the loop, perhaps when the dest:items().count stops increasing?
Maybe Jane can give us her thoughts?
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

User avatar
tatewise
Megastar
Posts: 16613
Joined: 25 May 2010 11:00
Family Historian: V6.2
Location: Torbay, Devon, UK
Contact:

Re: Problem with code snippet

Post by tatewise » 08 Nov 2019 15:03

This alteration to the repeat until loops seems to work.
If the dest:items().count reaches the items.count then it returns true.
Every 1 second it checks if count has stuck, and if so then returns false.
Does this look OK to you?

Code: Select all

require "luacom"

function extractZip(zipFile,folder)
    --  Create com object to work with Files and Folders
    local shell = luacom.CreateObject("Shell.Application")
    local source = shell:NameSpace(zipFile)  -- Get the zip file
    local items = source:items()
    local dest = shell:NameSpace(folder)   -- Get the destination folder file
    dest:CopyHere(items)    --  Copy the Zip to the Folder
    repeat                  --  Wait for copy to Zip to complete
        local count = dest:items().count
        if count == items.count then return true end
        fhSleep(1000,500)
    until count == dest:items().count
    return false
end

function buildZip(zipFile,folder)
    -- Create Empty Zip File
    local strZipHeader = 'PK'..string.char(5,6)..string.rep(string.char(0), 18)
    zip = assert(io.open(zipFile, 'w'))
    zip:write(strZipHeader)
    zip:close()
    --  Create com object to work with Files and Folders
    local shell = luacom.CreateObject("Shell.Application")
    local source = shell:NameSpace(folder)  -- Get the source folder
    local items = source:items()
    local dest = shell:NameSpace(zipFile)   -- Get the destination Zip file
    dest:CopyHere(items)    --  Copy the Source folder to the Zip file 
    repeat                  --  Wait for copy to Zip to complete
        local count = dest:items().count
        if count == items.count then return true end
        fhSleep(1000,500)
    until count == dest:items().count
    return false
end

if not extractZip(strzipfile,strfolder) then fhMessageBox("Extract Zip File Failed.") end

if not buildZip(strzipfile,strfolder) then fhMessageBox("Create Zip File Failed.") end

Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

User avatar
ColeValleyGirl
Megastar
Posts: 1367
Joined: 28 Dec 2005 22:02
Family Historian: V6.2
Location: Cirencester, Gloucestershire
Contact:

Re: Problem with code snippet

Post by ColeValleyGirl » 11 Nov 2019 10:43

Mike, that last code sniippet unerringly returns 'false' when extracting the Research Planner Help files...

User avatar
tatewise
Megastar
Posts: 16613
Joined: 25 May 2010 11:00
Family Historian: V6.2
Location: Torbay, Devon, UK
Contact:

Re: Problem with code snippet

Post by tatewise » 11 Nov 2019 11:07

That is odd, as it works fine for me if I substitute it into Research Planner V2.0 in place of its extractZip(...).

[ EDIT: However, on closer inspection, my proposed code only works reliably if the target folder is empty (as specified in the code snippet advice), and in the Research Planner that is not the case as that folder is Research Planner that already contains the .cfg and .fhf files, so your variant involving the files count is necessary. However, all it is actually checking is that the Research Planner Help 2.0 sub-folder gets created and not the actual contents.
I wonder if there is a way to check that sub-folder contents by using shell:NameSpace(subfolder) where subfolder is the ...\Research Planner\Research Planner Help 2.0 sub-folder? ]
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

User avatar
ColeValleyGirl
Megastar
Posts: 1367
Joined: 28 Dec 2005 22:02
Family Historian: V6.2
Location: Cirencester, Gloucestershire
Contact:

Re: Problem with code snippet

Post by ColeValleyGirl » 11 Nov 2019 12:04

This is weird -- it fails consistently for me if I do that.

The extraction completes before the loop is entered -- is that not the case for you?

If the directory is initially empty except for the zip file itself, dest:items().count is calculated after the extraction completes *including* the Help directory and the zip file , therefore 2 ; which is (a) always greater than items.count (1) and (b) never changes during the course of the loop, so appears to be stuck. So the extraction succeeds but claims it didn't.

User avatar
ColeValleyGirl
Megastar
Posts: 1367
Joined: 28 Dec 2005 22:02
Family Historian: V6.2
Location: Cirencester, Gloucestershire
Contact:

Re: Problem with code snippet

Post by ColeValleyGirl » 11 Nov 2019 12:05

We came to the same conclusion at the same time...

User avatar
tatewise
Megastar
Posts: 16613
Joined: 25 May 2010 11:00
Family Historian: V6.2
Location: Torbay, Devon, UK
Contact:

Re: Problem with code snippet

Post by tatewise » 11 Nov 2019 17:15

I believe I have a solution.
Part of the problem was that the code snippet did not take account of existing items in the target folder.
But that is further confounded by the Research Planner Plugin creating the Research Planner Help 2.0 subfolder before extracting the ZIP file. Therefore, extracting the ZIP adds no items to the target folder Research Planner.dat and so there is nothing to count in that folder to check the extraction has worked.

BTW: The earlier version of function extractZip(...), and the original code snippet, did not correctly detect or return a true/false indication of ZIP file extraction.

I have modified the function extractZip(...) code snippet to account for existing items.
I have deleted the lfs.mkdir(HelpFileDirectory) from local function GetHelpFile().
Now extractZip(...) correctly returns true or false depending on whether the extraction adds any item(s), which in this case is just the Research Planner Help 2.0 subfolder.
I will see if the function buildZip(...) needs a similar treatment.

Code: Select all

function extractZip(zipFile,folder)
    --  Create com object to work with Files and Folders
    local shell = luacom.CreateObject("Shell.Application")
    local source = shell:NameSpace(zipFile)  -- Get the zip file
    local items = source:items()
    local dest = shell:NameSpace(folder)   -- Get the destination folder file
    local files = dest:items().count + items.count -- Count of top level items already in folder plus those in zip file
    dest:CopyHere(items)    --  Copy the Zip to the Folder
    repeat                  --  Check that copy of Zip completed
        local count = dest:items().count
        if count == files then return true end
        fhSleep(1000,500)
    until count == dest:items().count
    return false
end
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

User avatar
ColeValleyGirl
Megastar
Posts: 1367
Joined: 28 Dec 2005 22:02
Family Historian: V6.2
Location: Cirencester, Gloucestershire
Contact:

Re: Problem with code snippet

Post by ColeValleyGirl » 11 Nov 2019 17:31

Thanks for that Mike -- it works.

Until I find the next bug :)

No, seriously, thank you.

User avatar
tatewise
Megastar
Posts: 16613
Joined: 25 May 2010 11:00
Family Historian: V6.2
Location: Torbay, Devon, UK
Contact:

Re: Problem with code snippet

Post by tatewise » 13 Nov 2019 20:36

I believe that the function buildZip(...) posted last Friday afternoon is satisfactory.

So I've updated the Knowledge Base > Zip File Create & Extract (code snippet) with the new Code scripts, and added testing of the returned values to the Usage section, but removed the requirement that destination folders must be empty.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

Post Reply