* Legacy Backup and Restore FH Settings plugin does not run in Linux/WINE

[Mark1834]

This is spun off from Testing plugin compatibility with Mac/Linux, posted as a new thread as it is about an issue with an existing plugin, not general compatibility issues.

The legacy backup and restore plugin loads normally in FH7 under Linux Mint/WINE, but when starting the actual backup, stops with the following error message:

Code: Select all

[string "C:\ProgramData\Calico Pie\Family Historian\Pl..."]:655: COM exception:(d:\my\lua\luacom-master\src\library\tluacom.cpp,382):
stack traceback:
	[C]: in metamethod '__index'
	[string "C:\ProgramData\Calico Pie\Family Historian\Pl..."]:655: in upvalue 'attributes'
	[string "C:\ProgramData\Calico Pie\Family Historian\Pl..."]:673: in local 'getFileList'
	[string "C:\ProgramData\Calico Pie\Family Historian\Pl..."]:685: in field 'GetFolderContents'
	[string "C:\ProgramData\Calico Pie\Family Historian\Pl..."]:4766: in upvalue 'intCopyFiles'
	[string "C:\ProgramData\Calico Pie\Family Historian\Pl..."]:5756: in function <[string "C:\ProgramData\Calico Pie\Family Historian\Pl..."]:5719>
	(...tail calls...)
	[C]: in function 'iuplua.MainLoop'
	[string "C:\ProgramData\Calico Pie\Family Historian\Pl..."]:2696: in field 'ShowDialogue'
	[string "C:\ProgramData\Calico Pie\Family Historian\Pl..."]:6154: in function 'GUI_MainDialogue'
	[string "C:\ProgramData\Calico Pie\Family Historian\Pl..."]:6185: in main chunk

Selecting Continue returns to the main plugin menu, but both the large backup button and Close Plugin buttons are greyed out and the entire menu is unresponsive. Clicking on the window Close (X) closes the plugin normally. The Family Historian, Application and Program Data folders have been created, but no files have been copied.

I strongly suspect that the FileSystemObject used to manage files and folders is the culprit. Unix-derived file systems (Mac OS, Linux, etc) have never been bound by the historic 8.3 limitation of 16-bit DOS, so file attributes do not support SHORTNAME, SHORTPATH, or TYPE, which are all Windows-only concepts. This causes issues for emulators (exactly the type of scenario that the proposed KB page is designed to alert to).

fhFileUtils() has exactly the same issue, but the next release of FH should include an updated library with support for these ancient features removed. This only fixes FH7, so if the plugin requires using FileSystemObject directly to support Unicode file paths in legacy versions of FH, the plugin author has to manage the issue directly.

It should be a simple hypothesis to test. Are there any other Store plugins that use FSO to get the attributes of existing files? They may display similar behavior.

[tatewise]

Thank you Mark, I'll investigate, but it won't be for a day or two...

[Mark1834]

This simple script illustrates the FSO issue. In Windows, it pops up a message box with the short path description of the active GEDCOM file, as expected. Run it in WINE, and WINE (and therefore FH) crashes completely.

Code: Select all

require('luacom')
FSO = luacom.CreateObject("Scripting.FileSystemObject")
File = fhGetContextInfo('CI_GEDCOM_FILE')
fileObj = FSO:GetFile(File)
fhMessageBox(fileObj.ShortPath, 'MB_OK', 'MB_ICONINFORMATION')

Calling the ShortName, ShortPath, or Type properties of the File object is being removed from fhFileUtils() getFileFolderDetails(...) for FH7 as it makes the function incompatible with WINE.

It may not be exact issue here, but I suspect it is at least related.

[Ron Melby]

its a pain, but I suppose one could do the command dir /X > sfn8_3.txt and cut it into a table of longfile name and shortfilename (8.3)

[tatewise]

Mark, it looks like you are correct and my plugin problem in WINE is caused by those unsupported FSO attributes.
My plugin doesn't need them but my library package, my version of fhFileUtils(), extracted them for completeness.

The link below to Backup and Restore Family Historian Settings has removed those unsupported FSO attributes.
Could you please check that it now runs in WINE and if so I can publish an updated plugin.

https://1drv.ms/u/s!ArcYtrv6ufQpmYBXdQd ... Q?e=O6YHsr

[Mark1834]

Unfortunately, it's still failing, with a similar error at the same place. I wonder if the root cause is somewhere in the luacom routine? Trying to debug the plugin is a non-starter, due to its complexity, but if you have a very short model script that uses the same underlying approach to generate a file list from a selected folder, I'm happy to see if that runs ok.

[tatewise]

Try this simplified library and basic use of the GetFolderContents(...) function that uses the FSO attributes.

Backup and Restore Experiments plugin: https://1drv.ms/u/s!ArcYtrv6ufQpo6gu24T ... A?e=jk3qLp

[Mark1834]

Still no joy. We know that fhFileUtils() works fine (once the unsupported attributes are removed). The most obvious difference is that fhFileUtils() uses luacom.pairs rather than the enumerator method, but it's not that on its own, as I know that the enumerator approach works ok in isolation.

Before we go too far, it is probably worth seeing what happens under Crossover. Standard Ubuntu/Mint WINE is more up-to-date than it was, but it still lags a version or two behind. I've got a USB3/SATA adapter and a couple of old spare mechanical HDDs, so I might set one up with MINT straight from the publishers to see how that compares, but it won't be for a while yet.

[tatewise]

My Backup and Restore FH Settings plugin has been using exactly the same code as in the recent experiment since Mar 2022, and I'm sure it will have been used in Crossover since then. If it had failed I'm certain I would have told about it, so I suspect Crossover is OK.

If I recall correctly, to get the code to work across FH v5 to v7 it had to use the enumerator method as luacom.pairs did not work.

Have you stepped through the code in debug mode to identify exactly where it crashes?

There is no desperate hurry to resolve this issue, so take whatever time you need.

In the meantime, perhaps a Crossover user will take pity on us and run the experiment or the full plugin

[Mark1834]

It's definitely at the line that gets file attributes, see screen grab below:

Screenshot from 2024-06-28 12-34-54.png (31.4 KiB) Viewed 933 times

Also if I change line 313 to simply local tblAttr = {} and change line 358 to local strPath = tblAttr.path or '' to allow for tblAttr.path not being defined, the plugin runs to completion (albeit with only a limited output, but there are no errors).

There is nothing fundamentally wrong with either FSO or luacom Enum, as the following code snippet generates the expected table of plugin files with the dates of modification:

Code: Select all

require("luacom")												-- To create File System Object
FSO = luacom.CreateObject("Scripting.FileSystemObject")

objFolder = FSO:GetFolder(fhGetContextInfo('CI_APP_DATA_FOLDER') .. '\\Plugins')

tblName, tblDate = {}, {}
tblEnum = luacom.GetEnumerator(objFolder.Files)
local tblAttr = tblEnum:Next()
while tblAttr do
	table.insert(tblName, tblAttr.Name)
	table.insert(tblDate, tblAttr.DateLastModified)
	tblAttr = tblEnum:Next()
end

fhOutputResultSetColumn('File', 'text', tblName,  #tblName)
fhOutputResultSetColumn('Date', 'text', tblDate,  #tblDate)

Screenshot from 2024-06-28 12-42-48.png (13.85 KiB) Viewed 933 times

That's probably about as much as I can do for the moment. As you rightly say, how it performs in Crossover is the key test, as that community is much more significant than Linux users. I'd still like to understand exactly what the problem is, though. Are there other store plugins that use the same custom library? It's probably worth testing one or two of those, as they probably don't have a readily available alternative.

(Edited to modify the code snippet, removing two superfluous lines used only for debugging)

[Valkrider]

Mike

I fired up my laptop with Crossover on it and ran your experimental plugin and it crashed FH see the attached screenshot.

Screenshot 2024-06-28 at 12.49.16.png (44.34 KiB) Viewed 931 times

[Mark1834]

Interesting - I remember commenting back in 2022 when the Windows backup plugin was developed that the original version didn't run under WINE, so it could be a long-standing issue that has not been diagnosed.

[Valkrider]

Mark

Your Windows based plugin works fine under Crossover I ran it after I ran Mike's test one just to make sure.

Screenshot 2024-06-28 at 13.44.40.png (64.67 KiB) Viewed 910 times

[tatewise]

It is a bit mysterious why line 313 fails.
It is essentially the same as the fhFileUtils(..) lines 74-82 that do the same thing.

The only difference is that my plugin uses colon ; separators instead of comma , separators.

I wonder if one of the other attributes is also not supported by WINE.

I have rewritten the experimental plugin to assign each attribute separately to see which one triggers the error.
You will have to step through in debug mode to find it.

I have also assigned the attributes using a table with comma separators instead of colons.

The name of the local table has been made different from the parameter to avoid any confusion.

Please try this update via the link below.
Backup and Restore Experiments plugin: https://1drv.ms/u/s!ArcYtrv6ufQpo6gu24T ... A?e=jk3qLp

[Valkrider]

No link @Mike

[tatewise]

I meant you to use yesterday's link but I've added it to today's post now.

[LornaCraig]

I meant you to use yesterday's link but I've added it to today's post now.

@Mike: For anyone who uses the default display order for posts (probably most people) yesterday’s link is ‘above’, not ‘below’. Hence it looked as if the link was missing, until you added it at the bottom of today’s post.

[Valkrider]

Sorry Mike no luck with that latest version either exactly the same fault.

[tatewise]

Unless you ran it in debug mode, I'm not surprised as it is essentially the same code in an adjusted form.
The idea was to step in debug mode to see which line fails.

[Valkrider]

Line 315

[Mark1834]

Sorted.

With parameters passed individually, it quickly became clear that the only two that were being accepted were Name and Path. That's clearly not right, as we know WINE fully supports file timestamps, so the problem must be in the function that populates your table of attributes.

TBH, I found your code very opaque and almost impossible to debug, as your variable names gave no clue to their contents, so I had to look back several statements to remind myself what they represented. In addition, you re-declare the same variable name within the same scope to contain a different type of data.

Once I rewrote the function using more transparent names (a more verbose version of the style used in both the MS documentation and fhFileUtils()), it was clear what the problem was. You were mixing files and folders in the same table, so use of that data was limited by whichever of the two was least supported in the emulator.

It turns out that folders are less supported than files, as per the snippet below (where col.. denotes a collection of that type, obj.. is an object such as a File object or Folder object, and tbl.. is reserved for only tables that I populate, not everything that is technically a table, etc....)

Code: Select all

require("luacom")												-- To create File System Object
FSO = luacom.CreateObject("Scripting.FileSystemObject")

objParentFolder = FSO:GetFolder(fhGetContextInfo('CI_APP_DATA_FOLDER'))
tblName, tblPath = {}, {}
colFolders = luacom.GetEnumerator(objParentFolder.SubFolders)
local objFolder = colFolders:Next()
while objFolder do
	table.insert(tblName, objFolder.Name)
	table.insert(tblPath, objFolder.Path)
	objFolder = colFolders:Next()
end
fhOutputResultSetColumn('Folder', 'text', tblName,  #tblName)
fhOutputResultSetColumn('Path', 'text', tblPath,  #tblPath)

Any attribute other than Name or Path either stops with an error or crashes WINE (and therefore FH) completely. In hindsight, that does make some sense, as there are significant technical differences between Windows Folders and Unix style directories (which are only a special kind of file), but it is perhaps surprising that Folder timestamps don't appear to be supported.

The fix is therefore to separate out file and folder management into separate functions.

[tatewise]

OK, so we have learnt something not mentioned before, that WINE folders support a much smaller subset of FSO attributes than WINE files. Presumably, that will be fed into the updated fhFileUtils.fh_lua library.

My function GetFolderContents(...) returns a similarly complex table structure as the fhFileUtils function getFolderContents(...) with all the FSO folder and file attribute values in the same table.

Please check my updated plugin via the link below.
Backup and Restore Experiments plugin: https://1drv.ms/u/s!ArcYtrv6ufQpo6gu24T ... A?e=jk3qLp

[Valkrider]

Mike that runs fine under Crossover, no output though which is the result you were looking for I assume and no crash.

[Mark1834]

I had to make one minor change to fully support the current "out-of-the-box" repository version of WINE, but that is specific to Linux, not Mac. Don't make additional changes to the library just to support Linux - it will be a small community of tech-savvy users, and they probably wouldn't use either version of the backup plugin unless they were transferring settings between PCs as it is simpler to backup both user data and all settings directly from the host OS.

[tatewise]

As shown by Colin's experiments, it is not just Linux/WINE that is affected but also Crossover.
I'm not sure whether that was Crossover on Linux or a Mac.
Perhaps, PlayOnLinux/Mac also needs to be examined.

The 'unsupported FSO attributes' may not be used by any of my plugins that use my library. I'll have to check.
The full set of FSO attributes was included only for completeness and to mirror LFS as much as possible.
I suspect the fhFileUtils.fh_lua library similarly included them just for completeness.

As I'm sure you know, transferring settings using just host OS files omits the Windows Registry settings, which include many of the Tools > Preferences, all the default core Diagram settings, and Records Window columns.