* sorting tables by v

Writing and using plugins for Version 5 and above.
User avatar
Ron Melby
Superstar
Posts: 457
Joined: 15 Nov 2016 15:40
Family Historian: V6.2

Re: sorting tables by v

Post by Ron Melby » 14 Aug 2019 01:13

general case, from 1 to n tbl (an array of tbl)
in this case:

tbl = {tblDATES, tblNAME, tblSEX)
(for the purposes of this code, assume that either tbl[1]
has equal or more entries than the other tables.
insofar as I can imagine tbl[1]...tbl[{n] will always
have equal number of entries.

tblDATES =
[1] '(1913 - 1989}'
[2] '(1927 - 2004)'
(and so on)

tblNAME =
[1] 'NELSON, AL '
[2} 'BARSNESS, CLIFFORD '
(and so on)

tblSEX =
[1] 'M'
[2] 'M'
(and so on)

function bldKLIST(tbl)
local tblKLIST{}

for k, v in pairs (tbl[1]) do
local fld =''
for t, #tbl do
fld = fld .. t.tbl[t]
enddo
table.insert(tblKLIST,fld)
endo
return tblKLIST
end

tblKLIST:

[1] '(1913 - 1989}NELSON, ALM'
[2] '(1927 - 2004)BARSNESS, CLIFFORDM'
(and so on)

it doesn't look ok to me, and there are slicker programmers on the forum
than I.

BTW do you have a server going casters up or have you BREXITED before time and the EUs unplugging you?
Pages are taking an interminable time to load here in Yankshire and you seem to be the only site.
FH V.6.2.7 Win 7 64 bit

User avatar
tatewise
Megastar
Posts: 16882
Joined: 25 May 2010 11:00
Family Historian: V6.2
Location: Torbay, Devon, UK
Contact:

Re: sorting tables by v

Post by tatewise » 14 Aug 2019 09:47

You don't explain the objective of the script in that last posting so it is difficult to review it.
How does it deal the crucial problem of multiple Wars per person?

Regarding the web site server going casters up, see FHUG website slow (17000).
Anyway tsoHost are based in the UK so BREXIT has no impact.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

User avatar
Ron Melby
Superstar
Posts: 457
Joined: 15 Nov 2016 15:40
Family Historian: V6.2

Re: sorting tables by v

Post by Ron Melby » 14 Aug 2019 12:07

multiple wars per person is not crucial, as I have explained before multiple wars get multiple records.

sort by war, sort by person

so some of the entries for war may be:

' '
American Revolution
War of 1912
Indian Wars
WWI
WWII
Korea
Vietnam

will use multiple entries.

sort those in order a.WAR < b.WAR.

then sort by name a. NAME < b. NAME
so that the wars are alpha and the names are put together alpha and wars should follow name in order.

then on print to file

if record id (not key) is same as previous id
write war and its ancillary stuff
else
write name stuff and war and its ancillary stuff
end
so it sort of looks like:

ABEL, Cain (1840 -1927) M Civil War US Army PVT 1st MN Infantry
............................................... King Phillips War Militia

dots used so you get the spacing idea.
Last edited by Ron Melby on 14 Aug 2019 12:26, edited 2 times in total.
FH V.6.2.7 Win 7 64 bit

User avatar
Ron Melby
Superstar
Posts: 457
Joined: 15 Nov 2016 15:40
Family Historian: V6.2

Re: sorting tables by v

Post by Ron Melby » 14 Aug 2019 12:14

once again the objective of the script is general in nature.

given 1 to n number of tables where the entries are k, v only

i want to take k order for every table

and for those one to n tables

i want to:
composite field = v.tbl one .. v.tbl two ... v. tbl three ... v. tbl n
table.insert(composite, composite field)

return the new table.

I thought I had written the thing pretty clearly.
FH V.6.2.7 Win 7 64 bit

User avatar
Ron Melby
Superstar
Posts: 457
Joined: 15 Nov 2016 15:40
Family Historian: V6.2

Re: sorting tables by v

Post by Ron Melby » 14 Aug 2019 17:30

tbl = {tblName, tblWar, tblDates}

function bldKLIST(tbl)
local KEYVAL = {}

for k, v in ipairs(tbl[1]) do
local KEY = ''
for K = 1, #tbl do
KEY = KEY .. v.tbl[K] ****** attempt to index field 'tbl' (a nil value)
end
table.insert(KEYVAL, KEY)
end
return KEYVAL
end


I am trying to code:

for as many entries in tblNames:
ipair tbl[1] (tblNames)

build a field by relative record number (k) in a table.
value of tbl[1] tblNames +
value of tbl[2] tblWar +
value of tbl[3] tblDates
write out the concat key in new table
FH V.6.2.7 Win 7 64 bit

User avatar
tatewise
Megastar
Posts: 16882
Joined: 25 May 2010 11:00
Family Historian: V6.2
Location: Torbay, Devon, UK
Contact:

Re: sorting tables by v

Post by tatewise » 14 Aug 2019 17:41

Make sure you use local tbl = ... and same for other tables so they appear in the lower right debug pane.
Then debug the for k, v in ipairs(tbl[1]) do loop.
What are the values of k and v each time round the loop ?

That should give you the clues to what is going wrong.
You really must learn to use the debug tools and understand Lua table structures, otherwise progress will be very difficult.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

User avatar
Ron Melby
Superstar
Posts: 457
Joined: 15 Nov 2016 15:40
Family Historian: V6.2

Re: sorting tables by v

Post by Ron Melby » 14 Aug 2019 17:53

v is correct in the first part of the loop before I execute the line in error, then I get the index error set off by ****
v is the value in v.tblname and we would expect that to be so because that is tbl[1] record[k=1]

ok thats fine, I dont need v.tbl[K] in code but now I need the loop to get
v from tbl[2]
and
v from tbl[3]
or however many tables I send in, from 1 ... n and glue it into one field.
might just as well say 2 .. n because:
if its one table, I already have a sorting v. but it is going to index starting at one.
FH V.6.2.7 Win 7 64 bit

User avatar
tatewise
Megastar
Posts: 16882
Joined: 25 May 2010 11:00
Family Historian: V6.2
Location: Torbay, Devon, UK
Contact:

Re: sorting tables by v

Post by tatewise » 14 Aug 2019 19:31

but now I need the loop to get
v from tbl[2]
and
v from tbl[3]
You know how to use ipairs to do that.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

User avatar
Ron Melby
Superstar
Posts: 457
Joined: 15 Nov 2016 15:40
Family Historian: V6.2

Re: sorting tables by v

Post by Ron Melby » 14 Aug 2019 20:11

I do not, know how to do this, because that is what I am trying to do:

for k, v in an unknown number of tables (whose names I will not tell you)
for each record{k]
I need v in tbl[from 1 to number of tables] that have the key[k]
do not know how to do that.


I can read one table, and I can get values from any number of KNOWN tables.

and I believe my code demonstrates that.
I am trying to do it, and am on the right track, but cannot make the code statement correctly
I am right up to the
fld = fld + value of tbl[K] having key [k] which is what I think I am writing but obviously am not.
FH V.6.2.7 Win 7 64 bit

User avatar
tatewise
Megastar
Posts: 16882
Joined: 25 May 2010 11:00
Family Historian: V6.2
Location: Torbay, Devon, UK
Contact:

Re: sorting tables by v

Post by tatewise » 14 Aug 2019 20:31

Saying you don't know is a negative attitude, when I know you do know.

Start at the beginning.
How do you loop though each integer entry k with value v for table tbl ?

You have done that lots of times, so you do know.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

User avatar
Ron Melby
Superstar
Posts: 457
Joined: 15 Nov 2016 15:40
Family Historian: V6.2

Re: sorting tables by v

Post by Ron Melby » 14 Aug 2019 20:43

for k, v in ipairs (tbl[1]) do

do not know how to loop thru multiple tables in an array in a loop repeatedly, k by k

for _, strType in ipairs ( { 'INDI'; 'FAM'; } ) do
is not what I want, if it is what you are asking....run thru [1] then [2] serially.
I want to run thru [1] and [2] in parallel
FH V.6.2.7 Win 7 64 bit

User avatar
tatewise
Megastar
Posts: 16882
Joined: 25 May 2010 11:00
Family Historian: V6.2
Location: Torbay, Devon, UK
Contact:

Re: sorting tables by v

Post by tatewise » 14 Aug 2019 21:14

for k, v in ipairs (tbl[1]) do will not loop though tbl[1], tbl[2], tbl[3] which is what you asked to do.

So how do you loop through tbl[1], tbl[2], tbl[3], et seq ?
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

User avatar
Ron Melby
Superstar
Posts: 457
Joined: 15 Nov 2016 15:40
Family Historian: V6.2

Re: sorting tables by v

Post by Ron Melby » 14 Aug 2019 21:28

this look familiar at all?


function bldKLIST(tbl)
local KEYVAL = {}
local tbl = tbl

for k, v in ipairs(tbl[1]) do
local KEY = ''
** for K = 1, #tbl do
** KEY = KEY .. v.tbl[K] <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<ERROR
** end
table.insert(KEYVAL, KEY)
end
return KEYVAL
end
third time I have posted it
** this is where I thought I was going to do it.

i AM TRYING TO BUILD A KEY WITH THAT STATEMENT SOMETHING LIKE

KEY = '' +
value.tblNames{1] +
value.tblDates[1] +
value. tblSEX[1]
write KEY
do the same for [2] to end of entries in tbl[1] (tblNames)
FH V.6.2.7 Win 7 64 bit

User avatar
tatewise
Megastar
Posts: 16882
Joined: 25 May 2010 11:00
Family Historian: V6.2
Location: Torbay, Devon, UK
Contact:

Re: sorting tables by v

Post by tatewise » 14 Aug 2019 21:58

It is getting us nowhere by repeatedly posting the same script that does not work the way you want.

for k, v in ipairs (tbl[1]) do will not loop though tbl[1], tbl[2], tbl[3] which is what you want to do.

So how do you write a snippet of Lua script to loop through tbl[1], tbl[2], tbl[3], et seq ?

Don't complicate things. Just consider the simple answer to that simple question.
It is in all the Lua user guides, and you have used it many times.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

User avatar
Ron Melby
Superstar
Posts: 457
Joined: 15 Nov 2016 15:40
Family Historian: V6.2

Re: sorting tables by v

Post by Ron Melby » 14 Aug 2019 22:58

yeah, I am well aware that for k, v in tbl{1] does not loop thru tbl1 2 and three

K=1 to #tbl do
this = tblK]
end

that revolves me around in the array of table names. fourth time posted.
FH V.6.2.7 Win 7 64 bit

User avatar
tatewise
Megastar
Posts: 16882
Joined: 25 May 2010 11:00
Family Historian: V6.2
Location: Torbay, Devon, UK
Contact:

Re: sorting tables by v

Post by tatewise » 15 Aug 2019 08:35

So why do you think for k, v in ipairs ( tbl[1] ) do does not loop through tbl[2], tbl[3], et seq ?

That statement is saying loop through the contents of only the tbl[1] entry, i.e. just the first entry.

What VERY simple change to that ipairs statement would loop through the contents of all the tbl entries ?
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

User avatar
Ron Melby
Superstar
Posts: 457
Joined: 15 Nov 2016 15:40
Family Historian: V6.2

Re: sorting tables by v

Post by Ron Melby » 16 Aug 2019 06:15

the intent was to get the number of entries really.

consider the statement as I was making it to mean loop thru the number of entries that are contained in the leftmost key, whether those other tables are longer or shorter.

anyway, sorted I did not use ipairs or pairs. it actually, really came to me in a dream.

I now have two sort methods, mine having some limitations. I am trying now to consider a slick way to overcome them. I suppose I will in time.
FH V.6.2.7 Win 7 64 bit

User avatar
Ron Melby
Superstar
Posts: 457
Joined: 15 Nov 2016 15:40
Family Historian: V6.2

Re: sorting tables by v

Post by Ron Melby » 16 Aug 2019 06:17

I must say the new server is very snappy.
FH V.6.2.7 Win 7 64 bit

User avatar
Ron Melby
Superstar
Posts: 457
Joined: 15 Nov 2016 15:40
Family Historian: V6.2

Re: sorting tables by v

Post by Ron Melby » 17 Aug 2019 16:21

this problem is the most perplexing for me yet.

requirement of the function rtv(): all tables involved are [rrn]'blahblahblahyaddayaddayadda' form (arrival, or array, whichever you like)

case1 = ({tblName; tblSVC; tblWar; tblDates})
rtx = rtv(case1)

case2 = ({tblName;})
rtx = rtv(case2)

case3 = tblName
rtx = rtv(tblName)


function rtv(tbl)

if case1
do this
end
if case2
do that
end
if case3
do theotherthing
end

end

simple methods loke:if #tbl
if tbl[2]
dont work and I am not sure that there is a difference in case 2 and 3
FH V.6.2.7 Win 7 64 bit

User avatar
tatewise
Megastar
Posts: 16882
Joined: 25 May 2010 11:00
Family Historian: V6.2
Location: Torbay, Devon, UK
Contact:

Re: sorting tables by v

Post by tatewise » 17 Aug 2019 21:31

Assuming tblName does not contain tables then this works:

Code: Select all

if #tbl == 4 then
	case = 1
elseif #tbl == 1 and type(tbl[1]) == "table" then
	case = 2
elseif #tbl == 0 or type(tbl[1]) ~= "table" then
	case = 3
else
	case = unknown
end
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

User avatar
Ron Melby
Superstar
Posts: 457
Joined: 15 Nov 2016 15:40
Family Historian: V6.2

Re: sorting tables by v

Post by Ron Melby » 17 Aug 2019 23:52

Code: Select all

 local function karg(tbl)
    local ct = 0 
    local ktyp = ''
    local vtyp = ''
    local typ = type(tbl)  

    if typ ~= 'table' then
      return nil
    end

    for k, v in pairs(tbl) do
      ktyp = type(k)
      vtyp = type(v)
      if ktyp ~= 'number' then
        if vtyp == 'table' then
          ct = ct + 1 
        elseif vtyp == 'string' then
          return 1 
        end
      end
    end 
    return ct
  end

  local function bldKLIST(ent, argv, tbl)
    local KEYLIST = {}

    for kfld = 1, ent do
      local KEY = ''
      for t = 1, argv do
        local ktbl = tbl[t] 
        KEY = KEY .. ktbl[kfld] 
      end
      table.insert(KEYLIST, KEY)
    end
    return KEYLIST
  end

function crtUSRIDX(tbl, KEYSORT)
  
  local KEYLIST = {}
  local argv = karg(tbl)

  if not argv then
    return nil
  end

  if argv > 1 then
    local ent = #tbl[1]
    tblsrt = bldKLIST(ent, argv, tbl)
  else
    tblsrt = tbl
  end

  for rrn in pairs(tblsrt) do
    table.insert(KEYLIST, rrn)
  end
  table.sort(KEYLIST, 
    function(a, b)
      return KEYSORT(tbl[a], tbl[b])
    end)
end
I am now all turned around

nowhere to go, I am so lost been bent over this for months.

alright, doesnt work for the real world...
but in FH, you generally build a table, build a table something on the order of

if indi is what I want
table.insert(tbl1, name)
table.insert)tbl2,dates)
table.insert(tbl3.relation)
end

I will probably never ask, what is the address of the date where my grandmothers, cats, owners inlaws, wife died..
but lets pretend, I would ask that.

so, I can build (within fh) a reasonably robust glue of tables from left to right and sort them.
SEE ABOVE
those tables can be:
a single table in arrival order (within fh)
a single table (within fh) in a table
multiple tables(within fh, joined by value

detecting those, each in their kind or just blowing up (not a bad way to go) innit?
i want a record table of that sort, such that iam delivered keys in order of sort without let or hinderance, I call them rrn, (relaive record number, where am I failing? expectq table of rrns[keys] back
FH V.6.2.7 Win 7 64 bit

User avatar
tatewise
Megastar
Posts: 16882
Joined: 25 May 2010 11:00
Family Historian: V6.2
Location: Torbay, Devon, UK
Contact:

Re: sorting tables by v

Post by tatewise » 18 Aug 2019 11:00

For one last time I advise you use a table record structure and if you don't want to go that way then you are on your own.

Code: Select all

local tblData = { }

function MakeData()
	ptrIndi = fhNewItemPtr()
	ptrIndi:MoveToFirstRecord('INDI')
	while ptrIndi:IsNotNull() do
		local strName = fhGetItemText(ptrIndi,"~.NAME:SURNAME_FIRST")
		local intRid = fhGetRecordId(ptrIndi)
		local strSex = fhGetItemText(ptrIndi,"~.SEX")
--	--	table.insert( tblData, { Name=strName; Rid=intRid; Sex=strSex; } )
		table.insert( tblData, { } )
		local intKey = #tblData
		tblData[intKey].Name = strName  -- same as tblData[intKey]["Name"] = strName
		tblData[intKey].Rid = intRid    -- same as tblData[intKey]["Rid"] = intRid
		tblData[intKey].Sex = strSex
		ptrIndi:MoveNext() 
	end
end
The MakeData() function constructs the tblData records with whatever field names are wanted.
Notice the two ways of building the record either as one composite table or field by field.
In this example the field names are Name, Rid and Sex.

Code: Select all

function SortData(...)

	local intArg = 1

	local function SortKey(tblA,tblB)
		local valA = tblA[arg[intArg]]
		local valB = tblB[arg[intArg]]
		if valA == valB and arg.n > intArg then
			intArg = intArg + 1
			return SortKey(tblA,tblB)
		end
		intArg = intArg - 1
		return valA < valB
	end

	table.sort( tblData, SortKey )

end

MakeData()

SortData("Sex","Name")

SortData("Rid","Name","Sex")
The SortData(...) function sorts tblData according to the field names supplied.
It uses a special vararg table parameter that can hold a variable number of arguments.
See http://www.lua.org/manual/5.1/manual.html#2.5.9 2.5.9 – Function Definitions.

The SortKey(tblA,tblB) function is called recursively until a field differs or the vararg is exhausted.

The tblData is accessed in sorted order using its integer key:

Code: Select all

for intKey, tblRec in ipairs (tblData) do	-- tblRec is same as tblData[intKey]
	local strName = tblRec.Name		-- same as tblRec["Name"]
	local intRid = tblRec.Rid		-- same as tblRec["Rid"]
	local strSex = tblRec.Sex
end
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

User avatar
Ron Melby
Superstar
Posts: 457
Joined: 15 Nov 2016 15:40
Family Historian: V6.2

Re: sorting tables by v

Post by Ron Melby » 19 Aug 2019 16:01

At the outset, way back when I was doing relations and that table build, it became clear that the 'RCD' build you advocated is the way to go, it is far and away number 1.

That is why I am changing the operation of my _STD_xxx modules. it will be my way forward.

However, until that is done, and in the forseeable future...

I need something for plugins where tables are separate like is usually done on a display, I had thought that for 'meatball surgery' in my plugins and the host of plugins in the store, where fhOutputResultSetColumn is the end result (and they can do some simple sorting table by table) that in those MANY MANY cases, if I were to want a printed report that varied significantly from the table, table, table... layout button to save as... that without having to rewrite the entire plugin I could simply drop in an include, 3 lines of structured code, and a customized (since you cant grab or point to an actual object (object name) in LUA, I could do a meatball report fairly handily (right now in .rtf, and .txt, and will expand to .doc)

I am close. Your code is very helpful. I am just frustrated after 2 days-- 20 hours and dream about for 4 debugging and rewriting and moving problems from here to there of that miserable little what 20 or 30 lines of code.... I probably shouldnt have posted that post, it was done in despare.

SO in order of long term ways of doing business:
1. rewrite standard modules to 'RCD' instead of individual tables, as you have demonstrated to me time and again Mike.
2. Investigate penlight library as Helen recommends.
way way last , my glue tables and sort code.

But there are reasons I am pig-headedly making this code as well.
FH V.6.2.7 Win 7 64 bit

User avatar
tatewise
Megastar
Posts: 16882
Joined: 25 May 2010 11:00
Family Historian: V6.2
Location: Torbay, Devon, UK
Contact:

Re: sorting tables by v

Post by tatewise » 19 Aug 2019 17:12

I gave you the interim solution a week ago, but you ignored it.
Presumably, since they are for fhOutputResultSetColumn, all the separate tables must be the same length.
So copy them all into one record table and sort that using SortData("Rid","Name","Sex") as explained yesterday.

Code: Select all

local tblData = {  }
for i = 1, #tblName do
	table.insert( tblData, { Name=tblName[i]; Rid=tblRid[i]; Sex=tblSex[i]; }
end
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

User avatar
Ron Melby
Superstar
Posts: 457
Joined: 15 Nov 2016 15:40
Family Historian: V6.2

Re: sorting tables by v

Post by Ron Melby » 20 Aug 2019 22:43

USRIDX has 194 entries as I expect, and in the form I expect. The range from 1 to 194 no gaps there are no entries that contain nil nor even ''

typical entries are:

[5] => (table .2)
KLIST => "MELBY, Ronald Nolan 1972-1975 (1954 - )"
IDX => 5
[6] => (table .2)
KLIST => "NEWTON, Lyle Delane (1952 - 2018)"
IDX => 6
[7] => (table .2)



USRIDX = {}
for rrn, klist in ipairs(tbl) do
table.insert(USRIDX, {KLIST=klist; IDX=rrn;} )
end

dir = dir or '<'
if dir ~= '<'
and dir ~= '>' then
return -1
end

table.sort(USRIDX,
function(a, b)
return a.KLIST .. ' ' .. dir .. ' ' .. b.KLIST ****
end)

**** attempt to index local 'a' (a nil value)

that is the code chunk as it stands, copied from working code, there is no intervening code in the chunk.

the table sort function is cut up for easier display of error.

I cannot find anything wrong with this code. I have searched the net and did the debugs, still no answer.

I should add it does some sorting (but so much of it is invisible... I looked at a and b quite a while and it seems working fine I changed direct (dir) to =< (but i know no case of equal exists) and since nothing can be added into the sort function, i cant do a running print.... but it still fails somewhere. I can provide the full table of course. If it is some arcane error....
FH V.6.2.7 Win 7 64 bit

Post Reply