* recursion, knowing where I am in a nested table, and my own movenext()

For users to report plugin bugs and request plugin enhancements; and for authors to test new/new versions of plugins, and to discuss plugin development (in the Programming Technicalities sub-forum). If you want advice on choosing or using a plugin, please ask in General Usage or an appropriate sub-forum.
Post Reply
User avatar
Ron Melby
Megastar
Posts: 917
Joined: 15 Nov 2016 15:40
Family Historian: V6.2

recursion, knowing where I am in a nested table, and my own movenext()

Post by Ron Melby »

I have been working on this code off and on since Oct 2020. Even before that, and abandoned and deleted it in frustration, but I am not going to give up. Someone here has done this though, for some reason. Mostly now though, I am on, and have been for about 3 weeks, several hours a day. miketate, you have always said I overthink my code, and have never done recursion correctly without your help. We are here again. I am trying to read _G (or _G._G) I would like to enumerate it somewhat like debug does. so, problems I am running across are: I cannot get the name of the original table (I can get its pointer). Then how do I know I am in a nested table, and where I am in it? then the next problem, is twofold, lets say I have a table of zero entries, how do I log it? and the final problem, how do I ignore tables such as iup._M* and move past them to the next entry, sort of like my own MoveNext()? here is my present code, and it works for some things but then goes horridly awry.

Code: Select all

-- require '_STD_SYS'
-- require '_STD_HDR'
-- require '##_STD_HEX' -- still debugging

--[[
scan all tables in _G for functions
try to build a reverse mapping of all functions pointers
string.sub() should not just be sub(), but the full name
]]

local _oblt = {}  -- object built
local _Gtbl = {}  -- enumeration of _G
local _gid  = 1   -- index of _Gtbl table

local _elvl = {}  -- table depth, i.e. iup.callback.
local _etbl = 0

local _dft  = {addr = '*NULL', dec = 0}

function rslv_G(_tbl, __nam)

  function get_addr(_gatyp)
    local _ga = _dft
    if type(_gatyp) == 'function'
    or type(_gatyp) == 'table' then
      _gatyp = (tostring(_gatyp))
      _gatyp = _gatyp:sub(_gatyp:find(' ') + 1):match('(%x*)')
      _ga.addr = _gatyp
      --  _ga.dec = hex_to_dec(_gatyp):gsub('%.0', '') or 0
    end
    return _ga
  end

  function wrt_obj(_wtyp, _wnam, _wpkg, _wga)

    for _, _dbgID in pairs({16, 189, 248, 255, 405}) do
      if _gid == _dbgID then
        _BREAKPOINT = true
        break
      end
    end

    wrt_type[(type(_wtyp))](_wtyp, _wnam, _wpkg, _wga)
  end

  wrt_type =
  {
    ['boolean'] = function(_otyp, _onam, _opkg, _oga)
      print('in boolean')

      local bool
      if _otyp then
        bool =  'true'
      else
        bool =  'false'
      end

      _Gtbl[_gid] =
      {
        id   = _gid,
        addr = _oga.addr,
        dec  = _oga.dec,
        typ  = type(_otyp),
        _cl  = string.format('%s%s', _opkg, _onam),
        pkg  = _opkg,
        name = _onam,
      }
      _gid = _gid + 1
    end, -- fn boolean

    ['thread'] = function(_otyp, _onam, _opkg, _oga)
      print('in thread')

      local _a, _b = pcall(string.dump, _otyp)

      _Gtbl[_gid] =
      {
        id   = _gid,
        addr = _oga.addr,
        dec  = _oga.dec,
        typ  = type(_otyp),
        _cl  = string.format('%s%s', _opkg, _onam),
        pkg  = _opkg,
        name = _onam,
      }
      _gid = _gid + 1
    end, -- fn thread

    ['userdata'] = function(_otyp, _onam, _opkg, _oga)
      print('in userdata')

      local _a, _b = pcall(string.dump, _otyp)

      _Gtbl[_gid] =
      {
        id   = _gid,
        addr = _oga.addr,
        dec  = _oga.dec,
        typ  = type(_otyp),
        _cl  = string.format('%s%s', _opkg, _onam),
        pkg  = _opkg,
        name = _onam,
      }
      _gid = _gid + 1
    end, -- userdata (generally fhptr)

    ['nil'] = function(_otyp, _onam, _opkg, _oga)
      print('in nil')

      _Gtbl[_gid] =
      {
        id   = _gid,
        addr = _oga.addr,
        dec  = _oga.dec,
        typ  = type(_otyp),
        _cl  = string.format('%s%s', _opkg, _onam),
        pkg  = _opkg,
        name = _onam,
      }
      _gid = _gid + 1
    end, -- fn nil

    ['string'] = function(_otyp, _onam, _opkg, _oga)

      _Gtbl[_gid] =
      {
        id   = _gid,
        addr = _oga.addr,
        dec  = _oga.dec,
        typ  = type(_otyp),
        _cl  = string.format('%s%s', _opkg, _onam),
        pkg  = _opkg,
        name = _onam,
        val  = _otyp
      }
      _gid = _gid + 1
    end, -- fn string

    ['number'] = function(_otyp, _onam, _opkg, _oga)

      local  _val  = ('%.1f'):format(_otyp)

      _Gtbl[_gid] =
      {
        id   = _gid,
        addr = _oga.addr,
        dec  = _oga.dec,
        typ  = type(_otyp),
        _cl  = string.format('%s%s', _opkg, _onam),
        pkg  = _opkg,
        name = _onam,
        val  = _val:gsub('%.0', ''),
      }
      _gid = _gid + 1
    end, -- fn number

    ['function'] = function(_otyp, _onam, _opkg, _oga)

      local _opkg = _opkg
      local _fh = string.sub(_onam, 1, 2)
      if  _fh == 'fh' then
        _opkg = ('%s%s.'):format(_opkg, _fh)
      end

      _Gtbl[_gid] =
      {
        id   = _gid,
        addr = _oga.addr,
        dec  = _oga.dec,
        typ  = type(_otyp),
        _cl  = string.format('%s%s', _opkg, _onam),
        pkg  = _opkg,
        name = _onam,
      }
      _gid = _gid + 1
    end, -- fn function

    ['table'] = function(_ityp, _inam, _ipkg, _iga)
      local ityp = type(_ityp)
      local inam = _inam
      local ipkg = _ipkg

      if _oblt[_iga.addr] then
        return  -- this prevents rereading recursive tables (but it dont)
      end
      _oblt[_iga.addr] = true

      for _, _ign in pairs({'_G.', '_G._G.', '_Gtbl.', 'iup._M.', 'package.loaded'}) do
        if _wpkg == _ign then
          _epkg = ''
          _ityp = ''
          return
        end
      end

      for _enam, _etyp in pairs(_ityp) do
        local etyp = type(_etyp)

        if etyp == 'table' then

          _epkg = '' --
          _elvl[#_elvl + 1] = ('%s.'):format(_enam)

          for  _ix, _ in ipairs(_elvl) do
            -- glue level
            _epkg = ('%s%s'):format(_epkg or '', _elvl[_ix])
          end

          _etbl = _etbl + 1

          -- type(_etyp) == 'table'
          local _ega = get_addr(_etyp)

          if next(_etyp) then
            wrt_obj(_etyp, _enam, _epkg or '', _ega)
          else
            _Gtbl[_gid] =
            {
              id   = _gid,
              addr = _ega.addr,
              dec  = _ega.dec,
              typ  = type(_etyp),
              _cl  = string.format('%s%s', _epkg, _enam),
              pkg  = _epkg,
              name = _enam,
            }
            _gid = _gid + 1
          end

        elseif etyp == 'function' then

          -- type(_etyp) == 'function'
          local _ega = get_addr(_etyp)
          local _enam = _enam
          if type(_enam) == 'number' then
            _enam = (('[%s]%s'):format(_enam, _inam))
          end

          wrt_obj(_etyp, _enam, _epkg or '', _ega)

          -- expansion later
          -- elseif _etyp == 'userdata' then  io. has them i.e. stdin
          -- elseif _etyp == 'string' then  'global constant'
          -- elseif _etyp == 'number' then  'global constant'
          -- elseif _etyp == 'boolean' then
          -- elseif _etyp == 'thread' then
          -- elseif _etyp == 'nil' then
        end
        -- type(_etyp) == 'table' 'function'
      end
      -- out of table
      -- how to manage nested tables correctly ??
      _epkg = '' -- clear for non tables
      _enam = '' -- clear for non tables
      _etbl = _etbl - 1  -- see 207
      if _etbl <=0 then
        _elvl = {}
      else
        _elvl[#_elvl] = nil
      end
      -- for _e = _etbl, #_elvl do
      --   _elvl[_etbl] = nil
      --   _elvl[_e] = nil
      -- end

      -- fn table
    end,
  }


--  strdbg()

  local _t = tostring(_tbl)
  _t = _t:sub(_t:find(' ') + 1):match('(%x*)')
  _t = _t:match('(%x*)')
  local _st = _dft
  _st.addr = _t
  wrt_obj(_tbl, _t, '', _st)  -- get name and addr put in _Gtbl 1 or ?
-- fn rslv_G
end

function main()
-- *NULL: landmark only
end

rslv_G(_G._G, '')

local tblnbr = {}
local tbllib = {}
local tblnam = {}
local tbltyp = {}
local tblhex = {}
local tbldec = {}

for k, _ in pairs(_Gtbl) do
  local g = _Gtbl[k]
  table.insert(tblnbr, g.id)
  table.insert(tbllib, g._cl)
  table.insert(tblnam, g.name)
  table.insert(tbltyp, g.typ)
  table.insert(tblhex, g.addr)
  table.insert(tbldec, g.dec)
end

fhOutputResultSetColumn('id ', 'integer', tblnbr,  #tblnbr,  24,  'align_right', 1)
fhOutputResultSetColumn('lib', 'text'   , tbllib,  #tbllib, 256,  'align_left')
fhOutputResultSetColumn('nam', 'text'   , tblnam,  #tblnam, 128,  'align_left')
fhOutputResultSetColumn('typ', 'text'   , tbltyp,  #tbltyp,  48,  'align_left')
fhOutputResultSetColumn('adr', 'text'   , tblhex,  #tblhex,  48,  'align_mid')
fhOutputResultSetColumn('dec', 'text'   , tbldec,  #tbldec,  12,  'align_right')

return
FH V.6.2.7 Win 10 64 bit
User avatar
tatewise
Megastar
Posts: 28341
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: recursion, knowing where I am in a nested table, and my own movenext()

Post by tatewise »

Sorry Ron, but I really don't understand the purpose of the script.
There are very few comments in the script describing what each part is doing.
Please explain in a bit more detail what you are trying to achieve and exactly why you think it is not working as expected.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
User avatar
Ron Melby
Megastar
Posts: 917
Joined: 15 Nov 2016 15:40
Family Historian: V6.2

Re: recursion, knowing where I am in a nested table, and my own movenext()

Post by Ron Melby »

I have tried to add some comments to this new code... but am very lost myself.
I am reading _G
if there is a table or function in _G I get a hex handle and its package. added to name this makes path
in the case of known fh functions I add fh. to the path
i.e debug.getmetatable, table.foreachi, up.callbacks.MAP_CB.[1]MAP_CB, up.BOX.parent.constructor, &c.

if you run the code, (it updates nor deletes anything, just reports, you will get an idea of what goes wrong,
I will assume your installation is somewhat different, but very close to this.

clearly my seminal problem is in recursion of nested tables. additionally, I have a problem in ignoring tables I dont want to tear apart, and empty tables.

42 fh.fhHasChildItem fhHasChildItem function
things are working well to line 42
then I obvs. run into _G which I want to skip over entirely but the artifact is left
43 _G.fh.fhSetValueAsLink fhSetValueAsLink function (so i have hit a table)
55 fh.fhNewItemPtr fhNewItemPtr function (back to functions things run ok for a couple entries)
56 _G.coroutine.resume resume function (I have hit a table coroutine. Artifact _G rears its ugly head)
186 _G.iup.callbacks.MAP_CB.[1]MAP_CB [1]MAP_CB function
my next problem surfaces, the iup.callbacks nesting works correctly but ...
187 ElementPropertiesDialog ElementPropertiesDialog function
I have lost my nesting level because I should still be in table iup. id est: iup.ElementPropertiesDialog &c.
222 _G.iup.BOX.parent.callback.hide hide function
223 setAttributes setAttributes function
same as above lost nesting again
253 _G.iup.BOX.WIDGET.TREEREFTABLE.TREEREFTABLE TREEREFTABLE table
here is my next problem. TREEREFTABLE is an empty table, I want to log it and move on, but now its a permanent artifact.
254 _G.iup.BOX.WIDGET.TREEREFTABLE.isSysXkey isSysXkey function
331 _G.iup.BOX.WIDGET.TREEREFTABLE._M.user user function
at this point I have hit another table I want to gloss over
iup._M it now adds to artifacts.
439 _G.iup.BOX.WIDGET.TREEREFTABLE.package.preload.loaded.string.debug.getupvalue getupvalue function
along about here, I am till the end of the file, being innundated with artifact from ignored recursions, and have completley lost the plot.

Code: Select all

-- require '_STD_SYS'
-- require '_STD_HDR'
-- require '##_STD_HEX' -- still debugging

--[[
scan all tables in _G for functions
try to build a reverse mapping of all functions pointers
string.sub() should not just be sub(), but the full name
]]

local _oblt = {}  -- object built
local _Gtbl = {}  -- enumeration of _G
local _gid  = 1   -- index of _Gtbl table

local _elvl = {}  -- table depth, i.e. iup.callback.
local _etbl = 0   -- table depth counter

local _dft  = {addr = '*NULL', dec = 0} -- hex and dec of function or table

function rslv_G(_tbl, __nam)  -- resolve _G

  function get_addr(_gatyp)
    -- gets hex address of function or table
    local _ga = _dft
    if type(_gatyp) == 'function'
    or type(_gatyp) == 'table' then
      _gatyp = (tostring(_gatyp))
      _gatyp = _gatyp:sub(_gatyp:find(' ') + 1):match('(%x*)')
      _ga.addr = _gatyp
      --  _ga.dec = hex_to_dec(_gatyp):gsub('%.0', '') or 0
    end
    return _ga
  end

  function wrt_obj(_wtyp, _wnam, _wpkg, _wga)

    -- for debugging certain records that are known to be incorrect, your numbers will be different
    for _, _dbgID in pairs({16, 189, 248, 255, 405}) do
      if _gid == _dbgID then
        _BREAKPOINT = true
        break
      end
    end
    -- this writes an entry in _Gtbl by type.
    wrt_type[(type(_wtyp))](_wtyp, _wnam, _wpkg, _wga)
  end

  wrt_type =
  {
    ['boolean'] = function(_otyp, _onam, _opkg, _oga)
      print('in boolean')

      local bool
      if _otyp then
        bool =  'true'
      else
        bool =  'false'
      end

      _Gtbl[_gid] =
      {
        id   = _gid,
        addr = _oga.addr,
        dec  = _oga.dec,
        typ  = type(_otyp),
        pth  = string.format('%s%s', _opkg, _onam),
        pkg  = _opkg,
        name = _onam,
      }
      _gid = _gid + 1
    end, -- fn boolean

    ['thread'] = function(_otyp, _onam, _opkg, _oga)
      print('in thread')

      local _a, _b = pcall(string.dump, _otyp)

      _Gtbl[_gid] =
      {
        id   = _gid,
        addr = _oga.addr,
        dec  = _oga.dec,
        typ  = type(_otyp),
        pth  = string.format('%s%s', _opkg, _onam),
        pkg  = _opkg,
        name = _onam,
      }
      _gid = _gid + 1
    end, -- fn thread

    ['userdata'] = function(_otyp, _onam, _opkg, _oga)
      print('in userdata')

      local _a, _b = pcall(string.dump, _otyp)

      _Gtbl[_gid] =
      {
        id   = _gid,
        addr = _oga.addr,
        dec  = _oga.dec,
        typ  = type(_otyp),
        pth  = string.format('%s%s', _opkg, _onam),
        pkg  = _opkg,
        name = _onam,
      }
      _gid = _gid + 1
    end, -- userdata (generally fhptr)

    ['nil'] = function(_otyp, _onam, _opkg, _oga)
      print('in nil')

      _Gtbl[_gid] =
      {
        id   = _gid,
        addr = _oga.addr,
        dec  = _oga.dec,
        typ  = type(_otyp),
        pth  = string.format('%s%s', _opkg, _onam),
        pkg  = _opkg,
        name = _onam,
      }
      _gid = _gid + 1
    end, -- fn nil

    ['string'] = function(_otyp, _onam, _opkg, _oga)

      _Gtbl[_gid] =
      {
        id   = _gid,
        addr = _oga.addr,
        dec  = _oga.dec,
        typ  = type(_otyp),
        pth  = string.format('%s%s', _opkg, _onam),
        pkg  = _opkg,
        name = _onam,
        val  = _otyp
      }
      _gid = _gid + 1
    end, -- fn string

    ['number'] = function(_otyp, _onam, _opkg, _oga)

      local  _val  = ('%.1f'):format(_otyp)

      _Gtbl[_gid] =
      {
        id   = _gid,
        addr = _oga.addr,
        dec  = _oga.dec,
        typ  = type(_otyp),
        pth  = string.format('%s%s', _opkg, _onam),
        pkg  = _opkg,
        name = _onam,
        val  = _val:gsub('%.0', ''),
      }
      _gid = _gid + 1
    end, -- fn number

    ['function'] = function(_otyp, _onam, _opkg, _oga)

      local _opkg = _opkg
      local _fh = string.sub(_onam, 1, 2)
      if  _fh == 'fh' then
        _opkg = ('%s%s.'):format(_opkg, _fh)
      end

      _Gtbl[_gid] =
      {
        id   = _gid,
        addr = _oga.addr,
        dec  = _oga.dec,
        typ  = type(_otyp),
        pth  = string.format('%s%s', _opkg, _onam),
        pkg  = _opkg,
        name = _onam,
      }
      _gid = _gid + 1
    end, -- fn function

    ['table'] = function(_ityp, _inam, _ipkg, _iga)
      local ityp = type(_ityp)
      local inam = _inam
      local ipkg = _ipkg

      if _oblt[_iga.addr] then -- object built by address
--        next(_ityp) -- movenextitem() doesnt work
        return  -- this prevents rereading already done but doesnt work
      end
      _oblt[_iga.addr] = true -- object is built

      -- these are tables I do not want to write, but will not be picked up above, because they have different addressses
      -- at some point I want to register the table in the file and then gloss over it, much like: 	_G => (table .110) (see previous)
      for _, _ignore in pairs({'_G.', '_G._G.', '_Gtbl.', 'iup._M.', 'package.loaded'}) do
        if _wpkg == _ignore then
          _epkg = ''
          _ityp = ''
          --  next(_ityp) -- movenextitem() doesnt work i.e. if I encounter _G_G move past the entire table and get next entry in _tbl. pretend it doesnt exist.
          return
        end
      end

      for _enam, _etyp in pairs(_ityp) do
        local etyp = type(_etyp)

        if etyp == 'table' then

          _epkg = ''

          --[[
          this builds the package path by nesting level
          i.e.
          iup.
          then iup.callbacks
          then iup.callbacks.action
          then iup.callbacks.action.epander
          as I descend into the nest.
          ]]

          _elvl[#_elvl + 1] = ('%s.'):format(_enam)
          for  _ix, _ in ipairs(_elvl) do
            -- glue level
            _epkg = ('%s%s'):format(_epkg or '', _elvl[_ix])
          end
          _etbl = _etbl + 1  -- add to table nesting level


          -- type(_etyp) == 'table'
          local _ega = get_addr(_etyp)

          if next(_etyp) then
            -- if the table is not empty, read it and write it.
            wrt_obj(_etyp, _enam, _epkg or '', _ega)

          else
            -- should be an empty table, for example iup. TREEREFTABLE => (table .0) but it does not work and clear path back to iup.
            _Gtbl[_gid] =
            {
              id   = _gid,
              addr = _ega.addr,
              dec  = _ega.dec,
              typ  = type(_etyp),
              pth  = string.format('%s%s', _epkg, _enam),
              pkg  = _epkg,
              name = _enam,
            }
            _gid = _gid + 1
          end

        elseif etyp == 'function' then

          -- type(_etyp) == 'function'
          local _ega = get_addr(_etyp)
          local _enam = _enam
          if type(_enam) == 'number' then
            _enam = (('[%s]%s'):format(_enam, _inam))
          end

          wrt_obj(_etyp, _enam, _epkg or '', _ega)

          -- expansion later
          -- elseif _etyp == 'userdata' then  io. has them i.e. stdin
          -- elseif _etyp == 'string' then  'global constant'
          -- elseif _etyp == 'number' then  'global constant'
          -- elseif _etyp == 'boolean' then
          -- elseif _etyp == 'thread' then
          -- elseif _etyp == 'nil' then
        end
        -- type(_etyp) == 'table' 'function'
      end
      -- out of table
      -- how to manage nested tables correctly ??
      _epkg = '' -- clear (ie. iup.callbacks.)
      _etbl = _etbl - 1  -- see 207 this should be my nesting level
      if _etbl <=0 then
        _elvl = {} -- if I am out of the nested table, clear the nesting array
      else
        _elvl[#_elvl] = nil -- i.e. if iup.callbacks.ACTION.expander clear expander to prepare for next table entry -- canvas.
      end
      -- this doesnt work, and therefore commented out
      -- for _e = _etbl, #_elvl do
      --   _elvl[_etbl] = nil
      --   _elvl[_e] = nil
      -- end

      -- fn table
    end,
  }


--  strdbg() landmark for debugging
-- run the table.

  local _t = tostring(_tbl)  -- get hex address cant get name
  _t = _t:sub(_t:find(' ') + 1):match('(%x*)') -- get only hex
  local _st = _dft -- put in address table
  _st.addr = _t

  -- if you debug here, and examine _tbl(_G,_G) you will sort of see what I want my table to look like, but I want to enumerate it a bit further.
  wrt_obj(_tbl, _t, '', _st)  -- get name and addr  (put in _Gtbl 1 or ?_

end -- fn rslv_G

function main()
-- *NULL: landmark only
end

rslv_G(_G._G, '')


-- print results
local tblnbr = {}
local tbllib = {}
local tblnam = {}
local tbltyp = {}
local tblhex = {}
local tbldec = {}

for k, _ in pairs(_Gtbl) do
  local g = _Gtbl[k]
  table.insert(tblnbr, g.id)
  table.insert(tbllib, g.pth)
  table.insert(tblnam, g.name)
  table.insert(tbltyp, g.typ)
  table.insert(tblhex, g.addr)
  table.insert(tbldec, g.dec)
end

fhOutputResultSetColumn('id ', 'integer', tblnbr,  #tblnbr,  24,  'align_right', 1)
fhOutputResultSetColumn('lib', 'text'   , tbllib,  #tbllib, 256,  'align_left')
fhOutputResultSetColumn('nam', 'text'   , tblnam,  #tblnam, 128,  'align_left')
fhOutputResultSetColumn('typ', 'text'   , tbltyp,  #tbltyp,  48,  'align_left')
-- fhOutputResultSetColumn('adr', 'text'   , tblhex,  #tblhex,  48,  'align_mid')
-- fhOutputResultSetColumn('dec', 'text'   , tbldec,  #tbldec,  12,  'align_right')

return
FH V.6.2.7 Win 10 64 bit
User avatar
tatewise
Megastar
Posts: 28341
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: recursion, knowing where I am in a nested table, and my own movenext()

Post by tatewise »

OK, my first question is: If you are analyzing a table the first thing I would expect to see somewhere is
for j, k in pairs ( table ) do

So the heart of the enumeration process should be something like this:

Code: Select all

local _Gtbl = {}  -- enumeration of _G
local depth = 0  -- Depth of nested table recursion
local names = {} -- Names of resolved tables to prevent infinite recursion 
function rslv_G(_tbl, __nam)
  depth = depth + 1
  names[__nam] = true
  for n, t in pairs (_tbl) do  -- Find name and type of each entry
    local type = type(t)
    table.insert(_Gtbl,{ depth=depth; name=n; type=type; })  -- This effectively enumerates the entries
    if type == "table" and not names[n] then
      rslv_G(t, n)  -- If nested table not already resolved then recursively analyse
    elseif type == "function" then
      -- Enumerate a function with name in n
    end
  end
  depth = depth - 1
end

rslv_G(_G, '_G')
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
User avatar
Ron Melby
Megastar
Posts: 917
Joined: 15 Nov 2016 15:40
Family Historian: V6.2

Re: recursion, knowing where I am in a nested table, and my own movenext()

Post by Ron Melby »

something on that order...

in the program code last sent:

line 307 calls function rslv_G line 20

which has functions and starts execution at line 299, calling line 35, which picks function at line 180, remembering a table is coming in, and starting line 202 is the reader. which looks something like your sample. (remember, it doesnt always have to be _G or _G,_G .... but if I can get that one right, its a pretty long ways to bulletproof.
FH V.6.2.7 Win 10 64 bit
User avatar
tatewise
Megastar
Posts: 28341
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: recursion, knowing where I am in a nested table, and my own movenext()

Post by tatewise »

Maybe, but it looks unnecessarily complex and 335 lines long.
I've updated my script to save and show the _Gtbl which outputs much the same as your script but with 47 lines.
Any final refinements I will leave to you.

Code: Select all

local _Gtbl = {}  -- enumeration of _G
local paths = {}  -- holds nested table names
local depth = 0  -- Depth of nested table recursion
local names = {} -- Names of resolved tables to prevent infinite recursion 
function rslv_G(_tbl, __nam)
  depth = depth + 1
  names[__nam] = true
  paths[depth] = __nam
  for n, t in pairs (_tbl) do  -- Find name and type of each entry
    local type = type(t)
    local path = table.concat(paths,".") .. "." .. n
    path = path:gsub('^_G.fh','fh.fh')
    table.insert(_Gtbl,{ depth=depth; path=path; name=n; type=type; })  -- This saves the entries
    if type == "table" and not names[n] then
      rslv_G(t, n)  -- If nested table not already resolved then recursively analyse
    end
  end
  paths[depth] = nil
  depth = depth - 1
end

rslv_G(_G, '_G')

-- print results
local tblnbr = {}
local tbllev = {}
local tblpth = {}
local tblnam = {}
local tbltyp = {}

for k, g in ipairs(_Gtbl) do
   table.insert(tblnbr, k)
   table.insert(tbllev, g.depth)
   table.insert(tblpth, g.path)
   table.insert(tblnam, g.name)
   table.insert(tbltyp, g.type)
end

fhOutputResultSetColumn('id ', 'integer', tblnbr,  #tblnbr,  24,  'align_right', 1)
fhOutputResultSetColumn('lev', 'text'   , tbllev,  #tbllev, 24,  'align_mid')
fhOutputResultSetColumn('path', 'text'   , tblpth,  #tblpth, 128,  'align_left')
fhOutputResultSetColumn('name', 'text'   , tblnam,  #tblnam, 128,  'align_left')
fhOutputResultSetColumn('type', 'text'   , tbltyp,  #tbltyp,  48,  'align_left')
-- fhOutputResultSetColumn('adr', 'text'   , tblhex,  #tblhex,  48,  'align_mid')
-- fhOutputResultSetColumn('dec', 'text'   , tbldec,  #tbldec,  12,  'align_right')

return
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
User avatar
Ron Melby
Megastar
Posts: 917
Joined: 15 Nov 2016 15:40
Family Historian: V6.2

Re: recursion, knowing where I am in a nested table, and my own movenext()

Post by Ron Melby »

it is indeed a great deal simpler, I will have to run this thru debugging about 10,000 times. since I do have to do a little more info on each type. Thanks.
FH V.6.2.7 Win 10 64 bit
User avatar
Ron Melby
Megastar
Posts: 917
Joined: 15 Nov 2016 15:40
Family Historian: V6.2

Re: recursion, knowing where I am in a nested table, and my own movenext()

Post by Ron Melby »

such as it is here it is, much shorter and simpler. could be shorter, I am unsure if I will use the hex value, its going to be part of my snazzy debugger, and I kept types in there because its how I do my table serializer, and now I can simplify that somewhat, so I have a working pattern. I have tried a few things to test it, but now I need a monster program to test it in, and I expect Mike, that MLF is it, do you want to tell me what line to call it on so that I can debug this table writer?

I would add and remove a require (easy where to put that) insert one line and add and remove that (not so easy where to put that where full program is built...

Code: Select all

local _Gtbl = {} -- enumeration of _G
local pkg   = ''
local pth   = {} -- holds nested table
local lvl   = 0  -- Depth of nested table recursion


local ignore =
{
  ['_G'] = '',
  ['Gtbl'] = '',
--  ['loaded'] = '',
--  ['loaders'] = '',
  ['_M'] = '',
} -- Names of resolved tables to prevent infinite recursion

local function wrt_entry(_pkg, _nam, _lvl, _typ, _hex)

  local  wrt_type =
  {
    ['userdata'] = function(_pkg, _nam, _lvl, _typ, _hex)

      _Gtbl[#_Gtbl + 1] =
      {
        pkg  = _pkg,
        name = _nam,
        lvl =  _lvl,
        typ  = _typ,
        hex = _hex,
      }
    end, -- userdata (generally fhptr)

    ['constant'] = function(_pkg, _nam, _lvl, _typ, _hex)

      _Gtbl[#_Gtbl + 1] =
      {
        pkg  = _pkg,
        name = _nam,
        lvl =  _lvl,
        typ  = _typ,
        hex = _hex,
      }
    end, -- fn string

    ['function'] = function(_pkg, _nam, _lvl, _typ, _hex)

      local _pkg = _pkg
      local _fh = string.sub(_nam, 1, 2)
      if  _fh == 'fh' then
        _pkg = ('%s%s.'):format(_pkg, _fh)
      end

      _Gtbl[#_Gtbl + 1] =
      {
        pkg  = _pkg,
        name = _nam,
        lvl  =  _lvl,
        typ  = _typ,
        hex  = _hex,
      }
    end, -- fn function

    ['table'] = function(_pkg, _nam, _lvl, _typ, _hex)
      _Gtbl[#_Gtbl + 1] =
      {
        pkg  = _pkg,
        name = _nam,
        lvl =  _lvl,
        typ  = _typ,
        hex = _hex,
      }
    end,
  }

  wrt_type[_typ](_pkg, _nam, _lvl, _typ, _hex)
end

function unravel(_tbl, _inam)

  local function hex_svc(xtyp)
    -- gets hex hexess of function or table
    local _xtyp = ''
    if type(xtyp) == 'function'
    or type(xtyp) == 'table' then
      _xtyp   = (tostring(xtyp))
      _xtyp   = _xtyp:sub(_xtyp:find(' ') + 1):match('(%x*)')
    end
    return _xtyp
  end

  for nam, ntyp in pairs (_tbl) do  -- Find name and type of each entry
    local otyp = type(ntyp)
    local _hex = hex_svc(nam)

    if otyp ~= 'table' then

      if otyp == 'number'
      or otyp == 'string' then
        otyp = 'constant'
      elseif otyp == 'function' then
        local fnam = nam
        if type(fnam) == 'number' then
          nam = (('[%s]%s'):format(fnam, _inam))
        end
      end

      wrt_entry(pkg, nam, lvl, otyp, _hex)
    end

    if otyp == 'table' then
      if ignore[nam] then
        wrt_entry(pkg, nam, lvl, otyp, _hex)
        -- If nested table not resolved recursively analyse
      elseif  not ignore[nam] then
        lvl      = lvl + 1
        pth[lvl] = ('%s.'):format(nam)
        pkg      = ''
        for  _ix, _ in ipairs(pth) do
          -- glue level
          pkg = ('%s%s'):format(pkg, pth[_ix])
        end
        ignore[nam] = _hex
        unravel(ntyp, nam)
      end -- ign
    end -- tbl
  end -- for
  pth[lvl] = nil
  lvl      = lvl - 1
  pkg      = ''
  for _, p in pairs(pth) do
    pkg = ('%s%s'):format(pkg or '', p or '')
  end
end

unravel(_G, '')
-- return _Gtbl

-- print results
local tblnbr = {}
local tbllvl = {}
local tblpkg = {}
local tblnam = {}
local tbltyp = {}
local tblhex = {}

for k, g in ipairs(_Gtbl) do
  table.insert(tblnbr, k)
  table.insert(tbllvl, g.lvl)
  table.insert(tblpkg, g.pkg)
  table.insert(tblnam, g.name)
  table.insert(tbltyp, g.typ)
  table.insert(tblhex, g.hex)
end

fhOutputResultSetColumn('id ', 'integer', tblnbr,  #tblnbr,  24,  'align_right')
fhOutputResultSetColumn('lvl', 'integer', tbllvl,  #tbllvl,  24,  'align_right')
fhOutputResultSetColumn('pkg', 'text'   , tblpkg,  #tblpkg, 128,  'align_left', 1)
fhOutputResultSetColumn('name', 'text'  , tblnam,  #tblnam,  96,  'align_left', 2)
fhOutputResultSetColumn('type', 'text'  , tbltyp,  #tbltyp,  48,  'align_left')
fhOutputResultSetColumn('hex',  'text'  , tblhex,  #tblhex,  48,  'align_mid')

return
FH V.6.2.7 Win 10 64 bit
User avatar
tatewise
Megastar
Posts: 28341
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: recursion, knowing where I am in a nested table, and my own movenext()

Post by tatewise »

I suspect you are mistaking complex operations for complex table structures.
I don't think any of my Plugins use complex tables, and certainly not as complex as the _G table.
However, I will have a look and see if I can find anything complex.
The table structures you have posted here in the past are far more complex than mine.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
User avatar
Ron Melby
Megastar
Posts: 917
Joined: 15 Nov 2016 15:40
Family Historian: V6.2

Re: recursion, knowing where I am in a nested table, and my own movenext()

Post by Ron Melby »

not only tables, but functions, constants, userdata and so on, as well as many more fh functions, then I have to see what to do with fh. env functions like you have. I said debugger earlier, but really it will be a profiler, with some debug functions. I have simply learned some tricks for my table serializer is all, so I can pack my table up a little more without so much wasted byte space in blanks and so on. My serializer operates a bit different than chill code, and so when I look for my api-key in _STD_GPS (a require which adds information to lat long entries in place and address, I have to return your table as [1] one thing, do you save functions in tables you want serialized? I have never done that successfully.
FH V.6.2.7 Win 10 64 bit
User avatar
tatewise
Megastar
Posts: 28341
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: recursion, knowing where I am in a nested table, and my own movenext()

Post by tatewise »

You only mentioned wanting to "debug this table writer" so I assumed you only wanted complex tables!

What exactly do you want your profiler to operate on?

I rarely save functions in tables and why would I want them serialized? Please explain.

From memory, I think 'Show Project Statistics' uses functions in tables to help analyse tags.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
User avatar
Ron Melby
Megastar
Posts: 917
Joined: 15 Nov 2016 15:40
Family Historian: V6.2

Post by Ron Melby »

I dont know why one would save a function, but while reviewing every serializer I could find to see if one would meet my needs, (presently pure lua) I saw many that claimed to do just that., and tried them out... #epicfail. Saw claims of nested deeply as you want, and #epicfail. (even serpent -- by the zerobrane guy PaulK) who claimed serialization and had some neat bells and whistles on his, when I wanted to save an array of tables and arrays i.e. [1] = ([epic] ={ 'e, p, i, c,) , [fail] = {fail, fail, fail fail}, ([1] = f, [2] = a, [3] = i, 4 = l}, }, [2] = .... n(repeat) decided that the index's 1 and 2 for the outer table was not necessary, because they werent sequential as in a real array (the indexes were FAM(fhgetrcdid) so, I thought they were kinda necessary. in any case, for a profiler I think memory and time spent in functional areas are important, and so this will catagorize them. I will run against _G in a program and debug, then since I cannot do a line by line (and not sure lua could handle that strain) I can somewhat classify ancillary functions, and separate some from others. at some point I will have a complete fh. then, and I can see no other way at the moment, I can hand code lua. (such as gcinfo, and assert) and that would be a perm table, and augment it with _G, and so I hope to come up with if it aint _G itself without a path, then it must be user program, function, constant, table, userdata, etc.

However, as I have said before, it is a great deal like my table serializer, and the exercise is also helpful for that, at present rather than blow up as chillcode does on some things, I will write a comment where the function save would be, where the userdata would be, and where the thread would be and so on, because actively, in your running program, you can put all those things in a table.
FH V.6.2.7 Win 10 64 bit
User avatar
tatewise
Megastar
Posts: 28341
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: recursion, knowing where I am in a nested table, and my own movenext()

Post by tatewise »

Sorry, that is a lot of words that mean nothing to me and does not explain what benefits serializing offers to me.

Here is an example from about line 3950 in 'Show Project Statistics' of how I use a table of functions.
For each SOUR subsidiary tag such as DATA or NOTE2 or OBJE2 the desired function is chosen from the dicWhat table.
The only caveat is that each such function must accept a compatible set of parameters.

Code: Select all

	local dicWhat =		-- Functions invoked by FoundCitation() for each tag
	{			-- These functions are defined in the script above (not shown here)
		DATA  = doDataDate ;
		NOTE  = FindCitations ;	
		NOTE2 = FindCitations ;
		OBJE  = UpdateMedia  ;
		OBJE2 = UpdateLMO ;
	}
	local function FoundCitation(ptrOld,tblGrd,strRow)	-- Process any Citation (SOUR)
		local ptrRef = fhNewItemPtr()			-- Loop through each tag
		ptrRef:MoveToFirstChildItem(ptrOld)
		while ptrRef:IsNotNull() do
			local action = dicWhat[fhGetTag(ptrRef)] or CheckIsUDF
			action(ptrRef:Clone(),tblGrd,strRow) 	-- Invoke function for each tag
			ptrRef:MoveNext()
		end
		UpdateCount(tblGrd,strRow,"Cites")		-- Count each Citation
	end -- local function FoundCitation
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
User avatar
Ron Melby
Megastar
Posts: 917
Joined: 15 Nov 2016 15:40
Family Historian: V6.2

Re: recursion, knowing where I am in a nested table, and my own movenext()

Post by Ron Melby »

serialize table = save table to file. deserialize (or materialize) is put the table back in your program.
FH V.6.2.7 Win 10 64 bit
Post Reply