* finding and parsing an attribute by TYPE

For users to report plugin bugs and request plugin enhancements; and for authors to test new/new versions of plugins, and to discuss plugin development (in the Programming Technicalities sub-forum). If you want advice on choosing or using a plugin, please ask in General Usage or an appropriate sub-forum.
Post Reply
User avatar
Ron Melby
Megastar
Posts: 917
Joined: 15 Nov 2016 15:40
Family Historian: V6.2

finding and parsing an attribute by TYPE

Post by Ron Melby »

--main
local ptrINDI = fhNewItemPtr()
local strTag
local strError
ptrINDI:MoveToFirstRecord("INDI")
while ptrINDI:IsNotNull() do
strTag,strError = fhGetFactTag("MILT", "Attribute", "INDI", false)
if strError then
fhMessageBox("Tag: " .. strTag)
else
fhMessageBox("Error: " .. strError)
end
ptrINDI:MoveNext()
end
This is what I have, but it needs much work...

what it looks for is ATTR-MILT.

I want to look for the TYPE MILT, and then parse the ATTR

here are a couple examples from the file:

1 _ATTR Civil War, US Army, PVT
2 TYPE MILT
2 NOTE Company G, 24th MI Volunteer Infantry

1 _ATTR Civil War, US Army
2 TYPE MILT
2 NOTE Civil War veteran, Enlisted August 15, 1861 Detroit. Served in Company E, 16th Engineer Regiment MI 7 Sept 1851.
3 CONT
3 CONT Served Regiment R. C. on July 1, 1863.

1 _ATTR US Army, SP4
2 TYPE MILT
2 DATE FROM 3 JAN 1956 TO 28 NOV 1958
2 PLAC , , , AK, USA

1 _ATTR WWII, US Air Force, SGT
2 TYPE MILT
2 DATE ABT 1941

It displays fine in the program and I want to report these for two reasons, to insure that the _ATTR in MILT is comma separated and ordinate so that I can report on 1,2,3,n (*all) as separate values, or 2,3 or variations like that, and to make a pdf to send to others so we are on the same page with military.
Let the problem statement for now be, how do I find TYPE MILT, and then go back to the ATTR record and read the "Gob" to parse it?
FH V.6.2.7 Win 10 64 bit
User avatar
tatewise
Megastar
Posts: 28333
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: finding and parsing an attribute by TYPE

Post by tatewise »

Ron, you are getting too involved with the GEDCOM tag structure.
A custom Attribute (_ATTR & TYPE) is treated as a single Fact the same as a standard Attribute (OCCU).

Contrary to your statement, fhGetFactTag(...) does not look for _ATTR-MILT facts associated with an Individual.
It composes the tag _ATTR-MILT from the parameters supplied so you know what to look for.
e.g. fhGetFactTag("Miltary", "Attribute", "INDI", false)

What you need instead is something like:

Code: Select all

	local ptrMILT = fhNewItemPtr()
	ptrMILT:MoveTo(ptrINDI,"~._ATTR-MILT")
	while ptrMILT:IsNotNull() do
		local strMILT = fhGetValueAsText(ptrMILT)
		ptrMILT:MoveNext("SAME_TAG")
	end
where strMILT will hold the Attribute value such as "Civil War, US Army, PVT" for each fact in turn

To parse that string you need to use the plugins:code_snippets:split_a_line_using_a_separator|> Split a Line using a Separator (code snippet).

BTW: Your strError test and fhMessageBox(...) alternatives are the wrong way round, and should be:

Code: Select all

	if strError then
		fhMessageBox("Error: " .. strError)
	else
		fhMessageBox("Tag: " .. strTag)
	end
i.e. When strError holds an error text it is true, and when nil it is false.

If you wanted, you use that strTag value in the earlier code:

Code: Select all

	ptrMILT:MoveTo(ptrINDI,"~." .. strTag)
Further to all that, you could probably do what you want with a Fact Query involving the =TextPart(...) function.
The Result Set can be saved as a PDF.

To achieve the same with a Plugin will require it to produce a Result Set.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
User avatar
Ron Melby
Megastar
Posts: 917
Joined: 15 Nov 2016 15:40
Family Historian: V6.2

Re: finding and parsing an attribute by TYPE

Post by Ron Melby »

OK, makes sense, I have done like that before, way back when I was needing to fix NICKs. (the actual string had me confused)
regards the parsing some of the comma cutting I can do, did that in my FaG table where I was looking for dupes, its just that I am unclear on at least two issues, one semi solved.
1) from TYPE MILT will I have to get Parent Pointer and read down the logical GOB? much like the getParentItem getChildItem we did when I was moving and deleting sources for fixing names (I realize tatewise, you do not keep all this in your head, you read fix and work on many other things whereas I lock my mind into what I have done before and sort of pick up from that bookmark)
2) I come from an explicit read language, that makes sense to me. so how would I go from this psuedo code to LUA inside FH?

Code: Select all

INDIptr:MoveToFirstRecord('INDI')
  while INDIptr:IsNotNull() do
      while records inside not null
          move to First record inside this INDI
          while records inside not null
              read inside
              do whatever comparison and computation 
              if found what I am looking for
                  next INDI 
              end
        end
   end
    move to next record inside this INDI
    ptr:MoveNextSpecial()
 end
end
FH V.6.2.7 Win 10 64 bit
User avatar
Jane
Site Admin
Posts: 8507
Joined: 01 Nov 2002 15:00
Family Historian: V7
Location: Somerset, England
Contact:

Re: finding and parsing an attribute by TYPE

Post by Jane »

Ron, you might find it useful to make use of the code snippets or the help examples in the Plugin help, also when I wrote a plugin the other day I put together some notes on my blog you might find helpful.

Also try looking at the Data Reference tool to work out the relationship between fields, when you see a + and move down you using :MoveToFirstChild() to get there

My Blog is https://www.taubman.org.uk/family/wp/20 ... historian/

If it does not help, I am sure Mike will jump in.
Jane
My Family History : My Photography "Knowledge is knowing that a tomato is a fruit. Wisdom is not putting it in a fruit salad."
User avatar
tatewise
Megastar
Posts: 28333
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: finding and parsing an attribute by TYPE

Post by tatewise »

May I refine your pseudo code:

Code: Select all

Move To First Individual Record
while such a Record exists do
	Move to first Military Attribute
	while such an Attribute exists do
		read its value
		do whatever comparison and computation
		Move to Next Military Attribute
	end
	Move to Next Individual record
end
Putting your original INDI loop and my Attribute loop together we get two nested loops.
The outer while ptrINDI:IsNotNull() do loop finds each Individual record.
The inner while ptrMILT:IsNotNull() do loop finds each Military Attribute per Individual.
There is no need for any Parent or Child navigation.
You don't need MoveNextSpecial() if only looking for specific facts with the SAME_TAG.

Code: Select all

local ptrINDI = fhNewItemPtr()
ptrINDI:MoveToFirstRecord("INDI")
while ptrINDI:IsNotNull() do				-- Process each Individual record
	local ptrATTR = fhNewItemPtr()
	ptrATTR:MoveTo(ptrINDI,"~._ATTR-MILT")		-- Would use "~.OCCU" for Occupation Attribute
	while ptrATTR:IsNotNull() do						-- Process each Military Attribute 
		local strATTR = fhGetValueAsText(ptrATTR)		-- Get value of Attribute
		-- do whatever needs to be done, e.g. Split a Line using a Separator (code snippet).
		ptrATTR:MoveNext("SAME_TAG")		-- Move to next Military Attribute
	end
	ptrINDI:MoveNext()						-- Move to next Individual record
end
Note how inside FH and Lua the GEDCOM lines 1 _ATTR and 2 TYPE in are merged into one Attribute that effectively is one line:
1 _ATTR-MILT Civil War, US Army, PVT
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
User avatar
Ron Melby
Megastar
Posts: 917
Joined: 15 Nov 2016 15:40
Family Historian: V6.2

Re: finding and parsing an attribute by TYPE

Post by Ron Melby »

Jane and Mike, that sort of stuff is going to get me around 80% home. It is simpler than I thought....Thank you both ever so much.

Thanks, that sort of clarifies it, I didnt know how to get there, and am still fuzzy on where the pointer lays, remembering that I need to check for records within the attribute of other types DATE, PLAC, NOTE and so on. I can output to tables and make a snazzy report, but since it will be more like a dictionary entry when I am done, I will probably write out a text file as I develop this further. Note that when I say gob it is a shorthand for describing all the records in the set comprising that event, attribute or whatever. that is from the 1st level tag to the nth level tag in order until the next 1st level tag. I have the problem that I can only work on these things in fits and starts, and for short periods, and that is not conducive to memorizing certain things. I have no idea how _ATTR-MILT could be discerned from a perusal of the documentation, I often wonder where the index of functions is, and I by dint of confusion have saved that somehow to my searches in help, so that issue is solved, I often wonder is there a builtin that may give me some relief for this and that situation, and I know I have stumbled across the list of builtins but cannot find them in the help. I would save such an index to my saved searches in help as well, if I can figure out how I did it.

Well, carry on, wot?
FH V.6.2.7 Win 10 64 bit
User avatar
Jane
Site Admin
Posts: 8507
Joined: 01 Nov 2002 15:00
Family Historian: V7
Location: Somerset, England
Contact:

Re: finding and parsing an attribute by TYPE

Post by Jane »

I have no idea how _ATTR-MILT could be discerned from a perusal of the documentation
As I said use the built-in Data Reference, the best one is on the Query Window, as you add them in it provides the full data reference and at the bottom of the list a description of the field.

All Lua functions for FH are listed in the How To Write a Plugin from the More option on the Plugin Window.
Jane
My Family History : My Photography "Knowledge is knowing that a tomato is a fruit. Wisdom is not putting it in a fruit salad."
User avatar
tatewise
Megastar
Posts: 28333
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: finding and parsing an attribute by TYPE

Post by tatewise »

Ron the built-in features you are looking for do exist. They assume you only know the name of the fact as shown inside FH and provide a Data Reference that is akin to GEDCOM. However, those Data References are not always identical to the Tags you will see in the GEDCOM file itself, and that may be your confusion. Custom Attributes are such an example, where your Data Ref is _ATTR-MILT but in GEDCOM it involves the _ATTR and TYPE tags. In this case, knowing the GEDCOM fine details can hinder understanding how FH and Lua work.

1) Data Reference Assistant
Jane's advice to use the built-in Data Reference Assistant (DRA) in FH to discover internal fact tags is perfect.
In the Query Window the DRA is more visible than elsewhere, and is on the Columns tab in the Fields pane on the left.
For Individual facts, use an Individual Query, then click on [+] Attributes to expand the list to choose your Military attribute however it is named inside FH, and add it to the Columns to see its Data Ref tag; then delete it to avoid disrupting the Query.
See how_to:understanding_data_references|> Understanding Data References for much more advice.

2) fhGetFactTag(...) Function
The built-in feature in Lua is the fhGetFactTag(...) function you started using in your original posting:
strTag, strErr = fhGetFactTag("Military", "Attribute", "INDI", false)
where Military needs to be whatever name is shown inside FH for that fact.
In other words it translates from names you know Military, Attribute, INDI to a GEDCOM style tag _ATTR-MILT you didn't know.

BTW: What name is shown inside FH for this MILT attribute?
i.e. What is its name on the Facts tab or in Tools > Fact Types?
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
User avatar
Ron Melby
Megastar
Posts: 917
Joined: 15 Nov 2016 15:40
Family Historian: V6.2

Re: finding and parsing an attribute by TYPE

Post by Ron Melby »

the display in FH for that tag is MilSvc.
FH V.6.2.7 Win 10 64 bit
User avatar
tatewise
Megastar
Posts: 28333
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: finding and parsing an attribute by TYPE

Post by tatewise »

So in any Data Reference Assistant look for an Attribute with the name MilSvc to find %INDI._ATTR-MILT[1]%.

In Lua strTag, strErr = fhGetFactTag("MilSvc", "Attribute", "INDI", false) should return _ATTR-MILT in strTag.

The attached Fact Query called Military Details should produce a Result Set listing much of what you want.
It filters on the Fact Label being MilSvc but if nothing is listed in the Result Set the Rows tab may need tweaking:
Add if =FactLabel(%FACT%) begins with MilSvc
either needs to use =FactName(%FACT%) or MilSvc needs correcting.

In each of those built-in features, the name you know MilSvc is translated to the GEDCOM compatible _ATTR-MILT tag.

The reason FH and Lua use just one pseudo tag such as _ATTR-MILT instead of the _ATTR & TYPE combo, is to ensure other fact related features such as sub-fields DATE and PLACe are handled the same as standard facts like BIRTh and OCCUpation.
i.e.
INDI._ATTR-MILT.DATE c.f. INDI.OCCU.DATE c.f. INDI.BIRT.DATE
INDI._ATTR-MILT.PLAC c.f. INDI.OCCU.PLAC c.f. INDI.BIRT.PLAC
=FactValue(%INDI._ATTR-MILT%) c.f. =FactValue(%INDI.OCCU%)
ptrFact:MoveTo(ptrINDI,"~._ATTR-MILT") c.f. ptrFact:MoveTo(ptrINDI,"~.OCCU") c.f. ptrFact:MoveTo(ptrINDI,"~.BIRT")

Can you see that if the _ATTR & TYPE tags were handled separately within FH and Lua then those operations would not be the simple equivalents as shown above.
Attachments
Military Details.fhq
(1.41 KiB) Downloaded 166 times
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
User avatar
Ron Melby
Megastar
Posts: 917
Joined: 15 Nov 2016 15:40
Family Historian: V6.2

Re: finding and parsing an attribute by TYPE

Post by Ron Melby »

function split(strTxt,strSep)
local tblFields = {}
local strPattern = string.format("([^%s]+)", strSep or ",")
strTxt = tostring(strTxt or "")
strTxt:gsub(strPattern, function(strField) tblFields[#tblFields+1] = strField end)
return tblFields
end -- function split

tblFields = split(strATTR, ",")

local War = tblFields, 1
local Branch = tblFields, 2
local Rank = tblFields, 3
local Other = tblFields, 4


two major problems with this, its inelegant and there may be 0....4 values there..., don't want nils, do I? and
it doesnt work anyway
any slick ways of doing this?
FH V.6.2.7 Win 10 64 bit
User avatar
tatewise
Megastar
Posts: 28333
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: finding and parsing an attribute by TYPE

Post by tatewise »

It does work ~ dozens of us have used it successfully it for many years.

Did you see the plugins:code_snippets:split_a_line_using_a_separator|> Split a Line using a Separator (code snippet) Usage guide?

It is easy to determine how many comma separated parts are found, and it is easy to avoid nil values.
local Parts = #tblFields -- tells you how many parts
local Rank = tblFields[3] or "" -- will replace nil with the empty string "" or whatever you want.

If it is too inelegant for you, then you are welcome to do your own thing...

Did the Query that I attached work for you?

Did our explanations of built-in features satisfy your needs?

Some feedback would be nice.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
User avatar
Ron Melby
Megastar
Posts: 917
Joined: 15 Nov 2016 15:40
Family Historian: V6.2

Re: finding and parsing an attribute by TYPE

Post by Ron Melby »

this is what I have so far, and having it, I need to fix data to standardize it, so I will put in buddy fields to go right to the MilSvc in question, there are artifacts and fred (for redoing) cleanup is later I cannot put up an image of the report it is too big even zip:

Code: Select all

--[[
@title: Find MilSvc
@author: Ron Melby
@lastupdated: Dec 2018
@version: 1.0
@description:
 
* INDI
* MilSvc
* CEMETERY
* SOURCE
* Page
* Plot
]]

function split(strTxt,strSep)
	local tblFields = {}
	local strPattern = string.format("([^%s]+)", strSep or ",")
	strTxt = tostring(strTxt or "")
	strTxt:gsub(strPattern, function(strField) tblFields[#tblFields+1] = strField end)
	return tblFields
end -- function split

-- Main Code
-- Tables for Result Set
tblINDI   = {} -- record pointer
tblName   = {} -- formatted name
tblDates  = {} -- Birth Death
tblWar    = {} -- 
tblBranch = {} -- 
tblRank   = {} -- 
tblOth    = {} --
tblNOTES  = {} -- 

local ptrINDI = fhNewItemPtr()
ptrINDI:MoveToFirstRecord("INDI")
while ptrINDI:IsNotNull() do                         -- Process each Individual record
  local ptrATTR = fhNewItemPtr()
  ptrATTR:MoveTo(ptrINDI,"~._ATTR-MILT")      
  while ptrATTR:IsNotNull() do                      -- Process each Military Attribute 
  
  -- Get INDI Record information
    Name = fhGetItemText(ptrINDI, "~.NAME:SURNAME_FIRST")
    NameSuffix = fhGetItemText(ptrINDI,"~.NAME.NSFX")
    Sortname = (Name.. ' ' ..NameSuffix)
    table.insert(tblName, Sortname)
    table.insert(tblINDI, ptrINDI:Clone())
    local Birth = fhGetItemText(ptrINDI,"~.BIRT.DATE:YEAR")
    local Death = fhGetItemText(ptrINDI,"~.DEAT.DATE:YEAR")
    local Dates = "(" .. Birth .. " - " .. Death .. ")"  
    table.insert(tblDates, Dates)
  
-- Get value of Attribute
-- Split into component parts
    local strATTR = fhGetValueAsText(ptrATTR)
    tblFields = split(strATTR, ",") 
    table.insert(tblWar,    tblFields[1] or " ")
    table.insert(tblBranch, tblFields[2] or " ")
    table.insert(tblRank,   tblFields[3] or " ")
    table.insert(tblOth,    tblFields[4] or " ")
 
    ptrATTR:MoveNext("SAME_TAG")                   -- Move to next Military Attribute
  end
  ptrINDI:MoveNext()                               -- Move to next Individual record
end


  fhOutputResultSetColumn("Person","text", tblName,  #tblName, 128, "align_left")
  fhOutputResultSetColumn("LnkNam","item", tblINDI,  #tblINDI, 128, "align_left", 2, true, "default", "buddy")
  fhOutputResultSetColumn("Dates", "text", tblDates, #tblDates, 48, "align_left", 1)
  fhOutputResultSetColumn("War",   "text", tblWar,   #tblWar,   48, "align_left")  
  fhOutputResultSetColumn("Branch","text", tblBranch,#tblBranch,48, "align_left")  
  fhOutputResultSetColumn("Rank",  "text", tblRank,  #tblRank,  48, "align_left")
  fhOutputResultSetColumn("Oth",   "text", tblOth,  #tblOth,   128, "align_left") 
-- fhOutputResultSetColumn("NOTES",    "item", tblNOTES, #tblNOTES,512, "align_left")
  return 

FH V.6.2.7 Win 10 64 bit
User avatar
tatewise
Megastar
Posts: 28333
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: finding and parsing an attribute by TYPE

Post by tatewise »

I'll ask again ~ Did the Query that I attached work for you?
It does most of what your Plugin achieves, and with a little tweak can do it all with much less effort.

In fact I have attached a tweaked version that does the same as your Plugin.
It also includes the Rec Id and if you click on the Dates will display the MilSvc attribute.
( I don't have any MilSvc attributes so it is actually showing Occupations here. )
Military Details.png
Military Details.png (28.42 KiB) Viewed 10411 times
Attachments
Military Details.fhq
(1.63 KiB) Downloaded 153 times
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
User avatar
Ron Melby
Megastar
Posts: 917
Joined: 15 Nov 2016 15:40
Family Historian: V6.2

Re: finding and parsing an attribute by TYPE

Post by Ron Melby »

Mike, I will answer again, yes the query works, and I expect the new query will work better. The issue is that this is all a setup to write out a file to convert to PDF that will look more like an encyclopedia entry. This little vignette was just a run thru for cleanup, and perhaps your query will cleanup faster, and then I have to get those entries under this Attribute that may have a DATE, NOTES, and so on.

Thank you for your massive help though so far.
FH V.6.2.7 Win 10 64 bit
User avatar
Ron Melby
Megastar
Posts: 917
Joined: 15 Nov 2016 15:40
Family Historian: V6.2

Re: finding and parsing an attribute by TYPE

Post by Ron Melby »

if Birth = "" then
Birth = "bbbb" (b representing blank for spacing)
end

in debug Birth = '""
if this is the case; I want Birth to be 4 blanks.

I have also tried not Birth and several other variations, none of which complain, and none of which work.

I am certainly missing something trivial here.
FH V.6.2.7 Win 10 64 bit
User avatar
tatewise
Megastar
Posts: 28333
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: finding and parsing an attribute by TYPE

Post by tatewise »

What I suggest is that you become familiar with looking up general Lua programming syntax.
When in the Plugin edit and debug window click Help > Lua Online Reference Manual.
In this case look down the Contents to 2.5.2 Relational Operators that should answer your question.

In other words, instead of giving you a bottle of water, I'm showing how to build a well :idea:

However, it does depend on whether Birth holds a text string or some other type of value. See 2.2 – Values and Types.

BTW: You can also use Help > Family Historian API to lookup all the FH specific features.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
User avatar
Ron Melby
Megastar
Posts: 917
Joined: 15 Nov 2016 15:40
Family Historian: V6.2

Re: finding and parsing an attribute by TYPE

Post by Ron Melby »

Code: Select all

if Birth == "" then <<<< (1 blank) 
      Birth = '       ' <<<<(8 blanks)
end 
Its drinking koolaid though.... 4:40a here, just got this as you were posting. Figured out c equality notation about an hour ago.
8 blanks to pad it out for a 4 character field. Now I will pdf it because I suspect wisinwig.

wisiwig. didn't find that in the reference manuals all night.
FH V.6.2.7 Win 10 64 bit
Post Reply