{"id":10704,"date":"2020-07-07T12:42:46","date_gmt":"2020-07-07T12:42:46","guid":{"rendered":"https:\/\/fhug.org.uk\/kb\/?post_type=kb_article&p=10704"},"modified":"2023-04-12T15:03:08","modified_gmt":"2023-04-12T15:03:08","slug":"understanding-lua-patterns","status":"publish","type":"kb_article","link":"https:\/\/www.fhug.org.uk\/kb\/kb-article\/understanding-lua-patterns\/","title":{"rendered":"Understanding Lua Patterns"},"content":{"rendered":"

Introduction<\/h2>\n

A pattern is a formula for matching text strings that conform to some definable criteria (experienced coders need to remember Lua patterns are similar but not identical to regular expressions<\/a>). Optionally, the pattern allows components of the string to be captured in order to compose a new text string incorporating those components.<\/p>\n

For example, the pattern\u00a0ABC%d%d%d<\/code>\u00a0(where\u00a0%d<\/code>\u00a0matches any digit) will match the letters\u00a0ABC\u00a0followed by any three digits, such as\u00a0ABC142\u00a0or\u00a0ABC897\u00a0or\u00a0ABC603, but will NOT match\u00a0ABC93X.<\/p>\n

So patterns comprise regular literal text characters that match themselves, such as\u00a0ABC\u00a0above, and\u00a0magic\u00a0or\u00a0meta\u00a0characters that have a special meaning, such as\u00a0%d<\/code>\u00a0above. The following sections focus on the meanings of those\u00a0magic\u00a0characters.<\/p>\n

Pattern Match Functions<\/h2>\n

Lua has some string functions that use patterns. Among them are:<\/p>\n\n\n\n\n\n\n
string.find(string, pattern)<\/td>\nfinds the first instance of\u00a0pattern\u00a0in\u00a0string\u00a0and returns its position<\/td>\n<\/tr>\n
string.gmatch(string, pattern)<\/td>\nwhen called repeatedly, returns each successive instance of\u00a0pattern\u00a0in\u00a0string<\/td>\n<\/tr>\n
string.gsub(string, pattern, repl)<\/td>\nreturns a string where all instances of\u00a0pattern\u00a0in\u00a0string\u00a0have been replaced with\u00a0repl<\/td>\n<\/tr>\n
string.match(string, pattern)<\/td>\nreturns the first instance of\u00a0pattern\u00a0in\u00a0string<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

Some of these functions have additional optional parameters, and behave slightly differently if the\u00a0pattern\u00a0defines\u00a0Captures<\/a>. For details, see the Lua reference manual (online) for the version that you’re using (the links are in Useful External Links below).<\/p>\n

Constructing a Pattern<\/h2>\n

So, how do we construct a pattern?<\/p>\n

At the foundation of patterns are\u00a0Character Classes<\/a>\u00a0that represent sets of characters. Each character class matches just one character that conforms to that class. For example, the character class\u00a0%d<\/code>\u00a0matches any one digit\u00a00-9, whereas the character class\u00a0A<\/code>\u00a0matches the single letter\u00a0A.<\/p>\n

More complex structures are\u00a0Pattern Items<\/a>\u00a0that represent combinations of characters. For example, the pattern item\u00a0%d+<\/code>\u00a0matches a sequence of one or more digits, and\u00a0A+<\/code>\u00a0matches a sequence of\u00a0A’s.<\/p>\n

Finally a\u00a0Pattern<\/a>\u00a0is a sequence of\u00a0Pattern Items<\/a>\u00a0that may contain\u00a0Captures<\/a>\u00a0of internal components whose purpose is explained later.<\/p>\n

Remember the\u00a0magic\u00a0characters\u00a0% . [ ] ^ $ ( ) * + - ?<\/code>\u00a0have special contextual meanings, so usually don’t match themselves.<\/p>\n

<\/a>Character Classes<\/h3>\n

Lua character classes are:<\/p>\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
Class<\/strong><\/td>\nMeaning<\/strong><\/td>\n<\/tr>\n
Y<\/td>\nrepresents the character\u00a0Y\u00a0itself as long as it is not a\u00a0magic\u00a0character<\/td>\n<\/tr>\n
.<\/td>\nrepresents any single character<\/td>\n<\/tr>\n
%a<\/td>\nrepresents all letters A-Z and a-z<\/td>\n<\/tr>\n
%c<\/td>\nrepresents all control characters such as Null, Tab, Carr.Return, Linefeed, Delete, etc<\/td>\n<\/tr>\n
%d<\/td>\nrepresents all digits 0-9<\/td>\n<\/tr>\n
%g<\/td>\nall printable characters except space (not Lua 5.1)<\/td>\n<\/tr>\n
%l<\/td>\nrepresents all lowercase letters a-z<\/td>\n<\/tr>\n
%p<\/td>\nrepresents all punctuation characters or symbols such as . , ? ! : ; @ [ ] _ { } ~<\/td>\n<\/tr>\n
%s<\/td>\nrepresents all white space characters such as Tab, Carr.Return, Linefeed, Space, etc<\/td>\n<\/tr>\n
%u<\/td>\nrepresents all uppercase letters A-Z<\/td>\n<\/tr>\n
%w<\/td>\nrepresents all alphanumeric characters A-Z and a-z and 0-9<\/td>\n<\/tr>\n
%x<\/td>\nrepresents all hexadecimal digits 0-9 and A-F and a-f<\/td>\n<\/tr>\n
%z<\/td>\nrepresents the character with code \\000 because embedded zeroes in a pattern do not work (\u0192h<\/span><\/span>6\/Lua 5.1)<\/td>\n<\/tr>\n
\\0<\/td>\nrepresents the character with code \\000 because embedded zeroes in a pattern do not work (\u0192h<\/span><\/span>7\/Lua 5.3)<\/td>\n<\/tr>\n
<\/td>\nThe upper case letter versions of the above reverses their meaning
\ni.e.\u00a0%A\u00a0represents all non-letters and\u00a0%D\u00a0represents all non-digits<\/td>\n<\/tr>\n
%Y<\/td>\nrepresents the character\u00a0Y\u00a0if it is any non-alphanumeric character
\nThis is the standard way to get a\u00a0magic\u00a0character to match itself
\nAny punctuation character (even a non\u00a0magic\u00a0one) preceded by a\u00a0%\u00a0represents itself
\ne.g.\u00a0%%\u00a0represents\u00a0%\u00a0percent and\u00a0%+\u00a0represents\u00a0+\u00a0plus<\/td>\n<\/tr>\n
[set]<\/td>\nrepresents the class which is the union of all characters in the\u00a0set
\nA range of characters is specified by separating first and last character of range with a\u00a0–\u00a0hyphen e.g.\u00a01-5
\nAll classes described above may also be used as components in the\u00a0set
\ne.g.\u00a0[%w~]\u00a0(or\u00a0[~%w]) represents all alphanumeric characters plus the\u00a0~\u00a0tilde<\/td>\n<\/tr>\n
[^set]<\/td>\nrepresents the complement of set, where set is interpreted as above
\ne.g.\u00a0[^A-Z]\u00a0represents any character except upper case letters<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

Many patterns are simply made up of a sequence of these classes.
\nFor example, to match the string\u00a0### Abc\u00a0(where\u00a0#\u00a0designates a digit), you would use the pattern\u00a0%d%d%d Abc<\/code>.<\/p>\n

Note: classes %a<\/code> to %x<\/code> and %A<\/code> to %X<\/code> above aren’t strictly necessary, being just shorthand for certain [set]<\/code> or [^set]<\/code> classes.
\ne.g.\u00a0%a<\/code>\u00a0is\u00a0[A-Za-z]<\/code>\u00a0,\u00a0%d<\/code>\u00a0is\u00a0[0-9]<\/code>\u00a0,\u00a0%l<\/code>\u00a0is\u00a0[a-z]<\/code>\u00a0,\u00a0%L<\/code>\u00a0is\u00a0[^a-z]<\/code>\u00a0,\u00a0%U<\/code>\u00a0is\u00a0[^A-Z]<\/code>\u00a0, and so on.<\/p>\n

<\/a>Pattern Items<\/h3>\n

By adding modifiers, you can extend the meaning of those Character Classes<\/a> represented by class below.<\/p>\n\n\n\n\n\n\n\n\n
Item<\/strong><\/td>\nMeaning<\/strong><\/td>\n<\/tr>\n
class<\/td>\nmatches any single character in the\u00a0class<\/td>\n<\/tr>\n
class*<\/td>\nmatches 0 or more repetitions of\u00a0class\u00a0and will always match the longest possible chain<\/td>\n<\/tr>\n
class+<\/td>\nmatches 1 or more repetitions of\u00a0class\u00a0and will always match the longest possible chain<\/td>\n<\/tr>\n
class-<\/td>\nmatches 0 or more repetitions of\u00a0class\u00a0and will always match the shortest possible chain<\/td>\n<\/tr>\n
class?<\/td>\nmatches 0 or 1 occurrence of\u00a0class<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

For example,\u00a0%d+<\/code>\u00a0will match a string of digits.<\/p>\n

There are also two special pattern items.<\/p>\n\n\n\n\n\n
Item<\/strong><\/td>\nMeaning<\/strong><\/td>\n<\/tr>\n
%N<\/td>\nfor\u00a0N\u00a0between\u00a01\u00a0and\u00a09\u00a0matches the\u00a0Nth captured substring (see\u00a0Captures\u00a0<\/a>below)<\/td>\n<\/tr>\n
%bXY<\/td>\nmatches a substring starting with\u00a0X\u00a0and ending with\u00a0Y\u00a0and the substring also has the same number of\u00a0X\u00a0as\u00a0Y
\nFor example,\u00a0%b()<\/code>\u00a0will match balanced nested pairs of\u00a0( )\u00a0parentheses<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

<\/a>Pattern<\/h3>\n

A Pattern is a sequence of Pattern Items that can be anchored to the start &\/or end of a string.<\/p>\n\n\n\n\n\n\n
Anchor<\/strong><\/td>\nMeaning<\/strong><\/td>\n<\/tr>\n
^<\/td>\nwhen at the beginning of a pattern, forces the pattern to match the start of a string<\/td>\n<\/tr>\n
$<\/td>\nwhen at the end of a pattern, forces the pattern to match the end of a string<\/td>\n<\/tr>\n
<\/td>\nWhen\u00a0^\u00a0or\u00a0$\u00a0is elsewhere in a pattern (except in\u00a0[^set]), it has no\u00a0magic\u00a0meaning and represents itself<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

For example,\u00a0^A%d+Z$<\/code>\u00a0will match a string starting with\u00a0A, ending with\u00a0Z, with only digits in between.<\/p>\n

<\/a>Captures<\/h3>\n

A Pattern<\/a> may have sub-patterns enclosed by ( )<\/code> parentheses, that are counted as matches are found. They can be accessed using the %N<\/code> item defined above. The empty capture ()<\/code> will return the string position where a match is found.<\/p>\n

For example,\u00a0^A(%d+)Z$<\/code>\u00a0will capture just the digits from any matching string, and subsequently\u00a0%1<\/code>\u00a0represents those captured digits.<\/p>\n

So the function\u00a0string.gsub(S, \"^A(%d+)Z$\", \"%1\")<\/code>\u00a0will return just the digits from string\u00a0S<\/code>\u00a0as long as it matches the pattern, otherwise it returns the original string\u00a0S<\/code>\u00a0unaltered.<\/p>\n

Examples<\/h2>\n

Here are a few basic examples of Lua Patterns that match Family Historian text.<\/p>\n\n\n\n\n\n\n
Lua Pattern<\/strong><\/td>\nMatching Text<\/strong><\/td>\n<\/tr>\n
^Q[1-4] %d%d%d%d$<\/td>\nQuarter date e.g. Q2 1987 or Q4 1876<\/td>\n<\/tr>\n
^[123]?%d %u%l+ %d%d%d%d%$<\/td>\nSimple date e.g.\u00a021 March 1765\u00a0or\u00a08 June 1678<\/td>\n<\/tr>\n
^[1-3]?%d %u%l+ %d%d%d%d% %(%l%l%l%)$<\/td>\nQualified date e.g.\u00a019 September 1765 (est)
\nNote how\u00a0magic\u00a0( )\u00a0parentheses need\u00a0%\u00a0prefix<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

The following\u00a0Code Snippets<\/a> use patterns, as does the Search and Replace<\/a> plugin.<\/p>\n