Writing an xLine Gleaner

A gleaner is a plug-in routine invoked by an xLine (PipeLine or FileLine) process. It pairs monitored information with the relevant InfoDictItem ID(s) and returns those pairs to the xLine process for filing in the InfoHub database. Typically, you use a FileLine Gleaner for active monitoring of a text file and a PipeLine Gleaner for active monitoring of stdout and/or stderr of a process.

A gleaner must have at least one of the following extrinsic expressions (functions):

Here are some tips for writing M code for a gleaner:

  1. If you need to return multiple pairs of <key><delimiter><value>, the return value should be the form of:

    <key1><delimiter><value1><delimiter><key2><delimiter><value2>...

    To ensure accurate filing of information in the InfoHub database, choose your delimiter such that it is never contained in any <key> or <value> portion of the returned string. $char(30) is used as a delimiter in the ULFM Reference Implementation. You can also specify the delimiter as the case-insensitive keyword "var", in which case the delimiter should be passed as the first character of the expression returned by the gleaner in the following pattern:

    <delimiter><key1><delimiter><value1><delimiter><key2><delimiter><value2>...
  2. Each <key> is a comma delimited list of canonical positive integers between 1 and 2**31-1 representing the path beneath the xLine to an appropriate InfoDictItems as defined in the InfoHub Configuration File.

  3. Ensure that <key>s never include the InfoHubID, PublisherID, or xLineID; they should only reflect the InfoDict hierarchy below the xLine's InfoDict.

  4. You should always use a NEW command to locally scope variables used by your gleaner; this avoids inadvertant conflicts with the local variables used in the InfoHub code. A NEW lasts only while the current scope of execution is active, and prevents accidental modifications of InfoHub local variables.

  5. All user-added or -modified code should also follow that same coding conventions as used with InfoHub.

Examples:

Gleaner Name

Type

Description

UptimeGleaner.m

PipeLine

Monitors information from the stdout of the uptime command in the ULFM Reference Implementation.

LogFileGleaner.m

FileLine

Monitors the system and authentication log files in the ULFM Reference Implementation.

FileLineSyslogGleaner.m

FileLine

A gleaner implementation whose InfoExpr function reads /var/log/messages line-by-line and converts GT.M-only messages to the form of <key><delimiter><value> pairs. FileLineSyslogGleaner.m is a part of the GT.M Monitoring Reference Implementation.

[Tip]InfoHub Internals

When an xLine receives a string consisting of key-value pairs, it processes each pair individually. First, InfoHub verifies the existence of the following node to ascertain whether there is a Path in the configuration which matches the <key>:

^InfoHubConf(<InfoHubID>,<InfoHubSeqNo>,"Paths",<PublisherID>,<xLineID>,<key>)

If the node exists and is of non-META type, InfoHub performs the following three updates:

  1. ^InfoHubInfo(<InfoHubID>,<PublisherID>,<PipeLineID>,<key>,<seqno>)=":<type>:<value>"

  2. ^InfoHubInfo(<InfoHubID>,"TimeToSeqNo",<PublisherID>,<PipeLineID>,<key>,<time>,<seqno>)=""

  3. ^InfoHubInfo(<InfoHubID>,"SeqNoToTime",<PublisherID>,<PipeLineID>,<key>,<seqno>)=<time>

where <value> is the one corresponding to the current key; <type> is as specified in the ^InfoHubConf(...,"Paths",...) node; and <seqno> is the current sequence number for this key. "TimeToSeqNo" and "SeqNoToTime" are ancillary nodes that InfoHub uses to perform book-keeping functions. InfoHub skips the first of these updates if the value for a particular key did not change from the last time. Otherwise, InfoHub "closes" the previous update by providing the ending sequence number in the value:

^InfoHubInfo(<InfoHubID>,<PublisherID>,<xLineID>,<key>,<seqno>)="<ending seqno>:<type>:<value>"

Suppose the prior value for key 9999 was .77 and the current value is .64. The following node represents the InfoHub database update at the time when key 9999 was .77:

^InfoHubInfo(11,112,1137,9999,87)=":String:.77"

The following nodes represent InfoHub database updates when the value corresponding to key 9999 changes to .64.

  • ^InfoHubInfo(11,112,1137,9999,87)="88:String:.77"

  • ^InfoHubInfo(11,112,1137,9999,88)=":String:.64"

Here 88 is the ending sequence number of the prior value. The change for key 9999's corresponding value to .64 also adds the following nodes:

  • ^InfoHubInfo(11,"SeqNoToTime",112,1137,9999,88)="1385382167772674"

  • ^InfoHubInfo(11,"TimeToSeqNo",112,1137,9999,1385382167772674,88)=""

In case of META nodes, it performs the following update in addition to the updates to the node causing the META action:

  • ^InfoHubInfo(<InfoHubID>,"Alerts",<PublisherID>",<xLineID>,<key>)="<time>:<SubscriptionID>:info:ref:<last non-meta path for this event >"

Related Information: