Config fields

Depending on when each field can be modified, they are said:

  • Clean : requires a fresh repo to change this value
  • Static : requires that no lmake is running to change this value
  • Dynamic : field can be changed any time.

The default value is mentioned in ().

disk_date_precision : Static (0.010)

This attribute instruct open-lmake to take some margin (expressed in seconds) when it must rely on file dates to decide about event orders. It must account for file date granularity (generally a few ms) and date discrepancy between executing hosts and disk servers (generally a few ms when using NTP).

  • If too low, there are risks that open-lmake consider that data read by a job are up to date while they have been modified shortly after.
  • If too high, there is a small impact on performance as open-lmake will consider out of date data that are actually up to date.

The default value should be safe in usual cases and user should hardly need to modify it.

heartbeat : Static (10)

Open-lmake has a heartbeat mechanism to ensure a job does not suddenly disappear (for example if killed by the user, or if a remote host reboots). If such an event occurs, the job will be restarted automatically.

This attribute specifies the minimum time between 2 successive checks for a given job. If None (discouraged), the heartbeat mechanism is disabled.

The default value should suit the needs of most users.

  • If too low, build performance will decrease as heartbeat will take significative resources
  • If too high, reactivity in case of job loss will decrease

heartbeat_tick : Static (0.1)

Open-lmake has a heartbeat mechanism to ensure a job does not suddenly disappear (for example if killed by the user, or if a remote host reboots). If such an event occurs, the job will be restarted automatically.

This attribute specifies the minnimum time between 2 successive checks globally for all jobs. If None (discouraged), it is equivalent to 0.

The default value should suit the needs of most users.

  • If too low, build performance will decrease as heartbeat will take significative resources
  • If too high, reactivity in case of job loss will decrease

local_admin_dir : Clean (-)

This variable contains a dir to be used for open-lmake administration in addition to the LMAKE dir.

It is guaranteed that all such accesses are performed by the host, hence a dir in a locally mounted disk is fine.

  • If unset, administration by user is simplified (no need to manage an external dir), but there may be a performance impact as network file systems are generally slower than local ones.
  • If set to a local dir, user has to ensure that lmake and other commands are always launched from the host that has this locaol file system.
  • If set to network dir, there is no performance gain and only added complexity.

Open-lmake fully handle symbolic links (cf data model).

However, there is an associated cost which may be useless in some situations.

ValueSupport level
'full'symbolic links are fully supported
'file'symbolic links are only supported if pointing to files
'none'symbolic links are not supported

max_dep_depth: Static (100)

The rule selection process is a recursive one. It is subject to infinite recursion and several means are provided to avoid it.

The search stops if the depth of the search reaches the value of this attribute, leading to the selection of a special internal rule called infinite.

  • If too low, some legal targets may be considered infinite.
  • If too high, the error message in case of infinite recursion will be more verbose.

max_error_lines : Dynamic (100)

When a lot of error lines are generated by open-lmake, other than copying the stderr of a job, only the first max_error_lines ones are actually output, followed by a line containing ... if some lines have been suppressed. The purpose is to ease reading.

network_delay : Static (1)

This attribute provides an approximate upper bound of the time it takes for an event to travel from a host to another.

  • If too low, there may be spurious lost jobs.
  • If too high, there may be a loss of reactivity.

The default value should fit most cases.

path_max : Static (200)

The rule selection process is a recursive one. It is subject to infinite recursion and several means are provided to avoid it.

The search stops if any file with a name longer than the value of this attribute, leading to the selection of a special internal rule called infinite.

reliable_dirs : Static (False if non-local backends are used)

This attribute specifies whether dir coherence is enforced when a file is created/modified/unlinked.

When not the case, open-lmake emit additional traffic to ensure the necessary coherency.

Known file systems :

File systemreliable dirs
NFSFalse
CEPHTrue
  • If uselessly set to False, there is a performance hit.
  • If wrongly set to True, builds will be unreliable with difficult to analyze patterns.

So if in doubt, leave False.

sub_repos : Static (())

This attribute provide the list of sub-repos.

Sub repos are sub-dirs of the repo that are themselves repos, i.e. they have a Lmakefile.py. Inside such sub-repos, the applied flow is the one described in it (cf Subrepos).

console.date_precision : Dynamic (None)

This attribute specifies the precision (as the number of digit after the second field, for example 3 means we see milli-seconds) with which timestamps are generated on the console output. If None, no timestamp is generated.

console.has_exec_time : Dynamic (True)

If this attribute is true, execution time is reported each time a job is completed.

console.history_days : Dynamic (7)

This attribute specifies the number of days the output log history is kept in the LMAKE/outputs dir.

console.host_len : Dynamic (None)

This attribute specifies the width of the field showing the host that executed or is about to execute the job. If None, the host is not shown. Note that no host is shown for local execution.

console.show_eta : Dynamic (False)

If this attribute is true, the title shows the ETA of the command, in addition to statistics about number of jobs.

console.show_ete : Dynamic (True)

If this attribute is true, the title shows the ETE of the command, in addition to statistics about number of jobs.

trace.size : Static (100_000_000)

While open-lmake runs, it may generate an execution trace recording a lot of internal events meant for debugging purpose.

The trace is handled as a ring buffer, storing only the last events when the size overflows. The larger the trace, the more probable the root cause of a potential problem is still recorded, but the more space it takes on disk.

This attributes contains the maximum size this trace can hold (open-lmake keeps the 5 last traces in case the root cause lies in a previous run).

trace.n_jobs : Static (1000)

While open-lmake runs, it generates execution traces for all jobs.

This attributes contains the overall number of such traces that are kept.

trace.channels : Static (all)

The execution trace @lmake generates is split into channels to better control what to trace.

This attributes contains a list or tuple of the channels to trace.

colors : Dynamic (reasonably readable)

Open-lmake generate colorized output if it is connected to a terminal (and if it understands the color escape sequences) (cf video-mode).

This attribute is a pdict with one entry for each symbolic color. Each entry is a 2-tuple of 3-tuple's. The first 3-tuple provides the color in normal video mode (black/white) and the second one the color in reverse video (white/black). Each color is a triplet RGB of values between 0 and 255.

backends : Dynamic

This attribute is a pdict with one entry for each active backend (cf backends).

Each entry is a pdict providing resources. Such resources are backend specific.

backends.*.interface : Dynamic (best guess)

When jobs are launched remotely, they must connect to open-lmake when they start and when they complete. The same is true if the job is launche locally but it launches sub-commands remotely (in this case it is the command that needs to connect to the job trampoline). This is done by connecting to a socket open-lmake has opened for listening, which requires that we must have a means to determine an IP address to connect to. The host running open-lmake may have several network interfaces, one of them (typically only one) being usable by such remote hosts. There is no generic way to determine this address, so in general, open-lmake cannot determine it automatically.

This value may be empty (using hostname for addresse look up), given in standard dot notation, as the name of an interface (as shown by ifconfig) or the name of a host (looked up as for ping). In case of ambiguity, local backend will use the loop-back address, remote backends will generate an error message showing the possible choices.

backends.*.environ : Dynamic ({})

Environment to pass when launching job in backend. This environment is accessed when the value mentioned in the rule is ....

backends.local.cpu : Dynamic (number of phyical CPU's)

This is a normal resource that rules can require (by default, rule require 1 cpu).

backends.local.mem : Dynamic (size of physical memory in MB)

This is the pysical memory necessary for jobs. It can be specified as a number or a string representing a number followed by a standard suffix such as k, M or G. Internally, the granularity is forced to MB.

backends.local.tmp : Dynamic (0)

This is the disk size in the temporary dir necessary for jobs. It can be specified as a number or a string representing a number followed by a standard suffix such as k, M or G. Internally, the granularity is forced to MB.

caches : Static

This attribute is a pdict with one entry for each cache.

Caches are named with an arbitrary str and are referenced in rules using this name.

By default, no cache is configured, but an example can be found in lib/lmake/config.py, commented out.

caches.*.tag : Static (-)

This attribute specifies the method used by open-lmake to cache values. In the current version, only 2 tags may be used:

  • none is a fake cache that cache nothing.
  • dir is a cache working without daemon, data are stored in a dir.

caches.<dir>.dir : Static

This attribute specifies the dir in which the cache puts its data.

The dir must pre-exist and contain a file LMAKE/size containing the size the cache may occupy on disk. The size may be suffixed by a unit suffix (k, M, G, T, P or E). These refer to base 1024.

caches.<dir>.reliable_dirs : Static (False)

Same meaning as config.reliable_dirs for the dir containing the cache.

caches.<dir>.group : Static (default group of user)

This attribute specifies the group used when creating entries.

debug

When ldebug is used, it consults this dict.

It maps debug keys to modules to import to implement the debug method (cf man ldebug). Values contain the module name optionnaly followed by a human description (that will appear with ldebug -h) separated with spaces.