jMax IRCAM - Centre Georges Pompidou

Variables



Introduction

Variables are a way to bind values to names; the scoping rules of jMax are similar to block oriented lexical scoping languages, like C, where the block is implemented by the patcher. The complexity of the implementation derive from supporting a real-time editing of the patch, so that the scoping rules and variable values must be evaluated incrementally for every small change of the patch; this is taken care by the variable propagation algorithm.

Propagation Algorithm

The basic goal of the propagation algorithm is to propagate the change in the definition of variable to all the objects using it, detecting and avoiding definition loops, and supporting all the scoping rules when a variable is redefined in a scope different from the original.

The algorithm is based on three basic concept: object recomputing, suspended variable, and object stealing.

Object recomputing means simply that when we have a new value ready for a variable, in order to the change to affect the object, we recompute it, i.e. we reinstantiate it starting from the same object description; see to the Objects chapter for details.

Suspended variables is a more subtle concept: a variable can actually have four states: not existing, undefined, suspended and restored:

  • a not existing variable is a variable that as not yet been defined *or* referred; there is no trace of this variable in the system.

  • a undefined variable is a variable that has been referred, but never defined; the variable name is registered, usually in the root patcher scope, and its value is a void atom to register the fact that the variable is not defined.

  • a suspended variable is a variable that has been defined, but its value cannot be accessed for expression evaluation; it is a temporary condition, existing only during the execution of the propagation algorithm; when a variable become suspended, all the existing objects depending on it will suspend the variable they define, and so on recursively. New objects instantiated while a variable is suspended will become error objects, but their dependency on the variable will be recorded correctly.

  • a restored or active variable is a variable defined and referred, whose value can be used for object instantiation; a variable is always created suspended, and become active by an explicit restore operation; when a suspended variable is restored, all the objects depending on this variable are recomputed, so they can be reinstantiated with the new variable value; as a side effect, all the recomputed object defining a variable will restore that variable, and so on recursively.

When an object is instantiated (or redefined), if it define a variable, the variable is created in the current scope and suspended; the object expression is evaluated, the object created, and at the end the variable value is set and the variable restored; since during the expression evaluation the object variable is suspended, we have the guarantee that all the variables depending on this object are suspended; if the expression refer one of these variables, we get an error in the expression evaluation; this mechanism guarantee that there are no loops.

Also, the recursive restore chain started by the variable restore will recompute all the objects depending directly or indirectly on this variable, in the correct order; this mechanism guarantee value propagation.

One mechanism is still needed to support scoping rule, that is the object stealing; when a variable is defined in a given scope, we look if the variable has been already defined in a scope that include the current one, and if this outside definition have users (i.e. object that depend on the variable) in the current scope; if it is the case, we move these users to the new variable in the current scope; the suspend/restore mechanism will do the rest.

If an object refer a variable that has not been defined yet, it is registered as user of an undefined variable stored in the outmost scope (the root patcher); in this way, the above object stealing work also for defining a previously undefined variable.

Architectural Limits of Variables

Variables are currently intended to be used in the object instantiation process, and while the current mechanisms could be extended toward a run time use, but many problems have to be solved; the main problem is that currently the referential integrity of the system are guaranted by the following:

  • Variable values have the same lifetime of objects
  • Variable values are only referred thru variable names
  • Variable values are private data to an object using them, and are not communicated to the external world with any means different than the variable binding.

These constraint assure that when an object is destroyed, the variable value is destroied, and all the object using this value are recomputed, and no pending reference to the destroyed value are left anywhere.

Supporting a run time use of variable values imply substituting this mechanisms with equivalent run-time mechanisms; note that in general garbage collection is not always sufficent, because a garbage collected based system do not work very well with object that have an explicit, intentional, close/destroy operation.

Immagine for example a device object that define a variable bound to an open device; destroying the device object imply, at least, closing the device; suppose that there are references to this device created passing a reference to this device at run time; the standard garbage collector technique would be to wait that this device is de-referenced before closing it; this is of course not possible (think of a sound file device where at the end you find an undetermined amount of silence added waiting for the garbage collector to close the file); all the references to this object must have a way to know when the object is closed/destroyed.

Garbage collectors techniques work well when the semantic of a value is not tied to its existence, but its use.

Solutions for this kind of problems exists, but their complexity is surely quite bigger than the current set of solutions and algorithm.

Note: the system currently have a bug in the referential integrity: the metaclass discrimination data base can store a permanent pointer to a value obtained thru a variable argument; in general, this pointer is not dereferenced, but it could with a special purpose equivalence function; this problem need a good solution.

An other limit in the current algorithm is that the almost everywhere in the code there is the assumption that a scope is closed by a patcher; inserting an explicit scope operator, like '::' in C++, like for example composed names ($foo.bar), will break this assumption, and many critical parts of the algorithm will not work any more, in particular the object stealing; the whole code base should be carefully reviewed before a scope operator can be added.

Main Data Structures

The basic types for the variable handling are defined in the file mess_types.h.
  • fts_binding_t : it is the implementation of the binding of a name to a value. i.e. the low level implementation of a variable; for each variable name and scope there can be at most one binding structure. A binding structure include the name of the variable, a flag to tell if the binding is suspended or not, the value of the variable, a list of the objects using the variable and a list of object defining a variable (to handle double definition errors).

  • fts_env_t: it is a list of bindings; represent the set of bindings existing inside a given scope; it is just a list of fts_binding_t.

  • in the fts_object_t: each object store the name of the variable the object define (if any) and the list of bindings the object refer.

  • in the fts_patcher_t: each patcher store an instance of fts_env_t that represent the definitions existing in the patcher.

Main Functions

The variables.c define three groups of functions, one for bindings (static functions named fts_binding_*), one for env (static functions named fts_env_*), and a public set of functions for variables themselves, named fts_variable_*.

Binding related functions
fts_binding_new
fts_binding_delete
Create and delete new Bindings

fts_binding_suspend
fts_binding_restore
fts_binding_is_suspended
Handle variable suspension and restoring at the binding level.

fts_binding_add_user
fts_binding_remove_user
Handle the binding user list, i.e. the list of objects referring this variable.

fts_binding_add_definition
fts_binding_remove_definition
fts_binding_defined_by
fts_binding_defined_only_by
Handle the binding definition list, i.e. the list of objects defining this binding; if there are more than one objects defining the same object, we are in an error situation (double definition).
Environment related functions
fts_env_init
Initialize a patcher environment.

fts_env_add_binding
fts_env_remove_bindings
Add and remove bindings from an environment.

fts_env_remove_suspended_bindings
Remove all the bindings belonging to a specif owner, that are currently suspended; used by the patcher to suppress old variables after a redefinition.

fts_env_suspend_bindings
suspend all the bindings in the environment belonging to an owner. Used by the patcher during patcher redefinition.

fts_env_get_binding
Find a binding for a variable, in the current binding.

Variable related, public functions
These functions are commented with more details in the variables.h file; the variable is specified with a name/scope pair, not with internal structures, and in general they look for the correct binding following the scoping rules.
fts_variable_define
fts_variable_can_define
fts_variable_undefine
fts_variables_undefine
fts_variables_undefine_suspended
Define and undefine variables in a given scope, as their bindings correspondent.

fts_variable_is_suspended
fts_variable_suspend
fts_variables_suspend
fts_variable_restore
Handle suspending and restoring of variables.

fts_variable_get_value
Access a variable value.

fts_variable_add_user
Add a user to a variable, i.e. an object referring it.

Related Files

The variable propagation algorithm is implemented parly in variables specific files, but is also supported by specific code in object instantiation and redefinition; so check also some of the object and expressions related files.

  • variables.c
  • variables.h