Name

Type

Usage

Default

Description

trackable

integer

M

Index of the trackable in the trackables array that will be used for this anchor.

requiresAnchoring

boolean

M

Indicates if AR anchoring is required for the rendering of the associated nodes.

If TRUE, the application shall skip the virtual assets attached to this anchor until the pose of this anchor in the real world is known.

if FALSE, the application shall process the virtual assets attached to this anchor

minimumRequiredSpace

vec3

O

(0,0,0)

Space required to anchor the AR asset (width, height, depth in meters). This space corresponds to an axis-aligned bounding box, placed at the origin of the trackable space. Width is aligned with x axis, height with y axis and depth with z axis, expressed in trackable local space. This value shall be compared to the bounding box of the real-world available space associated with the trackable as estimated by the XR runtime.

aligned

enumeration

O

NOT_USED

The aligned flag may take one of the following values: NOT_USED=0, ALIGNED_NOTSCALED=1, ALIGNED_SCALED=2.

If ALIGNED_SCALED is set, the bounding box of the virtual assets attached to that anchor is aligned and scaled to match the bounding box of the real-world available space associated with the trackable as estimated by the XR runtime.

actions

array(number)

O

N/A

Indices of the actions in the actions array of the interactivity extension to be executed once the pose of this anchor is determined. An example is a setTransform action to place the virtual assets attached to that anchor.

light

integer

O

N/A

Reference to an item in the lights array of the MPEG_lights_texture_based extension.

recommendedSpatialComputingConfig

object

O

N/A

Provides a set of recommended parameters specifying the needed spatial description. The semantics of this object are given in Table 8.1- 1.

Name

Type

Usage

Default

Description

scanOptions

array

O

N/A

Array of options (enumeration) for the scan computation: possible values are given in

Table 8.1- 2.

scanDetails

object

O

N/A

Specifies the required level of detail for the representation. The semantics are presented in Table 8.1- 3.

scanUpdate

object

O

N/A

Specifies the frequency at which the spatial description must be updated (Table 8.1- 4).

scanVolumes

array

O

N/A

Array of bounding volumes that determine the spaces where scanned objects must be provided. Real scan objects that intersect one or more of the bounding volumes should be provided, and all other objects ignored. The semantics for these volumes are presented in Table 8.1- 5.

realSemantic

array

O

N/A

Semantic descriptions of nodes that are needed (“table”, “room”, “chair”, “wall”, “light”, “freespace” …)

lightOptions

array

O

N/A

Array of option (enumeration) for the light extraction: possible values are given in Table 8.1- 6.

lightUpdate

object

O

N/A

Specifies the frequency at which the extraction of real light must be updated (Table 8.1- 4)

Enumeration value

Description

PLANE = 0

Request plane data for scanned objects

PLANAR_MESH

Request planar meshes for scanned objects

VISUAL_MESH

Request 3D visualization meshes for scanned objects

COLLIDER_MESH

Request 3D collider meshes for scanned objects

FREE_VOLUME

Request to get the available space around a trackable

BOUNDING_BOX

Request a simplified collider mesh

TEXTURED_MESH

Request mesh with a texture

Name

Type

Usage

Default

Description

primitivesNumber

number

O

N/A

Gives the quantity of geometric primitives per cubic meter (m³)

textureOption

enumeration

O

LOW_RES

Gives the resolution of the texture of textured meshes.
Possible values are 512X512 (0), 1024X1024 (1) or 2048X2048 (2).

Name

Type

Usage

Default

Description

occurrence

enumeration

O

ONCE

An occurrence value:

— ONCE = 0: the update is performed only once, for instance in case of a static real scene,

— N_FRAME: the update is performed periodically, every N rendering frames, N depending on the dynamism of the real scene. The N value is provided in the frameNumber parameter.

— AUTO: the update frequency is managed by the module that computes the representation. This module may perform an analysis to detect significant changes in the real world and start an update. This analysis may be performed from raw images data, like RGB images from a camera or depth images from a depth sensor.

numberOfFrames

number

O

N/A

Indicates the periodicity, in number of frames, of the update, when the occurrences value is N_FRAME.

Name

Type

Usage

Default

Description

type

enumeration

M

SPHERE=0, BOX, FRUSTUM

If (type == SPHERE)

center

array

M

3D coordinates of the center of the sphere

radius

number

M

Radius of the sphere in meters.

If (type == BOX)

pose

matrix

M

4*4 matrix representing the center position and orientation of the box

extents

array

O

Edge-to-edge length of the box along each dimension.

If (type == FRUSTUM)

pose

matrix

M

4*4 matrix representing the position and orientation of the tip of the frustum

fov

vec4

M

Angles of the four sides of the frustum

far

number

M

Positive distance of the far plane of the frustum

near

number

M

Positive distance of the near plane of the frustum

3D coordinates are expressed in the XR space related to trackable associated to the anchor.

Enumeration value

Description

DirectionalLight = 0

Request the extraction of directional lights

EnvLight

Request the extraction of environment lights

PointLight

Request the extraction of point lights

SpotLight

Request the extraction of spot lights

AreaLight

Request the extraction of area lights

if (type== TRIGGER_USER_INPUT){

userInputDescription

string

M

Describes the user body part and gesture related to the input. The format shall follow the OpenXR input path description as defined in [OpenXR] section 6. An example is: “/user/hand/left/grip”.

nodes

array

O

N/A

Indices of the nodes in the nodes array to be considered for this user input.

}

if (type== TRIGGER_USER_INPUT){

userInputDescription

string

M

 Describes the user body part and gesture related to the input. The format shall follow the OpenXR input path description as defined in [OpenXR] section 6. An example is: “/user/hand/left/grip”.

The annex I gives extra semantics for user input triggers involving multiple users.

nodes

array

O

N/A

Indices of the nodes in the nodes array to be considered for this user input.

}

Name 

Type 

Usage 

Default 

Description 

physics

object

O

N/A

Provides a set of parameters at scene level to be used for the physics simulation. The semantics of this object are given in table 8.2-1.

triggers

array 

M 

[] 

Contains the definition of all the triggers used in that scene

actions

array 

M 

[]

Contains the definition of all the actions used in that scene

behaviors

array 

M 

[]

Contains the definition of all the behaviors used in that scene. A behavior is composed of a pair of (triggers, actions), control parameters of triggers and actions, a priority weight and an optional interrupt action

Name 

Type 

Usage 

Default 

Description 

recommendedPhysicsHighPrecision

boolean

O

false

Determines whether the application should enable a more deterministic and precise physic simulation

gravity

number

O

-9.81

Determine the gravity for the whole scene. In meters per second square (m.s^-2), as defined in the international unit system.

recommendedPhysicsFrameRate

number

O

50

Provides the recommended frame rate at which the Physics Engine should operate. In frame per second, as defined in the international unit system.

bounceThreshold

number

O

1

A contact with a relative velocity below this threshold will not result in a bounce. In meter per second (m.s^-1), as defined in the international unit system.

Name

Type

Usage

Default

Description

type

enumeration

M

One element of Table 34 that defines the type of the trigger.

if (type == TRIGGER_COLLISION){

collider

integer

M

the index of the mesh element that provides the collider geometry for the current node.

The collider mesh may reference a material.

isStatic

boolean

M

If True, the collider is defined as a static collider.

physics

object

O

N/A

Provides a set of parameters at node level to be used for physics simulation. The semantics of this object are given in table 8.2-2.

primitives

array(Primitive)

O

N/A

List of primitives used to activate the proximity or collision trigger.

Semantics are presented in Table 35.

}

…

Name 

Type 

Usage 

Default 

Description 

needPreciseCollisionDetection

boolean

O

false

If true, the physics engine should handle the collision detection more accurately by increasing the detection rate for this node.

linearDamping

number

O

0

A non-negative value, in second^-1 (s^-1), as defined in the international unit system. It defines the linear drag coefficient which corresponds to the rate of decrease of the linear velocity over time.

It is used to compute a new velocity value V(t) at each simulation step (dt):

V(t+dt) = V(t)*(1-linearDamping*dt), the velocity being clamped to 0.

angularDamping

number

O

0

A non-negative value, in second^-1 (s^-1), as defined in the international unit system. It defines the angular drag coefficient which corresponds to the rate of decrease of the angular velocity over time.

It is used to compute a new velocity value V(t) at each simulation step (dt):

V(t+dt) = V(t)*(1-angularDamping*dt), the velocity being clamped to 0.

useGravity

boolean

M

Indicates if gravity affects the object

mass

number

M

Mass of the object in kilogram, as defined in the international unit system.

restitution

number

M

Provides the ratio of the final to initial relative velocity between two objects after they collide

staticFriction

number

M

Unitless static friction coefficient as defined in the Coulomb friction model [6]. Friction is the quantity which prevents surfaces from sliding off each other. Static friction is used when the object is lying still. It will prevent the object from starting to move.

dynamicFriction

number

M

Unitless static friction coefficient as defined in the Coulomb friction model. When a large enough force is applied to the object, a dynamic friction is used and will attempt to slow down the object while in contact with another.

}

init()

Initializes the external renderer by providing the related media source information and their corresponding buffers. It also establishes a session between the Presentation Engine and the external renderer.

The inputs to this method call should be:

— A media source object that contains a handler to the buffer(s), where the source media will be made available by the MAF. A description of the media source and the contents of each buffer shall also be provided.

configure()

Configures the external renderer to establish an initial alignment and synchronization between the Presentation Engine and the external renderer.

The parameters to this method may include:

— A mapping between the initial timestamp of the common Presentation Engine timeline and that of the media associated with the external renderer. It also provides information about the clock rate of the Presentation Engine.

— A list of mapped nodes in the source media rendered by the external renderer. This list shall at least contain one object with a mapping to the main camera of the main scene description. For audio renderers, this may be the audio listener. The information is provided by the MPEG_node_mapping extension in the scene description document. It should also provide the initial pose of the mapped nodes after applying the 4x4 matrix of the transform parameter provided in the node mappings.

The external renderer may then subscribe for updates to specific aligned nodes, or it may specifically ask for current state for these nodes, using the referenceId.

start()

pause()

resume()

stop()

Allows the Presentation Engine to control the playback of selected media sources associated with the external renderer for interactivity purposes.

update()

Used by the Presentation Engine to update node positions and orientations for which there is a mapping with the external renderer. Updates may result from received scene updates, user interactions, animations, physics simulations, or any other events. The Presentation Engine uses this update() method when one or more mapped nodes need to be spatially synchronized, depending on their synchronizationOccurrence and synchronizationOccurenceCombination parameters provided in their MPEG_node_mapping extension.

The parameters passed to this method are an array of objects consisting of:

— The referenceId of the node to which this update applies

— The current pose of the mapped node after applying the 4x4 matrix of the transform parameter provided in the corresponding MPEG_node_mapping

updateGraph()

The Presentation Engine uses the updateGraph function to add, update, or remove a set of nodes to the internal representation of the scene that is maintained by the external renderer. Only nodes that have a mapping with the external renderer can be passed through this method.

The parameters to this method are an array of objects that include:

— The graph operation: ADD, REMOVE, UPDATE

— For ADD: the referenceId and the initialization information for the associated media data to the object that is to be added.

— For REMOVE: the referenceId of the object to be removed.

— For UPDATE: the referenceId of the object to be updated, as well as a dictionary of attributes and their update values.

registerCallback()

The Presentation Engine may provide a callback function to the external renderer to allow it to query the status of certain parameters at any time. This may for example include asking for the current user pose.

The Presentation Engine shall register a callback function whenever possible.

interface GenericRenderControl {   
void init();
   void configure();
void start();
void pause();
void resume();
void stop();
update();
void updateGraph();
   void registerCallback();
};

mappings

array(object)

M

An array of mappings associated with the containing node.

component

string

“urn:mpeg:sd:component:default”

O

An identifier of the component associated with this mapping. The component may for instance be

“urn:mpeg:sd: component:audio-renderer” to indicate that the component is an audio renderer.

source

number

N/A

M

The index in the MPEG_media that provides the media resource that contains the mapped element.

referenceId

number

N/A

M

An identifier of the element in the referenced resource.

transform

array(number)

Identity

O

A 4x4 TRS matrix which transforms the 3D coordinates of the node having this glTF extension expressed in the glTF2.0 scene coordinate system to the 3D coordinates of the node of the external renderer graph referenced by the referenceId identifier expressed in the external renderer scene coordinate system.

If the mapped node is a child of an AR Anchor/Trackable, the 4x4 TRS matrix transforms the 3D coordinates of the node having this glTF extension expressed in the AR Anchor/Trackable coordinate system to the 3D coordinates of the node of the external renderer graph referenced by the referenceId identifier expressed in the AR Anchor/Trackable coordinate system.

updateRecommendation

object

N/A

O

Indicate update recommendations for the node. The semantics is given in table x2.

supportsInteractivity

boolean

false

O

Indicates if interactivity actions applied to the node should be exposed if an API is made available to the Presentation Engine by the renderer of the resource.

synchronizationOccurrences

array(enumeration)

[EVENT]

O

An array of synchronization occurrences. Each element of this array is an Enumerator with the following possible values:

— ONCE: the synchronization is done once at the configuration step,

— EVENT: the synchronization is done based on the activation of one or more triggers (e.g., visibility, proximity, collision, user input) as those defined in the MPEG_interactivity extension. The indices of the triggers, from the triggers array of the MPEG_scene_interactivity extension, are given in the events parameter,

— N_FRAME: the synchronization is periodic every N (1, 2, …) rendering frames. The N value is provided in the frameNumber parameter.

events

array(number)

N/A

0

Array of indices of triggers from the triggers array of the MPEG_scene_interactivity extension. Required when EVENT is mentioned in the synchronizationOccurrences array.

frameNumber

number

N/A

0

Indicate the periodicity, in number of frames, of the synchronisation, when N_FRAME is mentioned in the synchronizationOccurrences array.

synchronizationOccurrenceCombination

string

“|”

O

A set of logical operations to apply to the synchronization occurrences. A ‘#’ indicates the occurrence index, ‘&’ indicates a logical AND operation, ‘|’ a logical OR operation and ‘~’ a NOT operation. Parenthesis are used to group some operations. Such a syntax may give the following string: “#1&~#2|(#3&#4)”.

Name

Type

Usage

Default

Description

type

enumeration

M

One element that defines the type of the trigger.

…

extras

object

O

N/A

multiUsers

object

O

N/A

Additional parameters to address a TRIGGER_USER_INPUT trigger involving multiple users. See Table I.2- 1 for the details of this object.

Name

Type

Usage

Default

Description

multiCondition

enumeration

O

ALL

If ALL, the trigger’s condition must be met by all the users.

If ONLYN, the trigger’s condition must be met by exactly N users.

If ATLEASTN, the trigger’s condition must be met by at least N users.

duration

number

O

2

Give a time duration, in second.

After the trigger condition is met by a first user, it specifies a time window during which the multiCondition must be met.

number

number

O

1

Give the number of users when condition is ONLYN or ATLEASTN.