Usage of augmentedtree - detailed examples

This chapter shows more detailed examples than within the chapter Basic Usage. These simplified examples are taken from existing projects and resembles the major issues augmentedtree was written for.

**Accessing values** - Example on how to retrieve nested values. - Using the nested example data - items are selected using a single key. - Refining the selection retrieves exact values.

**Usage of Schemas** - Getting a quick view on the relevant (by your definition) values. - How to define schemas - How the data looks without schemas - How the data looks with schemas - How schemas effect selecting items

**Use-case of the *or*-conditional selection** in combination with setting multiple values.

Accessing values - ‘Where did I put it again?’

Did you ever wondered ‘Where did i put it?’? You were about to write an analysis script and put the configuration values into a JSON dump beforehand by another preliminary process. Or you can’t remember the structure of the nested output and its key names?

Nested example data

The exemplary data can be found within a json-file.

import json
from dicthandling import read_from_json_file

# load and show the nested data
nested_data = read_from_json_file("resources/nested_data_of_examples.json", "detailed/example-1")
print(json.dumps(nested_data, indent="  "))

{
  "section-name": {
    "7h3-P4r7-Y0u-C4n7-R3m-3mb3r": {
      "metatype": "my-man",
      "type": "worker",
      "metadata-1": 24,
      "metadata-2": "value needed for a function",
      "name-of-this-item": "Gerry",
      "tasks": {
        "class": "TaskCollection",
        "description": "Tasks 'Gerry' should do.",
        "items": [
          {
            "metatype": "task",
            "class": "WorkerTask",
            "name": "prepare-task",
            "arg-1": "Not this one.",
            "arg-2": "Not this one either."
          },
          {
            "metatype": "task_of_gerry",
            "class": "WorkerTask",
            "name": "get-cracking",
            "arg-1": "This one you want."
          }
        ]
      },
      "another-parameter": [
        1,
        2,
        3
      ]
    }
  },
  "etc.": "..."
}

Selecting specific items

In this example all items with a key arg-1 are selected using select(*path_parts). The desired value can be obtained by knowing the item’s index.

from augmentedtree import AugmentedTree, ALL_ITEMS

# augment the nested data
atree = AugmentedTree(nested_data)

# get a selection of all items with the key 'arg-1'
taskarg1_selection = atree.select("arg-1")

# take a look on the selected items using the explicit print method of
# the selection
taskarg1_selection.print()

# in this example get all values from the selection using a slice (ALL_ITEMS)
# equivalent to [:] (using ALL_ITEMS makes the code more understandable)
taskarg1_of_alltasks = taskarg1_selection[ALL_ITEMS]

# here it is known the desired arg-1 is at the end.
taskarg1_of_getcracking = taskarg1_of_alltasks[-1]

print("\narg-1 of 'get-cracking': {}".format(taskarg1_of_getcracking))

#0 Not this one.
#1 This one you want.

arg-1 of 'get-cracking': This one you want.

Refining the selection

In the prior example knowledge of the items occurrence is required to obtain the desired value. In cases where you don’t know the position of the item a refinement of the selection comes handy. Exact values/items can be retrieved using where(*path_parts).

arg1_selection = atree.select("arg-1")
# here: using 'where' on the prior made selection returns only one value
crackpoint = arg1_selection.where("get-cracking")

crackpoint.print()

# which can be accessed using the first index
taskarg1_of_getcracking = crackpoint[0]

print("\narg-1 of 'get-cracking': {}".format(taskarg1_of_getcracking))

#0 This one you want.

arg-1 of 'get-cracking': This one you want.

Usage of Schemas - Getting a quick view on the relevant values

With schemas a more semantic like behavior can be applied to the nested data. Schemas are defined using dictionaries and it planned to implement JSON-schemas.

Interpretation of metadata within this package

By using schemas values can be classified as metadata. In this context values are classified as metadata

• if these values are not essential for the impression of your data, therefore can be hidden from the view.
• Additional not essential values which can be understood as attributes of an entity. These will be used selecting values by where.
• Besides attributes there can be data, which is not directly related for the entity but is used for process control. E.g. an unique identifier generated at runtime.

The distinction is based on your interpretation. What do you want to tell the viewer?

Example: A blue tennis ball will be enough data for the majority to depict such an object. Since the default color of tennis balls is yellow, the color blue in that case is essential to forward this information, to make the deviation of the standard clear. Diameter, mass, manufacturer, production date, etc. are additional attributes of a tennis ball, but are needed for specific occasions only. (“Pass me the blue ACME tennis ball, which was made before 2020.02.02”). An ‘runtime metadata’ of a tennis ball is the store’s article number for the cashier system.

Using schemas

Schemas are defined by a dictionary with specific entries. The recommend way to define a schema is by using the construct-method of MappingSchemaBuilder. For further explanation see the section Schemas.

schema = MappingSchemaBuilder.construct(
    identifier=("key-within-the-target-mapping", "identifier"),
    primarykey="key-which-value-is-used-as-primekey",
    primaryname="key-which-value-is-used-as-primename",
    additional_metafieldkeys=["keys", "treadend", "as", "metadata"]
}

In the example below three schemas are used for the nested data, giving the output a different meaning.

Schemas have to be explicitly defined for usage using the use_MappingSchema_schema() method.

from augmentedtree import MappingSchema, use_mappingtree_schemas, MappingSchemaBuilder

schemas_as_kwargs = read_from_json_file(
    "resources/nested_data_of_examples.json",
    "detailed/example-1-schemas"
)
schemas = MappingSchemaBuilder.construct_from_collection(schemas_as_kwargs)

# How an example schema definition looks like.
GERRY_SCHEMA = {
                MappingSchema.IDENTIFIER: ("metatype", "my-man"),
                MappingSchema.PRIMARYKEY: "type",
                MappingSchema.PRIMARYNAME: "name-of-this-item",
                MappingSchema.METAFIELDKEYS: [
                    "metatype",
                    "type",
                    "name-of-this-item",
                    "metadata-1",
                    "metadata-2"
                ]
            }

use_mappingtree_schemas(GERRY_SCHEMA, *schemas)

Representation without schemas

This output shows the nested data in its ‘natural’ occurrence.

tree_like_it_is = AugmentedTree(nested_data, use_schemas=False)
tree_like_it_is.print()

{..}
  section-name:
    7h3-P4r7-Y0u-C4n7-R3m-3mb3r:
      metatype: my-man
      type: worker
      metadata-1: 24
      metadata-2: value needed for a function
      name-of-this-item: Gerry
      tasks:
        class: TaskCollection
        description: Tasks 'Gerry' should do.
        items:
          0.
            metatype: task
            class: WorkerTask
            name: prepare-task
            arg-1: Not this one.
            arg-2: Not this one either.
          1.
            metatype: task_of_gerry
            class: WorkerTask
            name: get-cracking
            arg-1: This one you want.
      another-parameter: [1, 2, 3]
  etc.: ...

Representation using schemas

Using schemas can reduce the needed vertical space and increase readability.

tree_using_schemas = AugmentedTree(nested_data)
tree_using_schemas.print()

{..}
  section-name:
    worker: Gerry
      tasks: Tasks 'Gerry' should do.
        prepare-task: WorkerTask
          arg-1: Not this one.
          arg-2: Not this one either.
        get-cracking: WorkerTask
          arg-1: This one you want.
      another-parameter: [1, 2, 3]
  etc.: ...

Impact of schemas on selecting items

Using schemas also makes selecting items more natural to he user.

atree = AugmentedTree(nested_data)

# get a selection of all 'arg-1' items with 'get-cracking' in the path
taskarg1_of_getcracking = tree_using_schemas.select("get-cracking", "arg-1")

taskarg1_of_getcracking.print()

#0 This one you want.

Schemas can be redefined setting the override_existing parameter of use_MappingSchema_schema to True. In the following example the field arg-1 is added to the metadata and will be hidden from the standard view (compare to prior output above). Metadata can be additionally listed using the additional_columns parameter of the tree item’s print-method.

TASK_SCHEMA = MappingSchemaBuilder.construct(
    identifier=("class", "WorkerTask"),
    primarykey="name",
    primaryname="class",
    additional_metafieldkeys=["metatype", "arg-1"]
)

use_mappingtree_schemas(TASK_SCHEMA, override_existing=True)

tree_using_schemas = AugmentedTree(nested_data)
tree_using_schemas.print(additional_columns=["@arg-1", "arg-2"])

                                              @arg-1                arg-2
{..}                                  '                    '
  section-name:                       '                    '
    worker: Gerry                     '                    '
      tasks: Tasks 'Gerry' should do. '                    '
        prepare-task: WorkerTask      ' Not this one.      ' Not this one either.
          arg-2: Not this one either. ' Not this one.      ' Not this one either.
        get-cracking: WorkerTask      ' This one you want. '
      another-parameter: [1, 2, 3]    '                    '
  etc.: ...                           '                    '

Use-case of the or-conditional selection

The next example combines the ‘usage of schemas’ for shortening the view, and setting multiple values at once, where a specific set of values need to be changed.

Here some script went wrong. After the ‘bug’ was fixed and a lot of job files has to be ‘reset’ to a specific configuration.

The nested data is a broadly simplified example. It consists of a list with 4 dictionaries resembling what could be simple task definitions.

from augmentedtree import use_mappingtree_schemas, AugmentedTree, MappingSchema, ALL_ITEMS
from dicthandling import read_from_json_file


task_list = read_from_json_file("resources/nested_data_of_examples.json", address="detailed/tasklist")
import json
print(json.dumps(task_list, indent="  "))

[
  {
    "metatype": "task",
    "state": "done",
    "args": "bread",
    "task": "buy"
  },
  {
    "metatype": "task",
    "state": "done",
    "args": "bread",
    "task": "take a slice of"
  },
  {
    "metatype": "task",
    "state": "done",
    "args": "sandwich",
    "task": "prepare"
  },
  {
    "metatype": "task",
    "state": "failed",
    "args": "sandwich",
    "task": "eat"
  }
]

By using schemas the meaning is be made easier to read and needed vertical space shortened.

schema_parameters = read_from_json_file("resources/nested_data_of_examples.json", address="detailed/tasklist-schemas")
schemas = MappingSchemaBuilder.construct_from_collection(schema_parameters)

use_mappingtree_schemas(*schemas)

task_tree = AugmentedTree(task_list)
task_tree.print()

[..]
  buy. bread
    state: done
  take a slice of. bread
    state: done
  prepare. sandwich
    state: done
  eat. sandwich
    state: failed

Now we get to part, where we want to reset 3 specific tasks, because the first task doesn’t need to be repeated. After the selection using the ‘or’ condition for the first path part, we check if the selection returned something.

tasks_to_repeat = task_tree.select(("take", "prepare", "eat"), "state")
tasks_to_repeat[ALL_ITEMS] = "to-do"

task_tree.print()

[..]
  buy. bread
    state: done
  take a slice of. bread
    state: to-do
  prepare. sandwich
    state: to-do
  eat. sandwich
    state: to-do