sequence of 'phases' object in rt-app config is undefined - should use an array instead?

[v0lker]

according to the JSON standard, the 'phases' object is unordered (http://www.json.org):
"An object is an unordered set of name/value pairs.[...] An array is an ordered collection of values. [...] A value can be a string in double quotes, or a number, or true or false or null, or an object or an array. These structures can be nested."

wouldn't it be necessary to use an array here, instead of:

so, instead of:

{
    "global": { ... },
    "tasks": {
        "thread1" : {
            "phases": {
                "phase1": { "run": 10, "sleep": 20 },
            }
        }
    }
}

or

{
    "global": { ... },
    "tasks": {
        "thread1" : {
            "run": 10,
            "sleep": 20
        }
    }
}

it would be

{
    "global": { ... },
    "tasks": {
        "thread1" : {
            "phases": [
                { "phase1": [ {"run": 10}, {"sleep": 20 }] }
            ]
        }
    }
}

or

{
    "global": { ... },
    "tasks": {
        "thread1" : [
            { "run": 10}, {"sleep": 20 }
        ]
    }
}

(updated the syntax above, because ironically i got it wrong..)

[v0lker]

one could possibly make it backwards compatible by accepting either... (but prefer the array, as in, warn if it isn't one)

[vingu-linaro]

If we want to go in this direction, not only "phases" should be an array but also each phase in "phases" as the events are ordered.
Then, "phases" is optional and events can be directly put in the task node ... which should become a array too.
If we want to move in that direction, we must keep backward compatibility:

• continue to support current syntax
• continue support the fact that "phases" is optional and event can be put directly in "thread1"

[credp]

There's an implicit ordering assumption in parsing the json inside rt-app - we build the execution tree as events are delivered, and the parser we use always delivers the events in the order they are present in the source file.

I don't really see much value in changing the syntax, but maybe we could document this expectation?

[v0lker]

yes, it's partially about the design working only accidentally:

• it's "just wrong"(tm), because it says the wrong thing about the data (suggesting the phases are not ordered)
• it's not parsed in rt-app but inside a library that could justifiably change behaviour (use a hash to store the dictionaries)
• when creating the JSON in e. g. python, have to make sure to use an OrderedDict, in order to get "sleep" before "run", for example, but a bit of a hassle if you're generating the config in a language that doesn't have ordered dicts.

there is also at least one actual limitation:

• can't have multiple steps with the same name within the same phase - e. g. {"run": 10, "sleep": 20, "run": 30, "sleep": 40}

[douglas-raillard-arm]

@credp

the parser we use always delivers the events in the order they are present in the source file.

Is there anything I missed in the source code backing that ?
Here is what I found for the phases, and it also applies for the events since it's using the same foreach macro.

From what I can see in parse_task_data():

		foreach(phases_obj, entry, key, val, idx) {
			log_info(PIN "Parsing phase %s", key);
			assure_type_is(val, phases_obj, key, json_type_object);
			parse_task_phase_data(val, &data->phases[idx], data, opts);
			/*
			 * Uses thread's current sched_data and taskgroup_data
			 * to detect policy/taskgroup misconfiguration.
			 */
			check_taskgroup_policy_dep(&data->phases[idx], data);
		}

We use this foreach macro that is defined as:

#define foreach(obj, entry, key, val, idx)				\
	for ( ({ idx = 0; entry = json_object_get_object(obj)->head;});	\
		({ if (entry) { key = (char*)entry->k;			\
				val = (struct json_object*)entry->v;	\
			      };					\
		   entry;						\
		 }							\
		);							\
		({ entry = entry->next; idx++; })			\
	    )

So the order is determined by the output of json_object_get_object(), itself declared as:

JSON_EXPORT struct lh_table *json_object_get_object(const struct json_object *obj);

So the lh_table structure decides of the order, which is a proper hashtable, which is (indirectly) created by json_object_new_object(). The hashtable itself is created at:

	    lh_kchar_table_new(JSON_OBJECT_DEF_HASH_ENTRIES, &json_object_lh_entry_free);

with a hash function for strings.
https://github.com/json-c/json-c/blob/master/json_object.c#L562

New keys are added by the JSON parser to objects using json_object_object_add(), which does not seem to do anything special to preserve the order:
https://github.com/json-c/json-c/blob/master/json_object.c#L620

EDIT: it does indeed preserve the order of insertion by maintaining a linked list of the items in the hash table:
https://github.com/json-c/json-c/blob/master/linkhash.c#L613