Currently configuration is validated whenever get_config_bundle is called. This was fine back when the primary way of moving the config dict around was just the raw dict (as the process would be to load data into the confman then immediately take the result out again - the state in the confman was just there to be used for another layer later).
Now with config bundles being used more though, it's becoming more useful (and clearer) to pass around the whole confman instance during initialisation. The problem here is then that the config gets validated every time something is pulled out to check a value during the initialisation and layer process.
Ideally, a better approach may be to shift the validation responsibility to the load() and load_overlay() steps, validating config as soon as it gets loaded. This has issues though, as often the config needs to be lazily validated (particularly where one config bundle has the details required to find the config specifications for the others).
Currently configuration is validated whenever `get_config_bundle` is called. This was fine back when the primary way of moving the config dict around was just the raw dict (as the process would be to load data into the confman then immediately take the result out again - the state in the confman was just there to be used for another layer later).
Now with config bundles being used more though, it's becoming more useful (and clearer) to pass around the whole confman instance during initialisation. The problem here is then that the config gets validated _every time_ something is pulled out to check a value during the initialisation and layer process.
Ideally, a better approach may be to shift the validation responsibility to the `load()` and `load_overlay()` steps, validating config as soon as it gets loaded. This has issues though, as often the config needs to be lazily validated (particularly where one config bundle has the details required to find the config specifications for the others).
A middle ground is probably to better isolate the source dict and the validated config dict. At the moment, the root_config dict sometimes has raw source values in it, and gets changed to valid config values (with defaults populated) only when get_bundle is called, which is less than ideal.
A solution to this would involve loading into a config_source dict, with calls to load() and load_overlay() just setting up this dict (overlaying the frozen config on top). Calls to get_config_bundle() would then first check if the bundle was present in root_config, and if not, deepcopy and validate the bundle from config_source. A validate_bundle() and validate_bundles() method would be provided to force a validation. To avoid loss of synchronisation, the load calls (or anything that modifies the config_source dict) should be careful to wipe the validated root_config dict.
A middle ground is probably to better isolate the source dict and the validated config dict. At the moment, the `root_config` dict sometimes has raw source values in it, and gets changed to valid config values (with defaults populated) only when `get_bundle` is called, which is less than ideal.
A solution to this would involve loading into a `config_source` dict, with calls to `load()` and `load_overlay()` just setting up this dict (overlaying the frozen config on top). Calls to `get_config_bundle()` would then first check if the bundle was present in `root_config`, and if not, deepcopy and validate the bundle from `config_source`. A `validate_bundle()` and `validate_bundles()` method would be provided to force a validation. To avoid loss of synchronisation, the `load` calls (or anything that modifies the `config_source` dict) should be careful to wipe the validated `root_config` dict.
novirium
added this to the v0.4 milestone 6 years ago
Currently configuration is validated whenever
get_config_bundleis called. This was fine back when the primary way of moving the config dict around was just the raw dict (as the process would be to load data into the confman then immediately take the result out again - the state in the confman was just there to be used for another layer later).Now with config bundles being used more though, it's becoming more useful (and clearer) to pass around the whole confman instance during initialisation. The problem here is then that the config gets validated every time something is pulled out to check a value during the initialisation and layer process.
Ideally, a better approach may be to shift the validation responsibility to the
load()andload_overlay()steps, validating config as soon as it gets loaded. This has issues though, as often the config needs to be lazily validated (particularly where one config bundle has the details required to find the config specifications for the others).A middle ground is probably to better isolate the source dict and the validated config dict. At the moment, the
root_configdict sometimes has raw source values in it, and gets changed to valid config values (with defaults populated) only whenget_bundleis called, which is less than ideal.A solution to this would involve loading into a
config_sourcedict, with calls toload()andload_overlay()just setting up this dict (overlaying the frozen config on top). Calls toget_config_bundle()would then first check if the bundle was present inroot_config, and if not, deepcopy and validate the bundle fromconfig_source. Avalidate_bundle()andvalidate_bundles()method would be provided to force a validation. To avoid loss of synchronisation, theloadcalls (or anything that modifies theconfig_sourcedict) should be careful to wipe the validatedroot_configdict.