Consider moving the validate stage in ConfigManager #10

Closed
opened 6 years ago by novirium · 1 comments
Owner

Currently configuration is validated whenever get_config_bundle is called. This was fine back when the primary way of moving the config dict around was just the raw dict (as the process would be to load data into the confman then immediately take the result out again - the state in the confman was just there to be used for another layer later).

Now with config bundles being used more though, it's becoming more useful (and clearer) to pass around the whole confman instance during initialisation. The problem here is then that the config gets validated every time something is pulled out to check a value during the initialisation and layer process.

Ideally, a better approach may be to shift the validation responsibility to the load() and load_overlay() steps, validating config as soon as it gets loaded. This has issues though, as often the config needs to be lazily validated (particularly where one config bundle has the details required to find the config specifications for the others).

Currently configuration is validated whenever `get_config_bundle` is called. This was fine back when the primary way of moving the config dict around was just the raw dict (as the process would be to load data into the confman then immediately take the result out again - the state in the confman was just there to be used for another layer later). Now with config bundles being used more though, it's becoming more useful (and clearer) to pass around the whole confman instance during initialisation. The problem here is then that the config gets validated _every time_ something is pulled out to check a value during the initialisation and layer process. Ideally, a better approach may be to shift the validation responsibility to the `load()` and `load_overlay()` steps, validating config as soon as it gets loaded. This has issues though, as often the config needs to be lazily validated (particularly where one config bundle has the details required to find the config specifications for the others).
novirium added the [status] Review [type] Enhancement labels 6 years ago
Poster
Owner

A middle ground is probably to better isolate the source dict and the validated config dict. At the moment, the root_config dict sometimes has raw source values in it, and gets changed to valid config values (with defaults populated) only when get_bundle is called, which is less than ideal.

A solution to this would involve loading into a config_source dict, with calls to load() and load_overlay() just setting up this dict (overlaying the frozen config on top). Calls to get_config_bundle() would then first check if the bundle was present in root_config, and if not, deepcopy and validate the bundle from config_source. A validate_bundle() and validate_bundles() method would be provided to force a validation. To avoid loss of synchronisation, the load calls (or anything that modifies the config_source dict) should be careful to wipe the validated root_config dict.

A middle ground is probably to better isolate the source dict and the validated config dict. At the moment, the `root_config` dict sometimes has raw source values in it, and gets changed to valid config values (with defaults populated) only when `get_bundle` is called, which is less than ideal. A solution to this would involve loading into a `config_source` dict, with calls to `load()` and `load_overlay()` just setting up this dict (overlaying the frozen config on top). Calls to `get_config_bundle()` would then first check if the bundle was present in `root_config`, and if not, deepcopy and validate the bundle from `config_source`. A `validate_bundle()` and `validate_bundles()` method would be provided to force a validation. To avoid loss of synchronisation, the `load` calls (or anything that modifies the `config_source` dict) should be careful to wipe the validated `root_config` dict.
novirium added this to the v0.4 milestone 6 years ago
novirium added [status] Available and removed [status] Review labels 6 years ago
novirium closed this issue 6 years ago
novirium added [status] Done and removed [status] Available labels 6 years ago
Sign in to join this conversation.
Loading…
There is no content yet.