Overview of the Settings Attribute#
OpenPNM objects all include a settings attribute which contains certain information used by OpenPNM. The best example is the algorithm classes, which often require numerous settings such as number of iterations and tolerance for iterative calculations. This tutorial will provide an overview of how these settings work, both from the user perspective as well as for developers.
[1]:
import openpnm as op
pn = op.network.Cubic([4, 4,])
geo = op.geometry.SpheresAndCylinders(network=pn, pores=pn.Ps, throats=pn.Ts)
air = op.phases.Air(network=pn)
phys = op.physics.Basic(network=pn, phase=air, geometry=geo)
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
Input In [1], in <cell line: 4>()
2 pn = op.network.Cubic([4, 4,])
3 geo = op.geometry.SpheresAndCylinders(network=pn, pores=pn.Ps, throats=pn.Ts)
----> 4 air = op.phases.Air(network=pn)
5 phys = op.physics.Basic(network=pn, phase=air, geometry=geo)
AttributeError: module 'openpnm' has no attribute 'phases'
Normal Usage#
This section is relevant to users of OpenPNM, while the next section is more relevant to developers
Let’s look an algorithm that has numerous settings:
[2]:
alg = op.algorithms.ReactiveTransport(network=pn, phase=air)
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
Input In [2], in <cell line: 1>()
----> 1 alg = op.algorithms.ReactiveTransport(network=pn, phase=air)
NameError: name 'air' is not defined
We can see that many default settings are already present by printing the settings attribute:
[3]:
print(alg.sets)
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
Input In [3], in <cell line: 1>()
----> 1 print(alg.sets)
NameError: name 'alg' is not defined
We can override these settings manually:
[4]:
alg.sets.prefix = 'rxn'
print(alg.sets)
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
Input In [4], in <cell line: 1>()
----> 1 alg.sets.prefix = 'rxn'
2 print(alg.sets)
NameError: name 'alg' is not defined
We could also have updated these settings when creating the algorithm object by passing in a set of arguments. This can be in the form of a dictionary:
[5]:
s = {"prefix": "rxn"}
alg = op.algorithms.ReactiveTransport(network=pn, phase=air, settings=s)
print(alg.sets)
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
Input In [5], in <cell line: 2>()
1 s = {"prefix": "rxn"}
----> 2 alg = op.algorithms.ReactiveTransport(network=pn, phase=air, settings=s)
3 print(alg.sets)
NameError: name 'air' is not defined
Or as a ‘dataclass’ style, which is how things are done behind the scenes in OpenPNM as described in the section:
[6]:
class MySettings:
prefix = 'rxn'
# alg = op.algorithms.ReactiveTransport(network=pn, phase=air, settings=MySettings())
# print(alg.sets)
One new feature on OpenPNM V3 is that the datatype of some settings is enforced. For instance the 'prefix' setting must be a str, otherwise an error is raised:
[7]:
from traits.api import TraitError
try:
alg.sets.phase = 1
except TraitError as e:
print(e)
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
Input In [7], in <cell line: 2>()
1 from traits.api import TraitError
2 try:
----> 3 alg.sets.phase = 1
4 except TraitError as e:
5 print(e)
NameError: name 'alg' is not defined
OpenPNM uses the traits package to control this behavior, which will be explained in more detail in the next section.
Advanced Usage#
The following sections are probably only relevant if you plan to do some development in OpenPN
In the previous section we saw how to define settings, as well as the data-type protections of some settings. In this section we’ll demonstrate this mechanism in more detail.
OpenPNM has two settings related classes: SettingsData and SettingsAttr. The first is a subclass of the HasTraits class from the traits package. It preceeded the Python dataclass by many years and offers far more functionality. For our purposes the main difference is that dataclasses allow developers to specify the type of attributes (i.e. obj.a must
be an int), but these are only enforced during object creation. Once the object is made, any value can be assigned to a. The traits package offers the same functionality but also enforces the type of a for all subsequent assignments. We saw this in action in the previous section when we tried to assign an integer to alg.sets.prefix.
The SettingsData and HasTraits Classes#
Let’s dissect this process:
[8]:
from openpnm.utils import SettingsData, SettingsAttr
from traits.api import Int, Str, Float, List, Set
---------------------------------------------------------------------------
ImportError Traceback (most recent call last)
Input In [8], in <cell line: 1>()
----> 1 from openpnm.utils import SettingsData, SettingsAttr
2 from traits.api import Int, Str, Float, List, Set
ImportError: cannot import name 'SettingsData' from 'openpnm.utils' (/home/runner/work/OpenPNM/OpenPNM/openpnm/utils/__init__.py)
[9]:
class CustomSettings(SettingsData):
a = Int()
b = Float(4.4)
c = Set()
d = List(Str)
s = CustomSettings()
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
Input In [9], in <cell line: 1>()
----> 1 class CustomSettings(SettingsData):
2 a = Int()
3 b = Float(4.4)
NameError: name 'SettingsData' is not defined
Now we can print s to inspect the settings. We’ll see some default values for things that were not initialized like a, while b is the specified value.
[10]:
print(s)
{'prefix': 'rxn'}
The traits package enforces the datatype of each of these attributes:
[11]:
s.a = 2
s.b = 5.5
print(s)
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
Input In [11], in <cell line: 1>()
----> 1 s.a = 2
2 s.b = 5.5
3 print(s)
AttributeError: 'dict' object has no attribute 'a'
Let’s look at the attribute protection in action again:
[12]:
try:
s.a = 1.1
except TraitError as e:
print(e)
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
Input In [12], in <cell line: 1>()
1 try:
----> 2 s.a = 1.1
3 except TraitError as e:
4 print(e)
AttributeError: 'dict' object has no attribute 'a'
The traits package also enforces the type of values we can put into the list stored in d:
[13]:
s.d.append('item')
try:
s.d.append(100)
except TraitError as e:
print(e)
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
Input In [13], in <cell line: 1>()
----> 1 s.d.append('item')
2 try:
3 s.d.append(100)
AttributeError: 'dict' object has no attribute 'd'
The first one works because we specified a list of strings, while the second fails because it is attempting to write an integer.
Also, we can’t accidentally overwrite an attribute that is supposed to be a list with a scalar:
[14]:
try:
s.d = 5
except TraitError as e:
print(e)
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
Input In [14], in <cell line: 1>()
1 try:
----> 2 s.d = 5
3 except TraitError as e:
4 print(e)
AttributeError: 'dict' object has no attribute 'd'
Gotcha With the HasTraits Class#
When defining a set of custom settings using the HasTraits or SettingsData class, you MUST specify a type for each attribute value. If not then it is essentially ignored.
[15]:
class MySettings(SettingsData):
a = Int(1)
b = 2
mysets = MySettings()
print(mysets)
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
Input In [15], in <cell line: 1>()
----> 1 class MySettings(SettingsData):
2 a = Int(1)
3 b = 2
NameError: name 'SettingsData' is not defined
However, if you create a custom class from a basic python object it will work:
[16]:
class MySettings:
a = 1
b = 2
mysets = MySettings()
print(mysets.a, mysets.b)
1 2
The SettingsAttr Class#
The problem with the HasTraits class is that there is are lot of helper methods attached to it. This means that when we use the autocomplete functionality of our favorite IDEs (spyder and jupyter), we will have a hard time finding the attributes we set amongst the noise. For this reason we have created a wrapper class called SettingsAttr which works as follows:
[17]:
S = SettingsAttr(s)
print(S)
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
Input In [17], in <cell line: 1>()
----> 1 S = SettingsAttr(s)
2 print(S)
NameError: name 'SettingsAttr' is not defined
Importantly only the the user-created attributes show up, which can be test using the dir() command:
[18]:
dir(S)
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
Input In [18], in <cell line: 1>()
----> 1 dir(S)
NameError: name 'S' is not defined
SettingsAttr has as few additional features. You can add a new batch of settings after instantiation as follows:
[19]:
s_new = {'a': 5, 'e': 6}
S._update(s_new)
print(S)
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
Input In [19], in <cell line: 2>()
1 s_new = {'a': 5, 'e': 6}
----> 2 S._update(s_new)
3 print(S)
NameError: name 'S' is not defined
We can see the updated value of a, as well as the newly added e. Because e contained an integer (6), the datatype of e will be forced to remain an integer:
[20]:
try:
S.e = 5.5
except TraitError as e:
print(e)
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
Input In [20], in <cell line: 1>()
1 try:
----> 2 S.e = 5.5
3 except TraitError as e:
4 print(e)
NameError: name 'S' is not defined
Note that the _update method begins with an underscore. This prevents it from appearing in the autocomplete menu to ensure it stays clean.
For the sake of completeness, it should also be mentioned that the CustomSettings object which was passed to the SettingsAttr constructor was stored under _settings. The SettingsAttr class has overloaded __getattr__ and __setattr__ methods which dispatch the values to the _settings attribute:
[21]:
S.d is S._settings.d
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
Input In [21], in <cell line: 1>()
----> 1 S.d is S._settings.d
NameError: name 'S' is not defined
Another aspect to keep in mind is that the _settings attribute is a HasTraits object. This means that all values added to the settings must have an enforced datatype. This is done on the fly, based on the type of value received. For instance, once you set an attribute to string for instance, its type is set:
[22]:
S.f = 'a string'
try:
S.f = 1.0
except TraitError as e:
print(e)
print(S)
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
Input In [22], in <cell line: 1>()
----> 1 S.f = 'a string'
2 try:
3 S.f = 1.0
NameError: name 'S' is not defined
Adding Documentation to a SettingsData and SettingsAttr Class#
One the main reasons for using a dataclass style object for holding settings is so that docstrings for each attribute can be defined and explained:
[23]:
class DocumentedSettingsData(SettingsData):
r"""
A class that holds the following settings.
Parameters
----------
name : str
The name of the object
id_num : int
The id number of the object
"""
name = Str('foo')
id_num = Int(0)
d = DocumentedSettingsData()
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
Input In [23], in <cell line: 1>()
----> 1 class DocumentedSettingsData(SettingsData):
2 r"""
3 A class that holds the following settings.
4
(...)
10 The id number of the object
11 """
12 name = Str('foo')
NameError: name 'SettingsData' is not defined
[24]:
print(d.__doc__)
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
Input In [24], in <cell line: 1>()
----> 1 print(d.__doc__)
NameError: name 'd' is not defined
Note that this docstring was written when we defined DocumentedSettingsData subclass and it attached to it, but we’ll be interacting with the SettingsAttr class. When a SettingsAttr is created is adopts the docstring of the received settings object. This can be either a proper SettingsData/HasTraits class or a basic dataclass style object. The docstring can only be set on initialization though, so any new attributes that are created by adding values to the object
(i.e. D.zz_top = 'awesome') will not be documented.
[25]:
D = SettingsAttr(d)
print(D.__doc__)
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
Input In [25], in <cell line: 1>()
----> 1 D = SettingsAttr(d)
2 print(D.__doc__)
NameError: name 'SettingsAttr' is not defined
This machinery was designed with the idea of inheriting docstrings using the docrep package. The following illustrates not only how the SettingsData class can be subclassed to add new settings (e.g. from GenericTransport to ReactiveTransport), but also how to use the hightly under-rated docrep package to also inherit the docstrings:
[26]:
import docrep
docstr = docrep.DocstringProcessor()
# This docorator tells docrep to fetch the docstring from this class and make it available elsewhere:
@docstr.get_sections(base='DocumentSettingsData', sections=['Parameters'])
class DocumentedSettingsData(SettingsData):
r"""
A class that holds the following settings.
Parameters
----------
name : str
The name of the object
id_num : int
The id number of the object
"""
name = Str('foo')
id_num = Int(0)
# This tells docrep to parse this docstring and insert text at the %
@docstr.dedent
class ChildSettingsData(DocumentedSettingsData):
r"""
A subclass of DocumentedSettingsData that holds some addtional settings
Parameters
----------
%(DocumentSettingsData.parameters)s
max_iter : int
The maximum number of iterations to do
"""
max_iter = Int(10)
E = ChildSettingsData()
print(E.__doc__)
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
Input In [26], in <cell line: 6>()
2 docstr = docrep.DocstringProcessor()
5 # This docorator tells docrep to fetch the docstring from this class and make it available elsewhere:
6 @docstr.get_sections(base='DocumentSettingsData', sections=['Parameters'])
----> 7 class DocumentedSettingsData(SettingsData):
8 r"""
9 A class that holds the following settings.
10
(...)
16 The id number of the object
17 """
18 name = Str('foo')
NameError: name 'SettingsData' is not defined
And we can also see that max_iter was added to the values of name and id_num on the parent class:
[27]:
E.visible_traits()
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
Input In [27], in <cell line: 1>()
----> 1 E.visible_traits()
NameError: name 'E' is not defined
Again, as mentioned above, this inherited docstring is adopted by the SettingsAttr:
[28]:
S = SettingsAttr(E)
print(S.__doc__)
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
Input In [28], in <cell line: 1>()
----> 1 S = SettingsAttr(E)
2 print(S.__doc__)
NameError: name 'SettingsAttr' is not defined
Attaching to an OpenPNM Object#
The SettingsAttr wrapper class is so named because it is meant to be an attribute (i.e. attr) on OpenPNM objects. These attached to the settings attribute:
[29]:
isinstance(alg.sets, SettingsAttr)
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
Input In [29], in <cell line: 1>()
----> 1 isinstance(alg.sets, SettingsAttr)
NameError: name 'alg' is not defined
OpenPNM declares SettingsData classes with each file where class is defined, then this is attached upon initialization. This is illustrated below:
[30]:
class SpecificSettings(SettingsData):
a = Int(4)
class SomeAlg:
def __init__(self, settings={}, **kwargs):
self.settings = SettingsAttr(SpecificSettings())
self.settings._update(settings)
alg = SomeAlg()
print(alg.settings)
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
Input In [30], in <cell line: 1>()
----> 1 class SpecificSettings(SettingsData):
2 a = Int(4)
5 class SomeAlg:
NameError: name 'SettingsData' is not defined
Or with some additional user-defined settings and overrides:
[31]:
s = {'name': 'bob', 'a': 3}
alg2 = SomeAlg(settings=s)
print(alg2.settings)
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
Input In [31], in <cell line: 2>()
1 s = {'name': 'bob', 'a': 3}
----> 2 alg2 = SomeAlg(settings=s)
3 print(alg2.settings)
NameError: name 'SomeAlg' is not defined