Metaclass conflict with Keras 2.3.0

[HunterMcGushion]

• Using HyperparameterHunter with the recently released Keras version 2.3.0 raises metaclass conflict error, breaking Keras compatibility
• TEMPORARY WORKAROUND: Use Keras 2.2.5

Error Traceback

• From executing examples/keras_examples/experiment_example.py

Traceback (most recent call last):
  File "keras_examples/experiment_example.py", line 1, in <module>
    from hyperparameter_hunter import Environment, CVExperiment
  File "/Users/Hunter/hyperparameter_hunter/hyperparameter_hunter/__init__.py", line 29, in <module>
    from .environment import Environment
  File "/Users/Hunter/hyperparameter_hunter/hyperparameter_hunter/environment.py", line 27, in <module>
    from hyperparameter_hunter.keys.makers import CrossExperimentKeyMaker
  File "/Users/Hunter/hyperparameter_hunter/hyperparameter_hunter/keys/makers.py", line 28, in <module>
    from hyperparameter_hunter.library_helpers.keras_optimization_helper import initialize_dummy_model
  File "/Users/Hunter/hyperparameter_hunter/hyperparameter_hunter/library_helpers/keras_optimization_helper.py", line 52, in <module>
    from keras.callbacks import Callback as base_keras_callback
  File "/Users/Hunter/hyperparameter_hunter/env/lib/python3.7/site-packages/keras/__init__.py", line 3, in <module>
    from . import utils
  File "/Users/Hunter/hyperparameter_hunter/env/lib/python3.7/site-packages/keras/utils/__init__.py", line 26, in <module>
    from .vis_utils import model_to_dot
  File "/Users/Hunter/hyperparameter_hunter/env/lib/python3.7/site-packages/keras/utils/vis_utils.py", line 7, in <module>
    from ..models import Model
  File "/Users/Hunter/hyperparameter_hunter/env/lib/python3.7/site-packages/keras/models.py", line 10, in <module>
    from .engine.input_layer import Input
  File "/Users/Hunter/hyperparameter_hunter/env/lib/python3.7/site-packages/keras/engine/__init__.py", line 8, in <module>
    from .training import Model
  File "/Users/Hunter/hyperparameter_hunter/env/lib/python3.7/site-packages/keras/engine/training.py", line 14, in <module>
    from . import training_utils
  File "/Users/Hunter/hyperparameter_hunter/env/lib/python3.7/site-packages/keras/engine/training_utils.py", line 17, in <module>
    from .. import metrics as metrics_module
  File "/Users/Hunter/hyperparameter_hunter/env/lib/python3.7/site-packages/keras/metrics.py", line 34, in <module>
    class Metric(Layer):
  File "/Users/Hunter/hyperparameter_hunter/env/lib/python3.7/site-packages/six.py", line 849, in wrapper
    return metaclass(cls.__name__, cls.__bases__, orig_vars)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/abc.py", line 126, in __new__
    cls = super().__new__(mcls, name, bases, namespace, **kwargs)
TypeError: metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases

Bug Hunt Notes

• Updating tracers.ArgumentTracer to descend from ABCMeta does resolve the metaclass conflict, but leads to different errors regarding the usage of __deepcopy__ in TensorFlow:
  • NotImplementedError: __deepcopy__() is only available when eager execution is enabled.
• Further changes to the Keras API cause some unrelated tests to fail
• Noteworthy changes in Keras 2.3.0 that will require HH fixes:
  • Introduced class-based losses
  • Introduced class-based metrics
  • Renamed lr to learning_rate for all optimizers
  • Deprecated decay argument for all optimizers
  • Changed how metric names are reported to use the exact string given by the user ("acc" vs. "accuracy")

[OverLordGoldDragon]

Subclassing ArgumentTracer(abc.ABCMeta) in hyperparameter_hunter.tracers.py, then applying the following fixes yields the error shown toward bottom -- also shown output for print(self.name) inserted right above the breaking code.

---

# keras_helper.py
# AttributeError: 'LossFunctionWrapper' object has no attribute '__name__'
try:
    compile_params["loss_function_names"] = [_.__name__ for _ in compile_params["loss_functions"]]
except:
    compile_params["loss_function_names"] = [_.name for _ in compile_params["loss_functions"]]

# AttributeError: 'Sequential' object has no attribute 'weighted_metrics'
try:
    compile_params["weighted_metrics"] = get_keras_attr(model, "weighted_metrics")
except:
    print('WARNING: could not fetch "weighted_metrics" attribute; skipping...')

---

Full error trace:

total:0 # print(self.name) outputs
count:0
UnreadVariable

Traceback (most recent call last):

  File "<ipython-input-1-1226ec5313a7>", line 56, in <module>
    execute()
  File "<ipython-input-1-1226ec5313a7>", line 49, in execute
    shuffle=True,
  File "C:\hyperparameter_hunter\hyperparameter_hunter\experiment_core.py", line 165, in __call__
    return super().__call__(*args, **kwargs)
  File "C:\hyperparameter_hunter\hyperparameter_hunter\experiments.py", line 749, in __init__
    target_metric=target_metric,
  File "C:\hyperparameter_hunter\hyperparameter_hunter\experiments.py", line 595, in __init__
    target_metric=target_metric,
  File "C:\hyperparameter_hunter\hyperparameter_hunter\experiments.py", line 302, in __init__
    self.preparation_workflow()
  File "C:\hyperparameter_hunter\hyperparameter_hunter\experiments.py", line 354, in preparation_workflow
    self._generate_hyperparameter_key()
  File "C:\hyperparameter_hunter\hyperparameter_hunter\experiments.py", line 450, in _generate_hyperparameter_key
    self.hyperparameter_key = HyperparameterKeyMaker(parameters, self.cross_experiment_key)
  File "C:\hyperparameter_hunter\hyperparameter_hunter\keys\makers.py", line 367, in __init__
    KeyMaker.__init__(self, parameters, **kwargs)
  File "C:\hyperparameter_hunter\hyperparameter_hunter\keys\makers.py", line 90, in __init__
    self.parameters = deepcopy(parameters)
  File "D:\Anaconda\envs\test_hh\lib\copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "D:\Anaconda\envs\test_hh\lib\copy.py", line 240, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "D:\Anaconda\envs\test_hh\lib\copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "D:\Anaconda\envs\test_hh\lib\copy.py", line 240, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "D:\Anaconda\envs\test_hh\lib\copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "D:\Anaconda\envs\test_hh\lib\copy.py", line 240, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "D:\Anaconda\envs\test_hh\lib\copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "D:\Anaconda\envs\test_hh\lib\copy.py", line 215, in _deepcopy_list
    append(deepcopy(a, memo))
  File "D:\Anaconda\envs\test_hh\lib\copy.py", line 180, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "D:\Anaconda\envs\test_hh\lib\copy.py", line 280, in _reconstruct
    state = deepcopy(state, memo)
  File "D:\Anaconda\envs\test_hh\lib\copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "D:\Anaconda\envs\test_hh\lib\copy.py", line 240, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "D:\Anaconda\envs\test_hh\lib\copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "D:\Anaconda\envs\test_hh\lib\copy.py", line 215, in _deepcopy_list
    append(deepcopy(a, memo))
  File "D:\Anaconda\envs\test_hh\lib\copy.py", line 161, in deepcopy
    y = copier(memo)
  File "D:\Anaconda\envs\test_hh\lib\site-packages\tensorflow_core\python\ops\resource_variable_ops.py", line 464, in __deepcopy__
    name=self._shared_name + "_copy",
  File "D:\Anaconda\envs\test_hh\lib\site-packages\tensorflow_core\python\ops\variables.py", line 1155, in _shared_name
    return self.name[:self.name.index(":")]

ValueError: substring not found