Snapshot parametrization

[pawamoy]

Follow-up of #65 (comment) and our exchanges on the web.

I'm trying to use inline-snapshot to store the result of a function that takes a dictionary of options and outputs HTML. The dictionary of options is built by combining several parametrizations together. This allows me to test a subset of all the possible combination of options and their values without actually hard-coding these combinations:

import pytest

from package import function

options = {
    "option1": ["value1", "value2"],
    "option2": [True, False, None],
}


# The following two parametrizations build tests for all the following combinations:
# option1="value1", option2=True
# option1="value1", option2=False
# option1="value1", option2=None
# option1="value2", option2=True
# option1="value2", option2=False
# option1="value2", option2=None
@pytest.mark.parametrize("option1", options["options1"])
@pytest.mark.parametrize("option2", options["options2"])
def test_combinations(option1, option2):
    final_options = {"option1": option1, "option2": option2}
    html = function(final_options)
    assert outsource(html, suffix=".html") == snapshot()

The issue with the code above is that snapshot will store results in external files for all tested combinations, but will only keep reference of the last one in the code when running pytest with --inline-snapshot=create.

You pointed me towards a solution in #65, but IIUC none of the two suggested solutions will work for me.

Solution 1:

calculate_results=snapshot({(1, 2): 3, (3, 4): 7, (0, 0): 0, (-1, 1): 0, (10, -5): 5})

Here it's either asking me to hard-code combinations, which I wanted to avoid in the first place, or use calcultate_results = snapshot() and let inline-snapshot fix it. But when I try that, transforming my dictionary of options into a tuple of tuples:

# outside of the test
snapshots = snapshot()

# at the end of the test
assert outsource(html, suffix=".html") == snapshots[tuple((k, v) for k, v in final_options.items())]

...it errors with (pytest output):

self = {}
index = (('annotations_path', 'brief'), ('show_signature', False), ('show_signature_annotations', False), ('signature_crossrefs', False), ('separate_signature', True), ('unwrap_annotated', False), ...)

    def __getitem__(self, index):
        global _missing_values
    
        if self._new_value is undefined:
            self._new_value = {}
    
        old_value = self._old_value
        if old_value is undefined:
            _missing_values += 1
            old_value = {}
    
        child_node = None
        if self._ast_node is not None:
            assert isinstance(self._ast_node, ast.Dict)
            if index in old_value:
                pos = list(old_value.keys()).index(index)
                child_node = self._ast_node.values[pos]
    
>       if index not in self._new_value:
E       TypeError: unhashable type: 'list'

.venv/lib/python3.11/site-packages/inline_snapshot/_inline_snapshot.py:569: TypeError

I'm not sure to understand why, because clearly index here is a tuple of tuples, not a list.

UPDATE: hmm, looks like I don't understand how locals() work 🤔 Let me double-check.

---

Solution 2:

@pytest.mark.parametrize(
    "x, y, expected_result",
    [
        (1, 2, snapshot(3)),
        (3, 4, snapshot(7)),
        (0, 0, snapshot(0)),
        (-1, 1, snapshot(0)),
        (10, -5, snapshot(5)),
    ],
)

Here it seems there's no other way than hard-coding combinations into a single parametrization, which I'd like to avoid.

[pawamoy]

OK, indeed I misunderstood locals() but additionally there was a side-effect that modified my dictionary. After fixing this, it works! Well, the resulting snapshot() call is huge, but it works 😅 And it's for only 6 options haha, a dozen more to add.

Huge snapshot list

snapshots = snapshot({(
    ("annotations_path", "source"),
    ("show_signature", False),
    ("show_signature_annotations", False),
    ("signature_crossrefs", False),
    ("separate_signature", False),
    ("unwrap_annotated", False),
): external("36a1a03a6364*.html"), (
    ("annotations_path", "brief"),
    ("show_signature", False),
    ("show_signature_annotations", False),
    ("signature_crossrefs", True),
    ("separate_signature", True),
    ("unwrap_annotated", True),
): external("da69cf4e2834*.html"), (
    ("annotations_path", "source"),
    ("show_signature", True),
    ("show_signature_annotations", False),
    ("signature_crossrefs", True),
    ("separate_signature", True),
    ("unwrap_annotated", False),
): external("c7fb0a797254*.html"), (
    ("annotations_path", "brief"),
    ("show_signature", False),
    ("show_signature_annotations", False),
    ("signature_crossrefs", False),
    ("separate_signature", False),
    ("unwrap_annotated", False),
): external("36a1a03a6364*.html"), (
    ("annotations_path", "brief"),
    ("show_signature", True),
    ("show_signature_annotations", True),
    ("signature_crossrefs", False),
    ("separate_signature", True),
    ("unwrap_annotated", True),
): external("d552c9951139*.html"), (
    ("annotations_path", "source"),
    ("show_signature", True),
    ("show_signature_annotations", False),
    ("signature_crossrefs", False),
    ("separate_signature", False),
    ("unwrap_annotated", False),
): external("011c334b854b*.html"), (
    ("annotations_path", "source"),
    ("show_signature", False),
    ("show_signature_annotations", True),
    ("signature_crossrefs", True),
    ("separate_signature", False),
    ("unwrap_annotated", False),
): external("36a1a03a6364*.html"), (
    ("annotations_path", "brief"),
    ("show_signature", False),
    ("show_signature_annotations", True),
    ("signature_crossrefs", False),
    ("separate_signature", True),
    ("unwrap_annotated", False),
): external("a1b8ff405bf2*.html"), (
    ("annotations_path", "brief"),
    ("show_signature", False),
    ("show_signature_annotations", False),
    ("signature_crossrefs", True),
    ("separate_signature", False),
    ("unwrap_annotated", True),
): external("36a1a03a6364*.html"), (
    ("annotations_path", "brief"),
    ("show_signature", True),
    ("show_signature_annotations", False),
    ("signature_crossrefs", False),
    ("separate_signature", True),
    ("unwrap_annotated", False),
): external("1531e41e8dbe*.html"), (
    ("annotations_path", "brief"),
    ("show_signature", True),
    ("show_signature_annotations", False),
    ("signature_crossrefs", True),
    ("separate_signature", False),
    ("unwrap_annotated", True),
): external("011c334b854b*.html"), (
    ("annotations_path", "brief"),
    ("show_signature", True),
    ("show_signature_annotations", True),
    ("signature_crossrefs", False),
    ("separate_signature", False),
    ("unwrap_annotated", False),
): external("bd14e6a60af2*.html"), (
    ("annotations_path", "brief"),
    ("show_signature", False),
    ("show_signature_annotations", False),
    ("signature_crossrefs", True),
    ("separate_signature", False),
    ("unwrap_annotated", False),
): external("36a1a03a6364*.html"), (
    ("annotations_path", "source"),
    ("show_signature", True),
    ("show_signature_annotations", True),
    ("signature_crossrefs", False),
    ("separate_signature", False),
    ("unwrap_annotated", False),
): external("bd14e6a60af2*.html"), (
    ("annotations_path", "source"),
    ("show_signature", True),
    ("show_signature_annotations", True),
    ("signature_crossrefs", False),
    ("separate_signature", True),
    ("unwrap_annotated", False),
): external("d552c9951139*.html"), (
    ("annotations_path", "brief"),
    ("show_signature", True),
    ("show_signature_annotations", False),
    ("signature_crossrefs", True),
    ("separate_signature", False),
    ("unwrap_annotated", False),
): external("011c334b854b*.html"), (
    ("annotations_path", "brief"),
    ("show_signature", True),
    ("show_signature_annotations", True),
    ("signature_crossrefs", True),
    ("separate_signature", True),
    ("unwrap_annotated", False),
): external("89c8c205249f*.html"), (
    ("annotations_path", "source"),
    ("show_signature", False),
    ("show_signature_annotations", False),
    ("signature_crossrefs", False),
    ("separate_signature", False),
    ("unwrap_annotated", True),
): external("36a1a03a6364*.html"), (
    ("annotations_path", "brief"),
    ("show_signature", False),
    ("show_signature_annotations", True),
    ("signature_crossrefs", False),
    ("separate_signature", False),
    ("unwrap_annotated", False),
): external("36a1a03a6364*.html"), (
    ("annotations_path", "source"),
    ("show_signature", False),
    ("show_signature_annotations", False),
    ("signature_crossrefs", True),
    ("separate_signature", False),
    ("unwrap_annotated", True),
): external("36a1a03a6364*.html"), (
    ("annotations_path", "brief"),
    ("show_signature", True),
    ("show_signature_annotations", True),
    ("signature_crossrefs", True),
    ("separate_signature", False),
    ("unwrap_annotated", True),
): external("bd14e6a60af2*.html"), (
    ("annotations_path", "source"),
    ("show_signature", False),
    ("show_signature_annotations", True),
    ("signature_crossrefs", False),
    ("separate_signature", False),
    ("unwrap_annotated", True),
): external("36a1a03a6364*.html"), (
    ("annotations_path", "brief"),
    ("show_signature", False),
    ("show_signature_annotations", False),
    ("signature_crossrefs", False),
    ("separate_signature", False),
    ("unwrap_annotated", True),
): external("36a1a03a6364*.html"), (
    ("annotations_path", "brief"),
    ("show_signature", True),
    ("show_signature_annotations", False),
    ("signature_crossrefs", True),
    ("separate_signature", True),
    ("unwrap_annotated", False),
): external("c7fb0a797254*.html"), (
    ("annotations_path", "brief"),
    ("show_signature", False),
    ("show_signature_annotations", True),
    ("signature_crossrefs", True),
    ("separate_signature", False),
    ("unwrap_annotated", True),
): external("36a1a03a6364*.html"), (
    ("annotations_path", "source"),
    ("show_signature", True),
    ("show_signature_annotations", True),
    ("signature_crossrefs", False),
    ("separate_signature", True),
    ("unwrap_annotated", True),
): external("d552c9951139*.html"), (
    ("annotations_path", "brief"),
    ("show_signature", False),
    ("show_signature_annotations", True),
    ("signature_crossrefs", False),
    ("separate_signature", True),
    ("unwrap_annotated", True),
): external("a1b8ff405bf2*.html"), (
    ("annotations_path", "source"),
    ("show_signature", True),
    ("show_signature_annotations", False),
    ("signature_crossrefs", False),
    ("separate_signature", True),
    ("unwrap_annotated", True),
): external("1531e41e8dbe*.html"), (
    ("annotations_path", "source"),
    ("show_signature", False),
    ("show_signature_annotations", True),
    ("signature_crossrefs", False),
    ("separate_signature", False),
    ("unwrap_annotated", False),
): external("36a1a03a6364*.html"), (
    ("annotations_path", "source"),
    ("show_signature", True),
    ("show_signature_annotations", False),
    ("signature_crossrefs", True),
    ("separate_signature", True),
    ("unwrap_annotated", True),
): external("c7fb0a797254*.html"), (
    ("annotations_path", "source"),
    ("show_signature", False),
    ("show_signature_annotations", False),
    ("signature_crossrefs", True),
    ("separate_signature", False),
    ("unwrap_annotated", False),
): external("36a1a03a6364*.html"), (
    ("annotations_path", "brief"),
    ("show_signature", False),
    ("show_signature_annotations", True),
    ("signature_crossrefs", True),
    ("separate_signature", True),
    ("unwrap_annotated", False),
): external("d944bd9f3d8c*.html"), (
    ("annotations_path", "brief"),
    ("show_signature", True),
    ("show_signature_annotations", False),
    ("signature_crossrefs", False),
    ("separate_signature", False),
    ("unwrap_annotated", True),
): external("011c334b854b*.html"), (
    ("annotations_path", "source"),
    ("show_signature", True),
    ("show_signature_annotations", True),
    ("signature_crossrefs", False),
    ("separate_signature", False),
    ("unwrap_annotated", True),
): external("bd14e6a60af2*.html"), (
    ("annotations_path", "brief"),
    ("show_signature", True),
    ("show_signature_annotations", False),
    ("signature_crossrefs", False),
    ("separate_signature", False),
    ("unwrap_annotated", False),
): external("011c334b854b*.html"), (
    ("annotations_path", "source"),
    ("show_signature", False),
    ("show_signature_annotations", True),
    ("signature_crossrefs", True),
    ("separate_signature", True),
    ("unwrap_annotated", True),
): external("d944bd9f3d8c*.html"), (
    ("annotations_path", "source"),
    ("show_signature", True),
    ("show_signature_annotations", True),
    ("signature_crossrefs", True),
    ("separate_signature", False),
    ("unwrap_annotated", False),
): external("bd14e6a60af2*.html"), (
    ("annotations_path", "source"),
    ("show_signature", True),
    ("show_signature_annotations", True),
    ("signature_crossrefs", True),
    ("separate_signature", True),
    ("unwrap_annotated", True),
): external("89c8c205249f*.html"), (
    ("annotations_path", "brief"),
    ("show_signature", True),
    ("show_signature_annotations", True),
    ("signature_crossrefs", True),
    ("separate_signature", True),
    ("unwrap_annotated", True),
): external("89c8c205249f*.html"), (
    ("annotations_path", "brief"),
    ("show_signature", True),
    ("show_signature_annotations", True),
    ("signature_crossrefs", True),
    ("separate_signature", False),
    ("unwrap_annotated", False),
): external("bd14e6a60af2*.html"), (
    ("annotations_path", "source"),
    ("show_signature", False),
    ("show_signature_annotations", False),
    ("signature_crossrefs", False),
    ("separate_signature", True),
    ("unwrap_annotated", False),
): external("3603a08e7614*.html"), (
    ("annotations_path", "source"),
    ("show_signature", True),
    ("show_signature_annotations", False),
    ("signature_crossrefs", True),
    ("separate_signature", False),
    ("unwrap_annotated", True),
): external("011c334b854b*.html"), (
    ("annotations_path", "brief"),
    ("show_signature", False),
    ("show_signature_annotations", False),
    ("signature_crossrefs", False),
    ("separate_signature", True),
    ("unwrap_annotated", False),
): external("3603a08e7614*.html"), (
    ("annotations_path", "source"),
    ("show_signature", True),
    ("show_signature_annotations", False),
    ("signature_crossrefs", False),
    ("separate_signature", False),
    ("unwrap_annotated", True),
): external("011c334b854b*.html"), (
    ("annotations_path", "source"),
    ("show_signature", True),
    ("show_signature_annotations", False),
    ("signature_crossrefs", True),
    ("separate_signature", False),
    ("unwrap_annotated", False),
): external("011c334b854b*.html"), (
    ("annotations_path", "brief"),
    ("show_signature", True),
    ("show_signature_annotations", False),
    ("signature_crossrefs", False),
    ("separate_signature", True),
    ("unwrap_annotated", True),
): external("1531e41e8dbe*.html"), (
    ("annotations_path", "brief"),
    ("show_signature", True),
    ("show_signature_annotations", True),
    ("signature_crossrefs", False),
    ("separate_signature", True),
    ("unwrap_annotated", False),
): external("d552c9951139*.html"), (
    ("annotations_path", "source"),
    ("show_signature", False),
    ("show_signature_annotations", False),
    ("signature_crossrefs", True),
    ("separate_signature", True),
    ("unwrap_annotated", True),
): external("da69cf4e2834*.html"), (
    ("annotations_path", "source"),
    ("show_signature", True),
    ("show_signature_annotations", True),
    ("signature_crossrefs", True),
    ("separate_signature", True),
    ("unwrap_annotated", False),
): external("89c8c205249f*.html"), (
    ("annotations_path", "brief"),
    ("show_signature", False),
    ("show_signature_annotations", True),
    ("signature_crossrefs", True),
    ("separate_signature", False),
    ("unwrap_annotated", False),
): external("36a1a03a6364*.html"), (
    ("annotations_path", "source"),
    ("show_signature", False),
    ("show_signature_annotations", True),
    ("signature_crossrefs", True),
    ("separate_signature", False),
    ("unwrap_annotated", True),
): external("36a1a03a6364*.html"), (
    ("annotations_path", "source"),
    ("show_signature", False),
    ("show_signature_annotations", True),
    ("signature_crossrefs", False),
    ("separate_signature", True),
    ("unwrap_annotated", False),
): external("a1b8ff405bf2*.html"), (
    ("annotations_path", "source"),
    ("show_signature", True),
    ("show_signature_annotations", True),
    ("signature_crossrefs", True),
    ("separate_signature", False),
    ("unwrap_annotated", True),
): external("bd14e6a60af2*.html"), (
    ("annotations_path", "source"),
    ("show_signature", False),
    ("show_signature_annotations", False),
    ("signature_crossrefs", False),
    ("separate_signature", True),
    ("unwrap_annotated", True),
): external("3603a08e7614*.html"), (
    ("annotations_path", "brief"),
    ("show_signature", False),
    ("show_signature_annotations", False),
    ("signature_crossrefs", False),
    ("separate_signature", True),
    ("unwrap_annotated", True),
): external("3603a08e7614*.html"), (
    ("annotations_path", "brief"),
    ("show_signature", True),
    ("show_signature_annotations", True),
    ("signature_crossrefs", False),
    ("separate_signature", False),
    ("unwrap_annotated", True),
): external("bd14e6a60af2*.html"), (
    ("annotations_path", "source"),
    ("show_signature", False),
    ("show_signature_annotations", True),
    ("signature_crossrefs", True),
    ("separate_signature", True),
    ("unwrap_annotated", False),
): external("d944bd9f3d8c*.html"), (
    ("annotations_path", "source"),
    ("show_signature", True),
    ("show_signature_annotations", False),
    ("signature_crossrefs", False),
    ("separate_signature", True),
    ("unwrap_annotated", False),
): external("1531e41e8dbe*.html"), (
    ("annotations_path", "brief"),
    ("show_signature", False),
    ("show_signature_annotations", True),
    ("signature_crossrefs", False),
    ("separate_signature", False),
    ("unwrap_annotated", True),
): external("36a1a03a6364*.html"), (
    ("annotations_path", "brief"),
    ("show_signature", False),
    ("show_signature_annotations", True),
    ("signature_crossrefs", True),
    ("separate_signature", True),
    ("unwrap_annotated", True),
): external("d944bd9f3d8c*.html"), (
    ("annotations_path", "source"),
    ("show_signature", False),
    ("show_signature_annotations", True),
    ("signature_crossrefs", False),
    ("separate_signature", True),
    ("unwrap_annotated", True),
): external("a1b8ff405bf2*.html"), (
    ("annotations_path", "brief"),
    ("show_signature", True),
    ("show_signature_annotations", False),
    ("signature_crossrefs", True),
    ("separate_signature", True),
    ("unwrap_annotated", True),
): external("c7fb0a797254*.html"), (
    ("annotations_path", "source"),
    ("show_signature", False),
    ("show_signature_annotations", False),
    ("signature_crossrefs", True),
    ("separate_signature", True),
    ("unwrap_annotated", False),
): external("da69cf4e2834*.html"), (
    ("annotations_path", "brief"),
    ("show_signature", False),
    ("show_signature_annotations", False),
    ("signature_crossrefs", True),
    ("separate_signature", True),
    ("unwrap_annotated", False),
): external("da69cf4e2834*.html")})

Just to be clear, here's the working code:

# outside of the test
snapshots = snapshot()

# at the end of the test
snapshot_key = tuple((k, v) for k, v in final_options.items())
assert outsource(html, suffix=".html") == snapshots[snapshot_key]

Then run with --inline-snapshot=create 💥