Evidence counts#
This document is not a regulatory record or certificate; bounds and estimates are as labelled.
This page counts what MIME’s records claim and what its specifications state, and says for each whether a test that CI
runs holds it. It is a count and a table, not a verdict on the models. The claims are those of the known-anomalies
registry (docs/validation/known_anomalies.yaml); the statements are the normative sentences of the specifications
(docs/specification/).
A bound is an upper bound on an error over a stated domain. An estimate is a value with an uncertainty that bounds nothing: a maximum over finitely many points is an estimate, however it was computed. An acceptance test agrees with a physical reference to ground a sign, a frame or a unit, and claims no accuracy, so it is never read as one. A limit is the stated edge of where a model applies. A rule is a normative sentence, which has no number and no oracle, so it has no basis: it prints with the tests that hold it. The groups below keep the roles apart.
Evidence is a test that ran and passed in the default lane. A test marked skip, xfail, slow or gpu is not run
by CI’s default lane, so it is not evidence, whatever it would have done. The version of this page that also runs the
slow and gpu tests, and says which device each ran on, is 0.3.0’s.
Counts#
Claims 1; with a test that ran and passed in the default lane 1; not run 0; failed 0; skipped 0; covered by no test 0; outcome not read 0.
Statements 44; with a test that ran and passed in the default lane 43; not run 0; failed 0; skipped 0; covered by no test 1; outcome not read 0.
A test’s body is not checked: a cited test is matched by name, and nothing ties it to the claim or statement that cites it.
Group |
Rows |
passed |
failed |
skipped |
not run |
no test |
not read |
|---|---|---|---|---|---|---|---|
Bounds |
0 |
0 |
0 |
0 |
0 |
0 |
0 |
Estimates |
1 |
1 |
0 |
0 |
0 |
0 |
0 |
Acceptance tests |
0 |
0 |
0 |
0 |
0 |
0 |
0 |
Limits |
0 |
0 |
0 |
0 |
0 |
0 |
0 |
Rules |
0 |
0 |
0 |
0 |
0 |
0 |
0 |
Specification statements |
44 |
43 |
0 |
0 |
0 |
1 |
0 |
The floor on those counts is pinned in docs/ratchets/evidence_floor.txt: at least 1 claim and 38 statements must stay, and a pinned one that is missing fails the build.
Outcomes are the default lane’s, read from its JUnit reports (8329 tests in 10 reports); the workflow declares the lane’s device, cpu, and a report records none, so the Device column is that one declared value for every row that ran.
It holds no claim with a measured basis, so it is a count and not a report.
Claims#
Bounds#
An upper bound on an error over its stated domain.
None.
Estimates#
A value with an uncertainty that bounds nothing; a maximum over finitely many points is one.
Claim |
Quantity |
Value |
Role |
Basis |
Oracle |
Evidence |
What it bounds and estimates |
Outcome |
Device |
|---|---|---|---|---|---|---|---|---|---|
|
relative swim-speed error of the drag surrogate against the MRS values the 0.2 line checked it against (FL-9 in the 1/4 in tube, at 10 Hz) |
0.162 1 |
estimate |
computed |
software oracle |
tests/oracles/test_drag.py::test_the_surrogate_against_the_mrs_values_it_was_checked_against: default lane; passed in default lane |
estimate; computed; against a software oracle oracle; from 5 points over offsets 0.20, 0.25, 0.28, 0.30 and 0.32 of the tube’s radius, bounding nothing. the largest error over five offsets, against a float32 MRS whose wall table was not checked: it estimates the surrogate’s error there and bounds nothing between or beyond those offsets, nor the error against the flow |
passed |
cpu (declared by the workflow) |
Acceptance tests#
Agreement with a physical reference that grounds sign, frame and units; it claims no accuracy.
None.
Limits#
The stated edge of applicability.
None.
Rules#
A normative sentence; it has no basis, only its cases.
None.
Specification statements#
Rules: each has its cases and no basis. The statement is in the specification it names.
Statement |
Specification |
Outcome |
Device |
Its cases |
|---|---|---|---|---|
|
passed |
cpu (declared by the workflow) |
tests/sharing/test_hostile_bundles.py::test_a_lock_that_names_a_file_outside_the_layout_is_refused_before_it_is_read: default lane; passed in default lane; tests/sharing/test_hostile_bundles.py::test_a_file_the_lock_does_not_name_is_refused_even_where_a_folder_holds_files: default lane; passed in default lane |
|
|
passed |
cpu (declared by the workflow) |
tests/errors/test_errors.py::test_loading_a_folder_misspelt_names_the_saved_ones_beside_it: default lane; passed in default lane |
|
|
passed |
cpu (declared by the workflow) |
tests/saved/test_document.py::test_a_python_tag_is_refused_and_constructs_nothing: default lane; passed in default lane; tests/saved/test_load_refusals.py::test_reference_outputs_holding_a_pickled_object_are_refused_and_run_nothing: default lane; passed in default lane |
|
|
passed |
cpu (declared by the workflow) |
tests/saved/test_load_refusals.py::test_a_document_that_cannot_be_read_is_refused_naming_it: default lane; passed in default lane |
|
|
passed |
cpu (declared by the workflow) |
tests/errors/test_errors.py::test_files_that_cannot_be_read_are_refused_with_their_codes: default lane; passed in default lane |
|
|
passed |
cpu (declared by the workflow) |
tests/saved/test_document.py::test_a_key_given_twice_is_refused: default lane; passed in default lane; tests/saved/test_load_refusals.py::test_a_document_edited_by_hand_is_refused_by_its_code[a-document-that-is-not-yaml]: default lane; passed in default lane |
|
|
passed |
cpu (declared by the workflow) |
tests/saved/test_document.py::test_an_unknown_key_and_a_value_of_another_type_are_refused_by_path: default lane; passed in default lane; tests/saved/test_schema.py::test_what_the_annotations_allow_the_check_accepts_and_a_change_it_refuses: default lane; passed in default lane; tests/saved/test_json_schema.py::test_the_readers_disagree_exactly_on_the_recorded_gaps: default lane; passed in default lane |
|
|
passed |
cpu (declared by the workflow) |
tests/saved/test_newer_form_saved.py::test_a_form_one_above_the_newest_is_refused_naming_the_file_the_version_and_the_advice: default lane; passed in default lane |
|
|
passed |
cpu (declared by the workflow) |
tests/saved/test_previous_form.py::test_a_previous_folder_loads_as_exploratory: default lane; passed in default lane; tests/saved/test_previous_form.py::test_the_control_a_previous_document_holding_a_key_of_version_two_is_refused: default lane; passed in default lane; tests/saved/test_terms_saved.py::test_an_earlier_version_given_a_terms_key_is_refused: default lane; passed in default lane; tests/saved/test_duration_saved.py::test_the_control_an_earlier_form_holding_the_key_of_version_five_is_refused: default lane; passed in default lane |
|
|
passed |
cpu (declared by the workflow) |
tests/saved/test_duration_saved.py::test_a_saved_folder_runs_as_long_as_it_declared_with_no_duration_given: default lane; passed in default lane; tests/saved/test_duration_saved.py::test_an_override_beats_the_saved_length: default lane; passed in default lane; tests/saved/test_duration_saved.py::test_the_control_a_saved_duration_edited_after_its_lock_is_refused: default lane; passed in default lane; tests/saved/test_duration_saved.py::test_the_control_a_folder_with_no_duration_still_needs_one: default lane; passed in default lane; tests/saved/test_duration_saved.py::test_an_earlier_form_loads_without_a_duration_and_still_needs_one: default lane; passed in default lane; tests/saved/test_duration_saved.py::test_the_control_a_duration_a_run_would_refuse_is_refused_at_the_build_and_never_saved: default lane; passed in default lane |
|
|
passed |
cpu (declared by the workflow) |
tests/saved/test_terms_saved.py::test_terms_edited_by_hand_are_refused_at_load_by_their_code: default lane; passed in default lane |
|
|
passed |
cpu (declared by the workflow) |
tests/saved/test_terms_saved.py::test_a_licence_the_installed_packaging_does_not_list_is_read_as_written_and_the_load_says_so: default lane; passed in default lane |
|
|
passed |
cpu (declared by the workflow) |
tests/saved/test_load_refusals.py::test_a_lock_that_is_not_json_is_refused: default lane; passed in default lane |
|
|
passed |
cpu (declared by the workflow) |
tests/assets/test_folder_assets.py::test_a_malformed_asset_record_in_a_lock_is_refused_by_the_saved_forms_reader: default lane; passed in default lane; tests/saved/test_schema.py::test_the_lock_is_read_only_throughout: default lane; passed in default lane |
|
|
passed |
cpu (declared by the workflow) |
tests/saved/test_form_v2.py::test_the_control_mixed_versions_are_refused: default lane; passed in default lane; tests/saved/test_previous_form.py::test_the_control_a_previous_document_beside_a_current_lock_is_refused: default lane; passed in default lane |
|
|
passed |
cpu (declared by the workflow) |
tests/sharing/test_hostile_bundles.py::test_a_lock_naming_a_distribution_or_version_that_is_not_one_is_refused_and_prints_nothing_raw: default lane; passed in default lane; tests/sharing/test_hostile_bundles.py::test_a_lock_that_names_a_file_outside_the_layout_is_refused_before_it_is_read: default lane; passed in default lane |
|
|
passed |
cpu (declared by the workflow) |
tests/saved/test_load_refusals.py::test_a_lock_holding_a_key_twice_is_refused_by_the_load_and_by_every_reader_of_the_lock: default lane; passed in default lane; tests/saved/test_load_refusals.py::test_every_module_reads_the_lock_through_the_one_reader: default lane; passed in default lane; tests/saved/test_json_schema.py::test_the_readers_disagree_exactly_on_the_recorded_gaps: default lane; passed in default lane |
|
|
passed |
cpu (declared by the workflow) |
tests/saved/test_newer_form_saved.py::test_a_document_and_a_lock_that_are_both_newer_are_each_refused_once: default lane; passed in default lane |
|
|
passed |
cpu (declared by the workflow) |
tests/sharing/test_push_publish.py::test_a_production_publish_rewrites_the_lock_so_pack_makes_the_published_bundle: default lane; passed in default lane; tests/sharing/test_push_publish.py::test_a_lock_changed_between_staging_and_publish_is_left_alone: default lane; passed in default lane |
|
|
passed |
cpu (declared by the workflow) |
tests/sharing/test_referenced.py::test_a_folder_missing_a_referenced_file_is_refused_with_the_command_that_fetches_and_places_it: default lane; passed in default lane |
|
|
passed |
cpu (declared by the workflow) |
tests/assets/test_records.py::test_a_full_record_round_trips_and_keeps_its_kind_role_and_identity: default lane; passed in default lane; tests/assets/test_records.py::test_control_a_record_that_writes_its_own_tier_is_refused: default lane; passed in default lane; tests/sharing/test_records_travel.py::test_the_lock_holds_the_declared_record_whole_and_never_this_machines_locations: default lane; passed in default lane |
|
|
passed |
cpu (declared by the workflow) |
tests/viz/test_visual_pins.py::test_the_lock_holds_the_manifest_and_the_folder_holds_none: default lane; passed in default lane |
|
|
passed |
cpu (declared by the workflow) |
tests/saved/test_form6.py::test_an_earlier_lock_holding_a_key_version_six_added_is_refused: default lane; passed in default lane; tests/saved/test_form6.py::test_control_an_upgrade_that_accepts_the_keys_in_an_earlier_lock_is_caught_every_run: default lane; passed in default lane |
|
|
passed |
cpu (declared by the workflow) |
tests/saved/test_lock_assets.py::test_the_pin_wins_a_lock_naming_the_table_under_another_hash_reads_that_hash: default lane; passed in default lane; tests/saved/test_lock_assets.py::test_a_shared_source_holding_another_version_of_the_table_is_reported_and_the_pin_kept: default lane; passed in default lane; tests/viz/test_visual_pins.py::test_the_saved_arm_loads_with_its_stage_rebuilt_from_its_pins: default lane; passed in default lane; tests/viz/test_visual_pins.py::test_control_a_rebuild_that_ignores_the_saved_stages_pins_is_refused_every_run: default lane; passed in default lane |
|
|
passed |
cpu (declared by the workflow) |
tests/errors/test_errors.py::test_files_that_cannot_be_read_are_refused_with_their_codes: default lane; passed in default lane |
|
|
passed |
cpu (declared by the workflow) |
tests/saved/test_load_refusals.py::test_reference_outputs_whose_arrays_changed_are_refused: default lane; passed in default lane |
|
|
passed |
cpu (declared by the workflow) |
tests/scene/test_saved_scene.py::test_a_stage_edited_after_its_lock_is_refused_at_load: default lane; passed in default lane; tests/scene/test_derived_prims.py::test_an_edit_beyond_the_derived_prims_is_refused: default lane; passed in default lane |
|
|
passed |
cpu (declared by the workflow) |
tests/scene/test_saved_scene.py::test_a_changed_asset_file_is_refused_at_load: default lane; passed in default lane |
|
|
passed |
cpu (declared by the workflow) |
tests/materials/test_pins.py::test_a_saved_record_file_edited_by_one_byte_is_refused_at_load: default lane; passed in default lane; tests/materials/test_pins.py::test_a_record_file_missing_from_the_folder_is_refused_at_load: default lane; passed in default lane |
|
|
passed |
cpu (declared by the workflow) |
tests/saved/test_lock_assets.py::test_control_a_lock_without_the_tables_name_is_refused_when_loaded_naming_it: default lane; passed in default lane; tests/saved/test_lock_assets.py::test_the_pin_wins_a_lock_naming_the_table_under_another_hash_reads_that_hash: default lane; passed in default lane |
|
|
passed |
cpu (declared by the workflow) |
tests/saved/test_load.py::test_an_entry_no_installed_package_registers_is_refused_with_the_line_to_install: default lane; passed in default lane; tests/saved/test_load.py::test_another_version_is_refused_with_the_line_to_install: default lane; passed in default lane |
|
|
no test |
none |
not yet covered |
|
|
passed |
cpu (declared by the workflow) |
tests/saved/test_load_refusals.py::test_a_document_edited_by_hand_is_refused_by_its_code[a-setting-in-another-unit]: default lane; passed in default lane; tests/saved/test_load_refusals.py::test_a_document_edited_by_hand_is_refused_by_its_code[an-initial-state-in-another-unit]: default lane; passed in default lane; tests/saved/test_load_refusals.py::test_the_controllers_rate_in_another_unit_is_refused: default lane; passed in default lane; tests/saved/test_body_side_document.py::test_a_load_refuses_an_edited_document_naming_what_to_do[a-declared-value-in-another-unit]: default lane; passed in default lane; tests/saved/test_load_refusals.py::test_a_document_edited_by_hand_is_refused_by_its_code[a-condition-in-another-unit]: default lane; passed in default lane; tests/saved/test_load_refusals.py::test_a_document_edited_by_hand_is_refused_by_its_code[the-step-in-another-unit]: default lane; passed in default lane |
|
|
passed |
cpu (declared by the workflow) |
tests/saved/test_load_refusals.py::test_a_document_edited_by_hand_is_refused_by_its_code[a-setting-the-entry-does-not-have]: default lane; passed in default lane; tests/saved/test_load_refusals.py::test_a_document_edited_by_hand_is_refused_by_its_code[a-group-no-package-registers]: default lane; passed in default lane; tests/saved/test_load_refusals.py::test_a_document_edited_by_hand_is_refused_by_its_code[a-setting-the-entry-needs-left-out]: default lane; passed in default lane |
|
|
passed |
cpu (declared by the workflow) |
tests/saved/test_load_refusals.py::test_a_document_edited_by_hand_is_refused_by_its_code[dynamics-a-medium-does-not-take]: default lane; passed in default lane; tests/saved/test_body_side_document.py::test_a_load_refuses_an_edited_document_naming_what_to_do[commands-on-a-medium]: default lane; passed in default lane; tests/scene/test_saved_scene.py::test_a_document_whose_python_first_thing_moved_prim_is_refused: default lane; passed in default lane |
|
|
passed |
cpu (declared by the workflow) |
tests/saved/test_load_refusals.py::test_a_document_edited_by_hand_is_refused_by_its_code[a-material-the-document-does-not-hold]: default lane; passed in default lane |
|
|
passed |
cpu (declared by the workflow) |
tests/saved/test_load_refusals.py::test_a_document_edited_by_hand_is_refused_by_its_code[an-id-its-bindings-do-not-give]: default lane; passed in default lane |
|
|
passed |
cpu (declared by the workflow) |
tests/saved/test_load.py::test_a_document_edited_after_its_lock_is_refused_naming_what_changed: default lane; passed in default lane; tests/saved/test_load.py::test_a_document_saved_before_a_version_changed_is_refused_naming_the_entry: default lane; passed in default lane |
|
|
passed |
cpu (declared by the workflow) |
tests/saved/test_load.py::test_a_folder_saved_by_another_mime_version_is_refused_and_one_saved_by_this_one_loads: default lane; passed in default lane |
|
|
passed |
cpu (declared by the workflow) |
tests/saved/test_stage_writer.py::test_the_control_a_lock_naming_another_writer_or_a_stage_neither_writes_is_refused: default lane; passed in default lane |
|
|
passed |
cpu (declared by the workflow) |
tests/saved/test_load.py::test_a_factory_that_builds_another_graph_under_the_same_version_is_refused: default lane; passed in default lane; tests/saved/test_load.py::test_in_another_environment_differences_are_reported_and_outputs_compared_within_tolerance: default lane; passed in default lane |
|
|
passed |
cpu (declared by the workflow) |
tests/saved/test_load.py::test_a_saved_experiment_loads_and_verifies_bit_for_bit: default lane; passed in default lane |
|
|
passed |
cpu (declared by the workflow) |
tests/saved/test_json_schema.py::test_the_readers_disagree_exactly_on_the_recorded_gaps: default lane; passed in default lane; tests/saved/test_json_schema.py::test_on_drawn_files_and_their_changes_the_readers_agree: default lane; passed in default lane |
|
|
passed |
cpu (declared by the workflow) |
tests/saved/test_json_schema.py::test_every_golden_document_and_lock_validates_as_the_loader_reads_it: default lane; passed in default lane |
Generation digest (sha256 of the text above): ce62e2b696e842e0aad96bb8070be7ea407f82166261b6381dde98f000d4cc2e