Repository navigation
Relax reproducibility restriction #244
Description
Requiring all Ops to be deterministic (same inputs produce the same output every time) excludes the possibility of wrapping probabilistic algorithms, which limits the scope of the Ops framework, and will detract from the value of the Ops index (#174). While I believe strongly in the importance of reproducibility, we can accommodate irreproducible-but-useful algorithms within Ops by adding a reproducible flag to the framework for each Op, false by default, which can be set when the Op author knows the algorithm will behave reproducibly. When this flag is set, the framework will know that outputs can be cached, which can be useful to improve time performance of repeated computation. And when the flag is unset, the framework can emit a warning explaining the problem to the user, thus raising awareness of the reproducibility concern.
This documentation and probably elsewhere as well will need adjustments to reflect this change.
Activity
gselzer commentedon Jul 31, 2024
I'm actually really in favor of this. As reproducibility is something that cannot actually be guaranteed within each Op, the SciJava Ops application-wide guarantee was always going to be an aspiration that we cannot be sure about.
I was initially disapproving of the
false-by-default reproducibility hint, however of course it's necessary, as you cannot be sure that an Op is reproducible - it's best to evaluate the claim and then add the hint when we have validated reproducibility.I believe that this change will help us focus on our goals of accessibility, and of course a hint will enable reproducibility for those who want it.
Yeah, this is currently not supported, but a
WeakHashMapwithin theOpWrapper(example: here) would make checking simple and space-efficient.gselzer commentedon Aug 8, 2024
#261 edits the spots in the docs that I could find - is there anything else we need here, @ctrueden?