Miguel Afonso Caetano on Nostr: #ML #Science #Reproducibility #DataLeakage: "To minimize errors in ML-based science, ...

#ML #Science #Reproducibility #DataLeakage: "To minimize errors in ML-based science, and to make it more apparent when errors do creep in, we propose REFORMS (Reporting standards for Machine Learning Based Science) in a preprint released today. It is a checklist of 32 items that can be helpful for researchers conducting ML-based science, referees reviewing it, and journals where it is submitted and published.

The checklist was developed by a consensus of 19 researchers across computer science, data science, social sciences, mathematics, and biomedical research. The disciplinary diversity of the authors was essential to ensure that the standards are useful across many fields. A majority of the authors were speakers or organizers at a workshop we organized last year titled "The Reproducibility Crisis in ML-Based Science." (Videos of the talks and discussions are available on the workshop page.)

The checklist and the paper introducing it are available on our project website. The paper also provides a review of past failures, as well as best practices for avoiding such failures."

https://www.aisnakeoil.com/p/introducing-the-reforms-checklist