Some Considerations on Trusted Resources

There is a large variety of contributed R packages, which can be overwhelming when performing their accuracy assessment. These packages can be developed by anyone and may differ in accuracy. The white paper mentions the possibility to define “trusted resources” to simplify the assessment for some of the R packages.

The idea follows vendor assessments / audits to explore the internal validation practices of the vendor for proprietary software. For open-source software such audits are not logistically feasible. However, based on information available in the open-source domain, it may still be possible to perform a virtual audit of a vendor and their practices. In this context, we encourage the publication of software development life cycles (SDLC) documents, which support the process of risk assessment and provide evidence of software trustworthiness.

Continue reading

Status Update: R Validation Hub at R/Pharma 2021

Our favorite meeting of the year is approaching: R/Pharma 2021 will be held virtually November 2-4th, 2021. Also watch out for workshops the week before. Since the R validation Hub is closely associated with R/Pharma, we would like to highlight some of the presentations inspired by the R validation Hub, partner initiatives, or generally related to the topic of validation.

Tuesday: November 2, 2021

• 11:00am “R Package Validation at Roche” by Coline Zeballos, Roche
• 11:20am “Statistical Analysis and Pathway to a Risk-based Assessment of R packages at Merck KGaA/EMD Serono” by Juliane Manitz, EMD Serono
• 1:50 PM “Panel Discussion – Validation” with representatives of the R validation Hub”

Wednesday: November 3, 2021

Continue reading

Status Update: MSA Framework by TransCelerate

TransCelerate has published “Modernization of statistical analytics (MSA) Framework”. With goals similar to the R Validation Hub, the TransCelerate MSA framework seeks to demonstrate software reliability by establishing principles of accuracy, traceability, and reproducibility for a modern analytical software environment.

The MSA framework is centered around risk-assessment and mitigation practices to demonstrate reliability of software. This framework suggests assessing the accuracy of a software library via a confidence measure built on risk metrics such as published source code, issue management, usage, maturity, etc. If confidence does not meet the highest standards, additional testing is recommended. In addition, the intended use of a particular software library and the impact to the broader business outcome determines the requirement for additional testing. Altogether, the MSA framework is in line with the suggestions published in the R Validation Hub 2020 white paper for R. While the TransCelerate authors suggest their principles apply to a broad range of software, e.g. SAS, R, Python, Julia, etc, they do not provide specific suggestions for the implementation of their framework. The R Validation Hub can support the implementation of MSA inspired features with the R package riskmetric and the respective shiny app.

Continue reading

Status Update: CRAN Release of `riskmetric`

We have reached a major milestone. The R package riskmetric has been released and is now available on CRAN.

What is riskmetric?

riskmetric is a collection of risk metrics to evaluate the quality of R packages following the framework suggested by the R validation hub (see our white paper for details). Various quality metrics are provided which evaluate best practices of software development, code documentation, community engagement and development sustainability. This package serves as a starting point for exploring the heterogeneity of code quality, and begin a broader conversation about the validation of R packages.

Continue reading

Status Update: Conferences 2021

The new year is starting with some good news for the R validation hub. We will be able to present our work at a number of different meetings in 2021:

• National Institute of Statistical Sciences (NISS; April 21, 2021): NISS-Merck Meet-Up on Open Source Software in Pharma

• Annual Meeting of Statisticians in the Pharmaceutical Industry (PSI; June 21-23, 2021): Contributed session accepted

• Joint Statistical Meetings (JSM; August 7-12, 2021): Topic-contributed session accepted

  Continue reading

Status Update: A summary of 2020

2020 was a busy year for the R Validation Hub. We released our white paper describing our current thinking on a risk based approach to using R for regulatory work. We started to support the implementation of the white paper with tools such as riskmetric and our risk assessment application. And we started a new sub-team with the aim of producing a follow-up white paper on testing. Throughout, we have continued to share and gain feedback on our proposed approach, presenting at User!; running a workshop at R/Pharma; and speaking at an EU Programming Heads meeting in June.

Continue reading

Risk Assessment Application

Background

Towards the end of 2019, the R Validation Hub received an additional grant from the R Consortium to progress the next phase of our road map and produce a risk assessment app to complement the riskmetric package. In early 2020, Fission Labs were selected as our partner to build the first iteration of the application.

Fission Labs is a software product development services company delivering product life-cycle management and high-end scalable technology solutions. Our mission is to deliver complex products through simple and efficient engineering.

Continue reading

Status Update July 2020

It’s time to bring some updates to you on the current status of the R validation hub and we are have plenty of great developments.

Fission has finished their work on the R package risk assessment app and made the source code is available on github. It is an interactive web application providing a front end for the collection of metrics for R packages via riskmetric package including visualizations and comparison metrics.

Continue reading

Status Update May 2020

In these uncertain times, we would like to provide you with some good news and update you on the progress from the R validation hub. It has been a little while since you heard from us, but that doesn’t mean we were less active.

Communication: You may have realized that there are no further re-occurring meetings scheduled for the R validation hub since February. Following previous discussions, we are planning regular update releases via the website and newsletters instead. Meetings for the validation hub will be scheduled less frequently as needed. However, we are dependent on your input and continue to encourage volunteers to collaborate on all the different parts of the project.

Continue reading

A Risk-based Approach for Assessing R package Accuracy within a Validated Infrastructure: White Paper Summary

In this article I summarise some of the key themes from the R Validation Hub’s white paper, A Risk-based Approach for Assessing R package Accuracy within a Validated Infrastructure.

What is the R Validation Hub?

The R Validation Hub is a cross-industry initiative whose mission is to enable the use of R by the Bio-Pharmaceutical Industry in a regulatory setting, where the output may be used in submissions to regulatory agencies. The group was initially formed in 2018 by members of PSI’s AIMS SIG but has now expanded to include just under 100 members from multiple organisations across the pharmaceutical sector.

Continue reading