Increasing awareness of problems with the reproducibility and integrity of research led the UK Parliament Science and Technology Committee to launch, in July 2021, an inquiry into reproducibility and research integrity. We recognise at least four potential reasons why attempts to replicate a research finding may be unsuccessful: false positive statistical analyses, low generalisability of findings, suboptimal study designs (research integrity), and deliberate malfeasance (researcher integrity). It is important to make a distinction between the contributions of research integrity and of researcher integrity to the reproducibility crisis. While the impact of an individual instance of compromised researcher integrity is substantial, the aggregate impact of more prevalent problems with research integrity is likely much greater. The research community will be most efficient when failed replication efforts are never due to issues of research integrity or of researcher integrity, as this would allow focus on the scientific reasons for why two apparently similar experiments should reach different conclusions. We discuss the role of funders, institutions and government in addressing the "reproducibility crisis" before considering which interventions might have a positive impact on academia's approach to reproducible research, and a possible role for a committee on research integrity.
Abstract The current report is the result of the recent EFSA‐EBTC initiative to advance the use of automation tools in systematic reviews. The working group members identified the need for data exchange standards to facilitate transferring data between SR automation tools without loss or changes in data as a first priority. Closely related objectives identified were the concurrent development of data guidance and data content standards, and a strategy for governance of the resulting data standards, including their maintenance and regular updates. These conversations led up to the organization of workshops to initiate the development of data standards, as described in the current report. The EFSA‐EBTC collaboration organized a four half‐day workshop for an extensive group of stakeholders actively involved in the development or use of SR automation tools. The main objectives were 1) to generate a common understanding of the needs and hurdles for data standards in SR automation, 2) to collate viewpoints from all relevant stakeholders on how such standards may be developed, and 3) to kick‐off the development process. Three preparatory workshops preceding the final workshop were organized to clarify the scope and objectives of the main workshop. This external scientific report contains the summary of the preparatory and main workshops, as well as the proposed strategy for the process of how these data standards could be developed. The recommended next steps for development focus on further refining the aims, analysis of the landscape, interoperability with existing standards, setting up the infrastructure (e.g., establishing a governance structure and working groups containing all essential stakeholders), and the process of actual data standard development, launching and monitoring adoption of the standards.
Abstract: The adoption and incentivisation of open and transparent research practices is critical in addressing issues around research reproducibility and research integrity. These practices will require training and funding. Individuals need to be incentivised to adopt open and transparent research practices (e.g., added as desirable criteria in hiring, probation, and promotion decisions, recognition that funded research should be conducted openly and transparently, the importance of publishers mandating the publication of research workflows and appropriately curated data associated with each research output). Similarly, institutions need to be incentivised to encourage the adoption of open and transparent practices by researchers. Research quality should be prioritised over research quantity. As research transparency will look different for different disciplines, there can be no one-size-fits-all approach. An outward looking and joined up UK research strategy is needed that places openness and transparency at the heart of research activity. This should involve key stakeholders (institutions, research organisations, funders, publishers, and Government) and crucially should be focused on action. Failure to do this will have negative consequences not just for UK research, but also for our ability to innovate and subsequently commercialise UK-led discovery.
The adoption and incentivisation of open and transparent research practices is critical in addressing issues around research reproducibility and research integrity. These practices will require training and funding. Individuals need to be incentivised to adopt open and transparent research practices (e.g., added as desirable criteria in hiring, probation, and promotion decisions, recognition that funded research should be conducted openly and transparently, the importance of publishers mandating the publication of research workflows and appropriately curated data associated with each research output). Similarly, institutions need to be incentivised to encourage the adoption of open and transparent practices by researchers. Research quality should be prioritised over research quantity. As research transparency will look different for different disciplines, there can be no one-size-fits-all approach. An outward looking and joined up UK research strategy is needed that places openness and transparency at the heart of research activity. This should involve key stakeholders (institutions, research organisations, funders, publishers, and Government) and crucially should be focused on action. Failure to do this will have negative consequences not just for UK research, but also for our ability to innovate and subsequently commercialise UK-led discovery.
In: Stewart , A J , Farran , E K , Grange , J A , Macleod , M , Munafo , M R , Newton , P & Shanks , D R 2021 , ' Improving research quality : The view from the UK Reproducibility Network Institutional Leads for research improvement ' , BMC Research Notes , vol. 14 , 458 . https://doi.org/10.1186/s13104-021-05883-3
The adoption and incentivisation of open and transparent research practices is critical in addressing issues around research reproducibility and research integrity. These practices will require training and funding. Individuals need to be incentivised to adopt open and transparent research practices (e.g., added as desirable criteria in hiring, probation, and promotion decisions, recognition that funded research should be conducted openly and transparently, the importance of publishers mandating the publication of research workflows and appropriately curated data associated with each research output). Similarly, institutions need to be incentivised to encourage the adoption of open and transparent practices by researchers. Research quality should be prioritised over research quantity. As research transparency will look different for different disciplines, there can be no one-size-fits-all approach. An outward looking and joined up UK research strategy is needed that places openness and transparency at the heart of research activity. This should involve key stakeholders (institutions, research organisations, funders, publishers, and Government) and crucially should be focused on action. Failure to do this will have negative consequences not just for UK research, but also for our ability to innovate and subsequently commercialise UK-led discovery.
While high risk of failure is an inherent part of developing innovative therapies, it can be reduced by adherence to evidence-based rigorous research practices. Numerous analyses conducted to date have clearly identified measures that need to be taken to improve research rigor. Supported through the European Union's Innovative Medicines Initiative, the EQIPD consortium has developed a novel preclinical research quality system that can be applied in both public and private sectors and is free for anyone to use. The EQIPD Quality System was designed to be suited to boost innovation by ensuring the generation of robust and reliable preclinical data while being lean, effective and not becoming a burden that could negatively impact the freedom to explore scientific questions. EQIPD defines research quality as the extent to which research data are fit for their intended use. Fitness, in this context, is defined by the stakeholders, who are the scientists directly involved in the research, but also their funders, sponsors, publishers, research tool manufacturers and collaboration partners such as peers in a multi-site research project. The essence of the EQIPD Quality System is the set of 18 core requirements that can be addressed flexibly, according to user-specific needs and following a user-defined trajectory. The EQIPD Quality System proposes guidance on expectations for quality-related measures, defines criteria for adequate processes (i.e., performance standards) and provides examples of how such measures can be developed and implemented. However, it does not prescribe any pre-determined solutions. EQIPD has also developed tools (for optional use) to support users in implementing the system and assessment services for those research units that successfully implement the quality system and seek formal accreditation. Building upon the feedback from users and continuous improvement, a sustainable EQIPD Quality System will ultimately serve the entire community of scientists conducting non-regulated preclinical research, by helping them generate reliable data that are fit for their intended use.
In: Bespalov , A , Bernard , R , Gilis , A , Gerlach , B , Guillén , J , Castagné , V , Lefevre , I A , Ducrey , F , Monk , L , Bongiovanni , S , Altevogt , B , Arroyo-Araujo , M , Bikovski , L , de Bruin , N , Castaños-Vélez , E , Dityatev , A , Emmerich , C H , Fares , R , Ferland-Beckham , C , Froger-Colléaux , C , Gailus-Durner , V , Hölter , S M , Hofmann , M C , Kabitzke , P , Kas , M J , Kurreck , C , Moser , P , Pietraszek , M , Popik , P , Potschka , H , Prado Montes de Oca , E , Restivo , L , Riedel , G , Ritskes-Hoitinga , M , Samardzic , J , Schunn , M , Stöger , C , Voikar , V , Vollert , J , Wever , K E , Wuyts , K , MacLeod , M R , Dirnagl , U & Steckler , T 2021 , ' Introduction to the EQIPD quality system ' , eLife , vol. 10 , e63294 . https://doi.org/10.7554/eLife.63294
While high risk of failure is an inherent part of developing innovative therapies, it can be reduced by adherence to evidence-based rigorous research practices. Supported through the European Union's Innovative Medicines Initiative, the EQIPD consortium has developed a novel preclinical research quality system that can be applied in both public and private sectors and is free for anyone to use. The EQIPD Quality System was designed to be suited to boost innovation by ensuring the generation of robust and reliable preclinical data while being lean, effective and not becoming a burden that could negatively impact the freedom to explore scientific questions. EQIPD defines research quality as the extent to which research data are fit for their intended use. Fitness, in this context, is defined by the stakeholders, who are the scientists directly involved in the research, but also their funders, sponsors, publishers, research tool manufacturers, and collaboration partners such as peers in a multi-site research project. The essence of the EQIPD Quality System is the set of 18 core requirements that can be addressed flexibly, according to user-specific needs and following a user-defined trajectory. The EQIPD Quality System proposes guidance on expectations for quality-related measures, defines criteria for adequate processes (i.e. performance standards) and provides examples of how such measures can be developed and implemented. However, it does not prescribe any pre-determined solutions. EQIPD has also developed tools (for optional use) to support users in implementing the system and assessment services for those research units that successfully implement the quality system and seek formal accreditation. Building upon the feedback from users and continuous improvement, a sustainable EQIPD Quality System will ultimately serve the entire community of scientists conducting non-regulated preclinical research, by helping them generate reliable data that are fit for their intended use.
In: Bespalov , A , Bernard , R , Gilis , A , Gerlach , B , Guillen , J , Castagne , V , Lefevre , I , Ducrey , F , Monk , L , Bongiovanni , S , Altevogt , B , Arroyo Araujo , M , Bikovski , L , de Bruin , N , Castaños-Vélez , E , Dityatev , A , Emmerich , C H , Fares , R , Ferland-Beckham , C , Froger-Colléaux , C , Gailus-Durner , V , Hölter , S M , Hofmann , M C , Kabitzke , P , Kas , M J H , Kurreck , C , Moser , P , Pietraszek , M , Popik , P , Potschka , H , Prado Montes de Oca , E , Restivo , L , Riedel , G , Ritskes-Hoitinga , M , Samardzic , J , Schunn , M , Stöger , C , Voikar , V , Vollert , J , Wever , K E , Wuyts , K , MacLeod , M R , Dirnagl , U & Steckler , T 2021 , ' Introduction to the EQIPD quality system ' , eLife , vol. 10 , e63294 . https://doi.org/10.7554/eLife.63294 ; ISSN:2050-084X
While high risk of failure is an inherent part of developing innovative therapies, it can be reduced by adherence to evidence-based rigorous research practices. Numerous analyses conducted to date have clearly identified measures that need to be taken to improve research rigor. Supported through the European Union's Innovative Medicines Initiative, the EQIPD consortium has developed a novel preclinical research quality system that can be applied in both public and private sectors and is free for anyone to use. The EQIPD Quality System was designed to be suited to boost innovation by ensuring the generation of robust and reliable preclinical data while being lean, effective and not becoming a burden that could negatively impact the freedom to explore scientific questions. EQIPD defines research quality as the extent to which research data are fit for their intended use. Fitness, in this context, is defined by the stakeholders, who are the scientists directly involved in the research, but also their funders, sponsors, publishers, research tool manufacturers and collaboration partners such as peers in a multi-site research project. The essence of the EQIPD Quality System is the set of 18 core requirements that can be addressed flexibly, according to user-specific needs and following a user-defined trajectory. The EQIPD Quality System proposes guidance on expectations for quality-related measures, defines criteria for adequate processes (i.e., performance standards) and provides examples of how such measures can be developed and implemented. However, it does not prescribe any pre-determined solutions. EQIPD has also developed tools (for optional use) to support users in implementing the system and assessment services for those research units that successfully implement the quality system and seek formal accreditation. Building upon the feedback from users and continuous improvement, a sustainable EQIPD Quality System will ultimately serve the entire community of scientists conducting non-regulated preclinical ...