COLLECTION: DATA MANAGEMENT PLANNING ACROSS DISCIPLINES AND INFRASTRUCTURES

A review at our institution and a number of other Australian universities was conducted to identify an optimal institutional-wide approach to Research Data Management (RDM). We found, with a few notable exceptions, a lack of clear policies and processes across institutes and no harmonisation in the approaches taken. We identiﬁed limited methods in place to cater for the development of Research Data Management Plans (RDMPs) across different disciplines, project types and no identiﬁable business intelligence (BI) for auditing or oversight. When interviewed, many researchers were not aware of their institution’s RDM policy, whilst others did not understand how it was relevant to their research. It was also discovered that primary materials (PM), which are often directly linked to the effective management of research data, were not well covered. Additionally, it was unclear in understanding who was the data custodian responsible for overall oversight, and there was a lack of clear guidance on the roles and responsibilities of researchers and their supervisors. These ﬁndings indicate that institutions are at risk in terms of meeting regulatory requirements and managing data effectively and safely. In this paper


INTRODUCTION
Academic institutions are generating vast amounts of research data across numerous disciplines.In the spirit of open science and promoting the highest levels of research integrity, institutions develop policies and procedures to clarify the requirements of researchers for data management, but also to guide them in meeting these.Like all such governance frameworks, they are only as good as the manner in which they are implemented and used.It has become common practice to make mandatory training a part of employment, and most researchers will complete this training on induction and then conduct annual refreshers or undertake new training modules as required.
Research Data Management Plans (RDMPs) are the cornerstone of good research practice, and all institutions require them (Bellgard 2020;Williams et al. 2017).There are abundant reasons why RDMPs are best practice and should be developed and used, but whilst some believe they are mandatory, they are not prescribed under any legislation.In Australia, the National Health and Medical Research Council (NHMRC) sets out the responsibilities institutions have with respect to responsible behaviour in the Australian Code for the Responsible Conduct of Research (2018), 'the Code' (Australian Code 2018).Whilst this is not a statute, since it is also endorsed by the Australian Research Council and Universities Australia, it acts as a de facto requirement for all institutions that receive federal funding.In practice, no academic organisation in Australia would not comply with it.The Code sets out a series of principles expected of institutions, although there is no mechanism in place to verify that they obey the principles.
The Code requires institutions to have policies for managing research data and primary materials, in addition to providing guidance and training to assist researchers applying the principles.The Code's principles cover data ownership, stewardship and control of research data and primary materials, although the Code does not provide explicit guidance on how this should be delivered-allowing a broad range of interpretations.The NHMRC has instead published supporting guidance for the Code in a supplementary guide (Univ.Australia et al.).The NHMRC strongly encourages the development and use of a management plan for each research project, but does not mandate this.To complement the Code, the NHMRC has also published an ethical guidance document called the National Statement (NS) on Ethical Conduct in Humans (2007( , as amended in 2018-the NS)-the NS).
Like the Code, the NS is not a legal instrument, but it intersects with some activities that require its use that are under legislation, such as the Therapeutic Goods Act (1989) for clinical trial activities and the various Commonwealth, State and Territory and private sector data privacy legislation.Like the Code, the NS also describes the need for RDMPs as best practice, particularly in the creation of research databanks, but does not mandate their usage.The Code and the NS are guidelines that act at a national level and are signed up to by all universities.Given their importance, there may be an expectation that there was some level of standardisation in their implementation and operation across Australia.

ENVIRONMENTAL SCAN
Given the importance of Research Data and Primary Material (RD+PM) management plans, and as we were about to embark on a refresh of the QUT RD+PM policy and procedure, we sought to understand how it was currently being done, what researchers thought of it and how they thought it might be improved.We undertook a review of the QUT approach to managing RDMPs through the evaluation of the existing policies, procedures, related websites and any work instructions or departmental guidance.The QUT library was primarily responsible for managing RDMPs and had constructed a website with an online tool.Whilst the library reported a high level of utilisation, it was revealed that as it was not a structured database, it could not be analysed for content, and there was no record of how the plans were actually being used or monitored.
In addition to our review of our own activity, we examined the various approaches to Research Data Management (RDM) across 20 Australian Universities through an examination of their online policies, procedures, guidance material and forms.We have not presented all of the findings of that review here, as the content rapidly changes, and several have been undertaking their own reviews and enhancement of their processes.However, we found that several of the websites did not provide clear guidance material, and there was practically no consistency between them in terms of an approach.

UNDERTAKING AN INTERNAL CONSULTATION PROCESS TO IDENTIFY THE UNMET NEED
Like all QUT policies, the RDM policy is subject to cyclical review every three years, and the Office of eResearch was charged with the responsibility of performing a review.We formed the Research Data Management Strategy-Implementation Group (RDMS-IG), which comprised representative key academics across science, biomedical science and the humanities, in addition to the Office of eResearch and the Office of Research Ethics and Integrity.The IG met virtually on a monthly basis to provide guidance to the QUT eResearch team working on revisions to the policy and procedure (P&P).The RDMS-IG identified collectively and through extensive one-onone interviews with a range of academic and professional staff, that, like many institutional policies and procedures, the DMP was not widely known amongst their colleagues, and they did not refer to it on a regular basis.Instead, they noted that staff and students were required to make themselves aware of and to abide by the relevant P&P, and that a number of internal processes were in place to ensure that this happens.Through this stakeholder engagement, it was agreed that Primary Material management plans should be included.
Whilst the knowledge and application of institutional policies is a requirement of employment compliance, they are regarded as a chore that must be done at the start of employment or enrolment-and can be largely forgotten about in between annual mandatory training requirements.This formal compliance requirement has had an unfortunate effect, moving researchers into a 'compliance-mentality,' motivated largely by the need to check off a box, rather than approaching RDM+PM management as an important core component of conducting research itself.It should be noted that this is observed in related areas such as ethical review, where the need to 'get ethical approval' has replaced the primary notion of being ethical and making sure that the research project itself is intrinsically ethical (De Peuter & Conix 2022).
However, it was clear that whilst Policies and Procedures (P&Ps) related to Occupational Health and Safety (OHS), and other key compliances had mandatory training through an Learning Management System (LMS), the DMP webpage was not seen as essential, and a review of content indicated that there was a need for a refresh.Indeed, a key finding from the IG activity was the need to tailor the content of a DMP to the type of research being conducted.A DMP for law would necessarily be quite different to one for earth sciences or civil engineering.
Anecdotally, it was revealed to us that a lack of mandatory requirements for DMPs.together with a-less-than-friendly user process meant that DMPs were not taken as seriously as the university would like.We also found that in each institution, there was not one person with responsibility for its oversight, and the majority of senior people, while committed to the ideals and objectives of responsible research practice, felt they were under-resourced to deliver training to their staff and students and monitor how RDM was being conducted in everyday situations.
We responded by developing an institutional framework for RDM+PM management, taking a different co-design approach.Importantly, we adopted the approach to include PM, not just RDM.We sought to explore how we could create policy and guidance material for RDM and PM that fulfils the principles set out in the Australian Code for the Responsible Conduct of Research and in other international consensus policies such as the FAIR principles (Wilkinson et al. 2016).
We recognised that there is a natural alignment, especially for a number of disciplines, and RDM and PM plans can come in different forms.To deliver to this approach, we developed an online RDM+PM checklist to support and direct researchers.This checklist currently asks seven questions, taking into consideration the different needs of disciplines spanning Arts, Law, Business, Humanities, Health, Environment and STEM.In this manuscript, we provide an overview of the checklist tool, its inherent ability to enhance the engagement between researchers, their research students and all key stakeholders across the institution, including: research grants, ethics, facilities management, audit and risk, faculties, schools and research centres.The checklist captures real-time business intelligence (BI) statistics that allows us to: i) understand key blockers for any given researcher need and identify stakeholders to engage with to resolve them; ii) continuous feedback on the policy and guidance material; and iii) the ability to capture real-time BI data that can be used to define a metric to quantify an institution's maturity in managing research data, which we refer to as its Data Quotient™.

METHODS: THE APPROACH TO DEVISING THE RDM+PM CHECKLIST TOOL
In providing simplified guidance to researchers, we ask seven questions at the outset of their research projects around the following topics: do you understand your obligations as per the university RDM+PM policy?; does your research involve humans and animals?; are you creating, collecting or receiving digital data?; are you creating, collecting or receiving physical (primary) materials?; will the data be available for publishing?; and are there any limitations to sharing your research data?
We decided to implement an online tool to ask these questions.We summarise that the aim of the RDM+PM Checklist is to: i) direct researchers so they can have relevant conversations with QUT Subject Matter Experts, depending on the nature and content of their specific research project; ii) create a skeleton or framework for the creation of RDMPs, relevant to the evolving requirements of their project; iii) provide researchers and research supervisors with a tool to identify, monitor, and report on the specific areas of data and primary materials governance that their project requires; and iv) establish a repository of checklists that will facilitate the usable reporting of research activities across the institution and the ability to share these with researchers external to the institution.Figure 1 outlines the process we followed.

DEVELOPMENT PRINCIPLES
The tool is written as a responsive web application with a React frontend and a stateless API backend.Frontend and backend components are written in JavaScript and leverage cloud native apis to service storage, authentication, authorisation and monitoring requirements.The following principles guided the development of the tool: • High availability: the tool will be available for use outside of scheduled maintenance periods • Accessible: the tool is accessible to all university users.Usability is reviewed against university guidelines and audited by external third parties • Low cost: the tool is low cost to run, and the cost of the provisioning, running, and maintaining the tool will not incur a cost barrier to the tool owners or its users • Self-service: users are able to perform all required interactions with minimal training, rather than a formal onboarding process

ARCHITECTURAL DESIGN OF THE RDM+PM CHECKLIST
Figure 2 provides an overview of the architectural design implemented in Amazon Web Services.Each of the architectural components are described in Table 1.

SUMMARY OF KEY FINDINGS FROM CONSULTATIONS
As part of our review to revise the policy and procedures for RDM, we asked standard questions including: i).Should every research project have institutional project data storage space created by default?
ii).What should be the standard storage environment?
iii).How do we identify and highlight restricted technology use as part of the Checklist, e.g.Clustered Regularly Interspaced Short Palindromic Repeats (CRISPR) policy, etc.For the top 20 Australian universities, we examined whether they had a current policy, who is responsible for its implementation and management, what resources are available for help with developing plans, how compliance was monitored and how plans integrated with research ethics and integrity (where applicable).We also examined whether they used any online tools to capture plans and whether these were able to provide a means to find data.The findings are summarised in Table 2. Finally, we explored whether institutions provided support, such as running courses in how to develop RDM or PM plans for undergraduates and staff, and whether this was required as part of policy compliance.Based on our analysis, we then focused our desktop review on each QUT faculty.The findings are summarised in Table 3.  • attach an exported copy of their plan as supplementary documentation for grant or ethics applications; • communicate project data requirements to external and industry partners.
To facilitate this, we implemented an online RDM+PM checklist.

RDM+PM CHECKLIST
The Once logged in, researchers (HDR or career) can create a checklist specific to their project.The interactive checklist will present the user with simple questions, dynamically updating the question list and directing researchers towards relevant conversations with QUT Subject Matter Experts, depending on the nature and content of the answers they provide.The checklist is shown in Figure 3.
Key functional features in the current version of the Checklist are shown in Table 4.
As part of our extensive stakeholder engagement and piloting of the checklist, we obtained the following feedback from users, as outlined in Table 5.Researchers can use the tool as a skeleton or framework for the creation of RDM+PM plans, relevant to the evolving requirements of their project.HDR supervisors can use the tool to review student progress in their data management and spark meaningful discussions around data and primary material storage.HDR supervisors are trained with the tool and are expected to ensure all HDR students use the checklist as a mandatory process when initiating projects-complementing the training researchers receive as part of their induction certificate in checklist usage.

Web-based
Available to any web accessible device

Device dynamic
Display dynamically reacts to type of device used for readability

Tiered Access
User groups are assigned appropriate roles, and read/write access levels are customised to appropriate groups

SSO integration
Integration with institution Single Sign-On capabilities

User Groups
Multiple types of User are defined in groups, with appropriate access/feature set applied

Customised Branding
System is able to be branded to institution specifics

Metadata capture
All checklists produce relevant metadata per project, as defined by the institution

Customised Questions
All questions are customisable to institution preferences

Customised Links
All informational direction links are customisable to institution preferences

Email trigger
Directional link can trigger email to appropriate group if required

Accessibility Standards
Platform meets Australian Accessibility standards

Active Directory connection
System connects to institution active directory for pre-fill fields

API integration
System is able to use API to connect to other institution systems, e.g.PURE

Predefined list
System is able to offer predefined answers for fields (as opposed to open text)

Prefill
System can prefill metadata based on other API integrations (e.g.SSO, PURE)

Form widgets
Platform offers web-form widgets for customisation of questions, e.g.calendar, short field, check box, radio button etc.

Save
User is able to save a checklist

Revert
User is able to revert answers to previous save

Edit
User is able to edit an existing checklist

Print to PDF
User is able to print checklist as PDF

Delete
User is able to delete existing checklist

Supervisor linkage
User is able to link checklist to their supervisor

Collaborator Linkage
User is able to link checklist to collaborator

Search -User
Admins are able to search all users and checklists in system

Monitoring
System creates monitoring logs for performance maintenance

Login/Logout
User is able to securely login and out of system

Search -Filter
User is able to use filter search to narrow view of existing checklists

Import
System can import data from other systems (e.g.PURE)

Conditional Rendering
Questions are dynamically displayed based on field input of user

Realtime analytics
Admin are able to view real-time analytics of checklist data and usage

Data export for analytics
Admin are able to export all data from platform for detailed analytics

Supervisor management view
Supervisors can view all checklists they have been linked to

Collaborator management view
Collaborators can view all checklists they have been linked to The RDM+PM checklist provides researchers and research supervisors with a tool to identify, monitor and report on the specific areas of data and primary materials governance that their project requires.The RDM+PM checklist has been made mandatory for all HDR students to complete from the launch date.
Business intelligence can be obtained in real-time.Since August 2022, the checklist became mandatory for new HDR students.At the time of taking this data snapshot, as shown in Table 5, the compliance rate was very high, which involved a new cohort of 142 HDR students who used the tool.We have developed training on RDM as part of their induction, in addition to access to a self-service training video.Only a small number of students have not answered particular questions, and these can be drilled down to assist particular students and if necessary, revise the guidance material for students.From Table 6, if we simply average the percentage answered across the seven questions, we arrive at 97%.This high percentage not only demonstrates the usefulness of the tool, but also provides a mechanism to ultimately measure the institution's maturity with regards to RDM and PM management.We refer to this measure as an institution's Data Quotient, and this will be detailed in a subsequent publication.Within the checklist, there is functionality to enable the analysis of data in real-time (analytics), as shown in Figure 4.A range of graphical and tabular representations are dynamically available.
'I like the simple look and feel of the checklist.''This will help guide researchers to ask the right questions at the start of their projects.'As data management needs vary widely between projects depending on their discipline and scope, the checklist provides a dynamic solution that adapts to fit the need of the researcher in real-time.Crucially, the checklist offers guidance for both data management and primary material management.It is simple to use and understand and includes links to all relevant Subject Matter Experts (SMEs) to ensure that correct guidance for specific management practices is provided.It will be integrated into the early stages of the researcher lifecycle to ensure information is understood in a timely manner.The checklist aligns to the objectives of FAIR and data sharing programs that aim to minimise waste, allow for the re-purposing of data, facilitate reproducibility tests, and so forth.Importantly, the checklist can also foster collaboration across institutions, where the key seven questions can be customised for individual institutional needs, socialising the opportunity to arrive at a 'common' set of RDM and PM questions, while promoting best practice and sharing knowledge.
The approach devised by QUT has been acknowledged nationally through both external research infrastructure funding and by shifting the narrative for institutions to guide researchers in research data management planning, rather than providing a single data management planning tool that is not fit for purpose for many research disciplines (ARDC).
The questions posed by the checklist have been designed to directly reflect the current policy and for the first time, provide a systematic approach to guide researchers.Importantly, statistics are immediately available in real-time to enable managers to begin measuring the institution's preparedness for RDM.It is envisaged that a score can be devised, which can provide weightings on the answers to specific questions and identify opportunities though ongoing feedback where checklist functionality can be enhanced.
At the time of writing this manuscript, it is mandatory for PhD student projects to use the checklist, and we believe this is a strong representative basis.This was a university-wide decision as an extension of the mandatory induction and onboarding training material for students.The derived benefits include: As PhD projects are diverse in nature, the checklist needs to be 'fit for purpose'; the checklist promotes RDM+PM skills literacy and training at an early career stage, as well as ensuring supervisors' responsibility in understanding RDM+PM policy.This is reflected in the high compliance results already captured with the tool.Updates to the tool are underway; the checklist is being scoped to support the development of RDM plans, and feedback is obtained, as the tool is currently undergoing customisation for deployment at another institution.This approach enables an agile iterative refinement of the tool through the sharing of best practice between institutions.Further extensions the tool's functionality is through the automated provisioning of digital and data storage and analytic tool environments to support specific research projects.

Figure 1
Figure 1 Process to develop RDM Plans via the RDM+PM Checklist.
: protects application and AC-06, AC-08 and AC-11 from common web attacks.AC-02 CDN Content Delivery Network: Fast transfer of statics assets and protects against common malicious attacks AC-01 AC-03 Front-end Distribution Static website resources fronted by AC-02, not directly accessible.
Office of eResearch at QUT has designed a web-based Research Data Management + Primary Materials (RDM+PM) checklist to support and direct researchers, taking into consideration the different needs of disciplines spanning Arts, Law, Business, Humanities, Health, Environment and STEM.The solution conception, design and development has been driven by the end-users, relying on the co-development cohort for iterative feedback and enhancement throughout.The tool utilises QUT Single Sign-On for authentication and tiered user access controls; HDRs, supervisors, and other researchers can all use the same platform for different use cases.The checklist tool will be open sourced.

Figure 3
Figure 3 Creating a new RDM+PM Checklist.

Figure 4
Figure 4 Stacked bar chart of the data captured in the Checklist for HDR students.

Table 2
Summary of various approaches to Research Data Management (RDM) across 20 Australian Universities.
Figure 2 Architectural design of the RDM+PM Checklist.

Table 4
Key functional features in the Checklist tool.

Table 5
Table showing the responses from new HDR students.

Table 6
Table showing the responses from new HDR students.