Proposal
Table of Contents
- The Rationale for and Aims of the Project
- Relationship to Research in the Area
- Current Status of the Proposed Project
- The Approach the Project Will Take
- The Anticipated Deliverables and Outcomes
- Conclusion
- Attachment 1 - Bibliography
- Attachment 3 - Time lines
- Attachment 4 - Project Team
- Attachment 5 - Screenshots of Prototype
- Attachment 6 - Proposed System Architecture
- Attachment 7 - Evaluation Plan
- Attachment 8 - Dissemination Plan
- Attachment 9 - Relationship to the project led by Dr Jo McKenzie (UTS)
The Approach the Project Will Take
This project will be conducted over a two year period beginning in August 2007 and concluding in July 2009. The iterative nature of the systems life cycle proposed for this project will involve formative evaluation throughout the project and revisions of the design of the tool in response to feedback from project team and reference group members, as well as academics who participate in trials of the tool. A summative evaluation will be conducted at the conclusion of the project. The various stages of the project are summarised in an attached timeline (Attachment 3).
Management of the project will be the responsibility of the project team leader (Dr Denise Wood) and will follow guiding principals of Australian Standard 3950: Guide to Managing Product Design as the main goal is to develop a new open source, Web-enabled software product. Project team members will meet regularly via Centra (an online voice enabled communication system) and extensive documentation will be maintained, updated regularly and circulated electronically to the team members each month in advance of scheduled online meetings. This documentation will include:
- Communication strategy report, which identifies the key stakeholders (Australian Learning and Teaching Council (ALTC), Reference Group, Implementation Team and Project Manager) and records the method of communication, frequency of communication and issues raised by team members and/or other stakeholders.
- Gantt chart showing each stage of the project, milestones and both current status and projected timelines.
- Status report documenting the status of work completed.
- Issues log which records problems identified, priorities, impact, resolution strategy and date issue is resolved.
This project will support the scholarship of online learning and teaching through the enhancement of a prototype peer review tool (see Attachment 5) developed at UniSA and already trialed in courses in the pharmacy, built environment and nursing areas as reported by Wood and George (2003), George and Wood (2004, 2005), and George, Wood and Wache (2004). The enhancement of the prototype peer review tool through open source will demonstrate that it can support a range of online learning platforms. The peer review tool, once developed, will be trialed and evaluated by the participating universities, other Australian universities and several institutions that are members of the North American National Universities Telecommunications Network (NUTN), as well as UK organisations including the Higher Education Academy, the Association of Learning Technology and the Joint Information Services Committee. The participation of US and UK international partners will ensure the project is benchmarked against international standards. A website will be established as a means for dissemination of project information and deliverables, and will provide a vehicle for recruiting universities wishing to participate in the project (see Attachment 6). The website will also provide a mechanism for ensuring transparency and public accountability throughout the project.
The tool will be based on an agreed set of criteria and standards of good practice. It will provide a comprehensive, integrated peer review system that enables staff to record their achievements in online learning and teaching. It will provide staff with information to support their applications for academic promotion. Therefore this project will:
- Work with online learning and teaching specialists in Australian universities to identify primary and source materials relevant to the peer review of online learning and teaching;
- Populate the peer review tool with additional criteria and examples of primary and source materials of online learning and teaching ;
- Build on previously developed criteria for the peer review of online learning and teaching;
- Enhance the prototype peer review tool to develop a Web-enabled tool that can be used across different online platforms;
- Pilot the Web-enabled peer review tool with staff and evaluate the value and effectiveness of the tool to academic staff being peer reviewed and those conducting the peer review;
- Develop and disseminate case studies of the use of the Web-enabled peer review tool to the ALTC higher education community;
- Develop, evaluate and disseminate documentation that supports the use of the Web-enabled peer review tool;
- Develop, evaluate and disseminate documentation on the technical aspects of the open source Web-enabled tool;
- Establish and enhance networks amongst the Australian university community with an emphasis on the cultural acceptance between institutions of the value of a Web-enabled peer review tool for online learning and teaching.
Project team members will undertake the following responsibilities:
- Extend the review of existing tools already undertaken to identify features to be incorporated in the tool.
- Assist in identification of criteria that need to be included in the system, locating relevant references to embed as explanatory links to each criterion and locating exemplars of best practice.
- Advise on the technical aspects of the tool throughout design and development stages of the project.
- Undertake trials of the tool, as well as promote and facilitate trials across different disciplinary area within their respective institutions and via their networks in the wider higher educational community.
- Assist in formative and summative evaluation.
- Promote dissemination of the project deliverables.
Evaluation Plan
Evaluation of the project will be conducted in a formative manner throughout the design and development stages, and a summative evaluation will be conducted at the completion of the project. A national and international reference group will be established to oversee the project and will monitor the outcomes at each evaluation milestone. The evaluation strategy will incorporate the evaluation of:
- The user’s experience as a peer reviewer;
- The user’s experience as a peer reviewee;
- How the findings from this project can contribute to the evaluation policy of individual institutions and the policy of institutions as a collective;
- The project overall and in particular, how the collaboration of institutions in this area can be taken forward;
- The enhanced open source Web-enabled peer review tool according its fitness for purpose and its technical adequacy.
The summative evaluation will involve a more formal approach using a structured questionnaire and interviews. These results will be analysed and further amendments made to the tool based on user feedback (see Attachment 7 for a detailed summary of the evaluation plan).
Dissemination Plan
Both individually and collectively, the partner institutions are well placed in the Australian higher education community to inform and create awareness of the project (see Attachment 8). The partners have professional connections with HERDSA; ASCILITE; ODLAA; AUSWEB, ATN, the North American National Universities Telecommunications Network and UK higher education committees, and have a wealth of experience in running effective dissemination and development programs on a range of learning and teaching issues. Dissemination will include the two-way sharing of the outcomes and learning resulting from the project at different stages with stakeholders and the community. Dissemination activities will be planned to occur throughout the project, each with a specific purpose, target audience, timing, and key message and include annual conferences, regular newsletters and email communication to interested parties. A website will be maintained to enable academics from any institution to have input into the development of the system, or to register their interest in participating in the trials.