The evaluation component of the Pacific RISA project will develop measurement tools and evaluate the role of Pacific RISA in advancing adaptation planning in the Pacific region and thus contribute to assessing the value of the overall program.
The aim of the evaluation work is to measure the progress of adaptation planning in Hawai‘i, improve Pacific RISA program performance, demonstrate successes to funding agencies, stakeholders, and the public, and contribute to the emerging field of evaluating participatory science programs. We plan a two-pronged, but not sequential evaluation effort. The first prong will comprise an external evaluation of the role of Pacific RISA in progressing adaptation planning in Hawai‘i and constitute a discrete project. We will focus on Hawaii in part for logistical and cost reasons but also because Hawaii is where much of our initial work on integrating biophysical and social science and indigenous/local knowledge will take place. In the second prong, we will develop self-evaluation criteria to be administered in each year of the program and use the findings to refine our goals and methods region-wide.
The Pacific RISA received a preliminary evaluation report conducted by Dr. Susanne Moser (NCAR) that assessed and described selected features of the first three years of existence of the Pacific RISA. In particular, this assessment focused on the past and possible future roles of the Pacific RISA within the network of various Hawaiian and Pacific Island climate service organizations. Recommendations of the report are being used to guide aspects of the development of the Pacific RISA. For instance, the Pacific RISA continues to broker information and facilitate stakeholder engagement by favoring interpersonal modes of interaction (e.g., face-to-face meetings) and “localizing” information and communication processes, as appropriate, across and within groups of islands. The Pacific RISA is developing an integrated evaluation system to assess the effectiveness of the stakeholder engagement process.
The Pacific RISA – Climate Adaptation Partnership for the Pacific goals. (Source: Susanne Moser)
Evaluation measures reflect several dimensions of participatory and deliberative stakeholder engagement, including: (1) stakeholder representation—the legitimacy and fairness of the participant selection process; (2) breadth—the number and variety of participants involved; (3) openness of design—the extent participants are involved at early stages of the process, the number of points at which they are involved, and their influence on designing the process; (4) transparency—the extent to which others can obtain information about aspects of the procedure; (5) intensity—the amount of time and effort participants put into the process and the amount of interaction that occurs among them; (6) influence—the extent to which the process provides mechanisms by which participants can affect how the topic is defined, considered, and acted upon; (7) overall satisfaction with process; and (8) overall satisfaction with outcomes.
External Evaluation of the Reach of the PIRCA
Dr. Susi Moser completed an external evaluation in 2012 of the Pacific RISA program’s success in reaching and influencing different audiences in Hawai‘i and the Pacific region. Dr. Moser conducted key informant interviews, deployed an online survey, and analyzed event, website, and media data to 91) ascertain the perceptions of interest in and usefulness of the Pacific RISA-led 2012 Pacific Islands Regional Climate Assessment (PIRCA; Keener et al. 2012); 92) determine the geographic spread and reach of different types of stakeholders and capture interest in PIRCA information; 93) identify indicators of how the PIRCA was received; and 94) identify future information needs and ways to improve future assessments. The evaluation revealed that the PIRCA has done extraordinarily well in its outreach and influence, with several indicators illustrating that the Pacific RISA has been instrumental in achieving exceptional results.
Dr. Moser’s final reportwas delivered in December 2013 and shared with Pacific RISA partners, the RISA program office, and NCA/USGCRP partners (Moser 2013). Recommendations from the evaluation suggest opportunities for future directions of PIRCA and NCA outreach and sustaining regional assessment efforts.
Resilient and sustainable Pacific Island communities using climate information to manage risks and support practical decision-making about climate variability and change.
Evaluating Pacific RISA Performance
The evaluation component of the Pacific RISA project will develop measurement tools and evaluate the role of Pacific RISA in advancing adaptation planning in the Pacific region and thus contribute to assessing the value of the overall program.
The aim of the evaluation work is to measure the progress of adaptation planning in Hawai‘i, improve Pacific RISA program performance, demonstrate successes to funding agencies, stakeholders, and the public, and contribute to the emerging field of evaluating participatory science programs. We plan a two-pronged, but not sequential evaluation effort. The first prong will comprise an external evaluation of the role of Pacific RISA in progressing adaptation planning in Hawai‘i and constitute a discrete project. We will focus on Hawaii in part for logistical and cost reasons but also because Hawaii is where much of our initial work on integrating biophysical and social science and indigenous/local knowledge will take place. In the second prong, we will develop self-evaluation criteria to be administered in each year of the program and use the findings to refine our goals and methods region-wide.
The Pacific RISA received a preliminary evaluation report conducted by Dr. Susanne Moser (NCAR) that assessed and described selected features of the first three years of existence of the Pacific RISA. In particular, this assessment focused on the past and possible future roles of the Pacific RISA within the network of various Hawaiian and Pacific Island climate service organizations. Recommendations of the report are being used to guide aspects of the development of the Pacific RISA. For instance, the Pacific RISA continues to broker information and facilitate stakeholder engagement by favoring interpersonal modes of interaction (e.g., face-to-face meetings) and “localizing” information and communication processes, as appropriate, across and within groups of islands. The Pacific RISA is developing an integrated evaluation system to assess the effectiveness of the stakeholder engagement process.
Evaluation measures reflect several dimensions of participatory and deliberative stakeholder engagement, including: (1) stakeholder representation—the legitimacy and fairness of the participant selection process; (2) breadth—the number and variety of participants involved; (3) openness of design—the extent participants are involved at early stages of the process, the number of points at which they are involved, and their influence on designing the process; (4) transparency—the extent to which others can obtain information about aspects of the procedure; (5) intensity—the amount of time and effort participants put into the process and the amount of interaction that occurs among them; (6) influence—the extent to which the process provides mechanisms by which participants can affect how the topic is defined, considered, and acted upon; (7) overall satisfaction with process; and (8) overall satisfaction with outcomes.
External Evaluation of the Reach of the PIRCA
Dr. Susi Moser completed an external evaluation in 2012 of the Pacific RISA program’s success in reaching and influencing different audiences in Hawai‘i and the Pacific region. Dr. Moser conducted key informant interviews, deployed an online survey, and analyzed event, website, and media data to 91) ascertain the perceptions of interest in and usefulness of the Pacific RISA-led 2012 Pacific Islands Regional Climate Assessment (PIRCA; Keener et al. 2012); 92) determine the geographic spread and reach of different types of stakeholders and capture interest in PIRCA information; 93) identify indicators of how the PIRCA was received; and 94) identify future information needs and ways to improve future assessments. The evaluation revealed that the PIRCA has done extraordinarily well in its outreach and influence, with several indicators illustrating that the Pacific RISA has been instrumental in achieving exceptional results.
Dr. Moser’s final report was delivered in December 2013 and shared with Pacific RISA partners, the RISA program office, and NCA/USGCRP partners (Moser 2013). Recommendations from the evaluation suggest opportunities for future directions of PIRCA and NCA outreach and sustaining regional assessment efforts.
Our Vision
Resilient and sustainable Pacific Island communities using climate information to manage risks and support practical decision-making about climate variability and change.
Tag Cloud
Follow Pacific RISA
Subscribe to our mailing list
Archives
Links