Scoring Rubrics
Read this next example of a scoring rubric, which guides grant reviewers through the scoring process by posing a series of questions about the grant proposal.
California Education Learning Lab
2019-2020 Request for Proposals
"Using Research and Technology to Transform Undergraduate STEM Education"
Evaluation Rubric for Innovation Grant Full Proposals
Section A: Project Narrative
Innovation Grants will support projects aiming to transform the culture of learning, improve learning outcomes, and close equity and achievement gaps. These grants will support both projects that focus on developing and implementing pedagogical/curricular innovations in lower-division STEM courses and projects that aim to improve learning outcomes in undergraduate STEM education through large-scale faculty professional development programs.
Applicant teams were asked to submit a project narrative (fifteen pages maximum) that identifies the problem that the project aims to solve, explains the project's approach to solving this problem (i.e., the pedagogical/curricular innovation or proposed faculty professional development program), and details the project's specific goals and research strategy.
Please evaluate the project narrative based on the following questions. These questions correspond to key elements/components that applicant teams were asked to address within the overall context and as integral parts of their project narrative. For each question, please provide a brief assessment of how fully and effectively the proposal addresses the key element/component. Please evaluate each component within the context and entirety of the overall proposal.
Question |
Reviewer Assessment (c. 200 words per response) |
| 1. Innovation: Is the proposed project genuinely innovative? Does the proposed project include approaches, methodologies, interventions, or resources that are original and that show significant promise for improving learning outcomes and closing equity gaps, or does it employ demonstrated approaches, methodologies, interventions, or resources in ways that substantially advance pedagogical/curricular practice? Please assess the extent to which the proposal contains genuinely innovative pedagogical/curricular approaches or practices, and/or a genuinely innovative large-scale program of faculty professional development. (Please note: the RFP specifies that proposals may focus either on developing and on implementing pedagogical/curricular innovations or on creating innovative large-scale professional development programs, but projects may also address both pedagogy/curricula and professional development.) |
|
| 2. Culture of Learning: Does the proposed project demonstrate potential to transform the culture of learning? For the purposes of this grant, "culture of learning" refers both to the relationship between learning/education-based research and teaching practice, and to classroom/disciplinary educational culture (especially as it relates to students' sense of belonging in the classroom/discipline and to their experience of the classroom/discipline). Based on the proposal, please assess the potential of the proposed innovation or proposed professional development program to bridge traditional divisions between learning research and teaching practice and to change disciplinary and/or classroom learning culture in lower-division STEM courses. |
|
| 3. Research Design: Does the project narrative include a clear research strategy, including an explicit hypothesis, measurable objectives, and a well-developed assessment plan for evaluating the implementation and effectiveness of the project approach? Does project have potential to advance existing research in the science of human learning or in discipline-based education research? Please evaluate the project's research strategy, including the quality of the hypothesis, measurable objectives, and assessment plan. The assessment plan should discuss how the project team will identify assessments that are valid and reliable within the context of the proposed project's courses or innovations. |
|
| 4. Data and Technology Tools: Does the project narrative include a well-conceived plan for using data and technology tools to improve learning outcomes and/or facilitate the collection of learning data? Please evaluate the proposal's plan for using data and technology tools and for using data collected to iteratively develop or improve the project approach. |
|
| 5. Student Engagement and Sociocultural Barriers: Is the proposal grounded in literature relating to sociocultural barriers to student learning, and does it include a well-informed approach to increasing student engagement, especially among students who may not identify as STEM proficient? Does it convincingly explain how the proposed approach will address aspects of "traditional" classroom or disciplinary culture that are barriers to student learning and their sense of belonging? Please evaluate the project's approach to increasing student engagement and to overcoming socio-cultural barriers to student learning, and the degree to which this approach is grounded in and informed by relevant research literature. |
|
| 6. Scalability and Lasting Impact: Does the proposed project demonstrate clear potential for replication and dissemination, and/or capacity to effect positive pedagogical/curricular change at scale? Please evaluate the proposal's plan for disseminating/scaling the proposed innovations or professional development program and for achieving lasting impact at scale. |
|
| 7. Project team: Does the proposed project include meaningful, well-balanced collaboration among partner institutions and within the project team? Does the project team include social scientists, behavioral scientists, discipline-based education researchers, instructional designers, team members with expertise in learning assessment, and/or others with relevant expertise, and do these team members have clear roles within the project? |
Section B: Responsiveness to Statutory Selection Criteria
Statute established the California Education Learning Lab as a competitive grantmaking program for intersegmental faculty teams to incorporate learning science and adaptive learning technology into their curriculum and pedagogy, with the express purpose of increasing learning outcomes and closing equity and achievement gaps in STEM and other disciplines.
The following questions (8-15) assess proposals based on how effectively they meet statutory selection criteria.
Please indicate whether the additional statutory strength is part of the proposal by answering "Yes" or "No" in your evaluation.
Category |
Optional Brief Comments |
| 8. Does the project display "potential for reducing achievement and equity gaps in the particular discipline that is the subject of the call for proposals"? (Citation: Government Code 65059.2 (b)(1)(A)) |
|
| Evaluation for Question 8 Yes No |
|
| 9. Does the project team contain meaningful "depth and breadth of expertise in the particular discipline and deployment of learning science or adaptive learning technologies"? (Citation: Government Code 65059.2 (b)(1)(B)) |
|
| Evaluation for Question 9: Yes No |
|
| 10. Does the proposed project demonstrate potential for "increasing equity and accessibility in quality STEM education and other disciplines that show high initial failure or dropout rates, including scaling access to the newly developed or redesigned course or course series in the future"? (Citation: Government Code 65059.2 (b)(1)(C)) |
|
| Evaluation for Question 10 Yes No |
|
| 11. Does the proposed project demonstrate "potential to incorporate real-time learning outcome data to improve the curriculum"? (Citation: Government Code 65059.2 (b)(1)(D)) |
|
| Evaluation for Question 11 Yes No |
|
| 12. Does the proposed project utilize, or demonstrate potential to utilize, a "common technology platform to deliver the course or course series"? (Citation: Government Code 65059.2 (b)(1)(E)) |
|
| Evaluation for Question 12 Yes No |
|
| 13. Does the project include "representation of all three public higher education segments on the proposal's faculty team"? Note: Projects are required to include representation from only two of the three segments of public higher education. The representation of all three segments is considered an additional strength of the proposal. (Citation: Government Code 65059.2 (b)(1)(F)) |
|
| Evaluation for Question 13 Yes No |
|
| 14. Is there "inclusion of career education and workforce pathways in the proposed project"? This can include, but is not limited to, discussion of engaging with student career goals in redeveloping curricula, and of relating curricula to student career interests. (Citation: Government Code 65059.2 (b)(1)(G)) |
|
| Evaluation for Question 14 Yes No |
|
| 15. Does the proposed project identify "opportunities to leverage nonstate funding"? (Citation: Government Code 65059.2 (b)(1)(H)) |
|
| Evaluation for Question 15 Yes No |
Section C: Overall Assessment of Impact
The following evaluation should be based on your overall assessment of the full proposal as a whole. Please assess the likelihood that the proposed project/program, as described in the full proposal, will have a powerful, sustainable, positive influence on teaching and learning, and broad and significant impacts in improving learning outcomes and reducing equity/achievement gaps in the discipline or disciplines represented. In evaluating overall impact, please consider the proposal's feasibility, degree of scalability, affordability, replicability, and degree of innovation in concepts, approaches, methodologies or interventions.
Please use the following key to evaluate this overall impact:
High (score 5 or 4)
- Application has a high likelihood of powerful, sustainable, positive influence on teaching and learning, and broad and significant impacts in improving learning outcomes and reducing 6 equity/achievement gaps in the discipline or disciplines represented, with no or minor weaknesses in the proposal.
Medium (score 3 or 2)
- Application seeks to have a powerful, sustainable, positive influence on teaching and learning, and broad and significant impacts in improving learning outcomes and reducing equity/achievement gaps in the discipline or disciplines represented, with weaknesses in the proposal.
- Application addresses a problem of moderate importance, with some or no weaknesses.
Low (score 1)
- Application seeks to address a problem of moderate importance, but weaknesses in the proposal reduce the overall impact to low.
- Application addresses a problem of low or no importance, with some or no weaknesses.
Overall Assessment of Impact |
Score |
| Please provide up to 300-350 words maximum explaining your overall assessment of the proposal's likely impact and highlighting the proposal's strengths and weaknesses, including any constructive suggestions. |
Source: Governor's Office of Planning and Research, State of California, https://opr.ca.gov/docs/20200114-Innovation_Grant_Full_Proposal_Rubric.pdf
This work is in the Public Domain.