Bias Mitigation Ideas & Strategies for Scholarship Application Reviewers
Thank you for serving on a scholarship selection committee! These tips and guidelines serve as general information to augment the specific instructions you may receive from your particular scholarship program. These guidelines also reflect the approach our office has developed over time in attempting to maximize student support and fairness in reviews.
While we have tried to pull together a wide range of helpful considerations, there are certainly important things we have missed and issues that go beyond the scope of what we can address here.
Maintaining focus in the review process
It can be hard to do this for long stretches of time. To reduce decision fatigue, some reviewers prefer to spread their reading out in short stints over multiple days. Other reviewers prefer to complete their reviews in one day, with several breaks, to help keep their reviews consistent.
Centering fairness in the review process
Consider how scores and comments change over the review period, pay attention to when you tend to get more or less generous. Re-review applications in batches by score type: i.e. re-read all the applications scored as ‘2’ to see if they match, and re-score as needed.
Reviewing applications takes time
Anticipate spending 10-20 minutes per application, though most reviewers get more efficient as they go along. You can also overthink an application, so if you find yourself unable to make a decision, pause and move on to the next application – plan to revisit it after enough time has passed to apply the selection criteria to the application with fresh eyes and consistently with the rest of the batch.
Applicants put time and effort into the application you are reading
Look for strengths, rather than weaknesses. Center equity in your reviews by considering the many ways in which selection criteria can be demonstrated, not just the “traditional.”
Consider your potential implicit biases
Are there types of experiences or applicant profiles you are very familiar with vs. those that are less familiar to you (could be based on academic field, personal identities, socio-economic status, shared experiences and/or a myriad of other factors)? Being aware of your biases can help you to identify where they might impact your evaluation of application.
Strategies and tips for the review process:
Rubrics & Reading
- Review the rubric before reviewing applications.
- Review applications with the rubric side-by-side.
- Review all application materials, looking for evidence of all the selection criteria in all parts of the application.
- Start with the written materials such as personal statements or short answer responses, to understand the applicant’s perspectives and context.
- Review CVs and/or transcripts.
- Using the review rubric, score each criteria, checking back with the application and the evidence you are finding.
Notes & Comments
- Write detailed notes, making them as specific as possible about the evidence you are finding in the application materials.
- Be consistent in what you comment on (e.g. discuss engagement first, then academics, etc.), to more easily track differences in applications and changes in your own reviews. Note anything unique or memorable.
- Writing thorough and specific notes helps you to participate in any committee discussions, advocate for strong applicants, and can support scholarship advisors and applicants in feedback discussions, if feedback is offered.
Breaking up the batch
- Read no more than 5-6 applications in one sitting to reduce fatigue.
- Read applications in a random order or Z-A (vs. the typical A-Z).
- Read each application as if it was the first one. Do not compare content between applications.
One strategy experienced reviewers have shared takes a three-step approach:
- Read the selection criteria and the review rubric carefully. Skim all the applications to get a sense of the overall group.
- Carefully read each application and evaluate, taking detailed notes.
- Re-read applications where needed, to confirm details and to spot check for consistency.
Strategies for evaluating specific application components:
Scholarship Application Essays
Each scholarship program will have its own essay prompts crafted to gather information from students in alignment with the scholarship’s selection criteria and mission. These essays help the reviewer gain an overall understanding of the applicant’s lived experience in the context of the scholarship’s goals.
Being cognizant of common biases that may impact the evaluation process, and relying most heavily on the review rubric provided, here are a few additional factors that generally make for strong essays. Does the applicant:
- Answer all question(s) posed in the prompt?
- Make a connection between the mission of the scholarship (the activity or interest the scholarship seeks to support) and the applicant’s academic, career, personal interests and/or goals?
- Articulate how they plan to achieve those interests/goals and how the scholarship/activity will fit into that plan?
- Describe the decisions they have made, clarifying and reflecting on relevant past experiences with specificity and detail?
CV, Resume or Experience Form
The CV (or resume or application form eliciting similar lists of experiences) is an important part of many scholarship applications. It can offer additional details about applicants’ experience and accomplishments, flesh out timelines, and provide contextual information.
For each activity and accomplishment listed on the CV, consider the following in your review process:
- Has the student provided details about each activity such as:
- Project description
- Duties and responsibilities
- Length of participation
- Hours contributed
- How does the activity relate to
- the student’s stated academic/career goals?
- the scholarship’s requirements or goals?
- other commitments the applicant may have (such as caring for family or working)?
UW resources for students working on CVs:
Letters of Recommendations
Letters of recommendation could be coming from any source: faculty, community members, supervisors, mentors, etc. Stellar recommendation letters always stand out. It is harder to assess those in the middle of the pack. As a reviewer, look for positive, meaningful comments and evaluate the letters according to the review rubric. Clarify, if needed, the instructions given to applicants and to letter writers for the scholarship you are reading, to avoid making assumptions and to avoid the influence of biases.
- Do the letters confirm what the student has said in their application?
- Do the letters provide context, giving specific examples or information about student’s characteristics and qualities?
- Is the letter specific to the student, or could it have been written for anyone?
- Do the letters answer questions raised by the application?
Letters of recommendation can be challenging for a number of reasons. If a letter is problematic, avoid holding that against the applicant. Can you glean any information relevant to the selection criteria from it? Some specific challenging circumstances:
What to do with poorly written letters?
The writers may or may not be strong writers in general or be most comfortable writing in English. They also may or may not have had adequate time to write or proofread. Please do not hold writers’ errors (wrong applicant or scholarship name, grammatical mistakes or typos) against the applicant. Attempt to read the letter for content, in alignment with the review rubric.
What to do with recommendations from the same author for multiple applicants that appear very similar, or actually are the very same letter?
This is no fault of the applicant and is likely more reflective of the harried writer. Attempt to read each letter on its own and consider the merits of each separately, in alignment with the review rubric.
What to do with specifically negative letters?
Unfortunately, from time to time, applicants do make errors in selecting their letter writers and this is born out in recommendation letters that are not “recommendations” at all and provide mostly negative comments.
- Should this impact an otherwise strong application? No, the same questions provided above should be considered in this case.
- Consider if the writer says anything relevant to the application, and whether the letter provides evidence to back up the negative statements. The letter may say more about the author than the applicant.
See Bias Mitigation Ideas & Strategies for Scholarship Administrators/Managers section for additional considerations regarding recommendation letters.
Academic potential can be demonstrated in numerous ways; many research, scholarship and fellowship opportunities are no longer so focused on grades or GPA. However, transcripts are still included in most applications as one way of understanding the student’s academic potential.
In evaluating an applicant’s transcript with an eye towards equity, consider the following:
- Details and contexts shared that have impacted their studies
- Trajectory of the GPA – is the trend or pattern increasing? Consistent?
- Challenging periods – transitional years, changing majors, balancing school and work or family commitments and other examples that can result in quarter(s) where grades are inconsistent with the overall history of the transcript
- Acknowledgement of those challenging periods in other materials (including recommendation letters), with information to help you contextualize those grades
- Challenging course content; independent study/research credits the applicant has listed
- Credits the applicant has taken within the graded period
- Grades in major courses or courses relevant to their future goals
- Other achievements, engagements, commitments
- Avoid sorting the applications by GPA before the review begins.
Note: These guidelines are intended to supplement, not supplant individual scholarship instructions. Rather, they are strategies for reviewers to approach reviews fairly and mitigate common biases during the evaluation.