Selection committee guide

Foreword

This document is a guide for members of the Banting Postdoctoral Fellowships selection committees of the three federal granting agencies: the Canadian Institutes of Health Research (CIHR), the Natural Sciences and Engineering Research Council (NSERC), and the Social Sciences and Humanities Research Council (SSHRC). It describes the selection process for members and chairs of these committees, as well as the policies, guidelines and deliverables that define each stage of the review process.

Members of the Banting Postdoctoral Fellowships selection committees are appointed from the Canadian and international academic communities and are familiar with the mandate, structures and programs of the federal granting agencies. New members and those appointed from other sectors should refer to Selecting the Appropriate Federal Granting Agency for more detailed information regarding federal granting agency mandates.

Although this Selection committee guide strives to be comprehensive, committee members may still have questions after reading it. Members are asked to direct all questions to the program administrator responsible for their selection committee.

Table of contents

  1. Overview of the Banting Postdoctoral Fellowships program
  2. Policies and Considerations Impacting Peer Review
  3. Roles and responsibilities
  4. Review process
  5. Selection committee membership

1. Overview of the Banting Postdoctoral Fellowships program

Announced in the 2010 Federal Budget and officially launched on July 6, 2010, the Banting Postdoctoral Fellowships (PDF) program seeks to attract and retain top-tier postdoctoral talent, both nationally and internationally, to develop their leadership potential and to position them for success as research leaders of tomorrow, positively contributing to Canada's economic, social and research-based growth through leadership in a research-intensive career. Canadian and international candidates are eligible to apply for a Banting Postdoctoral Fellowship. Each Banting Postdoctoral Fellowship is valued at $70,000 (taxable) per annum for two years.

Prospective candidates wishing to apply for a Banting Postdoctoral Fellowship must do so in close collaboration with the host institution at which they seek to conduct their postdoctoral training. In light of the small number of awards in the program, and the demands of both the application process and the review process, institutions have been asked to be highly selective in terms of the applicants they endorse, as Banting Postdoctoral Fellowships will ultimately be awarded to a prestigious group of researchers who have demonstrated both research excellence and leadership, and synergy with the institution's strategic priorities. Banting Postdoctoral Fellowships awards must be taken up and held at the host institution that provided the initial endorsement.

The 70 Banting Postdoctoral Fellowships awarded each year are equally divided among the three federal granting agencies, with the 70th award rotating annually.

The Vanier-Banting Secretariat, which is housed at CIHR, is responsible for the day-to-day administration of the program.

2. Policies and Considerations Impacting Peer Review

The selection committee members are asked to keep in mind the following policies and guidelines when reviewing applications.

2.1 Confidentiality of application material

When logged onto ResearchNet, users will be asked to read and agree to the Conflict of Interest and Confidentiality Agreement for Review Committee Members, External Reviewers, and Observers, which describes the expectations and requirements of the tri-agencies.

Application material is provided to committee members in confidence and should be used for review purposes only and kept in a secure location. For example, committee members should avoid saving confidential documents in shared networks and/or on cloud services. In addition, material that committee members no longer require should be destroyed in a secure manner (i.e., shredded). All binders, files, personal notes, assessments, etc. must be left with Vanier-Banting Secretariat staff at the conclusion of the meeting. If the Secretariat requires committee members' assistance with providing additional information on a particular case after the competition, committee members will be provided with new copies of the relevant material.

2.2 Confidentiality of recommendations

All funding recommendations are subject to approval by the TAP Steering Committee and may be changed for reasons related to budget, administrative error, or lack of full compliance with program and agency-specific policies.

All matters discussed during selection committee meetings or teleconferences are confidential. Notifying applicants of the results of committee deliberations is the responsibility of the Vanier-Banting Secretariat, following the official approval by the TAP Steering Committee.

Applications are confidential and must not be discussed with or divulged to others. Any release of information to an applicant must be done through the Secretariat.

Results must not be disclosed by committee members prior to the Secretariat's official release. If approached by an applicant and/or university representatives concerning a decision or any other matter, committee members should decline discussion and refer the individual to the Secretariat. Staff will act as the liaison between the Banting Postdoctoral Fellowships selection committee and the applicant.

2.3 Privacy Act and Access to Information Act

Canada's Privacy Act stipulates that personal information provided by applicants must be used only for the purpose of assessing applications and making funding decisions. The use or disclosure of such information for any other purpose is illegal.

The information gathered for this purpose must be collected directly from the individual. It may be collected from other sources only as part of the formal review process. For this reason, Banting Postdoctoral Fellowships selection committee members must not use or consider information about an applicant that has been obtained independently.

In light of the large number of applications to the program, selection committee members are not required to provide written assessments of individual applications.

As per the Access to Information Act, applicants have a right to access information that is held by the Secretariat about their application/themselves. A written opinion of a reference about a application (for example, Referee assessment) is available to the applicant. However, they do not have the right to access the names of those who reviewed their application.

Selection committee membership will be published on this website approximately 60 days after funding decisions have been announced.

It is important that committee members adhere strictly to the guidelines set out in the Conflict of Interest and Confidentiality Policy of the Federal Research Funding Organizations.

2.4 Canadian Human Rights Act

The activities of federal granting agencies are subject to the Canadian Human Rights Act. The purpose of the Act is to give effect to the principle that all individuals should have an opportunity equal with other individuals to make for themselves the lives that they are able and wish to have, consistent with their duties and obligations as members of society, without being hindered in or prevented from doing so by discriminatory practices based on race, national or ethnic origin, colour, religion, age, sex, sexual orientation, gender identity or expression, marital status, family status, genetic characteristics, disability and conviction for an offence for which a pardon has been granted or in respect of which a record suspension has been ordered. Where the ground of discrimination is pregnancy or childbirth, the discrimination shall be deemed to be on the ground of sex.

2.5 Gender-neutral and gender-inclusive language in reviews

Reviewers are encouraged to adopt gender-neutral and gender-inclusive language in their reviews. In reviews, both written and spoken, gender-neutral and gender-inclusive language is more accurate and more respectful when discussing the science and the applicant.

When discussing the science remember that gender is non-binary. When research is meant to include all people, avoid binary statements like "men and women." Instead, consider phrases like "men, women, and gender-diverse people" or "people of all genders." In addition, be mindful of word choice. Below are a few example of words that could be replaced with more gender-neutral and gender-inclusive terms:

When referring to the applicant use gender neutral pronouns or phrases. For example, use "they" or "the applicant," rather than "he" or "she". Remain mindful of word choice, as some words could be replaced with more gender-neutral and gender-inclusive terms:

2.6 Official languages

The federal granting agencies, like all other federal institutions, play a key role in implementing the Official Languages Act. The federal granting agencies have an obligation to ensure that:

The federal granting agencies must ensure that their committees/panels and their staff are fully aware of their obligations and rights regarding official languages by providing documentation on official languages to employees and selection committee members and by including relevant guidelines in the instructions to selection committees.

In accordance with an active offer of bilingual service to the public, the Vanier-Banting Secretariat will try to appoint as many experts as possible with the appropriate language capabilities to serve on selection committees.

Selection committees must ensure that all applications receive a full evaluation (subject to the guidelines regarding level of effort for lower quality applications as noted above), regardless of the official language of presentation. On occasion, this may entail consultation with Secretariat staff to identify Banting Postdoctoral Fellowships selection committee members with adequate linguistic capability.

2.7 San Francisco Declaration on Research Assessment

The agencies are a signatory of the San Francisco Declaration on Research Assessment (DORA), which "recognizes the need to improve the ways in which the outputs of scholarly research are evaluated". DORA recognizes that scholarly outputs include a broad range of contributions and impacts and are not limited to published journal articles.

Reviewers are to assess productivity and progress broadly by taking into consideration:

Metrics such as number of publications and citations, and size/number of research grants should not be used in isolation to assess productivity. Reviewers should not use journal-based metrics (e.g., Journal Impact Factors) as surrogate measures of quality and impact of individual research publications. As stated in DORA, the "scientific content of a paper is much more important than publication metrics or the identity of the journal in which it was published".

2.8 Ethical considerations

Committee members play an important role in alerting staff to concerns they may have with respect to the ethics of the research being proposed. Examples of problems include the following:

If a member has any concerns with respect to ethical matters, these should be discussed immediately with Vanier-Banting Secretariat staff to determine if there is a means of resolving any apparent problems quickly or if the award of a fellowship should be delayed pending the resolution of the problem. While such concerns are being addressed, the review of applications should continue without being influenced by this issue.

2.9 Responsible conduct of research

The three federal granting agencies have defined their expectations with respect to scientific integrity in the Tri-Agency Framework: Responsible Conduct of Research.

Research

Canada’s federal granting agencies—the Canadian Institutes of Health Research, the Natural Sciences and Engineering Research Council of Canada, and the Social Sciences and Humanities Research Council of Canada—are committed to fostering and maintaining an environment that supports and promotes the responsible conduct of research. The Tri-Agency Framework: Responsible Conduct of Research sets out the responsibilities and corresponding policies for researchers, institutions, and the agencies that together help support and promote a positive research environment.

The federal granting agencies expect the highest standard of integrity in the research they fund. The electronic submission of an application to the Secretariat commits the applicant to a number of principles, including compliance with the Tri-agency Framework: Responsible Conduct of Research.

Should selection committee members identify what appears to be a lack of integrity, they should bring it to the attention of Secretariat staff at the earliest opportunity. The Secretariat will refer any allegations to the Secretariat for the Responsible Conduct of Research for follow-up. Such allegations should not be a consideration during the review process, nor should they be part of the committee's evaluation discussions, unless they impact the scientific quality of the application.

A selection committee member may, during the review process, encounter possible misconduct situations (e.g., apparent misrepresentation of publications and/or data, plagiarism and other problems such as a lack of appropriate control/monitoring within the institution itself or undue restriction on the dissemination of research supported by federal funds). Selection committee members should alert Secretariat staff of these situations at the earliest opportunity.

Selection and review process

The Secretariat expects the highest standards of integrity in the review process that it manages. The Tri-Agency Framework: Responsible Conduct of Research indicates that information provided by candidates for the purpose of selection and review cannot be used by reviewers without the author's permission. In addition, a reviewer should reveal to the federal granting agencies any material conflict of interest, financial or other, that might influence the Secretariat's decision on whether the reviewer should review nominations. Selection committee members are responsible for respecting the confidentiality of nomination material and for declaring material conflicts of interest. Should selection committee members become aware of a situation that violates the integrity of the review process, they should discuss this immediately with Secretariat staff.

2.10. Open Access Policy and Research Data Management Policy

The federal granting agencies believes research results produced using public funds belong, to the fullest extent possible, in the public domain. Grant holders must comply with the tri-agency Open Access Policy on Publications, by ensuring that any peer-reviewed journal publications arising from the tri-agency funded research are freely accessible within 12 months of publication. Additionally, in accordance with the tri-agency Research Data Management Policy, the tri-agency expects grant holders to responsibly and securely manage their research data and, where ethical, legal and commercial obligations allow, make it available for reuse by others.

3. Roles and responsibilities

3.1 Selection committee

3.1.1 Chair

Selection committee chairs play an important oversight role on the committee. Each committee has one chair. In the role of moderator, it is the Chair's responsibility, in collaboration with the VBS staff, to ensure that the review committee functions smoothly, effectively and objectively, and that a positive, constructive, fair-minded environment in which applications are evaluated is established and maintained.

The chair will:

The chair is not responsible for reviewing applications.

Note: The Secretariat asks that selection committee members not make formal recommendations for a future chair (i.e., no formal letters of recommendation are required). Instead, informal suggestions about potential chairs should be made to the Banting PDF program administrator.

3.1.2 Evaluators

All evaluators are responsible for:

Each evaluator can be assigned one or more of the four roles listed below.

3.1.2.1 Reviewers

A primary or secondary reviewer is a committee member who:

Primary and secondary reviewers are responsible for:

3.1.2.2 Third readers

A third reader is a member who:

Third readers are responsible for:

3.1.2.3 Guest readers

A guest reader is a selection committee member who:

Guest readers are responsible for:

For more information, see 4.1.5 (b) Identifying applications for review by guest readers.

3.1.2.4 Guest experts

A guest expert is a selection committee member who:

Guest experts are responsible for:

For more information, see 4.1.5 (c) Identifying applications for guest experts.

3.1.2.5 Equity, Diversity, and Inclusion (EDI) Champions

EDI champions are selection committee members who:

EDI champions are responsible for:

EDI is a collective responsibility of the committee as a whole and should be encouraged as a best practice for all reviewers

3.2 Program administrator

The main responsibilities of the Banting PDF program administrator during the competition include:

The program administrator is not a committee member and does not have voting rights on the committee.

3.3 Executive Director, Deputy Director, VBS Lead and Team Lead

The main responsibilities of the Banting PDF Executive Director, Deputy Director, VBS Lead and Team Lead include:

Communication of secure documents

Throughout the competition cycle, Vanier-Banting Secretariat staff may need to provide selection committee members with documents that contain sensitive information. In most cases, these documents will be shared by posting them on the appropriate pages within ResearchNet, the electronic web portal used for submitting reviews.

4. Review process

The review process consists of the following ten phases, which are further described below:

Prior to the selection committee meeting

During the selection committee meeting

After the selection committee meeting

4.1 Prior to the selection committee meeting

4.1.1 Receiving and assigning applications

Applications are received by the Vanier-Banting Secretariat through ResearchNet, and the program administrator assigns them to committee members for review.

Each agency-specific selection committee is multidisciplinary. Committee members are therefore asked to review and score applications in a number of different research areas related to the federal granting agency's mandate, including applications in areas that may not correspond to the members' exact areas of expertise. From a non-specialist's perspective, reviewers are asked to assess the intellectual challenge of the research in which the candidate will be involved. Although reviewers may not be familiar with the field, we ask that each reviewer complete their reviews from the point of view of a generalist. Each application is evaluated by two reviewers: one serves as the primary reviewer, and the other serves as the secondary reviewer.

The program administrator assigns a subset of applications to each reviewer using ResearchNet, the electronic web portal used for submitting reviews. In doing so, the program administrator seeks to balance workload among committee members while also taking into consideration members' language abilities and areas of expertise, as well as potential conflicts of interest between members and applications. Each committee member is responsible for reviewing their assigned applications.

As soon as the applications are assigned, each committee member is granted access to ResearchNet to review their Banting Postdoctoral Fellowships assignments. Members will receive an email notification advising them that their assignments are available.

4.1.2 Identifying conflicts of interest

Selection committee members identify additional conflicts of interest with those applications to which they were assigned.

Members are asked to use ResearchNet to identify potential conflicts with reviewing the applications assigned to them. A list of conflicts is provided below. Since this list is not exhaustive, reviewers should consult the Confidentiality and Conflict of Interest Guidelines that appear as the first mandatory task in ResearchNet. Should a member feel, for any reason, that they are unable to review an application that they initially accepted as an assignment, they should contact the program administrator immediately, and the application will be re-assigned to another reviewer as appropriate.

Are you in conflict with the applicant?

The following guidelines governing conflicts of interest apply to the evaluation of Banting Postdoctoral Fellowships applications. Guidelines of this nature cannot foresee all possible situations, and the federal granting agencies rely on the judgment of committee members in identifying conflicts. Since both the applicant and the research environment are evaluated, a committee member may not act as a reviewer for an application if:

When a committee member is in conflict with a particular application, they will be asked to leave the room during the selection committee meeting before the deliberations on that application begin.

It is the responsibility of committee members to declare any conflicts of interest prior to the review of an application. In cases where the guidelines do not clearly describe a situation, or where the committee or chair has difficulties making a decision about a particular situation, Vanier-Banting Secretariat staff has the responsibility to rule.

4.1.3 Completing the Equity, Diversity and Inclusion Self-identification Questionnaire

Selection committee members must complete the Equity, Diversity and Inclusion Self-identification Questionnaire in ResearchNet before accessing their assigned applications. For more information about the Equity, Diversity and Inclusion Self-identification Questionnaire, please refer to the Frequently Asked Questions and How-To Instructions on the CIHR website.

4.1.4 Pre-scoring applications (as reviewers)

Once conflicts of interest have been identified and the Equity, Diversity and Inclusion Self-identification Questionnaire has been completed, committee members may begin reviewing and pre-scoring their assigned applications. Reviewers use ResearchNet to pre-score all of their assigned applications and to submit their pre-scores electronically to the Vanier-Banting Secretariat prior to the selection committee meeting. A specific submission deadline will be communicated to committee members in due course.

Note: The review of applications must be based solely on the information provided in the applications. Members are asked not to obtain extra information (such as seeking updates to the publication status of research papers or visiting external websites linked to within the free-form sections) for the applications they are reviewing

a) Introduction

The review process is, by nature, subjective. Bias may manifest itself in any number of ways and could be based on one or more of the following factors, among others:

Reviewers are cautioned against using inappropriate judgment of applications based on such factors and are asked to constantly guard against the possibility of explicit or implicit bias influencing the decision-making process.

To avoid implicit bias related to the alphabetical order of the applications in ResearchNet, reviewers are invited to randomize the applications by application number by clicking on the application number column.

Can we improve equity and reduce bias in the review process?

The agencies are actively engaged in promoting equity and diversity, as well as in decreasing unconscious bias in their review processes. To this end, the Secretariat has curated a list of resources for selection committee members regarding equity, diversity and inclusion considerations. All members must review the material, especially the tri-agency unconscious bias training module.

Considerations for reviewing applications in which the proposed research respectfully involves and engages Indigenous communities

Applicants whose proposed research respectfully involves and engages Indigenous communities are asked to include "This research respectfully involves and engages Indigenous communities" at the beginning of their lay abstract.The program administrator for each selection committee (CIHR, NSERC, SSHRC) will endeavour to ensure that the primary or secondary reviewer on these applications has expertise in Indigenous research.

Primary and secondary reviewers are asked to identify any application whose research respectfully involves and engages Indigenous communities and that are not identified as such in the lay abstract. This will ensure that applications whose research respectfully involves Indigenous communities receive due attention.

All committee members assigned to these applications (primary reviewer, secondary reviewer and, if applicable, third reader, guest reader and/or guest expert) are expected to consult and take into consideration the guidelines outlined for reviewing applications respectfully involving Indigenous research. These guidelines, along with other important considerations, are available through the Equity, Diversity and Inclusion page.

Specifically the following two documents will provide guidance on the review of these applications:

The research proposal should reflect that the applicant and their supervisor/host institution are aware of and referring to relevant principles and protocols established for this kind of research. There should be evidence of engagement with the community in a meaningful and culturally safe manner.

Is the application eligible for support?

Vanier-Banting Secretariat staff is responsible for screening all applications against eligibility requirements to ensure that the candidate is eligible to apply to the program. Committee members are invited to bring any concerns or questions regarding the eligibility of an application to the attention of Secretariat staff but should proceed with evaluating the application while the situation is being assessed.

Is the proposed research contrary to the Tri-Agency Framework: Responsible Conduct of Research?

If a committee member suspects that an applicant's proposed research is contrary to the Tri-Agency Framework: Responsible Conduct of Research, or if the member has any ethical concerns with respect to the application, they should inform the program administrator of their concerns as soon as possible and proceed with evaluating the application while the situation is being assessed.

b) Selection criteria

Applications are evaluated and selected based on the following three criteria, weighted equally:

  1. Applicant's research excellence and leadership in the research domain
  2. Quality of the applicant's proposed research program
  3. Institutional commitment and demonstrated synergy between applicant and institutional strategic priorities

Note: The Application guide summarizes the steps applicants should follow to complete and submit their application, and includes information on how each component of the application relates to the selection criteria.

Members should base their reviews only on the information included in the application.

Criterion Indicators Sources

Applicant's research excellence and leadership in the research domain

Demonstrated capacity for research excellence based on track record as defined by the quality of the applicant's research contributions, and demonstrated capacity for leadership in the research domain as defined by the sphere of influence achieved to date by the applicant.

 Research history
  • Applicant's CCV

Impact of their activities in their research community

  • research programs
  • institutions (e.g., research, cultural)
  • research communities (e.g., local, international)
  • society at large (e.g., regional, national)
  • Clarity with which the applicant writes their proposal to a multi-disciplinary committee (i.e., non-specialist audience)
  • Description of the significance of up to three research contributions
  • Description of the significance of up to three leadership contributions
  • Three referee assessments (Applicant's research excellence and demonstrated leadership section)
  • Supervisor's statement; discussion of the applicant's research background and contributions
The stage and nature of the applicant's career path.
  • Special circumstances (if applicable)
    • Justification of career or research delays
    • Justification for remaining in PhD research environment
    • Justification for remaining in the same research environment (other than PhD)

Notes:

The agencies have signed the San Francisco Declaration on Research Assessment (DORA), which recognizes that research contributions are not limited to published journal articles but can include a broader range of contributions (e.g., research publications, reports, books, guidelines, datasets, code, tools, standards, software and commercialized products, article preprints, protocols, knowledge mobilization activities) and impacts (e.g., influence on policy and practice, societal outcomes, distinctions-based, meaningful and culturally safe research). In alignment with DORA, reviewers should:

  1. Assess excellence and productivity broadly (i.e., not just based on publications). Consider individual workstyles, contributions, commitments, variations in disciplines, and community and cultural standards. Collaboration, teamwork and mentoring are important and valid contributions to research and to training highly qualified personnel.

    • The gender of the applicant should not have an impact on how these contributions are valued.
    • Similar expectations apply to single-authored and multi-authored publications.
    • The applicant's track record should be reviewed in the context of research/leadership opportunities available to the applicant. Research/leadership opportunities refer to how an applicant's productivity and contributions correspond to the opportunities that have been available to them.
    • If applicable, consideration of the merit of non-academic contributions for research respectfully involving Indigenous Peoples must be taken into consideration.

  2. Assess excellence and productivity commensurate with the applicant’s context and personal circumstances (e.g., career stage, parental leave, child rearing, illness, disability, cultural, community or family responsibilities, socio-economic factors, access to research/leadership opportunities or relocation of research environment) that may have led to delays in research or in results dissemination. These personal circumstances must be considered on a case-by-case basis.

    • In the case of health professionals, standards of research productivity should be considered in relation to the applicant's level of experience and qualifications, and comparisons with applicants who have obtained a PhD should reflect these differences in standards. For applicants who have relevant work experience, scientific productivity prior to graduate school should be considered.

  3. Guard against placing too much value on the number of contributions; focus must be on the impact and significance of the contributions. Reviewers should not use journal-based metrics such as the Journal Impact Factor as surrogate measures of quality and/or excellence as they introduce bias into the review process. Citation rates vary between disciplines and contexts; members must be mindful of this when considering them as part of their evaluation. As stated in DORA, the "scientific content of a paper is much more important than publication metrics or the identity of the journal in which it was published".
Criterion Indicators Sources

Quality of applicant's proposed research program

Potential of the proposed research program, executed in the proposed institutional environment, to position the applicant to have significant impact through a research-intensive career.

The quality of the proposal in terms of its novelty/originality, feasibility and significance
  • Research proposal (including SGBA+ considerations)
  • Lay abstract of research proposal
  • Bibliography and citations included in the research proposal
  • Supervisor's statement
  • Three referee assessments (Merit of proposed research section)
The environment(s) in which the proposed research will be conducted
  • Research proposal
  • Supervisor's statement (discussion of research environment and support)
Research respectfully involving Indigenous Peoples (if applicable) – community engagement
  • Research proposal
  • Supervisor's statement (discussion of engagement, reciprocal benefit, institutional support etc.). Supervisor must corroborate details provided in the research proposal

Notes:

  1. Sex- and Gender Based Analysis Plus (SGBA+): In assessing the quality of the applicant's research proposal, consideration of sex, gender and diversity in the research design must be considered, if applicable:

    • rationale and methodology for including sex, gender and diversity in the research (from its design to the analysis of research findings) are clearly described;
    • should be evaluated in terms of scientific rigour and usefulness of the proposed research.

  2. Applications involving Indigenous communities must be assessed by its academic merit as usual, with additional review on the Indigenous research component. In order to be funded, these applications must fulfill the pillars for respectful research engaging with Indigenous Peoples. See the Equity, Diversity and Inclusion page for more information.
Criterion Indicators Sources

Institutional commitment and demonstrated synergy between applicant and institutional strategic priorities

Institutional commitment to champion the development of the applicant's research and leadership

Professional development opportunities offered by the institution
  • Supervisor's statement
    • provide evidence that the institution and supervisor are well-positioned to provide the required support to the applicant in relation to the proposed research
    • describe their academic and research background, and outline key contributions and accomplishments to date
    • describe the fit between the research interests and background of the supervisor and applicant, and the anticipated mutual benefits of working together
Institution's research capacity in the area of the applicant's proposed research program, providing an intellectually stimulating environment to position the applicant as a research leader
  • Supervisor's statement
    • clearly state the host institution's commitment in terms of the personnel, funding, facilities and/or other resources that will be available to support the applicant),
    • describe any collaborations with other communities (e.g, research, cultural) involved in the proposed research;

Alignment and synergy between the applicant's research ambitions and the institution's strategic priorities

Institutional potential to benefit strategically from engaging with the applicant.
  • Three referee assessments (Suitability of the research environment section)

Notes:

The evaluation on this criterion should not be based on an assessment of the institution per se but on the institution's:

  • commitment to the applicant;
  • capacity to enable the applicant to become a future leader in their chosen field;
  • potential to build upon its own strategic priorities through engagement with the applicant; and
  • awareness and support of any collaborations involved in the proposed research (e.g., research respectfully involving and engaging Indigenous communities).
c) Pre-scoring

Members are expected to give a pre-score (i.e., a score that is given before the selection committee meeting and that may or not coincide with the final score) between 0.1 and 9.0 (in increments of 0.1, with 9.0 being strong and 0.1 being weak) to their assigned applications for each of the three selection criteria. ResearchNet will automatically calculate the overall pre-score of the application by averaging the primary and secondary reviewers' pre-scores for each criterion.

In order to encourage reviewers to differentiate between highly promising applicants, and in order to ensure that the full range of the scale is used, a forced binning system has been implemented. Reviewers must ensure that their assigned applications fall within each of the three bins identified in the table below, distributing the applications according to the proportions indicated, and allocating the remaining 20% as they deem appropriate to compensate for a relatively strong or a relatively weak subset of applications:

Funding recommendation Score Proportion
Recommended 7.5 – 9.0 10%
Could be recommended 5.1 – 7.4 50%
Not recommended 0.1 – 5.0 20%

In addition, within the top bin, members must assign a minimum of 1 and a maximum of 2 applications between 8.0 and 9.0.

Reviewers are strongly encouraged to review a number of applications before entering pre-scores, as this will provide a meaningful frame of reference for pre-scoring all of their assigned applications. Members will be provided with an electronic worksheet, which can facilitate the task of pre-scoring and binning.

In order for this system to work effectively, it is essential that the entire range within a bin be used. Therefore, reviewers should make every effort to distinguish between applications within a given bin in order to avoid ties.

Reviewer pre-scores must be submitted electronically to the Vanier-Banting Secretariat via ResearchNet prior to the selection committee meeting. The exact submission deadline will be confirmed by the program administrator. It is critical that this deadline be respected, as Secretariat staff cannot determine which applications will be reviewed at the selection committee meeting or which applications will require a third reader until all of these pre-scores have been received. Members are encouraged to contact Secretariat staff at any time if they need assistance during the pre-scoring process.

Reminder: Committee members who have been assigned applications in which the proposed research respectfully involves Indigenous communities are required to consult and take into consideration the guidelines outlined for reviewing applications involving this type of research. These guidelines, along with other important considerations, are available through the Equity, Diversity and Inclusion page. Members are expected to read the material before they begin the pre-scoring process.

4.1.5 Conducting preliminary ranking of applications and assigning third readers, guest readers, guest experts

Once all of the primary and secondary reviewers' pre-scores have been received, the program administrator calculates the average of the two reviewers' overall pre-scores and ranks the applications from strongest to weakest.

The program administrator identifies applications where there is a discrepancy of 2.0 or more points between the primary and secondary reviewers' pre-scores and where one of those pre-scores is higher than the overall score of the 24th-ranked application. (The exact number of "discrepant" applications will depend on the number of discrepancies.)

The program administrator then provides the selection committee with a preliminary ranked list of all the applications, identifying those that will be discussed at the meeting, namely:

This preliminary list will not include the pre-scores assigned to the applications.

Upon receiving this list, committee members are asked to promptly identify any of the following:

a) Identifying additional applications to be discussed at the meeting and reviewed by third readers

Identifying additional applications is not necessary and, in any case, should be done judiciously in order to ensure that only the strongest applications and the most contentious are discussed. (Note: A given application may be flagged only by its primary or secondary reviewer.)

The program administrator will then assign a third reader to review the following set of applications:

b) Identifying applications for review by guest readers

On occasion, a research proposal spans the mandate boundaries of more than one selection committee (CIHR, NSERC, SSHRC). If expertise from another committee would be beneficial, an application can be flagged by the primary and/or secondary reviewer for evaluation by a guest reader serving on one of the other selection committees.

The program administrator will assign a guest reader to review these flagged applications.

c) Identifying applications for guest experts

In the event that a selection committee requires additional expertise in Indigenous research, the program administrator assigned to that committee will assign a guest expert to any application whose research respectfully involves Indigenous communities.

d) Declaring conflicts of interest

Third readers, guest readers and guest experts must contact the Secretariat's program administrator to identify any conflicts of interest with the applications assigned to them.

4.1.6 Pre-scoring applications (as third readers)

The third reader is responsible for reviewing and pre-scoring their assigned applications. This evaluation should be based on the same three criteria, the same scoring system (i.e., minding the scale; comparing applications against those they evaluated as a primary or secondary reviewer), and the same guidelines outlined above for the primary and secondary reviewers. Third readers should, however, pay particular attention to the third selection criterion: Institutional commitment and demonstrated synergy between applicant and institutional strategic priorities.

Third readers must contact the Secretariat's program administrator to identify any conflicts of interest with the applications assigned to them.

Third readers are to submit their pre-scores electronically to the program administrator, respecting the deadline provided. (The exact deadline will be confirmed by the program administrator.)

Pre-scores provided by the readers are not calculated into the overall scores of the applications. Rather, these pre-scores represent additional feedback to help inform discussion during the meeting. Readers are expected to be ready during committee deliberations to provide comments on the application's strengths and weaknesses.

4.1.7 Providing updated pre-score ranked list of applications

Prior to the selection committee meeting, the Secretariat's program administrator prepares an updated pre-score ranked list of applications—including pre-scores by the primary and secondary reviewers and, where applicable, by the third reader—and makes the list available to the selection committee members. This list of applications that will be discussed at the selection committee meeting will consist of:

No other applications are expected to be discussed at the meeting.

4.2 During the selection committee meeting

4.2.1 Meeting

Note: The committee is encouraged to use gender-neutral and gender-inclusive language when presenting reviews and engaging in discussions.

Note: Committee members are required to bring to the meeting their personal notes on applications assigned to them. All notes must be securely destroyed at the end of the meeting.

Members convene for the selection committee meeting, the dates of which are provided by the program administrator.

At the meeting, the following applications are reviewed one at a time, in rank order from strongest to weakest:

The total number of applications to be discussed depends on the number of discrepancies and the number of additional applications flagged by reviewers.

The procedure for reviewing an application during the meeting consists of the following steps:

  1. The primary reviewer introduces the application briefly to describe the research topic, career stage/path and any special circumstances indicated in the application. The reviewer will then verbally summarize the strengths and weaknesses of the application by addressing each criterion in turn and speaking to points raised in the three referee assessments. This summary should take approximately three minutes.
  2. The secondary reviewer provides comments, as needed, to highlight agreement or disagreement with the primary reviewer's analysis. These comments should take approximately two minutes.
  3. The third reader, if applicable, then adds any other information that they feel is required by the committee in order to assess the application, focusing especially on the third selection criterion (Institutional commitment and demonstrated synergy between applicant and institutional strategic priorities). These additional details should take approximately one minute.
  4. The guest reader, if applicable, then adds any other information that they feel is required by the committee in order to assess the application. These additional details should take approximately two minutes.
  5. The guest expert, if applicable, then adds any other information that they feel is required by the committee in order to assess the application, focusing on the Indigenous components of the application.
  6. If applicable and otherwise omitted from discussion, EDI champions raise relevant equity, diversity, and inclusion considerations for the assessment of the file (such as a candidate's research trajectory, opportunities or lack thereof and/or obstacles; personal circumstances that may have affected the record of research achievement or productivity of a candidate; the diversity in perspective and lived experiences; how Sex and Gender Based Analysis Plus (SGBA+) is considered in the research’s design, methods, analysis and interpretation, and/or dissemination of findings).
  7. Other committee members may make comments or raise questions in order to clarify information presented by the reviewers, readers and/or guest experts, which may lead to committee discussion. Members must ensure that career stage/path of the applicants and any special circumstances (e.g. unusual types of research contributions, research/leadership opportunities that were available to the applicant etc.) have been considered in the assessment of their track record.
  8. The committee comes to a consensus on the application's final score, taking into consideration the feedback provided by the primary and secondary reviewers as well as by any readers and/or guest experts. The selection committee is expected to decide on a consensus score for each of the three selection criteria, and these scores are then averaged to arrive at a final score for the application. Note: For an application to be considered eligible for funding, it must attain an average score of at least 5.1 in each of the three criteria.
  9. Once all applications have been discussed and scored, the ranking of the applications is reviewed to determine a quality cut-off line for funding (below which applications are deemed to be not strong enough to receive funding in the eventuality that funds become available) and to resolve any ties in scores between the 23rd-ranked application—or, in the case of the selection committee offering the 70th fellowship, the 24th-ranked application—and applications at the quality cut-off line.

Following the competition, the committee's final rating on each criterion for an individual application is made available to the applicant. This constitutes feedback to the applicant on the relative strengths and weaknesses of their application and on the application's relative ranking. Committee members do not submit written comments on individual applications.

The final ranked list of all applications is submitted to the TAP Steering Committee for approval.

Any comments related to the selection process should be brought to the attention of the committee's program administrator.

4.2.2 Establishing funding recommendations

Each agency-specific selection committee finalizes its ranked list and establishes a quality cut-off line for funding.

The final ranked list of all applications identified by each agency-specific committee, including the quality cut-off line for potentially funding alternate applications, is submitted to the TAP Steering Committee for approval.

In the event that Vanier-Banting Secretariat staff offer an award to a next-in-line applicant (after another applicant has declined their offer of award), staff will not go below the quality cut-off lines established by the selection committees.

Approval from the TAP Steering Committee results in funding 70 Banting Postdoctoral Fellowship recipients.

4.2.3 Discussing policy

At the end of the selection committee meeting, once the committee has made its recommendations, committee members are asked to participate in a policy discussion. The discussion generally includes commenting on the quality of the applications reviewed, suggesting improvements to the review process and program policies, identifying future membership needs, addressing competition and travel logistics (including hotel accommodations), and providing feedback on program administration by the Vanier-Banting Secretariat.

4.3 After the selection committee meeting

4.3.1 Preparing and submitting the chair's report

Following the meeting and in consultation with selection committee members and Vanier-Banting Secretariat staff, the chair prepares and submits their report.

This report serves to communicate the committee's comments and recommendations for improvements to the competition processes and program policies.

In addition to the final ranked list of all applications, the report should include relevant comments on such matters as:

The report is presented to the TAP Steering Committee.

5. Selection committee membership

5.1 Recruitment process

The federal granting agencies regularly solicit nominations for their selection committees from universities, industry and/or the government sector. Nominations are also put forward by current and previous committee members and by Vanier-Banting Secretariat staff.

The slate of nominees for each committee is prepared by Secretariat staff and is subject to final approval by Vanier-Banting Secretariat management. Chosen in a similar way, the committee chair has usually already served on a committee for one or more competitions.

5.2 Committee member selection criteria

The most important criterion governing committee membership is academic and research excellence. The value of the review process rests on the credibility of committee members and their recognized expertise and productivity in their fields. Although each committee is representative of the community it serves, members themselves are not expected to act as representatives of any particular group, institution, region or country. Committees are structured to ensure:

5.3 Term of membership

Membership rotation and renewal are both essential to ensuring the vitality of the selection committee process. Membership rotation is critical because it allows for broader representation of institutions and complementary expertise. However, the continuity of membership renewal is also desirable, since the presence of experienced members on a committee promotes consistency in the selection process and assists in the orientation of new members.

Selection committee members usually serve no more than three consecutive years. To maintain stability in membership, while providing a mechanism for membership renewal, a rotational system has been established for one third of the membership on a yearly basis. The extension of a term beyond three years for one additional year may be accepted in exceptional circumstances. Members who have completed their three-year term on the committee may be appointed as chair.

These membership terms also address the benefits of renewing the membership so that new perspectives are continually incorporated into the peer review process.

Date modified: