Processes and Policies

What is the purpose of reporting?

As part of our three-year LISA 2020 Network grant agreement with USAID, we provide certain data about the labs and USAID-funded projects. We formally report to USAID every 6 months, and ask labs to report data to us every 3 months in order to stay on track.

How were these metrics determined?

At the beginning of the grant period, we chose some outcomes that were suggested by USAID, and created some other custom outcomes, and these are the columns of information that you see in your spreadsheets. We also created some target numbers for these outcomes over the project period of three years, which we can revise from time to time as we see how the labs are performing—the goal of this being to objectively evaluate whether the LISA 2020 grant is meeting with success.

What are the metrics the LISA 2020 Network reports to USAID?

• Number of publications (peer-reviewed and non-peer reviewed)

  • Also title, date, type of publication, authors, and gender of authors

  • USAID is interested specifically in publications that results from projects directly funded by USAID1, but please record all lab project-related publications in this spreadsheet; we will use the additional information about the publications to determine whether to report them to USAID

  • I have recently added a new column asking for confirmation about whether a publication is lab-project related or not; lab-project related means work done with a domain expert/lab client, and an example of something that is not project related is a more theoretical publication that you undertake as an academic staff member

• Number of projects with individual domain experts (researchers, businesses, policy actors)

  • USAID is interested specifically in projects that are funded directly by USAID, but please record all projects in this spreadsheet (we will determine which to report to USAID based on our administrative records)

• Number of policy recommendation meetings

• Number of stat lab trainees of different types

• Number of educational events provided by the lab (short courses, 1-day workshops, multi-day workshops) and number of attendees of different types

• Number of labs in the LISA 2020 Network2

• Number of program and policy changes influenced by USAID-funded projects2

1 Currently in Nigeria, Ghana, and India, and will include TEACH fund projects once these are underway

2 Not recorded in spreadsheets

What should we be especially aware of when recording data?

1. Everything reported in the “Data” tab (main spreadsheet) should be a number. If anything requires an additional written explanation, please send a separate document, or document it in the email when you return the spreadsheet. The symbol “#” is meant to indicate that a number goes in the box, although it turns out that this is primarily a North American use of this symbol.

2. The second tab in the spreadsheet contains explanations of all metrics you are meant to record. Please review this if you are not sure what should go under any heading.

3. Areas that require additional information in order for us to count them are publications and policy recommendation convenings. For us to count these metrics, additional information must completed in the spreadsheet on another tab (either the “Publications” tab or the “Policy Recommendations Convenings” tab).

4. What is a policy recommendation convening? This is a meeting with policy actors who have the power to implement a public policy change. This meeting must involve a recommendation for public policy, based on data and analysis that you have already completed. It is NOT any meeting with a client/domain expert or policy actor.

What if our lab has achievements that are not covered by these metrics?

Many labs have operations and objectives that are not covered by the metrics we report to USAID. We encourage all labs to create their own metrics in addition to those that we collect for USAID, and record them in a separate spreadsheet of your own design. You may structure it in a different way that works for you.

The goal is for you to be able to measure your own progress and success, and additionally to be able to share these metrics with your stakeholders. If you do this, please send us your lab’s personal spreadsheet when you report your USAID metrics. We would still love to be able to share these successes with the LISA 2020 network. If you would like help designing these spreadsheets, please contact Kim Love at kim@krloveqcc.com. She is the LISA 2020 MEL (Measurement, Evaluation and Learning) Specialist. This is part of the technical assistance that the LISA 2020 Network is intended to provide.

You are also welcome to provide us with a narrative describing your lab’s successes in addition to the USAID spreadsheet. Programs that are working directly on USAID funded activities (the original three labs, plus teach fund awardees) are required to do this each quarter, though this has looked different depending on the lab’s activities that quarter.

Who do we ask if we have questions?

You can reach out to any of the USAID project leads. However, the best person to contact is Kim Love at kim@krloveqcc.com. She is the LISA 2020 MEL (Measurement, Evaluation and Learning) Specialist.

Challenge: Incentivizing Collaborators

• Certificates

  • Provide certificates to collaborators leading or supporting workshops and other stat lab events.

  • Provide quarterly/semesterly/yearly certificates to collaborators for meeting with domain experts or serving as walk-in consultants.

  • Network Level: Establish the LISA 2020 Certificate Program.

• Impact of Collaboration Experience

  • Demonstrate the impact of the collaboration experience on the growth and development of student members and on their future employability.

    • Use the opportunity to participate in the stat lab as a recruitment tool for the Department.

  • Provide opportunities to participate in staff exchanges with other network stat labs.

  • Seek out trainees and interns from other local institutions of higher education that do not have stat labs, but could benefit from an internship at a stat lab.

• Token Funding Support

  • For short courses, workshops, and collaborations in which funds are raised, share a token amount with participating collaborators.

• Do not rely on faculty to serve as the stat lab’s only collaborators. Involve students!

• Consider how service as a collaborator can meet educational degree requirements (e.g. a collaboration course, etc.).

• Encourage clients to include collaborators as co-authors on papers where they have made a significant contribution.

Challenge: Insufficient Funding

• Demonstrate the impact of the stat lab on the University, College/Faculty, and Department to secure additional funding or other forms of support (e.g. relief from other duties for faculty, administrative support).

  • One related opportunity is for the Research Office or Vice President at the Institution to rely on the stat lab for review of proposals.

  • Network Level: Continue to publish papers on the reach and impact of stat labs and the LISA 2020 Network to support these efforts.

• Seek out and submit proposals for funding opportunities. Share funding opportunities with other stat labs.

• Charge small fees for workshops or collaboration services.

  • Several stat labs are not allowed to charge fees due to institutional restrictions, potential solutions from network stat labs include 1) accepting donations instead of charging fees or 2) setting the lab up as an external entity separate from the university.

• Consider funding opportunities available from governmental sources.

• Network Level: Hold a workshop on locating and submitting funding proposals, support collaboration on proposals between network stat labs, share funding opportunities.

Challenge: Too Few Projects

• Advertise the stat lab services via social media and flyers/hand-bills.

• Encourage satisfied domain experts to refer their colleagues to the stat lab.

• Give talks to other Departments or Units within the institution to increase awareness of the stat lab.

• Seek out collaboration opportunities outside of the institution in the government and private sectors.

• Reach out to local institutions of higher education that do not have their own stat lab, but may benefit from collaboration with a stat lab.

• Recognize the impacts of the COVID-19 pandemic on research programs in general and the stat lab specifically.

Challenge: Need for Training in Statistical Areas and Techniques

• Reach out to network stat labs that have expertise in statistical areas that your collaborators need more experience with.

  • Invite experienced statisticians and data scientists to give invited talks or lectures to your stat lab either in person or remotely.

  • Provide opportunities for collaborators to participate in staff exchanges with other network stat labs.

• Attend LISA 2020 Network Symposia or review talks from previously held symposia.

• Network Level:

  • Create the LISA 2020 Network Shared Repository so that network stat labs can share educational resources and best practices.

  • Encourage stat labs that have expertise in certain statistical areas to present at the LISA 2020 Network Zoom meetings.

  • Continue to host annual symposia to provide opportunities to learn and improve skills

  • Continue to post recorded conference talks and short courses on the LISACollaboration YouTube channel.

Challenge: Collaboration Training

• Provide opportunities for collaborators to participate in staff exchanges with other network stat labs.

• Invite experienced collaborators to give invited talks or lectures to your stat lab either in person or remotely.

• Attend LISA 2020 Network Symposia or review talks from previously held symposia.

• Network Level:

  • Develop training materials related to collaboration skills to be shared widely for network stat labs to use in preparing statistical collaborators.

  • Create the LISA 2020 Network Shared Repository so that network stat labs can share educational resources and best practices.

  • Continue to post recorded conference talks and short courses on the LISACollaboration YouTube channel.

Other Insights from the Sustainability Assessment Responses

• Individual stat labs may face their own unique challenges not summarized above but reflected in their sustainability assessment response.

• Many stat labs have learned best practices or sought guidance and advice from their mentor or other stat labs in the network.

• The COVID-19 pandemic has affected the strength and sustainability of many organizations and network stat labs have also been impacted. Even so, network stat labs have been resilient and sought other avenues to continue their services.

• Staff exchanges and symposia have been a powerful opportunity for network members as reflected in the potential solutions above.