Processes and Policies

1. Identify a potential Director or Coordinator of the Stat Lab and a mentor from within the Network who will help the potential Director or Coordinator receive sufficient training and guidance in the non-technical skills needed to move between theory and practice to apply statistics and data science to solve real problems as well as to learn from other Stat Labs and share best practices.

2. Gather and document support from within the department and across the university for the Stat Lab in the form of letters from senior university officials supporting the official creation of such a Stat Lab.

3. Complete and submit via email to LISA2020@colorado.edu the Full Lab Plan/Proposal to become a “Proposed Member” of the Network. The proposal will document the purpose/mission of the Stat Lab, a name for it, and identify a physical location with enough space to meet with domain experts. It will also include detailed statements about the lab’s Context/Environment/Leadership; Mission, Goals, and Objectives; Activities; Personnel; Budget; Expected Outcomes (metrics); and Desired Impacts (metrics), as well as how the lab plans to connect with the Network. The Full Lab Plan/Proposal can be downloaded, edited, and returned via email.

4. Respond to a regional review committee’s feedback on the Full Lab Plan/Proposal. If the response to the feedback is satisfactory, the lab will be granted “Transitional Member” status in the Network.

5. Open the Stat Lab: a) Train your students, b) Provide research infrastructure for local domain experts, c) Teach short courses/workshops to improve statistical skills and literacy widely, d) Report on your Stat Lab’s activities, outcomes, and impacts (metrics).

6. Stay connected with the network via regular Zoom meetings, symposia, quarterly reports of Stat Lab activities and numbers, and other channels.

7. Report a full quarter of metrics and present about the lab to the Network.

Typically, labs will complete steps 1-4 before step 5, though some of the steps may be taken out of order. For example, a Stat Lab can open before they have completed step 3. We encourage labs to become connected with the Network (step 6) at the early stages of the process.

In general, labs that complete steps 1-3 are considered “Proposed Members”. Labs successfully responding to the feedback on the Full Lab Plan/Proposal (step 4) will become “Transitional Members.” The next step would be to begin operation of the Stat Lab. The final steps would be for the transitional lab to report a full quarter of metrics and have introduced their lab to the Network at a regular Zoom meeting. The Officer for Membership for the region that the transitional Stat Lab would reside in would then take a two-thirds vote of the Regional Member Stat Labs to determine if the lab would be submitted to the Vice President for Membership for final ratification of full membership. If accepted, they would become “Full Members.”

Download these instructions in PDF format.

Video instructions for Recording Metrics for the LISA 2020 Global Network. Download this quick guide as a pdf.

What is the purpose of reporting?

The goal is for individual Stat Labs to be able to measure their own progress and success, and additionally to be able to share these metrics with their stakeholders. The LISA 2020 Global Network collects these metrics from the individual Stat Labs and combines them to assess the reach and impact of the network as a whole.

What are the metrics the LISA 2020 Global Network collects?

• Number of publications (peer-reviewed and non-peer reviewed)

  • Also title, date, type of publication, authors, and gender of authors

  • USAID is interested specifically in publications that results from projects directly funded by USAID1, but please record all lab project-related publications in this spreadsheet; we will use the additional information about the publications to determine whether to report them to USAID

  • I have recently added a new column asking for confirmation about whether a publication is lab-project related or not; lab-project related means work done with a domain expert/lab client, and an example of something that is not project related is a more theoretical publication that you undertake as an academic staff member

• Number of projects with individual domain experts (researchers, businesses, policy actors)

• Number of policy recommendation meetings

• Number of stat lab trainees of different types

• Number of educational events provided by the lab (short courses, 1-day workshops, multi-day workshops) and number of attendees of different types

• Number of labs in the LISA 2020 Network1

1 Not recorded in spreadsheets

What should we be especially aware of when recording data?

1. Everything reported in the “Data” tab (main spreadsheet) should be a number. If anything requires an additional written explanation, please send a separate document, or document it in the email when you return the spreadsheet. The symbol “#” is meant to indicate that a number goes in the box, although it turns out that this is primarily a North American use of this symbol.

2. The second tab in the spreadsheet contains explanations of all metrics you are meant to record. Please review this if you are not sure what should go under any heading.

3. Areas that require additional information in order for us to count them are publications and policy recommendation convenings. For us to count these metrics, additional information must completed in the spreadsheet on another tab (either the “Publications” tab or the “Policy Recommendations Convenings” tab).

4. What is a policy recommendation convening? This is a meeting with policy actors who have the power to implement a public policy change. This meeting must involve a recommendation for public policy, based on data and analysis that you have already completed. It is NOT any meeting with a client/domain expert or policy actor.

What if our lab has achievements that are not covered by these metrics?

Many labs have operations and objectives that are not covered by the metrics we collect. We encourage all labs to create their own metrics in addition to those that we collect, and record them in a separate spreadsheet of your own design. You may structure it in a different way that works for you.

The goal is for you to be able to measure your own progress and success, and additionally to be able to share these metrics with your stakeholders. If you do this, please send us your lab’s personal spreadsheet when you report your network metrics. We would still love to be able to share these successes with the LISA 2020 Global Network. If you would like help designing these spreadsheets, please contact Kim Love at kim@krloveqcc.com. She is the Vice President for Monitoring and Evaluation.

You are also welcome to provide us with a narrative describing your lab’s successes in addition to the metrics spreadsheet.

Who do we ask if we have questions?

The LISA 2020 Global Network Vice President for Monitoring and Evaluation Kim Love (kim@krloveqcc.com).

Challenge: Incentivizing Collaborators

• Certificates

  • Provide certificates to collaborators leading or supporting workshops and other stat lab events.

  • Provide quarterly/semesterly/yearly certificates to collaborators for meeting with domain experts or serving as walk-in consultants.

  • Network Level: Establish the LISA 2020 Certificate Program.

• Impact of Collaboration Experience

  • Demonstrate the impact of the collaboration experience on the growth and development of student members and on their future employability.

    • Use the opportunity to participate in the stat lab as a recruitment tool for the Department.

  • Provide opportunities to participate in staff exchanges with other network stat labs.

  • Seek out trainees and interns from other local institutions of higher education that do not have stat labs, but could benefit from an internship at a stat lab.

• Token Funding Support

  • For short courses, workshops, and collaborations in which funds are raised, share a token amount with participating collaborators.

• Do not rely on faculty to serve as the stat lab’s only collaborators. Involve students!

• Consider how service as a collaborator can meet educational degree requirements (e.g. a collaboration course, etc.).

• Encourage clients to include collaborators as co-authors on papers where they have made a significant contribution.

Challenge: Insufficient Funding

• Demonstrate the impact of the stat lab on the University, College/Faculty, and Department to secure additional funding or other forms of support (e.g. relief from other duties for faculty, administrative support).

  • One related opportunity is for the Research Office or Vice President at the Institution to rely on the stat lab for review of proposals.

  • Network Level: Continue to publish papers on the reach and impact of stat labs and the LISA 2020 Network to support these efforts.

• Seek out and submit proposals for funding opportunities. Share funding opportunities with other stat labs.

• Charge small fees for workshops or collaboration services.

  • Several stat labs are not allowed to charge fees due to institutional restrictions, potential solutions from network stat labs include 1) accepting donations instead of charging fees or 2) setting the lab up as an external entity separate from the university.

• Consider funding opportunities available from governmental sources.

• Network Level: Hold a workshop on locating and submitting funding proposals, support collaboration on proposals between network stat labs, share funding opportunities.

Challenge: Too Few Projects

• Advertise the stat lab services via social media and flyers/hand-bills.

• Encourage satisfied domain experts to refer their colleagues to the stat lab.

• Give talks to other Departments or Units within the institution to increase awareness of the stat lab.

• Seek out collaboration opportunities outside of the institution in the government and private sectors.

• Reach out to local institutions of higher education that do not have their own stat lab, but may benefit from collaboration with a stat lab.

• Recognize the impacts of the COVID-19 pandemic on research programs in general and the stat lab specifically.

Challenge: Need for Training in Statistical Areas and Techniques

• Reach out to network stat labs that have expertise in statistical areas that your collaborators need more experience with.

  • Invite experienced statisticians and data scientists to give invited talks or lectures to your stat lab either in person or remotely.

  • Provide opportunities for collaborators to participate in staff exchanges with other network stat labs.

• Attend LISA 2020 Network Symposia or review talks from previously held symposia.

• Network Level:

  • Create the LISA 2020 Network Shared Repository so that network stat labs can share educational resources and best practices.

  • Encourage stat labs that have expertise in certain statistical areas to present at the LISA 2020 Network Zoom meetings.

  • Continue to host annual symposia to provide opportunities to learn and improve skills

  • Continue to post recorded conference talks and short courses on the LISACollaboration YouTube channel.

Challenge: Collaboration Training

• Provide opportunities for collaborators to participate in staff exchanges with other network stat labs.

• Invite experienced collaborators to give invited talks or lectures to your stat lab either in person or remotely.

• Attend LISA 2020 Network Symposia or review talks from previously held symposia.

• Network Level:

  • Develop training materials related to collaboration skills to be shared widely for network stat labs to use in preparing statistical collaborators.

  • Create the LISA 2020 Network Shared Repository so that network stat labs can share educational resources and best practices.

  • Continue to post recorded conference talks and short courses on the LISACollaboration YouTube channel.

Other Insights from the Sustainability Assessment Responses

• Individual stat labs may face their own unique challenges not summarized above but reflected in their sustainability assessment response.

• Many stat labs have learned best practices or sought guidance and advice from their mentor or other stat labs in the network.

• The COVID-19 pandemic has affected the strength and sustainability of many organizations and network stat labs have also been impacted. Even so, network stat labs have been resilient and sought other avenues to continue their services.

• Staff exchanges and symposia have been a powerful opportunity for network members as reflected in the potential solutions above.

Download this synthesis in PDF format.

Download the PDF of this policy.

Naming Convention for Statistical Collaboration Laboratories

Names of stat labs should be unique. If a stat lab has completed the full membership process within the LISA 2020 Global Network, it has initial rights to the name it established via the proposal and membership voting process. New stat labs will not be allowed to request names already in use. A stat lab will forfeit its right to a name if it is inactive for over one year and a new location has interest in its name. Stat labs that have not submitted quarterly metrics or annual reports; attended semi-monthly Zoom meetings, Symposium events, or other full network activities; and have not responded to email and other contact attempts from the Network Leadership for one year will be considered inactive. The Network Leadership will make all efforts to regain contact with stat labs with full membership in the LISA 2020 Global Network to try to ensure that this situation does not occur except in situations where previous stat labs are no longer functional.

Stat Lab Leadership Changes & Director/Coordinator Moves

The Leadership of the LISA 2020 Global Network understands that the leadership of current LISA 2020 stat labs may change to new Directors/Coordinators. Also, Directors/Coordinators may move to other institutions and wish to establish a stat lab at their new institution using similar processes and structures of those they developed at their previous institution. This policy describes the process for both of these common situations. In the case that a stat lab Director/Coordinator needs to temporarily step away from their duties for professional or personal reasons, the LISA 2020 Global Network Leadership should be notified of the interim Director/Coordinator and the timeline for the expected return of the recognized Director/Coordinator.

Leadership Change at a LISA 2020 Recognized Stat Lab

When the leadership of a LISA 2020 recognized stat lab changes, “Full Membership” status will be temporarily halted until the following process is completed to confirm the continuance of the lab.

1. Identify the new Director or Coordinator of the stat lab. Verify that the current mentor from within the LISA 2020 Global Network wishes to continue serving as mentor for the new Director/Coordinator or locate a new mentor within the network. This mentor will help the new Director/Coordinator receive sufficient training and guidance in the non-technical skills needed to move between theory and practice to apply statistics and data science to solve real problems as well as to learn from other stat labs and share best practices.

2. The new Director/Coordinator will gather and document continued support from within the department and across the University for the Stat Lab in the form of letters from senior university officials supporting the continued operation of such a stat lab.

3. Complete and submit via email to LISA2020@Colorado.EDU the Full Lab Plan/Proposal. This proposal can be based on the originally approved proposal for the stat lab, but should incorporate any changes that the new Director/Coordinator will implement plus revisions and updates that have occurred since the submission of the original proposal or that will improve the status and impact of the stat lab. As with the original proposal, this will document the purpose/mission of the stat lab, a name for it, and identify a physical location with enough space to meet with domain experts. It will also include detailed statements about the lab’s Context/Environment/Leadership; Mission, Goals, and Objectives; Activities; Personnel; Budget; Expected Outcomes (metrics); and Desired Impacts (metrics), as well as how the lab plans to connect with the LISA 2020 Network.
   A blank version of the Full Lab Plan/Proposal can be downloaded online. If you do not have a copy of the previously approved proposal, please contact LISA2020@Colorado.EDU and we will share it with you.

4. Respond to a review committee’s feedback on the submitted Full Lab Plan/Proposal. If the response to the feedback is satisfactory, the lab will be granted a temporary continuance of full membership status in the LISA 2020 Global Network.

5. Continue operations of the stat lab: a) Train your students, b) Provide research infrastructure for local domain experts, c) Teach short courses/workshops to improve statistical skills and literacy widely, d) Report on your stat lab’s activities, outcomes, and impacts (metrics).

6. Stay connected with the network via semi-monthly Zoom meetings, annual symposia, quarterly reports of stat lab activities and numbers, and other channels.

7. Report a full quarter of metrics and present about the lab to the LISA 2020 Network.

After having completed steps 1-4, previously established labs are granted a temporary continuance of full membership status. After demonstration of continued operation via the submission of activities, outcomes, and impacts (metrics) report, the new Director/Coordinator will be asked to provide a lab update presentation to the LISA 2020 Global Network at a regular Zoom meeting. If accepted by general consent of the LISA 2020 Global Network, full membership status would be reestablished and a new letter and certificate of membership will be issued.

After departure of a currently approved lab Director/Coordinator, if the new Director/Coordinator does not submit a revised Full Lab Plan/Proposal within 60 days, respond to review committee feedback within 15 days, or submit metrics during the next applicable quarter, Full Membership status may be withdrawn and institutions requested to complete the entire full membership process from the beginning.

LISA 2020 Recognized Stat Lab Director/Coordinator Moves to a New Institution

When a successful stat lab Director or Coordinator moves to a new institution, we understand that they may plan to implement many of the strategies they previously developed as they establish a new stat lab. We also understand that each institution is unique and that strategies will need to be tailored to fit the new environment. The following process is completed to establish a stat lab at the new institution. The Stat Lab Director should complete the full membership application process described above in the Seven Steps for Joining the Network section.