Before launch
Start with the decision the pilot needs to support.
“See whether teachers like it” is too vague to guide a rollout. Name the fluency problem, the classrooms involved, the starting information available, and what the district will decide at the end. A bounded question makes it easier to protect staff time and harder to mistake activity for impact.
- Which students, grades, operations, or instructional setting are in scope?
- What current practice or pain point is the pilot trying to improve?
- Which adoption, learning, and staff-experience signals will be reviewed?
- Who makes the expand, adjust, pause, or stop decision?
Prepare the conditions
A clean roster is not the same as a ready school.
Implementation also depends on role clarity, schedules, classroom expectations, support ownership, and product policies. Teachers should know why the pilot exists and what is expected before students encounter it. Technology and privacy teams should have time to review the data flow and controls before launch day.
- Name a school or district owner and a day-to-day teacher contact.
- Confirm classrooms, rosters, account access, and support paths.
- Agree on which collaborative, competitive, social, or optional features are appropriate.
- Give staff a short, usable orientation tied to the intended classroom routine.
During the pilot
Measure whether people used it and what changed. Those are different stories.
Adoption data can show whether the pilot reached classrooms. Practice and progress signals can show what happened inside the product. Teacher and student feedback can explain why. Keeping those evidence types separate produces a more honest review than collapsing everything into one success number.
- Adoption: access, participation, sessions, assignments, and classroom consistency.
- Learning signals: starting placement, accuracy, progression, and identified fluency needs.
- Experience: teacher workload, student response, support requests, and barriers to use.
Evidence boundary
A change observed during a pilot is not automatically caused by FlashMath. Stronger outcome claims require an evaluation design that can support them.
At the decision point
Make room for “adjust” and “stop,” not just “expand.”
A useful review asks who participated, who did not, where teachers found value, what created friction, and whether the product addressed the original problem. A district can then expand what worked, change the conditions, narrow the use case, or decide the fit is not strong enough.
- Compare results with the success criteria agreed on before launch.
- Look for variation across classrooms instead of relying only on an average.
- Record the support and policy changes an expansion would require.
- Share a plain-language summary with the staff who did the work.