Bring the same three scenarios to every vendor: one client’s SOP, one program from request to close-out, and, for agencies, one planner switching between two clients. Ask each vendor to show where every rule is checked, who decided, and what the record keeps. A feature list can’t answer those questions; a live program can.
Bring the same three scenarios
- One client’s SOP. Ask the vendor to show the moment each rule is checked, and what happens when it fails. Use the SOP checklist to pick the rules.
- One speaker program from request to spend per HCP, with a walk-in and one exception along the way.
- For agencies: one planner moving between two clients, each with its own rules.
Label all sample data. Keep what the product showed apart from what a customer achieved.
Score the questions
| Evaluation area | Question to ask | Evidence to request |
|---|---|---|
| SOP rules | Where is each rule in the client's SOP checked, and how strict is it? | The rule, the moment it is checked, and whether it blocks, warns, escalates, asks for proof, or monitors.BlockWarnEscalateRequire proofMonitor |
| Event-type configuration | What actually changes between two program types or formats that run differently? | Configured fields, approvals, roles, participant surfaces, spend behavior, and close requirements. |
| Client boundaries | What changes when an agency user switches clients, and what can someone at the client see? | Separate sample workspaces, role-scoped views, and cross-organization access controls. |
| Decisions and exceptions | Where are missing requirements, eligibility, rules, and exceptions reviewed? | The decision point, responsible role, retained reason, actor, and time. |
| HCP identity | How is each registration matched to one HCP record, and who confirms the match? | The lookup sources, the suggested match and its confirmation, and how one fixed record updates every report row. |
| Participant operations | How do invitation, registration, check-in, virtual attendance, walk-ins, no-shows, and reconciliation relate? | One attendee record across every registration and attendance path. |
| Spend and close-out | How are budget availability, estimates, actuals, reimbursements, and recipient-level transfers distinguished? | Separate financial views and the close-readiness requirements. |
| Integrations | What data moves, in which direction, and how are failures handled? | The fields that move, a dry run, and how failures are seen and retried. |
| Implementation | What inputs, owners, testing, and acceptance decisions are required? | A scoped responsibility matrix and representative testing plan. |
| Trust | Which security mechanisms and procurement facts are documented today? | Direct answers, architecture boundaries, access controls, encryption, and requested contractual materials. |
Separate four types of proof
- Product evidence: the mechanism is visible in the product or technical documentation.
- Operating evidence: the vendor can run the agreed scenario and show the resulting record.
- Customer evidence: a customer you can speak to confirms a dated outcome.
- Contractual evidence: the applicable agreement, security exhibit, DPA, SLA, or certification status states the commitment.
Questions that separate a demo from a deployment
- Was the program type in the demo set up for this client, or for every client? Ask how a new type is added.
- Ask to see the Open Payments file. Is it ready for the client’s aggregate-spend team to submit, or a spreadsheet someone reworks?
- When the AI proposes a change, who confirms it, and where is that recorded?
- Do the budget view, program spend, and per-HCP spend answer different questions?
- Fix one HCP’s record. Do the spend rows and the client’s CRM follow, or does someone fix each row?
- For each CRM connection: which fields move, in which direction, and what happens when one fails?
- Does a client switch change rules and access, or only the logo?
Use a written decision record
For every demonstration, record the scenario, what was visible in the product, what required configuration or customer input, the evidence supplied, open assumptions, the responsible owner, and the date by which the question must be resolved.
This is an evaluation framework, not a comparison of named vendors.
Common questions
What should an agency ask a speaker-program platform vendor?
Ask the vendor to run one client's SOP on a live program: where each rule is checked, what happens when it fails, and who decided. Then ask what a planner sees after switching to a second client.