Research Glossary
Definitions of key terms used in Benchline Reports research, methodology documentation, and editorial policy. This glossary exists to make Benchline's terminology explicit and consistent, reducing ambiguity in how conclusions are cited and interpreted.
Terms are used consistently across all published research. If a report uses a term defined here, that definition applies.
Core research terms
Category research
Structured analysis of a defined market segment, covering what the category means, who the participants are, how they differ, and what criteria a decision-maker should use to evaluate options. Distinct from product reviews (which assess specific products) and market sizing (which quantifies market value).
Category benchmark
A research output that applies documented evaluation criteria to a defined category of options, vendors, service providers, or tools, and reports findings with source notes and limitations. A Benchline benchmark does not produce a ranked list; it applies criteria and documents what the evidence supports for each option evaluated.
Sector analysis
Broader market-level research examining a category's structure, competitive dynamics, buyer demand patterns, and evaluation complexity, without assessing specific options. Provides the market context that category benchmarks operate within.
Research primer
A category-definition report that explains what a market segment covers, who it serves, what evaluation questions matter, and what proof decision-makers should require. Front-loads criteria before naming options. Useful when the category definition itself is contested or unclear.
Comparative analysis
A report that places two or more specific options side-by-side against documented criteria, enabling direct comparison of scope, capability, pricing signals, proof quality, and limitations. Distinct from a full category benchmark in that it focuses on a specific comparison set rather than the whole category.
Due diligence guide
A structured framework for evaluating a specific option in depth, covering scope verification, proof requirements, implementation assessment, risk factors, and selection criteria. Most applicable to professional services and high-stakes software decisions where vendor-level assessment matters.
The seven evidence classes
Benchline classifies all research inputs into seven evidence classes. Each class reflects a different type of proof and interpretive weight. Full definitions are in the research methodology.
Class 1, Direct documentation
Official materials published by the organization being assessed: website, product documentation, pricing pages, technical specifications, help center content. Highest reliability for factual claims about what the organization offers; lowest reliability for subjective capability claims without independent corroboration.
Class 2, Independent review signals
Aggregated patterns from independent review platforms (G2, Capterra, Trustpilot). Assessed at the aggregate level, volume, recency, and sentiment trends, not individual review content. Moderate interpretive weight; useful for directional signals, not definitive conclusions.
Class 3, Market and analyst references
Published market analysis, industry reports, and observations from independent sources with a track record of category coverage. Moderate to high weight depending on source independence and recency.
Class 4, Community and practitioner discussion
Published discussions in professional communities, forums, and practitioner platforms (Reddit, LinkedIn, Hacker News, professional Slack communities). Low to moderate weight; useful for identifying unmet needs and practitioner concerns not visible in official documentation.
Class 5, Search and market signals
Search demand patterns and market interest data. Used to establish that demand for a category or option exists, not to make capability claims. Low weight for product assessment; moderate weight for coverage initiation decisions.
Class 6, Submitted evidence
Documentation submitted by vendors, operators, or other interested parties for editorial consideration. Reviewed against the same standards as other classes. Moderate weight if corroborated by independent sources; low weight if sole source. Submission origin is disclosed in source notes.
Class 7, Editorial analysis
Interpretive conclusions drawn by the Benchline Editorial Desk based on the evidence above. Always labeled as editorial interpretation, never presented as documented fact. Required when evidence is ambiguous, criteria must be weighted, or implications must be drawn from incomplete information.
Research output types
Market report
A dated research page that defines a category, summarizes market signals, documents evaluation criteria, and explains how decision-makers should assess their options. Includes: direct answer, category definition, market overview, criteria rationale, source notes, limitations, and correction pathway.
Category benchmark
See definition above. Applies documented criteria to a specific set of options in a category, with findings reported per option including source basis and limitations.
Research primer
See definition above. Category-definition report; criteria-first structure.
Comparative analysis
See definition above. Focused two-or-more-option comparison against documented criteria.
Editorial terms
Documented fact
A claim directly supported by Class 1 to 5 evidence from independent sources. The most citable element of a Benchline report. Distinguished from editorial interpretation in all published research.
Editorial interpretation
A conclusion that involves judgment about what evidence means, how criteria should be weighted, or what the implications are for a specific decision context. Always labeled as editorial interpretation, never presented as a documented fact. Subject to correction if the interpretive basis is disputed with counter-evidence.
Evidence gap
A question the research cannot answer from available public sources. Documented explicitly in the limitations section, not filled with speculation or editorial assertion. Evidence gaps are a normal part of any research output and do not disqualify a report from publication.
Limitation
A constraint on the scope, currency, or completeness of a research output. Every Benchline report includes a limitations section that states what was not reviewed, what has changed since the research was conducted, and where editorial judgment was applied rather than documented evidence.
Source note
A disclosure in a published report that identifies which specific sources informed which conclusions. Distinguishes between independently discovered sources and submitted materials. Required on all Benchline research outputs.
Correction
A factual change to a published report based on new evidence or identification of an error. Corrections are reflected in the report's update timestamp and may be noted in the update history. Distinguished from editorial disagreements, which are noted but do not automatically trigger revisions.
Evaluation criteria terms
Category fit
Whether an option demonstrably serves the specific use case being evaluated, not merely an adjacent or related use case. Assessed by comparing stated capabilities against the research question's specific context.
Proof quality
The degree to which an option's capability claims are supported by independently verifiable, specific evidence, named customers, dated case studies, technical documentation, audit results, rather than marketing language or unverifiable assertions.
Pricing transparency
Whether pricing is publicly disclosed in sufficient detail for a decision-maker to assess fit without requiring a sales call. High pricing transparency means published pricing pages with clear tier definitions. Low transparency means "contact us for pricing" with no public guidance.
Risk factors
Specific conditions identified in the research that create evaluation risk for a decision-maker, pricing opacity, implementation friction, unverifiable capability claims, category mismatch, support model gaps, or contractual constraints not visible in public documentation.