Lexicon Engine Benchmark Suite
Independent scores on a fixed 1,050-sentence corpus (350 JFLEG, 350 BEA-2019, 350 LOCNESS). Lexicon tiers and publicly available Free Grammarly and QuillBot interfaces are measured under the protocols on this page. Figures describe this suite only. They are not a claim that any product is better or worse in every real-world use. All Lexicon scoring uses en-US; BEA and LOCNESS lean British, so dialect mismatches can affect local models. See the disclaimer.
How to read these scores
Grammar correction rewards careful edits more than aggressive ones. We report the usual GEC metrics below, plus corpus hit rates for fluency and grammar.
- Precision
- Of the edits the engine made, how many were actually correct? High precision means fewer wrong or noisy changes.
- Recall
- Of the real errors in the text, how many did the engine catch? High recall means fewer missed mistakes.
- F₀.₅
- A single combined score that weights precision about twice as heavily as recall, the standard for grammar error correction, because a bad rewrite is usually worse than leaving text alone.
- Clean FPR
- How often the engine “fixes” sentences that were already correct. Lower is better; 0% means it stayed quiet on clean prose.
- Latency
- Average time to process one sentence. Lex tiers: sequential local inference. Cloud grammar checkers: live time to first underlines on a single test sentence. Lower is better for live typing.
- JFLEG & BEA
- Corpus hit rates on the full suite: JFLEG measures fluency; BEA/LOCNESS measure grammar and usage. Shown as correct / expected edits.
Standard 4B: strong local F₀.₅ on this corpus with ~233 ms sequential inference latency.
Quality 27B: highest Lexicon F₀.₅ on this suite (51.7% recall, 4.3% clean FPR), fully offline.
LanguageTool: instant syntax at keystroke speed with zero clean false alarms.
Entire JFLEG + BEA-2019 + LOCNESS pool scored once per engine.
Full-Corpus Ladder
Pareto Efficiency Frontier
Evaluation Leaderboard
Scores for 6 configurations on this 1,050-sentence corpus (Free cloud tools where noted). Toggle to Detailed Matrix for edit counts, JFLEG/BEA hit rates, and latency. Rankings are for this suite only.
Independent evaluation of Free interfaces on this suite. Not affiliated with Grammarly or QuillBot. Expand for full disclaimer.
These results are an independent evaluation published by Lexicon for informational and comparative research purposes. They were produced by me (Aashish Harishchandre) using the methods described on this page. Lexicon is not sponsored by, affiliated with, endorsed by, or certified by Grammarly Inc., QuillBot, their parents, or any other third-party vendor named here. Product names and marks are trademarks of their respective owners and appear only under nominative fair use to identify the publicly available tools that were tested.
Third-party rows refer to Free web experiences available at the time of testing (for example Grammarly Free Docs with Pro-locked suggestions dismissed, and QuillBot Free Accept-all). They do not measure paid Pro, Business, Enterprise, or other premium SKUs, and they are not claims about those products. Scores depend on corpus choice, scoring protocol, product version, feature entitlements, network conditions, device performance, and how suggestions were accepted or dismissed. Vendor products, pricing, and UIs change over time. Results on this page may become outdated and may not match other labs or vendor marketing.
Latency for cloud tools comes from limited live probes (time to first underlines on sample text), not a full-corpus timing study. Lexicon latency uses instrumented local sequential inference and is not an identical protocol. Quality metrics for Lexicon and third-party Free outputs use the same corpus and scoring rules described on this page; that still does not make every metric perfectly comparable across product designs.
Nothing here is a warranty, guarantee, or promise of accuracy, completeness, superiority, or real-world performance for Lexicon or any compared product. Do not treat these numbers as the sole basis for purchasing, business, academic, or legal decisions. Additional terms appear in the Terms of Service.