Leyline Sorcery
← Catalogue
evalintake

41 Open-Source LLMs Evaluated on 19 Benchmarks

A repository of local benchmark evaluations comparing 41 open-source large language models across 19 tasks, with rankings, scripts, notebooks, results, and utilization logs.

MCP not recordedAPI not recordedCost unknownLicence unknown
Plan with 41 Open-Source LLMs Evaluated on 19 BenchmarksOpen homepage

[ Observed sample ]

Frequency in Leyline's observed sample

The count is distinct observed stacks. It is not a Score input.

This component is not in Leyline's observed sample.

Record cmp_41-open-source-llms-evaluated-on-19-benchmarks

Open ranks

[ Score explain ]

Why the Score is 3

How Score works

Score v2 uses only observed signals. The displayed contribution is weight × value × 100. Missing factors are insufficient evidence.

Score factor

GitHub health

18% weight. insufficient evidence

insufficient evidence

insufficient evidence

insufficient evidence

insufficient evidence

Score factor

Agent use

16% weight. insufficient evidence

insufficient evidence

insufficient evidence

insufficient evidence

insufficient evidence

Score factor

Human notes

14% weight. insufficient evidence

insufficient evidence

insufficient evidence

insufficient evidence

insufficient evidence

Score factor

Composability

16% weight. insufficient evidence

insufficient evidence

insufficient evidence

insufficient evidence

insufficient evidence

Score factor

Checklist

10% weight. No MCP surface is declared. No API is declared. Cost is unknown. 0 of 3 checklist fields are complete.

Value 0.00 · 0.0 points

10% weight × 0.00 value

0.0 points

No MCP surface is declared. No API is declared. Cost is unknown. 0 of 3 checklist fields are complete.

Score factor

Evidence quality

14% weight. 1 sources cover 1 evidence types. The latest dated evidence is 2026-11-04. Freshness is omitted unless observed as a signal.

Value 0.18 · 2.5 points

14% weight × 0.18 value

2.5 points

1 sources cover 1 evidence types. The latest dated evidence is 2026-11-04. Freshness is omitted unless observed as a signal.

Score factor

Benchmarks

12% weight. insufficient evidence

insufficient evidence

insufficient evidence

insufficient evidence

insufficient evidence

Total2.54 → 3 after rounding

GitHub health: insufficient evidence

Agent use: insufficient evidence

Human notes: insufficient evidence

Composability: insufficient evidence

Benchmarks: insufficient evidence

[ Claims ]

Publisher-asserted record

Claims are what the publisher declares. Score uses observed signals, not these assertions, except for checklist fields derived from interfaces. MCP, API, cost, and license claims are marked verified, unverified, or contradicted from intake evidence.

Kind
eval
Interfaces
None recorded
Functions

Function

Rank

order candidates by trust or fit

Rank Browse Rank

Function

Evaluate

score quality against a bench

Evaluate Browse Evaluate

Function

Generate

produce new text, code, or media

Generate Browse Generate

Function

Reason

plan or chain steps

Reason Browse Reason
Tags
None recorded
mcp
unverified · asserted false · https://github.com/jayminbhan/41-llms-evaluated-on-19-benchmarks
api
unverified · asserted false · https://github.com/jayminbhan/41-llms-evaluated-on-19-benchmarks
cost
unverified · asserted unknown · https://github.com/jayminbhan/41-llms-evaluated-on-19-benchmarks
license
unverified · asserted · https://github.com/jayminbhan/41-llms-evaluated-on-19-benchmarks

[ Capability slices ]

Compose the useful parts.

evaluate

LLM benchmark evaluation

Evaluates open-source language models across reasoning, math, commonsense, natural-language inference, knowledge, and reading tasks using lm-evaluation-harness.

UnknownCompose with this
rank

Model leaderboard ranking

Provides overall and category rankings calculated from task scores, including mean scores and runtime fields.

UnknownCompose with this

The repository provides scripts, notebooks, CSV and Excel tables, JSON results, stdout logs, and GPU utilization logs that can serve as static evaluation artifacts. The page does not state an API or MCP interface.

[ Relations ]

Derived alternatives and shared slots

  • Fills the same slot as
    MiniMax Shared functions generate, reason. Can fill the same function slot.
  • Fills the same slot as
    Grok (xAI) Shared functions generate, reason, search. Can fill the same function slot.
  • Fills the same slot as
    Alibaba Cloud Model Studio Shared functions generate, reason. Can fill the same function slot.
  • Fills the same slot as
    BytePlus ModelArk Shared functions generate, reason. Can fill the same function slot.
  • Fills the same slot as
    Hunter Shared functions evaluate, search. Can fill the same function slot.
  • Fills the same slot as
    TypeSafe Jev Shared functions evaluate, reason. Can fill the same function slot.
  • Fills the same slot as
    Toloka Shared functions evaluate, generate. Can fill the same function slot.
  • Fills the same slot as
    kogiQA MCP Web Browser Shared functions generate, search. Can fill the same function slot.

[ Signals ]

Observed evidence and source claims

Signals carry source, method, and observedAt. Source review dates and evidence basis are shown separately. Unobserved metrics stay insufficient evidence.

  • web
    executor

    Evidence basis unknown · Check date unknown · Source date 2026-11-04

No social source is stored.

Suggest a correction

Propose a change with a public source. Corrections wait for review before the card changes.

[ Notes ]

No notes are stored.