Leyline Sorcery

Component catalogue

156 records
AllRisingIn demandScore 70+MCPAPI

8 of 156 match

Clear

[ Comparison ]

build

41 Open-Source LLMs Evaluated on 19 Benchmarks leads with Score 3. Cost is unknown. The licence is unknown. Agent and composability metrics are insufficient evidence.

Leaders: 41 Open-Source LLMs Evaluated on 19 Benchmarks · LangSmith · awesome

Browse filters

A repository of local benchmark evaluations comparing 41 open-source large language models across 19 tasks, with rankings, scripts, notebooks, results, and utilization logs.

Function

Rank

order candidates by trust or fit

Rank Browse Rank

Function

Evaluate

score quality against a bench

Evaluate Browse Evaluate

Function

Generate

produce new text, code, or media

Generate Browse Generate

Function

Reason

plan or chain steps

Reason Browse Reason

Matched evaluate

GitHub

No signal yet

Agent use

No signal yet

Reputation

Human and agent evidence, together.

Score factors

41 Open-Source LLMs Evaluated on 19 Benchmarks · 3

Score v2 uses observed signals only. Missing factors read No signal yet.

  • GitHub healthNo signal yet
  • Agent useNo signal yet
  • Human notesNo signal yet
  • ComposabilityNo signal yet
  • Checklist0.0 pts
  • Evidence quality2.5 pts
  • BenchmarksNo signal yet
Full Score explanation
Observed signals only
evalseed
LangSmith

Tracing and evals for LLM apps. Observe and evaluate slices that feed Score later.

Function

Observe

watch a system

Observe Browse Observe

Function

Evaluate

score quality against a bench

Evaluate Browse Evaluate

Function

Monitor

alert on drift

Monitor Browse Monitor

Interface

http

HTTP API

Matched evaluate

GitHub

Observed

Agent use

No signal yet

Reputation

Human and agent evidence, together.

Score factors

LangSmith · 15

Score v2 uses observed signals only. Missing factors read No signal yet.

  • GitHub health5.4 pts
  • Agent useNo signal yet
  • Human notesNo signal yet
  • ComposabilityNo signal yet
  • Checklist6.7 pts
  • Evidence quality2.5 pts
  • BenchmarksNo signal yet
Full Score explanation
Observed signals only
evalintake
awesome

139 lines (106 loc) · 17.5 KB

Function

Memory

keep context across turns

Memory Browse Memory

Function

Code

write or edit programs

Code Browse Code

Function

Evaluate

score quality against a bench

Evaluate Browse Evaluate

Function

Index

make retrieval cheap

Index Browse Index

Function

Policy

allow, deny, or bound an action

Policy Browse Policy

Interface

mcp

Model Context Protocol server

Interface

http

HTTP API

Matched evaluate

GitHub

No signal yet

Agent use

No signal yet

Reputation

Human and agent evidence, together.

Score factors

awesome · 9

Score v2 uses observed signals only. Missing factors read No signal yet.

  • GitHub healthNo signal yet
  • Agent useNo signal yet
  • Human notesNo signal yet
  • ComposabilityNo signal yet
  • Checklist6.7 pts
  • Evidence quality2.5 pts
  • BenchmarksNo signal yet
Full Score explanation
Observed signals only
resourceintake
Toloka

Human-in-the-loop training data, RL environments, and evaluation for AI agents and LLMs.

Function

Evaluate

score quality against a bench

Evaluate Browse Evaluate

Function

Classify

assign a category

Classify Browse Classify

Function

Generate

produce new text, code, or media

Generate Browse Generate

Function

Observe

watch a system

Observe Browse Observe

Interface

http

HTTP API

Matched evaluate

GitHub

No signal yet

Agent use

No signal yet

Reputation

Human and agent evidence, together.

Score factors

Toloka · 12

Score v2 uses observed signals only. Missing factors read No signal yet.

  • GitHub healthNo signal yet
  • Agent useNo signal yet
  • Human notes1.2 pts
  • ComposabilityNo signal yet
  • Checklist6.7 pts
  • Evidence quality4.6 pts
  • BenchmarksNo signal yet
Full Score explanation
Observed signals only
appintake
Hunter

Searches business contacts and domains, finds and verifies email addresses, and enriches records.

Function

Enrich

add facts to a known person, company or record

Enrich Browse Enrich

Function

Evaluate

score quality against a bench

Evaluate Browse Evaluate

Interface

http

HTTP API

Open spec

Interface

mcp

Model Context Protocol server

Open spec

Matched evaluate

GitHub

No signal yet

Agent use

No signal yet

Reputation

Human and agent evidence, together.

Score factors

Hunter · 9

Score v2 uses observed signals only. Missing factors read No signal yet.

  • GitHub healthNo signal yet
  • Agent useNo signal yet
  • Human notesNo signal yet
  • ComposabilityNo signal yet
  • Checklist6.7 pts
  • Evidence quality2.5 pts
  • BenchmarksNo signal yet
Full Score explanation
Observed signals only
modelintake
TypeSafe Jev

Returns typed judgments and probabilities from software state and narrowly defined questions.

Function

Classify

assign a category

Classify Browse Classify

Function

Evaluate

score quality against a bench

Evaluate Browse Evaluate

Function

Reason

plan or chain steps

Reason Browse Reason

Interface

http

HTTP API

Open spec

Matched evaluate

GitHub

No signal yet

Agent use

No signal yet

Reputation

Human and agent evidence, together.

Score factors

TypeSafe Jev · 7

Score v2 uses observed signals only. Missing factors read No signal yet.

  • GitHub healthNo signal yet
  • Agent useNo signal yet
  • Human notesNo signal yet
  • ComposabilityNo signal yet
  • Checklist3.3 pts
  • Evidence quality3.7 pts
  • BenchmarksNo signal yet
Full Score explanation
Observed signals only
mcpintake
Lore

Open-source shared persistent memory for AI agents, backed by Notion. A Lore vault is a Notion page with Projects, Topics, Memories, Entities, and Facts databases so later sessions and people can reuse experiential knowledge, follow-up tasks, decision records, and procedures. Age

Function

Memory

keep context across turns

Memory Browse Memory

Function

Extract

pull a fact or field out

Extract Browse Extract

Function

Store

persist state

Store Browse Store

Function

Connect

speak a protocol (MCP, API, CLI)

Connect Browse Connect

Interface

mcp

Model Context Protocol server

Matched evaluate

GitHub

No signal yet

Agent use

No signal yet

Reputation

Human and agent evidence, together.

Score factors

Lore · 6

Score v2 uses observed signals only. Missing factors read No signal yet.

  • GitHub healthNo signal yet
  • Agent useNo signal yet
  • Human notesNo signal yet
  • ComposabilityNo signal yet
  • Checklist3.3 pts
  • Evidence quality2.5 pts
  • BenchmarksNo signal yet
Full Score explanation
Observed signals only

Portable agent skill that edits drafts to remove 20+ AI-slop patterns (min effective edit + self-check via eval.md) or detects patterns with quoted lines — without claiming AI-authorship detectors.

Function

Evaluate

score quality against a bench

Evaluate Browse Evaluate

Interface

skill

Agent skill file

Matched evaluate

GitHub

Observed

Agent use

No signal yet

Reputation

Human and agent evidence, together.

Score factors

petergyang/no-ai-slop · 6

Score v2 uses observed signals only. Missing factors read No signal yet.

  • GitHub health2.7 pts
  • Agent useNo signal yet
  • Human notesNo signal yet
  • ComposabilityNo signal yet
  • Checklist0.0 pts
  • Evidence quality3.2 pts
  • BenchmarksNo signal yet
Full Score explanation
Observed signals only

Evidence

Leyline ranks components. It does not proxy inference or resell tokens.

Human evidence

Observed stars, notes, maintainer trust, and public sources. Missing signals read No signal yet.

Agent evidence

Observed installs, success, resolve wins, and benches. Unobserved usage is never guessed or zeroed.