Midas reports BEAM memory benchmark with zero LLM calls
A Reddit user said they ran their first BEAM benchmark with Midas. The post says Midas reached 0.56 recall@k on BEAM 100K and 0.51 on BEAM 500K. The writer says this used 0 LLM calls, $0 API spend, and 0 data egress. They said 1M and 10M tiers are next.
Key points
- Midas was tested on the BEAM benchmark, according to the post.
- The reported BEAM 100K score was 0.56 recall@k.
- The reported BEAM 500K score was 0.51 recall@k.
- The writer says the run used no LLM calls, no API spend, and no data egress.
- The next planned tests are the 1M and 10M tiers.
Quick term guide
- BEAM benchmark
- A test for how well an AI agent can remember and find information over a long context or memory store.
- benchmark
- A test used to compare speed, quality, or cost.
- API spend
- Money paid to use an outside software or AI service through its API.
- AI agents
- AI agents are AI tools that can carry out steps toward a goal, not just answer once.
- model call
- One request sent to an AI model to get an answer.
- local-first
- An app design where your data is mainly stored and controlled on your own device.
- long-term memory
- The capability of an AI to store and retrieve information from previous interactions over extended periods.
- token cost
- The money or usage spent when sending text to an AI model and getting text back.