AI Agents Aim for $10 Profit; Memory Accuracy Benchmarks Revealed

In the rapidly evolving AI landscape, new experiments are pushing the boundaries of artificial intelligence capabilities and performance measurement. One user has tasked their AI agents, powered by DeepSeek models, with a challenge: to earn $10 online ethically, building upon a previous experiment where they successfully generated $1. These agents have developed infrastructure and coding tools, and are now embarking on a more ambitious goal, with the user providing additional funds for the next phase of their development.

Concurrently, detailed memory accuracy tests have been conducted on smaller AI models, including Gemma 2 9B and Gemma 3 4B and 12B. These benchmarks, utilizing the LoCoMo long-term conversation memory dataset, reveal varying performance levels depending on the judge model used. While initial results might appear high, a more stringent evaluation using larger judge models indicates a more realistic accuracy range, highlighting the challenges in assessing long-term memory retention in models of this size compared to larger, frontier models.