Two tiny models on my RTX 3060
LFM3 1.7B and Spark-X2.6 1.7B answer at about 170 and 135 tokens per second on a five year old card, and they extracted the data from an invoice perfectly. But for code, none of these twelve files works as asked. They are here exactly as the models wrote them, with nothing fixed, so you can check them yourself. Twelve files are a sample, not a benchmark.
The builds, as written
The closest of all: a ball that bounces and a score that goes up. But the paddles are giant blocks and the game makes no sense.
Something moves on the canvas and the score is there. The paddle is a huge white block.
Stops on an error: it calls a function that does not exist.
No game canvas: the page just prints the code as text.
Stops on an error: it uses a variable it never defined.
Stops on an error: a key name written as a variable.
The title and the menu appear, then a syntax error stops the script.
A good looking interface, but it repeated itself until the 24000 token limit and the file never closes.
Loads a 3D library the wrong way: a blank page.
A script with an import in the wrong place: a blank page with two buttons.
The same variable declared twice: the script does not start.
The same variable declared twice: only the menu appears.
What each run took
| Model | Build | Tokens | Tokens/s | Stopped because |
|---|---|---|---|---|
| LFM3 1.7B | Pong, try 2 | 1550 | 180.5 | finished |
| LFM3 1.7B | Pong, try 1 | 1848 | 179.7 | finished |
| LFM3 1.7B | Pong, try 3 | 1827 | 180.2 | finished |
| Spark-X2.6 1.7B | Pong, try 1 | 1199 | 142.3 | finished |
| Spark-X2.6 1.7B | Pong, try 2 | 1365 | 142.2 | finished |
| Spark-X2.6 1.7B | Pong, try 3 | 1613 | 142.0 | finished |
| LFM3 1.7B | Music studio | 5186 | 190.0 | finished |
| Spark-X2.6 1.7B | Music studio | 24000 | 138.7 | token limit |
| LFM3 1.7B | Life film | 1988 | 192.3 | finished |
| Spark-X2.6 1.7B | Life film | 3405 | 148.4 | finished |
| LFM3 1.7B | Voxel castle | 3537 | 190.3 | finished |
| Spark-X2.6 1.7B | Voxel castle | 8244 | 145.8 | finished |
With thinking on
My first pass on the three big prompts left thinking on. Nearly all of the 24000 tokens went to reasoning, and four of the six answers came back empty. These builds are with thinking off.
The exact prompts are for Ruocco Academy members. The Academy is not open yet: join the waiting list and you get one email when it opens. The demos on this page are free to use for everyone.
Join the waiting list