Extend to matrices and high-order tensors

[jli05]

First clean-up:

• accessory image files moved into assets/
• demo notebooks moved into demos/
• use Python's built-in unittest package for unit tests, one dependency (on pytest) less

Next, I plan to extend the engine to matrix or higher-order tensors, and make the backward() more efficient so that it won't have to recompute the operator topology for each piece of new input data.

[dnparadice]

Why do you keep making PRs? The repo is fine the way it is. If you re-organize it, it will not match the video demos that go along with it.

[jli05]

This is an extension to matrices and higher-order tensors. The notebook demos still run.

To see how it is like, check out https://github.com/brief-ds/micrograd @dnparadice

[jli05]

@karpathy could you review? You can check it out at https://github.com/brief-ds/micrograd:

• extend the Value class and backward derivation to matrices and higher-order tensors
• add a minimal set of operators
• split the tests into tests/test_engine.py to test deterministically (no dependency on torch) and tests/test_vs_torch.py

The Value class and tensordot function are exported at the package level:

from micrograd import Value, tensordot

[jli05]

@karpathy now the demos/demo_scalar.ipynb and demos/demo_vector.ipynb produce exactly the same 100 steps of training trace.

[jli05]

@karpathy you can check how it looks on https://github.com/brief-ds/micrograd

• 20kb core code at present (__init__.py, engine.py and optim.py)
• as portable as Python and NumPy
• loads 10x faster than PyTorch, 30x faster than TensorFlow
• the tensor-capable demo notebook demos/demo_vector.ipynb trains instantly, producing same intermediate results over 100 steps as your original demo notebook demos/demo_scalar.ipynb

[jli05]

@karpathy this version works with tensors.

As the core is just one 500-line Python file micrograd/engine.py, the learning curve is almost zero.

This blog post TensorFlow, Apple's MLX and our micrograd explains in terms of install size and performance micrograd is in par with MLX, but micrograd is extra easy to learn, play with and profile.

Would you consider merging?

[jli05]

I made micrograd tensor-capable with in mind the idea to try simplifying the attention mechanism. Below is a proposal. Any thought? @karpathy

https://www.brief-ds.com/2026/02/10/roadmap-att.html

Only micrograd's characteristics makes the study possible (simplicity, opening up forward and backward methods). Feel free to merge this PR. :)