Playground

This page runs aniate 2.1.0 in the browser through Pyodide, a build of Python for WebAssembly, and no code leaves the page. The first start downloads Python, numpy, scipy and matplotlib, which takes several seconds; later runs begin at once. The last expression of a program is displayed as a table, as in a Jupyter notebook, and every figure the program creates is drawn below its output.

A five-state corridor: solve it, simulate 10,000 episodes and draw the four standard plots.

import aniate as an
from aniate import vis

m = an.from_functions(
    states=range(5), actions=["left", "right"],
    transition=lambda s, a: {max(s - 1, 0): 1.0} if a == "left" else {min(s + 1, 4): 0.8, s: 0.2},
    reward=lambda s, a, s_next: 10.0 if s_next == 4 else -1.0,
    gamma=0.9, terminal=[4], initial=0,
)

sol = m.solve()                                   # optimal policy, its exact value, an error bound
runs = an.simulate(m, sol, episodes=10_000)       # 10,000 episodes, run in parallel
vis.overview(runs)                                # the four standard plots

sol                                               # the last expression is displayed as a table

Python starts when this page opens.Ctrl or ⌘ + Enter runs · Tab indents · Esc, then Tab, leaves the editor