Star Execution Engine

A streaming-first, asynchronous Python DSL interpreter.

Standard Python exec() is great, but it has a major limitation: it needs the full string of code before it can do anything.

Star Execution is different. It parses and executes code as a stream of tokens.

I built this engine to handle scenarios where code is generated piece-by-piece (like from an LLM or a slow network socket) and needs to run immediately, line-by-line, without waiting for the end of the file. It implements a custom recursive descent parser that handles Python control flow, scoping, and complex expressions on the fly.

Why use this?

Most people use eval or exec. You should use Star Execution if:

1. You are streaming code: You want to execute block A while block B is still being generated/downloaded.
2. You need a Sandbox: You want to strictly control what functions and variables are available to the user. This engine doesn't use Python's internal compiler; it uses a custom parser, so you can block specific keywords (like class or async) at the syntax level.
3. You need "Dry Runs": The engine has a prototype_mode that parses code, checks for syntax errors, and resolves scopes without actually executing side effects or heavy math.
4. You need deep feedback: It generates rich logs for syntax errors and runtime issues, pinpointing exactly where things broke in the stream.

Features

• Async-Native: Built on asyncio. The parser yields control while waiting for the next token from your stream.
• Custom Scope Management: Handles global, nonlocal, and closure variables manually. It knows the difference between defining a variable in a function vs. the global scope.
• Robust Parsing: It's not a regex hack. It fully parses logic like for loops, if/elif/else chains, list comprehensions, f-strings, and complex operator precedence.
• Security: It runs in a "predefined environment." If you don't give the script access to import, it can't import. It explicitly blocks exec, eval, and defining classes to keep things simple and safe.

Installation

It's a single file. No complex build steps.

1. Requirements:

   pip install numpy nest_asyncio

   (Note: The code imports syntax_highlighter. You'll need to provide that module or comment out the formatting bits if you don't have it.)

2. Setup: Copy star_execution.py into your project. That's it.

Usage

The Basics

You don't pass a string to the executor; you pass an async generator. This simulates the "streaming" aspect.

import asyncio
from star_execution import StarExecutor

# 1. Define what the script is allowed to touch
# Only give it access to 'print' and a starting variable 'x'
safe_env = {'x': 10, 'print': print}

# 2. Create the executor
executor = StarExecutor(variables=safe_env)

# 3. Simulate a stream of code arriving char-by-char
async def my_code_stream():
    code_to_run = "result = x * 5\nprint(result)"
    for char in code_to_run:
        yield char
        await asyncio.sleep(0.01) # Simulating network lag

# 4. Run it
if __name__ == "__main__":
    loop = asyncio.new_event_loop()
    loop.run_until_complete(executor.run(my_code_stream()))
    # Check the state after execution
    print(f"Final State: {safe_env['result']}") # Output: 50

Advanced: Functions and Closures

The engine is smart enough to handle function definitions and state retention.

async def function_stream():
    # It parses the def, stores it, and can call it later in the stream
    yield "def adder(a, b):\n"
    yield "    return a + b\n"
    yield "val = adder(10, 20)"

# After running this, safe_env['val'] will be 30.

How it works (Under the Hood)

The core logic lives in _StarExecutorInternal.

• StreamingCode: Wraps your generator. It buffers tokens and allows the parser to "peek" ahead without consuming the stream.

• CodeParser: Breaks the incoming text into logical blocks (like a full for loop block) based on indentation.

• Recursive Descent: The engine parses expressions recursively. It knows that * has higher precedence than +, and that lambda has the lowest precedence.

• It's a DSL: While it looks and acts like Python, it is a Domain Specific Language tailored for this executor.

• No Classes: You cannot define class MyObject: inside the script. This is by design to keep the execution model functional and predictable.

• No Async in Script: The script itself cannot use async/await keywords (though the executor itself is async).