Add control-flow statement ASYNC, make the endiannes (?) of BYTES configureable via optional argument

Author: python-processing-unit

SPECIFICATION.html

@@ -137,6 +137,8 @@
 
 Blocks group one or more statements and are enclosed in curly brackets: `{ statement1 ... statementN }`. Curly braces must match (that is, `{` closes with `}`). Blocks serve as the bodies of control-flow constructs and functions.
 
+ASYNC blocks: The `ASYNC` statement introduces a background task whose body is a regular block: `ASYNC{ ... }`. The statements inside the block execute synchronously with respect to each other but asynchronously relative to the rest of the program: the interpreter begins executing the block in a background task and immediately continues with the next statement in the current thread. The block shares the main program namespace (it executes with the same lexical environment), so assignments and mutations performed inside an `ASYNC` block are visible to the rest of the program immediately. Runtime errors raised inside an `ASYNC` block are reported via the interpreter's error hooks and recorded in the state log; they do not synchronously abort the main thread. Note that language invariants still apply inside the `ASYNC` block (for example, `RETURN` outside a function is a runtime error).
+
 Assignments have the syntax `TYPE : identifier = expression` on first use, where TYPE is `INT`, `STR`, or `TNS`. Spaces around the colon and equals sign are optional. Subsequent assignments to an existing identifier may omit the type but must preserve the original type. Variables are deallocated only when `DEL(identifier)` is executed.
 
 Tensor elements can be reassigned with the indexed form `identifier[i1,...,iN] = expression`. The base must be a previously-declared `TNS` binding. The indices must match the tensor's dimensionality, follow the same one-based/negative-index rules as ordinary indexing, and must reference anexisting position. The element's original type cannot change: attempting to store a different type at that position is a runtime error. Indexed assignment mutates the `TNS`.
@@ -166,7 +168,7 @@
 
 ## 6. Variables and Memory Model
 
-A variable is created only when it is first assigned with an explicit type annotation of the form `TYPE : name = expression`, where TYPE is `INT`, `STR`, or `TNS`. For example: `INT` : counter = 0` or `STR` : message = "hi"`. Subsequent assignments to the same name must match the declared type and may omit the type annotation (`counter = ADD(counter,1)`). Assigning to an undeclared name without a type annotation is a runtime error. A variable exists until `DEL(name)` is executed. Referencing a variable that has never been declared, or that has been deleted, is a runtime error.
+A variable is created only when it is first assigned with an explicit type annotation of the form `TYPE : name = expression`, where TYPE is `INT`, `STR`, or `TNS`. For example: `INT : counter = 0` or `STR : message = "hi"`. Subsequent assignments to the same name must match the declared type and may omit the type annotation (`counter = ADD(counter,1)`). Assigning to an undeclared name without a type annotation is a runtime error. A variable exists until `DEL(name)` is executed. Referencing a variable that has never been declared, or that has been deleted, is a runtime error.
 
 The language assumes at least a global typed environment mapping identifiers to (type, value) pairs. Function calls create new environments for parameters and local variables, as described in Section 6.2; the precise details of name resolution depend on the chosen scoping rules.
 
@@ -343,7 +345,7 @@
 - `EXIT()` or `EXIT(INT:code)` — Requests immediate termination of the interpreter. If an integer `code` is supplied, it is used as the interpreter's process exit code; otherwise `0` is used. Execution stops immediately when `EXIT` is executed (no further statements run), and an entry is recorded in the state log to make deterministic replay possible. Using `EXIT` inside a function terminates the entire program (not just the function).
 - `INT(STR: x):INT` ; `STR` -> `INT` using empty/01/other rules; `INT` passthrough
 - `STR(INT: x):STR` ; `INT` -> binary-spelled `STR`; `STR` passthrough
-- `BYTES(INT: n):TNS` — Converts a non-negative integer into its big-endian byte representation. The result is a one-dimensional `TNS` whose elements are `INT` values in the range `0..11111111` (0-255 decimal), ordered most-significant byte first. The tensor length is `max(1, ceil(bit_length(n)/8))`; `BYTES(0)` returns a single zero byte. Supplying a negative integer is a runtime error.
+- `BYTES(INT: n, endian = "big"):TNS` — Converts a non-negative integer into its byte representation. The optional `endian` parameter controls byte order: if `endian` is `"little"` the result is little-endian (least-significant byte first); if `endian` is `"big"` the result is big-endian (most-significant byte first). The result is a one-dimensional `TNS` whose elements are `INT` values in the range `0..11111111` (0-255 decimal). The tensor length is `max(1, ceil(bit_length(n)/8))`; `BYTES(0)` returns a single zero byte. Supplying a negative integer is a runtime error. Supplying an `endian` value other than `"big"` or `"little"` is a runtime error.
 - Binary literal: optional leading `-` (spaces/tabs/CR allowed after the dash) then `{0,1}+`
 - String literal: `"` characters `"` (no escapes, no newlines)
 - `EXIST(SYMBOL: x):INT` ; returns `1` if the identifier `x` exists in the current environment (searching enclosing environments), else `0`. The argument must be an identifier; supplying an expression that is not a plain identifier is a runtime error.

asm-lang.exe

@@ -8,6 +8,7 @@
 import tempfile
 import codecs
 import numpy as np
+import threading
 from dataclasses import dataclass, field
 from typing import Any, Callable, Dict, List, Optional, Tuple, Union
 from numpy.typing import NDArray
@@ -24,6 +25,7 @@
     ExpressionStatement,
     ForStatement,
     FuncDef,
+    AsyncStatement,
     GotoStatement,
     GotopointStatement,
     Identifier,
@@ -338,7 +340,6 @@ def _as_bool(value: int) -> int:
 class Builtins:
     def __init__(self) -> None:
         self.table: Dict[str, BuiltinFunction] = {}
-        # Numeric operators that support INT and FLT (no mixing).
         self._register_custom("ADD", 2, 2, self._add)
         self._register_custom("SUB", 2, 2, self._sub)
         self._register_custom("MUL", 2, 2, self._mul)
@@ -412,7 +413,7 @@ def __init__(self) -> None:
         self._register_custom("TYPE", 1, 1, self._type)
         self._register_custom("ROUND", 1, 3, self._round)
         self._register_custom("READFILE", 1, 2, self._readfile)
-        self._register_custom("BYTES", 1, 1, self._bytes)
+        self._register_custom("BYTES", 1, 2, self._bytes)
         self._register_custom("WRITEFILE", 2, 3, self._writefile)
         self._register_custom("EXISTFILE", 1, 1, self._existfile)
         self._register_custom("CL", 1, 1, self._cl)
@@ -1916,6 +1917,17 @@ def _bytes(
         number = self._expect_int(args[0], "BYTES", location)
         if number < 0:
             raise ASMRuntimeError("BYTES expects a non-negative integer", location=location, rewrite_rule="BYTES")
+        # Optional endian argument: default is big-endian
+        endian = "big"
+        if len(args) >= 2:
+            endian_val = args[1]
+            if endian_val.type != TYPE_STR:
+                raise ASMRuntimeError("BYTES expects string endian argument", location=location, rewrite_rule="BYTES")
+            endian = str(endian_val.value).strip().lower()
+        # Validate endian
+        if endian not in {"big", "little"}:
+            raise ASMRuntimeError("BYTES endian must be 'big' or 'little'", location=location, rewrite_rule="BYTES")
+
         if number == 0:
             data = np.array([Value(TYPE_INT, 0)], dtype=object)
             return Value(TYPE_TNS, Tensor(shape=[1], data=data))
@@ -1925,7 +1937,9 @@ def _bytes(
         while temp > 0:
             octets.append(temp & 0xFF)
             temp >>= 8
-        octets.reverse()
+        # octets currently little-endian (LSB first); reverse for big-endian
+        if endian == "big":
+            octets.reverse()
 
         data = np.array([Value(TYPE_INT, b) for b in octets], dtype=object)
         return Value(TYPE_TNS, Tensor(shape=[len(octets)], data=data))
@@ -3105,6 +3119,38 @@ def _execute_statement(self, statement: Statement, env: Environment) -> None:
         if isinstance(statement, GotoStatement):
             target = self._evaluate_expression(statement.expression, env)
             raise JumpSignal(target)
+        if isinstance(statement, AsyncStatement):
+            # Execute the block synchronously inside a background thread
+            block = statement.block
+            loc = statement.location
+
+            def _async_worker() -> None:
+                frame = self._new_frame("<async>", env, loc)
+                self.call_stack.append(frame)
+                try:
+                    self._emit_event("async_start", self, frame, env)
+                    try:
+                        self._execute_block(block.statements, env)
+                    except Exception as exc:
+                        # Notify hooks about the error so it can be recorded/handled.
+                        try:
+                            self._emit_event("on_error", self, exc)
+                        except Exception:
+                            pass
+                finally:
+                    # Pop the async frame and emit end event.
+                    try:
+                        self.call_stack.pop()
+                    except Exception:
+                        pass
+                    try:
+                        self._emit_event("async_end", self, frame, env)
+                    except Exception:
+                        pass
+
+            t = threading.Thread(target=_async_worker, daemon=True, name=f"asm_async_{self.frame_counter}")
+            t.start()
+            return
         raise ASMRuntimeError("Unsupported statement", location=statement.location)
 
     def _execute_if(self, statement: IfStatement, env: Environment) -> None:
@@ -3495,17 +3541,26 @@ def _evaluate_expression(self, expression: Expression, env: Environment) -> Valu
                 )
             try:
                 if keyword_args:
+                    # Allow specific named keywords for certain builtins.
                     if expression.name in {"READFILE", "WRITEFILE"}:
                         allowed = {"coding"}
-                        unexpected = [k for k in keyword_args if k not in allowed]
-                        if unexpected:
-                            raise ASMRuntimeError(
-                                f"Unexpected keyword arguments: {', '.join(sorted(unexpected))}",
-                                location=expression.location,
-                                rewrite_rule=expression.name,
-                            )
-                        if "coding" in keyword_args:
-                            positional_args.append(keyword_args.pop("coding"))
+                        key = "coding"
+                    elif expression.name == "BYTES":
+                        allowed = {"endian"}
+                        key = "endian"
+                    else:
+                        allowed = set()
+                        key = None
+
+                    unexpected = [k for k in keyword_args if k not in allowed]
+                    if unexpected:
+                        raise ASMRuntimeError(
+                            f"Unexpected keyword arguments: {', '.join(sorted(unexpected))}",
+                            location=expression.location,
+                            rewrite_rule=expression.name,
+                        )
+                    if key and key in keyword_args:
+                        positional_args.append(keyword_args.pop(key))
                     if keyword_args:
                         raise ASMRuntimeError(
                             f"{expression.name} does not accept keyword arguments",

interpreter.py

@@ -26,6 +26,7 @@ class Token:
     "WHILE",
     "FOR",
     "FUNC",
+    "ASYNC",
     "RETURN",
     "POP",
     "BREAK",

lexer.py

@@ -118,6 +118,11 @@ class ContinueStatement(Statement):
     pass
 
 
+@dataclass
+class AsyncStatement(Statement):
+    block: Block
+
+
 class Expression(Node):
     pass
 
@@ -231,6 +236,8 @@ def _parse_statement(self) -> Statement:
             return self._parse_func()
         if token.type == "IF":
             return self._parse_if()
+        if token.type == "ASYNC":
+            return self._parse_async()
         if token.type == "WHILE":
             return self._parse_while()
         if token.type == "FOR":
@@ -364,6 +371,11 @@ def _parse_continue(self) -> ContinueStatement:
         self._consume("RPAREN")
         return ContinueStatement(location=self._location_from_token(keyword))
 
+    def _parse_async(self) -> AsyncStatement:
+        keyword = self._consume("ASYNC")
+        block: Block = self._parse_block()
+        return AsyncStatement(location=self._location_from_token(keyword), block=block)
+
     def _parse_block(self) -> Block:
         opening = self._peek().type
         if opening == "LBRACE":