feat: add human-like smooth typing with optional typo injection#201
feat: add human-like smooth typing with optional typo injection#201ulziibay-kernel merged 6 commits intomainfrom
Conversation
Adds smooth typing mode to POST /computer/type that types text in word-sized chunks with variable intra-word delays and natural inter-word pauses via xdotool, following the same Go-side sleep pattern as doMoveMouseSmooth. Optionally injects realistic typos (adjacent-key, doubling, transpose) using geometric gap sampling (O(typos) random calls, not O(chars)) with QWERTY adjacency lookup, then corrects them with backspace after a "realization" pause. New API fields on TypeTextRequest: - smooth: boolean (default false) - enable human-like timing - typo_chance: number 0.0-0.10 (default 0) - per-char typo rate New package: server/lib/typinghumanizer with word chunking, QWERTY adjacency map, and typo position generation. Made-with: Cursor
- typing_demo.html: Dark-themed page with a textarea and keystroke timing visualization (bar chart of inter-key intervals) - demo_smooth_typing.py: Records three phases (instant, smooth, smooth+typos) via the kernel API for comparison GIF/MP4 Made-with: Cursor
jarugupj
left a comment
There was a problem hiding this comment.
Really cool stuff, the typinghumanizer package is super clean. The realization pause before backspace is a great detail, and the typo types cover the common cases really well -- adjacent key, doubling, transpose, extra char feels like it captures how people actually mess up.
One small thing I noticed -- in typeChunkWithTypo, the transpose branch and the else branch do the same thing:
if typo.Kind == typinghumanizer.TypoTranspose && typoLocalPos+1 < len(chunkRunes) {
correctText = string(chunkRunes[typoLocalPos:])
} else {
correctText = string(chunkRunes[typoLocalPos:])
}Could probably just be correctText = string(chunkRunes[typoLocalPos:]) without the if/else?
Both branches produced identical correctText. Simplified to a single expression as suggested by jarugupj. Made-with: Cursor
server/openapi.yaml
Outdated
| type: number | ||
| description: | | ||
| Probability (0.0-0.10) of a typo per character, which is then | ||
| corrected with backspace. Requires smooth to be true. Set to 0 |
There was a problem hiding this comment.
nit: "Requires smooth to be true" — this wording implies the server would return an error, but it actually silently ignores it. consider "Ignored when smooth is false" to match the actual behavior. no schema-level constraint enforces this either, so aligning the docs avoids confusion.
| return adj | ||
| } | ||
|
|
||
| // GenerateTypoPositions computes typo positions using geometric gap sampling. |
There was a problem hiding this comment.
nit: comment says "geometric gap sampling" but the implementation uses uniform gap sampling — halfGap + Intn(avgGap) produces a uniform distribution over [halfGap, halfGap+avgGap). geometric sampling would use exponentially distributed gaps (e.g. rng.ExpFloat64()). the behavior is fine, just the label is inaccurate.
| } | ||
| total := counts[TypoAdjacentKey] + counts[TypoDoubling] + counts[TypoTranspose] + counts[TypoExtraChar] | ||
| require.Greater(t, total, 0) | ||
| adjPct := float64(counts[TypoAdjacentKey]) / float64(total) |
There was a problem hiding this comment.
this only asserts TypoAdjacentKey (~60%). the other three kinds (TypoDoubling ~20%, TypoTranspose ~15%, TypoExtraChar ~5%) are counted but never checked — if two kinds got accidentally merged the test would still pass. consider asserting all four buckets.
masnwilliams
left a comment
There was a problem hiding this comment.
solid feature — clean separation between the humanizer library and the integration layer. main notes are nits on docs wording and test coverage. the identical if/else branches in typeChunkWithTypo were already flagged.
- Default smooth typing when the field is omitted (match moveMouse/dragMouse); OpenAPI default true and updated descriptions; typo_chance doc says ignored without smooth. - Apply every generated typo per word chunk via smoothTypeChunkWithTypos / executeTypoMistake (fixes dropped typos in long chunks). - typinghumanizer: document uniform gap sampling; assert all four typo kinds in distribution test. - Post-codegen patch script: allow empty JSON body (io.EOF) for screenshot and recording endpoints; restore omitempty on pointer response fields. - Makefile: run patch after oapi-codegen alongside existing SSE patch. Made-with: Cursor
Range and default remain in minimum/maximum/default; fuller semantics live in public docs only. Made-with: Cursor
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 2 potential issues.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit c32989e. Configure here.
scripts/oapi had two package main files in one directory, so go vet ./... failed with 'main redeclared'. Move each tool to its own subdirectory and update Makefile go run paths. Made-with: Cursor
masnwilliams
left a comment
There was a problem hiding this comment.
looks good — all first-round comments addressed. one thing to follow up on: phantom corrections when a typo lands on a non-letter character (AdjacentKey returns the char unchanged, so the backspace-correct cycle fires with no visible typo). not blocking — typo_chance defaults to 0 so it only hits opt-in users. nice work on the multi-typo rewrite.
PR #201 inadvertently downgraded oapi-codegen from v2.6.0 to v2.5.1 because the Makefile used `go install @latest`. Pin the version using Go's tool directive in go.mod (same pattern as kernel/kernel) so that `go tool oapi-codegen` always uses the declared version. Also removes the patch_strict_optional_json post-processing script since v2.6.0 natively generates io.EOF handling and omitempty tags. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
PR #201 inadvertently downgraded oapi-codegen from v2.6.0 to v2.5.1 because the Makefile used `go install @latest`. Pin the version using Go's tool directive in go.mod (same pattern as kernel/kernel) so that `go tool oapi-codegen` always uses the declared version. Also removes the patch_strict_optional_json post-processing script since v2.6.0 natively generates io.EOF handling and omitempty tags. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
## Summary - Adds `tool github.com/oapi-codegen/oapi-codegen/v2/cmd/oapi-codegen` directive to `server/go.mod`, pinning oapi-codegen at v2.6.0 via go.sum - Switches `server/Makefile` from `go install ...@latest` to `go tool oapi-codegen` so the pinned version is always used - Removes the `OAPI_CODEGEN` binary variable and install target from the Makefile (no longer needed) This is the root cause fix for the version regression in #201 (where `@latest` resolved to v2.5.1 instead of v2.6.0). PR #209 already restored the generated code; this PR prevents it from happening again by adopting the same `go.mod` tool directive pattern used in `kernel/kernel`. ## Test plan - [x] `go vet ./...` passes - [x] Unit tests pass (`go test -race` on all non-e2e packages) - [x] `go tool oapi-codegen -config ./oapi-codegen.yaml ./openapi-3.0.yaml` uses pinned v2.6.0 - [ ] CI server-test workflow passes 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Low Risk** > Low risk build tooling change: it only affects OpenAPI code generation and Go module dependency resolution, not runtime behavior. Main risk is unexpected dependency/tooling differences if the pinned tool or updated transitive deps behave differently across environments. > > **Overview** > **Pins `oapi-codegen` for reproducible OpenAPI generation.** `server/go.mod` now declares `oapi-codegen` as a Go `tool` dependency (and updates module sums), and `server/Makefile` switches `oapi-generate` from installing `@latest` into `bin/` to invoking `go tool oapi-codegen`. > > This removes the Makefile’s local `oapi-codegen` install target/variable and updates Go module dependencies (including bumping `github.com/getkin/kin-openapi` to `v0.133.0`) to reflect the pinned toolchain. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit d7632bf. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY --> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Summary
smoothandtypo_chancefields toPOST /computer/typefor human-like typing via xdotoolsmooth=true, text is typed in word-sized chunks with variable intra-word delays ([50, 120]ms) and inter-word pauses ([80, 200]ms, 1.5x at sentence boundaries)typo_chanceis set (0.0-0.10), realistic typos are injected using geometric gap sampling (O(typos) random calls, not O(chars)) and corrected with backspace after a "realization" pauseNew package:
server/lib/typinghumanizerSplitWordChunks-- word chunking with trailing delimiters attached to preceding wordUniformJitter-- random duration in a range, clamped to minimumIsSentenceEnd-- sentence boundary detection for longer pausesAdjacentKey-- QWERTY neighbor lookup from static[26][]bytearray, O(1)GenerateTypoPositions-- geometric gap sampling for typo placementTypo types (weighted distribution)
Related: #169 (plan document)
Demo
Test plan
smooth: trueon a running instancesmooth: true, typo_chance: 0.03to verify typo/correction behaviorMade with Cursor
Note
Medium Risk
Changes the
POST /computer/typebehavior by defaulting to a new smooth-typing path that introduces randomized delays and optional typo/backspace correction, which could affect automation timing and determinism. Also modifies the generated OpenAPI client/server glue via new post-generation patching, which can subtly impact request/response handling if the codegen output format changes.Overview
Adds human-like typing to
POST /computer/typevia new request fieldssmooth(defaults to true) andtypo_chance(0–0.10), enabling variable per-chunk delays, inter-word pauses, and optional typo injection with backspace correction.Introduces
server/lib/typinghumanizer(with tests) to chunk text, generate jittered timings, and compute realistic typo positions/adjacent-key substitutions, and wires this into the API implementation.Updates OpenAPI (
openapi.yaml+ regeneratedlib/oapi/oapi.go) and theserver/Makefileto run an additional codegen patch step (patch_strict_optional_json) to tolerate empty JSON bodies for optional-body endpoints and restoreomitemptytags; adds a small demo page + Python script to record side-by-side typing behavior.Reviewed by Cursor Bugbot for commit d63ab1d. Bugbot is set up for automated code reviews on this repo. Configure here.