Controlled code generation

Code generation safety

Make explicit policy for “where we can write, what we can execute, which networks we can reach”—allow paths and command allowlists beat vague “please don’t break things.” Keys, certs, .env, and prod config belong on deny lists, with secret scanning in pre-commit / CI as backup.

• Execution surface: generate and test in isolated workdirs or containers; do not inject repo tokens or cloud creds into agent context.
• Dependency surface: new packages must be auditable (lockfile, license, provenance); reject unpinned “just install this.”
• Data surface: redact logs and eval data; do not send stacks or dumps with PII back to the model.
• Human in the loop: two-person review for sensitive modules; tag generated or machine-edited files (comments, OWNERS, CODEOWNERS) for traceability.

Handlebars template code generation example (generating a REST Controller):

// templates/controller.hbs (Handlebars template)
// @generated by codegen v1.2.0 — DO NOT EDIT MANUALLY
// @generated-source: schemas/{{resourceName}}.yaml
// @generated-at: {{generatedAt}}

import { Router, Request, Response } from 'express';
import { {{ResourceName}}Service } from '../services/{{resourceName}}.service';
import { create{{ResourceName}}Schema } from '../schemas/{{resourceName}}.schema';

const router = Router();
const service = new {{ResourceName}}Service();

{{#each endpoints}}
router.{{method}}('{{path}}', async (req: Request, res: Response) => {
  {{#if hasBody}}
  const body = create{{../ResourceName}}Schema.parse(req.body);
  {{/if}}
  const result = await service.{{operationId}}({{params}});
  res.json(result);
});
{{/each}}

export default router;

// Generator script (Node.js)
const Handlebars = require("handlebars");
const fs = require("fs");
const yaml = require("js-yaml");

function generate(schemaPath, templatePath, outputPath) {
  const schema = yaml.load(fs.readFileSync(schemaPath, "utf8"));
  const template = Handlebars.compile(fs.readFileSync(templatePath, "utf8"));
  const code = template({ ...schema, generatedAt: new Date().toISOString() });
  fs.writeFileSync(outputPath, code);
  console.log(`Generated: ${outputPath}`);
}

Diff review

Prefer unified diff (or platform-equivalent line diffs). Review order: intent of deletes/changes → edge cases → tests and types. Split large agent patches into small PRs, each with one narratable goal, for bisect and rollback.

• Paths and file count first: within declared allowlist? touching generated-only or machine-only files?
• Behavior next: public API, serialization, auth checks, error codes vs docs and callers.
• Noise last: unrelated formatting, import reorder, mass renames → separate PR; do not mix with security fixes.

Gatekeeping: let CI auto-fix format/lint; humans focus on semantics and risk. Golden diffs, contract tests, or mutation tests help catch “compiles but wrong” generations.

Generated region annotation (preventing overwrites of manual changes) and review flow:

// ✅ Generated region annotation: use comments to mark machine-managed sections

// user.service.ts
export class UserService {
  // @codegen-start — code below is managed by the generator, do not edit manually
  async findById(id: string) { ... }
  async findAll(filter: UserFilter) { ... }
  async create(data: CreateUserDto) { ... }
  // @codegen-end

  // ✋ Manual extension below; will not be overwritten by generator
  async findByEmailWithProfile(email: string) {
    // Complex query, not suitable for auto-generation
  }
}

// Generator script: only replace annotated regions
const CODEGEN_START = "// @codegen-start";
const CODEGEN_END = "// @codegen-end";
function replaceGeneratedSection(fileContent, newCode) {
  const start = fileContent.indexOf(CODEGEN_START);
  const end = fileContent.indexOf(CODEGEN_END) + CODEGEN_END.length;
  if (start === -1 || end === -1) throw new Error("Generated markers not found");
  return fileContent.slice(0, start) + CODEGEN_START + "\n" +
    newCode + "\n  " + CODEGEN_END + fileContent.slice(end);
}

// Generation review flow (CI pipeline)
// 1. codegen --dry-run --output-diff     # output diff only, do not write files
// 2. git diff --exit-code                # CI verifies no unexpected file changes
// 3. human review diff PR
// 4. npm test / go test ./...            # tests must pass
// 5. merge (squash merge preserves @generated annotation)

Allow-path preview

Before wiring CI or agent config, use the tool below to check whether allow prefixes cover intended paths. Rules use prefix match: one rule per line, forward slashes; trailing * means “any path under this prefix.” Lines starting with # are comments. Deny wins: matching any deny prefix rejects the path.

SKILL draft

Put allow paths, diff conventions, and required CI checks into SKILL preconditions to reduce conversational drift.

Test the generator itself and the generated output separately:

// 1. Test the generator (snapshot tests)
import { generate } from "./codegen";

test("generating UserController should match snapshot", () => {
  const code = generate("./schemas/user.yaml");
  expect(code).toMatchSnapshot();  // snapshot changes require human review
});

test("generated code should include all CRUD endpoints", () => {
  const code = generate("./schemas/user.yaml");
  expect(code).toContain("router.get('/users'");
  expect(code).toContain("router.post('/users'");
  expect(code).toContain("router.put('/users/:id'");
  expect(code).toContain("router.delete('/users/:id'");
});

// 2. Test generated output (integration test, do not modify generated files)
// Test the generated Controller via supertest
import request from "supertest";
import app from "./app";

describe("Generated UserController", () => {
  it("GET /users should return user list", async () => {
    const res = await request(app).get("/users");
    expect(res.status).toBe(200);
    expect(Array.isArray(res.body)).toBe(true);
  });
});