Skip to content
Autowright

AI Configuration

Fixwright uses Claude AI via the Claude Agent SDK to analyze failures and apply fixes. This page covers AI configuration options.

Basic Configuration

import { FixingLoop, FileFailureSource } from '@autowright/fixwright';


const source = new FileFailureSource('./failures');
const fixingLoop = new FixingLoop({
  ai: {
    apiKey: process.env.ANTHROPIC_API_KEY!,
    model: 'claude-sonnet-4-20250514',
    maxTokens: 8192,
    temperature: 0,
  },
  // ... other config
}, source);

Configuration Options

apiKey (required)

Your Anthropic API key:

ai: {
  apiKey: process.env.ANTHROPIC_API_KEY!,
}

model

Claude model to use. Default: claude-sonnet-4-20250514

ai: {
  model: 'claude-sonnet-4-20250514',  // Recommended for code fixes
}

Available models:

  • claude-sonnet-4-20250514 - Best balance of speed and capability
  • claude-3-5-sonnet-latest - Latest Sonnet version
  • claude-opus-4-20250514 - Most capable, slower

maxTokens

Maximum tokens in the response:

ai: {
  maxTokens: 8192,  // Default
}

Increase for complex fixes that require more explanation.

temperature

Controls randomness in responses:

ai: {
  temperature: 0,  // Deterministic (recommended for code)
}

AgentFixer Configuration

For direct control over the agent, use AgentFixer:

import { AgentFixer } from '@autowright/fixwright';


const fixer = new AgentFixer({
  model: 'claude-sonnet-4-20250514',
  maxTurns: 20,    // Max tool use turns
  timeout: 300000, // 5 minute timeout
  verbosity: 'verbose',
});

maxTurns

Maximum number of tool interactions:

{
  maxTurns: 20,  // Default
}

A typical fix uses 5-10 turns (read file, analyze, edit, verify).

timeout

Maximum time for a fix attempt:

{
  timeout: 300000,  // 5 minutes (default)
}

verbosity

Control output detail level:

fixingLoop.setVerbosity('quiet');
// Minimal output - only results

Available Tools

Claude has access to these tools for fixing:

ToolDescription
ReadRead file contents
EditMake targeted code edits
WriteWrite new files
GlobFind files by pattern
GrepSearch file contents
BashRun shell commands (limited)

Tool Usage Example

fixingLoop.on('agent:tool_use', (toolName, input) => {
  switch (toolName) {
    case 'Read':
      console.log('Reading:', input.file_path);
      break;
    case 'Edit':
      console.log('Editing:', input.file_path);
      break;
    case 'Grep':
      console.log('Searching for:', input.pattern);
      break;
  }
});

Bash Restrictions

For security, Bash is limited to safe commands:

Allowed:

  • cat, ls, pwd, echo, grep, find, head, tail
  • npm test, npm run, npx, pnpm, yarn, node

Blocked:

  • rm -rf, sudo, chmod, chown
  • Pipe to shell (curl | sh)
  • Direct device access

Prompt Engineering

System Prompt

Claude receives a system prompt guiding its behavior:

You are an expert Playwright test automation engineer. Your task is to
analyze and fix failing Playwright tests.


Guidelines:
1. Read the full file first to understand context
2. Common failure causes: selector changes, timing issues, flow changes
3. Fix strategies: update selectors, add waits, update assertions
4. Best practices: prefer stable selectors, explicit waits

Context Provided

Each fix attempt includes:

  1. Failure Information

    • Script name and failed step
    • Error type and message
    • Stack trace

  2. Source Context

    • File path and line numbers
    • Failed code snippet

  3. Artifacts

    • Screenshot (if available)
    • DOM snapshot (if available)
    • Console logs

  4. Previous Attempts (if any)

    • What was tried
    • Why it failed
    • “Try a different approach”

Cost Management

Monitoring Usage

Track API costs:

fixingLoop.on('agent:complete', (result) => {
  console.log('Cost:', result.totalCostUsd);
  console.log('Duration:', result.duration, 'ms');
});

Cost Optimization Tips

  1. Use Sonnet - Claude Sonnet is more cost-effective than Opus
  2. Limit attempts - Reduce maxAttempts if fixes often fail
  3. Good artifacts - Better context means fewer retry attempts
  4. Filter failures - Don’t process obviously unfixable failures
{
  fixing: {
    maxAttempts: 2,  // Reduce from default 3
  },
  ai: {
    model: 'claude-sonnet-4-20250514',  // More economical
  },
}

Error Handling

API Errors

fixingLoop.on('agent:error', (error) => {
  if (error.message.includes('rate_limit')) {
    console.log('Rate limited, retrying...');
  } else if (error.message.includes('invalid_api_key')) {
    console.error('Invalid API key');
  }
});

Timeout Handling

fixingLoop.on('agent:complete', (result) => {
  if (!result.success && result.error?.includes('timeout')) {
    console.log('Fix attempt timed out');
  }
});

Environment Variables

Terminal window
# Required
ANTHROPIC_API_KEY=sk-ant-...


# Optional overrides
FIXWRIGHT_MODEL=claude-sonnet-4-20250514
FIXWRIGHT_MAX_TURNS=20
FIXWRIGHT_TIMEOUT=300000

Next Steps

API Reference