smoke-test▌
mastra-ai/mastra · updated Apr 8, 2026
MDX-style export adds YAML metadata + attribution linking explainx.ai and this canonical listing URL.
Creates a new Mastra project using create-mastra@<tag> and performs smoke testing of the Mastra Studio in Chrome.
Smoke Test Skill
Creates a new Mastra project using create-mastra@<tag> and performs smoke testing of the Mastra Studio in Chrome.
This skill is for Claude Code with Chrome MCP server. For MastraCode with built-in browser tools, use mastracode-smoke-test instead.
Usage
/smoke-test --directory <path> --name <project-name> --tag <version> [--pm <package-manager>] [--llm <provider>]
/smoke-test -d <path> -n <project-name> -t <version> [-p <package-manager>] [-l <provider>]
Parameters
| Parameter | Short | Description | Required | Default |
|---|---|---|---|---|
--directory |
-d |
Parent directory where project will be created | Yes | - |
--name |
-n |
Project name (will be created as subdirectory) | Yes | - |
--tag |
-t |
Version tag for create-mastra (e.g., latest, alpha, 0.10.6) |
Yes | - |
--pm |
-p |
Package manager: npm, yarn, pnpm, or bun |
No | npm |
--llm |
-l |
LLM provider: openai, anthropic, groq, google, cerebras, mistral |
No | openai |
Examples
# Minimal (required params only)
/smoke-test -d ~/projects -n my-test-app -t latest
# Full specification
/smoke-test --directory ~/projects --name my-test-app --tag alpha --pm pnpm --llm anthropic
# Using short flags
/smoke-test -d ./projects -n smoke-test-app -t 0.10.6 -p bun -l openai
Step 0: Parameter Validation (MUST RUN FIRST)
CRITICAL: Before proceeding, parse the ARGUMENTS and validate:
- Parse arguments from the ARGUMENTS string provided above
- Check required parameters:
--directoryor-d: REQUIRED - fail if missing--nameor-n: REQUIRED - fail if missing--tagor-t: REQUIRED - fail if missing
- Apply defaults for optional parameters:
--pmor-p: Default tonpmif not provided--llmor-l: Default toopenaiif not provided
- Validate values:
pmmust be one of:npm,yarn,pnpm,bunllmmust be one of:openai,anthropic,groq,google,cerebras,mistraldirectorymust exist (or will be created)nameshould be a valid directory name (no spaces, special chars)
If validation fails: Stop and show usage help with the missing/invalid parameters.
If -h or --help is passed: Show this usage information and stop.
Prerequisites
This skill requires the Chrome MCP server (Claude-in-Chrome) for browser automation. Ensure it's configured and running.
The Chrome MCP server provides tools like tabs_create_mcp, tabs_context_mcp, navigate_mcp, click_mcp, type_mcp, and screenshot_mcp.
Execution Steps
Step 1: Create the Mastra Project
Run the create-mastra command with explicit parameters to avoid interactive prompts:
# For npm
npx create-mastra@<tag> <project-name> -c agents,tools,workflows,scorers -l <llmProvider> -e
# For yarn
yarn create mastra@<tag> <project-name> -c agents,tools,workflows,scorers -l <llmProvider> -e
# For pnpm
pnpm create mastra@<tag> <project-name> -c agents,tools,workflows,scorers -l <llmProvider> -e
# For bun
bunx create-mastra@<tag> <project-name> -c agents,tools,workflows,scorers -l <llmProvider> -e
Flags explained:
-c agents,tools,workflows,scorers- Include all components-l <provider>- Set the LLM provider-e- Include example code
Being explicit with all parameters ensures the CLI runs non-interactively.
Wait for the installation to complete. This may take 1-2 minutes depending on network speed.
Step 2: Verify Project Structure
After creation, verify the project has:
package.jsonwith mastra dependenciessrc/mastra/index.tsexporting a Mastra instance.envfile (may need to be created)
Step 2.5: Add Browser Agent for Browser Testing
To test browser functionality, add a browser-enabled agent:
- Install browser packages:
<pm> add @mastra/stagehand
# or for deterministic browser automation:
<pm> add @mastra/agent-browser
- Create browser-agent.ts in
src/mastra/agents/:
import { Agent } from '@mastra/core/agent';
import { Memory } from '@mastra/memory';
import { StagehandBrowser } from '@mastra/stagehand';
export const browserAgent = new Agent({
id: 'browser-agent',
name: 'Browser Agent',
instructions: `You are a helpful assistant that can browse the web to find information.`,
model: '<provider>/<model>', // e.g., 'openai/gpt-4o'
memory: new Memory(),
browser: new StagehandBrowser({
headless: false,
}),
});
- Update index.ts to register the browser agent:
import { browserAgent } from './agents/browser-agent';
// In Mastra config:
agents: { weatherAgent, browserAgent },
Step 3: Configure Environment Variables
Based on the selected LLM provider, check for the required API key:
| Provider | Required Environment Variable |
|---|---|
| openai | OPENAI_API_KEY |
| anthropic | ANTHROPIC_API_KEY |
| groq | GROQ_API_KEY |
GOOGLE_GENERATIVE_AI_API_KEY |
|
| cerebras | CEREBRAS_API_KEY |
| mistral | MISTRAL_API_KEY |
Check in this order:
-
Check global environment first: Run
echo $<ENV_VAR_NAME>to see if the key is already set globally- If set globally, the project will inherit it - no
.envfile needed - Skip to Step 4
- If set globally, the project will inherit it - no
-
Check project
.envfile: If not set globally, check if.envexists in the project and contains the key -
Ask user only if needed: If the key is not available globally or in
.env:- Ask the user for the API key
- Create the
.envfile with the provided key
Only check for the ONE key matching the selected provider - don't check for all providers.
Step 4: Start the Development Server
Navigate to the project directory and start the dev server:
cd <directory>/<project-name>
<packageManager> run dev
The server typically starts on http://localhost:4111. Wait for the server to be ready before proceeding.
Step 5: Smoke Test the Studio
Use the Chrome browser automation tools to test the Mastra Studio.
5.1 Initial Setup
- Get browser context using
tabs_context_mcp - Create a new tab using
tabs_create_mcp - Navigate to
http://localhost:4111
5.2 Test Checklist
Perform the following smoke tests using the Chrome automation tools:
Navigation & Basic Loading
- Studio loads successfully (page contains "Mastra Studio" or shows agents list)
- Take a screenshot of the home page
Agents Page (/agents)
- Navigate to agents page
- Verify at least one agent is listed (the example agent from
--default) - Take a screenshot
Agent Detail (/agents/<agentId>/chat)
- Click on an agent to view details
- Verify the agent overview panel loads
- Verify model settings panel is visible
- Take a screenshot
Agent Chat
- Send a test message to the agent (e.g., "What's the weather in Tokyo?")
- Wait for response
- Verify response appears in the chat
- Take a screenshot of the conversation
Browser Agent (/agents/browser-agent/chat) - if browser agent was added
- Navigate to the browser-agent
- Send a message: "Go to example.com and tell me what you see"
- Verify the agent launches a browser and extracts content
- Verify response includes page content
- Take a screenshot
Tools Page (/tools)
- Navigate to tools page
- Verify tools list loads (should show get-weather tool)
- Take a screenshot
Tool Execution (/tools/get-weather)
- Click on the get-weather tool to open detail page
- Find the city input field and enter a test city (e.g., "Tokyo")
- Click Submit button
- Wait for execution to complete
- Verify JSON output appears with weather data (temp, condition, etc.)
- Take a screenshot
Workflows Page (/workflows)
- Navigate to workflows page
- Verify workflows list loads (should show weather-workflow)
- Take a screenshot
Workflow Execution (/workflows/weather-workflow)
- Click on the weather-workflow to open detail page
- Verify visual graph displays (shows workflow steps)
- Find the city input field and enter a test city (e.g., "London")
- Click Run button
- Wait for execution to complete
- Verify steps show success (green checkmarks)
- Click to view JSON output modal
- Verify execution details with timing appear
- Take a screenshot
Settings Page (/settings)
- Navigate to settings page
- Verify settings page loads
- Take a screenshot
Observability Page (/observability)
- Navigate to observability page
- Verify traces list shows recent activity (from previous tests)
- Click on a trace to view details
- Verify timeline view shows steps and timing
- Take a screenshot
Scorers Page (/evaluation?tab=scorers)
- Navigate to
/evaluation?tab=scorers(NOT/scorers- that route doesn't exist) - Verify scorers list loads (shows 3 example scorers)
- Take a screenshot
Additional Pages (verify load only)
- Templates page (
/templates) - Gallery of starter templates - Request Context page (
/request-context) - JSON editor - Processors page (
/processors) - Empty state OK - MCP Servers page (
/mcps) - Empty state OK
5.3 Report Results
After completing all tests, provide a summary:
- Total tests passed/failed
- Any errors encountered
- Screenshots captured
- Recommendations for issues found
Quick Reference
| Step | Action |
|---|---|
| Create Project | cd <directory> && npx create-mastra@<tag> <name> -c agents,tools,workflows,scorers -l <provider> -e |
| Install Deps | Automatic during creation |