Skip to main content

Best Practices for AI Command Types

This guide outlines the four primary AI command types in our testing framework. Each section provides clear examples of what works and what doesn't, helping you write more effective test automation.

AI Assertion​

Purpose: Verify that the application is in the expected state at specific points in test execution.

AI Assertions check if specific conditions are true in the application's current state. They function as test validation points and always return a boolean result (true/false).

When to use​

  • After navigation actions to confirm the correct page loaded
  • Before critical actions to ensure preconditions are met
  • After data operations to verify changes were applied correctly
  • To check for the presence or absence of UI elements

Writing effective assertions​

✅ Do❌ Don't
Use clear, binary questions
Is the account balance displayed as $1,250.00?
Simple yes/no question that's easy to evaluate		Ask vague or compound questions
Is everything correct on the page?
Too broad and checks multiple conditions
Include specific reference points
In the confirmation dialog, is the deposit amount $100?
Provides context since AI lacks step history		Assume context from previous steps
Is the amount correct?
AI doesn't know what "amount" or "correct" refers to
Test one condition per assertion
Is the error message visible?
Does the error message contain 'Invalid credentials'?
Atomic assertions for better error isolation		Combine multiple checks
Is the error message visible and does it say 'Invalid credentials'?
Harder to debug which condition failed
Always end with a question mark
Are there exactly 5 items in the shopping cart?
Consistent format improves clarity		Use statements instead of questions
There are 5 items in the cart
Not clearly an assertion
Include both positive and negative checks
Is the success message visible?
Is the error message absent from the screen?
Comprehensive test coverage		Only test for presence
Is there a message?
Doesn't verify the correct state

Key guidelines​

  • Formulate binary questions that the AI can evaluate based solely on displayed information
  • Be specific about expected values or states you're verifying
  • Use consistent formats like "Is [expected condition] true?" for clearer assertions
  • Provide context in each assertion since AI lacks memory of previous steps

AI Command​

Purpose: Manipulate the application interface by interacting with specific elements or controls.

AI Commands instruct the AI to perform specific, discrete actions within the application interface, similar to how a user would interact with UI elements.

When to use​

  • For basic UI interactions like clicking, typing, selecting, and scrolling
  • When you need precise control over individual steps in a workflow
  • For test setup operations that prepare the application state
  • To interact with specific UI components that require direct manipulation

Writing effective commands​

✅ Do❌ Don't
Use descriptive references, not positions
Click on the row where the status shows 'completed'
Works even if row order changes		Rely on fixed positions
Click on the third row
Breaks when data changes
Include distinguishing characteristics
Click the 'Submit' button with the green background in the payment form
Precise targeting reduces ambiguity		Be vague about the target
Click the button
Multiple buttons may exist
Add sequencing hints when needed
After the loading indicator disappears, click the 'Continue' button
Handles timing issues gracefully		Ignore timing dependencies
Click the 'Continue' button
May fail if element isn't ready
Use semantic identifiers
Click on the shopping cart icon
Readable and maintainable		Reference technical selectors
Click on element with ID 'btn-cart-123'
Brittle and hard to understand
Specify exact input format
Enter 'test@example.com' in the email field
Type '(555) 123-4567' in the phone number field
Clear expectations for data format		Be ambiguous about input
Enter an email
AI may not know the expected format
Identify items by content, not position
Select the product with name 'Wireless Mouse'
Resilient to list reordering		Use ordinal positions in lists
Select the second product
Fragile when list changes

Key guidelines​

  • Prepare for dynamic scenarios by using content-based references rather than positions
  • Be precise and specific about both the action and the target element
  • Include waiting conditions when elements may not be immediately available
  • Consider edge cases where elements might be conditionally present

AI Task​

Purpose: Execute complete user journeys or business workflows with a single instruction.

AI Tasks represent high-level business operations that may involve multiple steps and decisions. They operate at a higher abstraction level than individual commands.

When to use​

  • For end-to-end workflow testing that mimics real user journeys
  • When you want to abstract away implementation details
  • For data setup that requires complex business operations
  • To test complete business processes rather than individual UI interactions

Writing effective tasks​

✅ Do❌ Don't
Write intent-based actions with specific values
Deposit $100 into Checking Account
Clear business workflow with concrete details		Be vague about the operation
Make a deposit
Missing critical details like amount and account
Include contextual details
Transfer $500 from Savings to Checking, ensuring sufficient funds are available
Reduces ambiguity with preconditions and business rules		Omit important context
Transfer money between accounts
Unclear which accounts and validation rules
Structure complex workflows with Gherkin
Given I'm on the booking page
When I search for flights from NYC to London
Then select the cheapest option for next Friday
Clear preconditions, actions, and outcomes		Write run-on instructions
Go to booking page and search flights NYC to London next Friday and pick cheapest
Hard to parse and understand
Break down into focused sub-tasks
1. Add three items to cart
2. Proceed to checkout
3. Select standard shipping
4. Complete payment
Easier to understand and execute		Combine too many steps
Add items, checkout, ship, and pay
Overwhelming and prone to errors
Include success criteria
Register a new user and verify the welcome email is sent
AI knows when the task is complete		Leave completion ambiguous
Register a user
Unclear what constitutes success
Specify data requirements
Create a new customer with email format: firstname.lastname@example.com
Clear data generation rules		Assume data format
Create a new customer
AI may generate unsuitable data

Key guidelines​

  • Author intent-based actions that clearly represent complete business workflows
  • Include contextual details like preconditions, account types, or expected outcomes
  • Use Gherkin syntax (Given/When/Then) for complex workflows to improve clarity
  • Break down complex processes into smaller, focused tasks
  • Specify business rules or validation criteria that should be considered
  • Define success criteria so the AI knows when the task is complete

AI Extract Data​

Purpose: Dynamically retrieve and store data from the application interface for verification or subsequent operations.

AI Extract Data commands capture specific information from the application for storage and later use in test execution.

When to use​

  • When you need to capture dynamic data generated during test execution
  • For retrieving values that need to be verified later in the test
  • To gather data needed for subsequent test steps
  • For comparing values across different parts of the application
  • To extract information for reporting or debugging purposes

Writing effective extract commands​

✅ Do❌ Don't
Be precise about the data location
Create parameter ORDER_ID and assign the order number from the confirmation message
Specific location and clear parameter name		Be vague about where to find data
Get the order number
Unclear where to look and no parameter name
Specify the expected format
Extract the total amount as a number without currency symbol and store in TOTAL
Clear format expectations		Assume format handling
Extract the total amount
Unclear if "$1,234.56" or "1234.56" is expected
Use descriptive parameter names
Create parameter USER_EMAIL and assign the email from the profile section
Self-documenting parameter name		Use generic names
Create parameter DATA1
Unclear what the parameter contains
Provide context for identification
Extract the price from the row where product name is 'Wireless Mouse'
Handles multiple similar items		Assume uniqueness
Extract the price
May extract wrong value if multiple prices exist
Clarify single vs multiple values
Extract all product names from the search results into PRODUCT_LIST
Clear that multiple values are expected		Be ambiguous about quantity
Extract product names
Unclear if one or many values
Ensure data visibility
Scroll to the footer, then extract the copyright year into YEAR
Ensures data is in viewport		Extract from hidden areas
Extract the copyright year
May fail if footer isn't visible
Reference structural landmarks
From the 'Order Summary' section under 'Shipping Address', extract the ZIP code
Uses headers to guide extraction		Ignore document structure
Extract the ZIP code
Ambiguous in forms with multiple addresses
Specify fallback behavior
Extract the discount amount, or set to 0 if no discount is applied
Handles edge cases gracefully		Assume data always exists
Extract the discount amount
Fails when discount isn't present

Key guidelines​

  • Make commands precise and specific to the data you want to extract
  • Ensure data is visible in the viewport and legible before extraction
  • Use descriptive parameter names that clearly indicate what data they contain
  • Specify the expected format (text, number, date, etc.) and any formatting rules
  • Provide context for identifying the correct data when similar items appear multiple times
  • Clarify single vs multiple values when extracting from lists or tables
  • Reference structural landmarks (headers, sections) in structured documents
  • Define fallback behavior if the expected data cannot be found

Summary​

Effective AI commands share common characteristics:

  • Clarity: Use precise language that leaves no room for ambiguity
  • Context: Provide enough information since AI lacks memory of previous steps
  • Specificity: Include concrete values, locations, and expected outcomes
  • Resilience: Design commands that work even when the application state varies

By following these best practices, you'll create more reliable, maintainable, and effective test automation.