The Qarmiq platform

Everything your team needs
to keep the product stable.

Define what must always work. Verify it. Catch and investigate problems early. Build on what your team learns. Qarmiq brings the process together in one place.

See Qarmiq in action

Define what must never break.

Turn product expectations and requirements into clear scenarios and checks. Build a shared understanding of how the product should behave, grounded in knowledge your team confirms.

ScenarioRed line

A paid order always produces an invoice.

Area: Checkout · 4 checks · version 3
Paying with a valid card creates an invoiceVerified in a browser
happy path
A declined card never creates an orderVerified in a browser
negative
The invoice total matches the order totalVerified by API or database
edge case
An invoice is visible only to its own accountVerified by API or database
security

Illustrative example.

Scenarios and checks

Define what your product must always get right.

A scenario states one thing that must always be true, in a single sentence. Its checks say what must be verified to prove it. Nothing in them is a click or a selector, so a redesign does not invalidate them.

  • Checks tagged happy path, negative, edge case, security, accessibility or performance
  • Red lines for what is non-negotiable
  • Full version history, with restore
See Scenarios and checks →
Project knowledge4 items
Checkout help pagePulled from a URL, weekly
Confirmed
Refund policyUploaded as Markdown
Needs confirming
Release notes 2.4Disagrees with the refund policy on partial refunds
Conflict
/account/billingNothing explains this page. What is it for?
Question

Illustrative example.

Knowledge

Give Qarmiq the context your product depends on.

Build project knowledge from your own documents, and have your team confirm each item before anything relies on it. Conflicts are flagged, and Qarmiq asks about pages nothing explains.

  • Paste text, upload Markdown, text or HTML, or pull from a URL
  • A person confirms each item
  • Conflicts flagged, gaps asked about
See Knowledge →

Verify that it still works.

Combine human testing, autonomous AI agents, repeatable scripts, and supported external verifiers to check critical behavior as the product evolves.

Release 2.4 smoke runStaging · Desktop · macOS · locked
CheckChromeSafari
Accepts a valid card and confirms the orderverified by a script
passedpassed
Rejects an expired card with a clear messageverified by a person
passedfailed
Never charges twice on a double-clickverified by the agent · defect raised
failedfailed
Locks the form after five failed attemptsverified by a person
blockedblocked
Applies a discount code only onceverified by a script
skippedskipped

Illustrative example.

Runs and verdicts

Know exactly what passed, what failed, and where.

A run tests a fixed set of scenarios on one environment, across several browsers, and records pass, fail, blocked or skipped for each check. It keeps the exact wording it tested.

  • Per check, per browser, on a named environment
  • Each verdict shows who produced it
  • Defects filed from a failed cell
See Runs and verdicts →
staging.yourapp.com/checkout
Pay now
A paid order always produces an invoice.
Clicked “Pay now”
POST /api/orders returned 500
Console error: invoice is undefined

Illustrative example. Typed values are never recorded.

Witness

Make every manual test more useful.

A small widget in your own app. Testers test as they normally do, record pass or fail and file defects in place, while Witness keeps the evidence of everything they do.

  • Captures clicks, network calls, console errors and performance
  • Never records typed values, and is not video
  • One script tag in your app
See Witness →
Agent traceChrome · real browser
  1. Opens the checkout with a test card set to decline once
  2. Retries with a valid card. Payment confirmed, order created
  3. Opens the new order and looks for its invoice
  4. No invoice is present
Judge: failedCites steps [2] [3] [4]
failed
With nothing cited, the same answer would be marked inconclusive and handed to a person.

Illustrative example.

Autonomous testing

Let AI investigate. Make every verdict earn your trust.

Choose Test autonomously on one check or a whole run and an AI agent tests it in a real browser. A separate AI judge decides pass, fail or inconclusive from the evidence.

  • An answer that cites no evidence is inconclusive
  • A step-by-step trace and a video with typed text masked
  • Chrome-family browsers
See Autonomous testing →
Script: invoice after paymentverified
  1. The agent’s test passedA Playwright script is written from it
  2. Run several timesEvery run passed
  3. Run against a deliberately broken variantIt failed, as it should
  4. Next change: the script goes firstIt fails on a moved button
  5. The agent re-checks, and the product still worksThe script is updated, not recorded as a failure

Illustrative example.

Verified scripts

Turn verified behavior into repeatable checks.

A passing autonomous test produces a Playwright script. It is run several times, plus a deliberately broken variant, before it counts as verified, and it updates itself when the product still works.

  • Proven by repeated runs and a broken variant
  • Goes first, and repairs itself
  • Downloadable
See Verified scripts →
Release 2.4 smoke runVerified by
A paid order always produces an invoiceBrowser check
the AI agentpassed
The support assistant never invents a refund policyDeepEval: 2 of 40 refund prompts drifted
DeepEvalfailed
A customer can always sign inVerified script
a scriptpassed

Illustrative example.

Connectors

Bring every verification result into one view.

Verifier connectors bring other tools’ verdicts into the same run grid as everything else. The DeepEval connector, for testing AI features, works end to end.

  • Results labeled with the tool that produced them
  • DeepEval for AI features
  • Data-feed connectors on the roadmap
See Connectors →

Find problems and understand their impact.

Connect failures to the expected behavior, capture evidence, and give developers the context needed to investigate. Review new findings and get a consolidated view of product stability.

SHOP-D1High Open

The card is charged twice when Pay is double-clicked.

Breaks: A customer is never charged twice · Release 2.4 smoke run · Safari 18
Second charge
Developer brief
  • HappenedTwo charges within one second of a double-click.
  • ExpectedOne charge, and the button disabled after the first click.
  • ReproduceSteps taken from the captured session.
  • UnknownWhether the payment provider de-duplicates.

Illustrative example.

Defects and briefs

Every bug should come with a clear next step.

Defects carry severity, status, duplicates and attachments, linked to the scenario, check and run where they were found. An AI developer brief turns the captured evidence into something an engineer or a coding agent can act on.

  • Linked to the scenario, check and run
  • An annotated screenshot from Witness
  • A developer brief on request
See Defects and briefs →
Decision Board3 waiting
Learning from the AI agent

Refund after a partial shipment also works. Keep as a learning for this scenario?

A refund is only ever issued once

AcceptDismiss
New check confirmed in 4 passing runs

A declined card never creates an order. Add it as a check?

A paid order always produces an invoice

AcceptDismiss
Route change from the crawl

The checkout form lost its “company name” field. Still a promise?

Invoices carry the billing details

AcceptDismiss

Illustrative example.

Decision Board

Let AI discover more. Keep your team in control.

Everything Qarmiq proposes waits here, one question at a time, for someone to accept or dismiss: learnings, new checks, route changes and merges.

  • Proposals change nothing until accepted
  • A dismissed finding is not raised again
  • One question at a time
See Decision Board →
Stability readingWatch
Pass rate96 Regression-free92 Consistency74 Defect-free88 Stable areas90 Coverage83 Freshness95 Effort on risk86
Consistency at 74 and defect-free at 88 are holding the reading back.

Illustrative reading. Bands: Unstable below 70, Watch to 90, Stable above.

Insights

Know the state of your product, not just the state of your tests.

A radar of eight measured signals gives a Stable, Watch or Unstable verdict, set by a rule rather than an average: five signals measure stability, and three show how much evidence the reading rests on. It covers the last 60 days.

  • Eight signals from real runs
  • An executive summary and a stabilization plan
  • Defect trend, coverage map and risk heatmap
See Insights →
Project assistantZoe
Can we ship the new checkout on Friday?
Not yet. Checkout is on Watch. In the last run, 41 of 44 checks passed. The 3 that failed are two open defects:
• Declined card shows no error on Safari (high)
• Invoice total is a cent off on discounted orders (medium)
Both break “A paid order always produces an invoice.”
What is holding the overall reading back?
Two signals: consistency (74) and defect-free (88). Consistency is low because 3 checks in sign-in pass on one run and fail on the next. Everything else is healthy.
Draft a defect for the Safari problem.
Drafted. Nothing is submitted until you review it.
Draft defect · Declined card shows no error on Safari
Expected: a clear error. Seen: the page stays silent. Review and submit

Illustrative example.

Zoe

Ask your product quality data a question.

Zoe is an in-app assistant that answers questions about your project’s runs, scenarios and quality data. She can draft a defect or a run for a person to submit, but never changes anything herself.

  • Answers from your project’s own data
  • Drafts a defect or run for you to submit
  • Never changes anything herself
See Zoe →

Bring stability into your development workflow.

Connect supported AI coding tools through MCP so developers and agents can read findings, understand existing checks, and work with the product context before making changes.

you    Add coverage for refunds after a partial shipment

agent  orient            what is this project?
agent  find “refund”     1 scenario, 2 open defects
agent  explain SHOP-D1  evidence and developer brief
agent  author_scenario  planned, awaiting confirmation

you    confirm
agent  author_scenario  written, and logged

Illustrative example. Writes are planned, confirmed and logged.

MCP server

Give your coding agent the context to build without breaking what works.

Qarmiq’s MCP server lets AI coding tools read project and defect context and write scenarios and checks, with an API key. Every write is planned, confirmed and logged.

  • Read context, suggest scenarios, file defects, create runs
  • Nothing can be deleted through it
  • Keys scoped to read, authoring or execution
See MCP server →
One product

A growing body of verified knowledge.

Every verified behavior, finding, and accepted learning contributes to a more useful picture of how your product should work. Qarmiq helps your team build on that knowledge rather than repeatedly starting from scratch.

One record per scenario

Verdicts, scripts, defects and decisions all attach to the scenario they came from, whoever or whatever produced them.

See how it works →

No self-grading

A separate judge rules from cited evidence, and a result with nothing to show for it is inconclusive. Not even Qarmiq’s own AI grades its own work.

Why a verdict can be trusted →

What passed stays passing

A passing test becomes a verified script that goes first next time, and repairs itself when the product still works. The suite gets cheaper to keep, not dearer.

How regression stays accurate →

See how the pieces work together.

We’ll walk you through Qarmiq using a product scenario, from defining the expected behavior to detecting an issue and understanding the evidence.

See Qarmiq in action See pricing