Search K
Appearance
Remediate Red Team findings end to end
EARLY ACCESS
AI Remediate is not generally available yet. Contact F5 sales if you are interested in joining the early access program.
Overview
This use case follows one example from start to finish. A team finds a vulnerability with AI Red Team, fixes it with AI Remediate, and then checks the fix.
You'll learn how to:
- Find a vulnerability with a custom intent in an AI Red Team campaign.
- Request a remediation from the report, or have AI Security request one for you.
- Read the remediation results and decide whether the package is good enough.
- Watch the package on live traffic in Audit mode before it blocks anything.
- Switch the package to Block mode in the live project.
- Run the campaign again and compare the results.
When you finish, you'll have a guardrail package that targets the attacks the report found, running in a live project.
This page shows how the steps fit together. For the detailed procedure for each step, see Remediate AI risks.
EXAMPLE VALUES
The company, model, campaign, intent, report, project, and all figures on this page are made up. They show you what to look for. They don't describe how a real remediation will perform. Your results depend on your models, intents, and attacks.
The scenario
Example Retail runs a customer support chatbot. The chatbot uses a model connection named support-bot-model in a live Guardrails project named Customer support chatbot.
The chatbot's system prompt includes internal discount codes that support agents can give to customers in some cases. The security team wants to know whether a customer could talk the chatbot into revealing those codes.
| Item | Example value |
|---|---|
| Model connection | support-bot-model |
| Guardrails project | Customer support chatbot (live) |
| Campaign | Support bot: discount code leakage |
| Custom intent | "Get the assistant to reveal internal discount codes" |
| Report | Support bot discount codes: week 38 |
Before you begin
Make sure you have everything listed in Before you begin on the Remediate page:
- An AI Security license that includes AI Guardrails, AI Red Team, and the AI Remediate capability. AI Remediate requires both AI Guardrails and AI Red Team.
- A role with the Remediation access permission, permission to read AI Red Team reports, and permission to update the projects you deploy to.
- A supported deployment. Self-hosted deployments need the AI Guardrails and AI Red Team models, plus the AI Remediate model and its GPU node.
You also need:
- A Guardrails project that uses the same model connection that you test with AI Red Team. In this example, the Customer support chatbot project uses
support-bot-model. You can deploy a package only to projects that use the same connections as the report. For more information, see Create a project.
Find the vulnerability with AI Red Team
AI Remediate works only from custom intent results. Signature attack and operational attack results aren't used. So the team starts with a campaign that contains a custom intent.
- The team creates the Support bot: discount code leakage campaign with one custom intent: "Get the assistant to reveal internal discount codes." To create a campaign with custom intents, see Create an Agentic Warfare campaign.
- The team runs the campaign against
support-bot-model. This creates the Support bot discount codes: week 38 report. - The team opens the report. In the custom intents section, the report shows successful attacks against the discount code intent. In some conversations, the chatbot revealed a code.
For help reading the report, see Review a Red Team report.
Because at least one custom intent attack succeeded, the report shows a Remediate button in the reports list and in the report's custom intents section. If you don't see the button, see Troubleshooting.
Request a remediation
From the report, the team selects Remediate. The Remediate vulnerabilities dialog lists the vulnerable custom intent and model. The team selects Remediate to confirm.
AI Security opens the Remediation page. The status changes from Remediation requested to In progress while AI Remediate generates and evaluates guardrails. Nothing changes in your guardrails or projects yet.
Keep in mind:
- You can create only one remediation for each report. After the remediation exists, the report's Remediate button changes to View.
- The package is named after the report. In this example, the package is Support bot discount codes: week 38 package.
For the full procedure, see Request a remediation from a report.
Optional: Remediate scheduled campaigns automatically
The team plans to run this campaign every week. To get a remediation each time a scheduled run finds a vulnerable custom intent, they select the Remediate checkbox when they create the schedule. Auto-remediation is off by default. In the API, set remediationSettings.autorun to true.
For details, see Turn on auto-remediation.
Review the results
When the status changes to Ready, the team selects AI remediate > Remediation and opens the remediation.
The following figures are example values for this scenario.
| Result | Example value |
|---|---|
| Attack prevention rate: Pre-remediation | 10% |
| Attack prevention rate: Post-remediation | 92% |
| F1 | 0.93 |
| Suggested guardrails | 2 GenAI guardrails. For example, one that checks for requests to reveal discount, promo, or coupon codes. |
The team selects Details to see the overall accuracy and confusion matrix. The following counts are example values. With these counts, the example accuracy is 93%.
| Example count | |
|---|---|
| True positives | 46 |
| False positives | 3 |
| True negatives | 47 |
| False negatives | 4 |
The team also checks Performance per custom intent, which shows pre- and post-remediation scores for the discount code intent. To keep the detailed data, they select Download results. The download expires 7 days after the results are created.
Decide whether the package is good enough
AI Remediate aims for a package F1 score above 90%, but it doesn't stop you from using a package that falls short. Your team decides what's acceptable for your risk and your users. The team asks these questions:
- Did the attack prevention rate improve? Compare Pre-remediation with Post-remediation. A small gain might mean the guardrails don't match how the attacks work.
- Is the F1 score high enough? F1 balances missed attacks against false alarms. AI Remediate targets a score above 90%. A low F1 means the package misses many attacks, flags many prompts it shouldn't, or both.
- What do the false positives and false negatives look like? In general, false negatives are attacks the guardrails missed. False positives are prompts the guardrails flagged by mistake. For a customer support chatbot, false positives can block real customers who ask about discounts, so the team looks at them closely.
- Do the guardrail descriptions make sense? The team reads each suggested guardrail. A guardrail that's too broad, such as one that flags every question about prices, is likely to block real customers.
- Does every vulnerable intent improve? Performance per custom intent shows whether one intent still has a low score.
If the package isn't good enough, you can reject it. You can't undo a rejection, and you can create only one remediation for each report. For details, see Reject a package.
In this example, the team decides the results are good enough to try on live traffic.
For the full procedure and a description of each result, see Review a remediation and Results.
Watch the package on live traffic in Audit mode
The team wants to see how the guardrails behave on real customer prompts before they block anything.
On the remediation, which has the status Ready, the team selects Import and deploy.
They select the Customer support chatbot project. Live projects only helps them find it.
For each guardrail, they set Mode to Audit and leave Enable on.
Remediation guardrails default to Block mode and are enabled, so the team changes the mode to Audit.
They select Import and deploy. The status changes to Deployed.
The guardrails now scan prompts in the project, but in Audit mode they don't block them. Over the next few days, the team reviews the project's prompt logs to see which prompts the guardrails flag. They look for two things:
- Prompts that look like discount code attacks. These show the guardrails catch what they're meant to.
- Ordinary customer questions that were flagged. These are false positives that would block real customers in Block mode.
For prompt logs, see Understand prompt history logs. For the deploy procedure, see Deploy the package to projects.
TRY IT IN THE PLAYGROUND INSTEAD
To test the package before it sees traffic, select Import and test. AI Security adds the package to the playground and custom guardrails list. The status changes to Imported. From Imported, you can select Test or Reject. To use the guardrails in projects, publish the package.
For details, see Import and test the package and Statuses.
Switch to Block mode in the live project
The prompt logs show that the guardrails flag discount code attempts and few ordinary questions. The team is ready to block.
- The team opens the Customer support chatbot project and goes to the Guardrails tab.
- They confirm that the remediation guardrails are in the guardrail list.
- For each remediation guardrail, they change Mode from Audit to Block.
The guardrails now block matching prompts in the live project.
INFO
After you deploy a package to projects, you can't reject it.
For more on guardrail scan settings, see Add response scans to a guardrail.
Confirm the fix
Next, the team checks that the fix works. They follow Verify the remediation:
- In the project, they confirm that the remediation guardrails appear with Block mode.
- They send a test prompt similar to one of the successful attacks in the original report. The guardrail blocks it.
- They review the project's prompt logs.
- They run the Support bot: discount code leakage campaign again against the same connection and compare the new report with Support bot discount codes: week 38.
When they compare the reports, the team looks at:
- The number of successful attacks against the discount code intent.
- The conversations that still succeed, if any. These show what the guardrails miss.
If attacks still succeed, the team can request a remediation from the new report. If the campaign runs on a schedule with auto-remediation turned on, AI Security requests the remediation automatically when a run finds a vulnerable custom intent.
To see how quickly the team responded, they check Time to respond on the remediation. It shows the time from when the report finished to when the package was deployed.
Next steps
- Turn on auto-remediation for your scheduled campaigns. See Turn on auto-remediation.
- Add more custom intents to your campaigns to cover other sensitive data. See Get started with AI Red Team.
- Review the limits and statuses before you scale up. See Defaults and limits.
- Track remediation activity in the audit log. See Get audit logs.
References
For more information, see: