Manage campaigns

AI RED TEAM

This article is meant for AI Red Team users.

ROLES AND PERMISSIONS

To complete the tasks described in this section, make sure you have the required permissions.

• Learn more about permissions in AI Security

Use AI Security to manage your campaigns.

• Learn more about campaigns

Get a list of campaigns

You can list all campaigns available in the system.

To list campaigns:

1. Add your token to the following code:

   from calypsoai import CalypsoAI
   import json
   
   # Define the URL and token for CalypsoAI
   CALYPSOAI_URL = "https://www.us1.calypsoai.app"
   CALYPSOAI_TOKEN = "ADD-YOUR-TOKEN-HERE"
   
   # Initialize the CalypsoAI client
   cai = CalypsoAI(url=CALYPSOAI_URL, token=CALYPSOAI_TOKEN)
   
   # Get a list of all campaigns
   campaigns = [campaign.model_dump_json() for campaign in cai.campaigns.iterate()]
   
   # Print the response
   print(campaigns)

2. Run the script.

3. Review the response.

   The following response sample is a simplified successful response that highlights key details for this request.

   [
     {
       "attacks": [
         {
           "converters": [
             "base64"
           ],
           "pack": "2025-05",
           "severity": 1,
           "technique": "static_content",
           "vector": "fictional_context_change"
         },
         (...)
       ],
       "description": "Test Run",
       "id": "019744f1-5121-70d2-b315-fc31f3d62129",
       "name": "Test run"
     },
     (...)
   ]

   The response includes the following key parameters:

   • attacks: A list of the attacks included in each campaign.
   • attacks > converters: A list of converters used by each attack.
   • attacks > severity: The severity of a successful attack.
   • attacks > technique: The technique used by each attack. This value defines the attack type. In this case, static_content is a signature attack.
   • attacks > vector: The vector used by the attack. This value defines the attack vector, for example, fictional context change.
   • name: The name of the attack campaign.
   To view the full response, select this link.

   [
       {
           "attacks": [
               {
                   "converters": [
                       "base64",
                       "leetspeak",
                       "unicode_confusable",
                       "caesar",
                       "repeat_token",
                       "single_character"
                   ],
                   "pack": "2025-05",
                   "severity": 1,
                   "technique": "static_content",
                   "vector": "fictional_context_change"
               },
               (...)
           ],
           "description": "Test Run",
           "id": "019744f1-5121-70d2-b315-fc31f3d62129",
           "name": "Test run",
           "orgId": null,
           "vendored": false
       },
       (...)
   ]

Update a campaign

After you create a campaign, you can update it with the campaign ID.

You can update the following campaign properties:

• name
• description
• List of attacks

To update a campaign:

1. Edit the following sample.

   from calypsoai import CalypsoAI
   from calypsoai.datatypes import StaticContentAttack, DynamicSingleTurnContentAttack,
   DynamicMultiTurnContentAttack, OperationalAttack, Vector1, Vector, PromptConverter,
   OperationalAttackVector
   
   # Define the URL and token for CalypsoAI
   CALYPSOAI_URL = "https://www.us1.calypsoai.app"
   CALYPSOAI_TOKEN = "ADD-YOUR-TOKEN-HERE"
   
   # Initialize the CalypsoAI client
   cai = CalypsoAI(url=CALYPSOAI_URL, token=CALYPSOAI_TOKEN)
   
   # Update the campaign
   cai.campaigns.update(
     campaign='ADD-YOUR-CAMPAIGN-ID-HERE',
     name="ADD-YOUR-NEW-CAMPAIGN-NAME-HERE",
     description="ADD-YOUR-NEW-CAMPAIGN-DESCRIPTION-HERE",
     attacks=[
       StaticContentAttack(vector=Vector1.DAN, converters=[PromptConverter.BASE64]),
       DynamicSingleTurnContentAttack(vector=Vector1.FICTIONAL_CONTEXT_CHANGE,
   converters=[PromptConverter.CAESAR]),
       DynamicMultiTurnContentAttack(vector=Vector.CRESCENDO,
   converters=[PromptConverter.LEETSPEAK]),
       OperationalAttack(vector=OperationalAttackVector.TLS),
     ]
   )

   • Add your token value.

   • In cai.campaigns.update, provide:

     • The ID of the campaign you want to update.
     • A new name for the campaign.
     • An updated description.
     • A list of attacks.

     UPDATING A CAMPAIGN

     The name, description, and attacks properties are optional. Include only the properties you want to update.

2. Run the script.

Delete a campaign

If you no longer need a campaign, you can delete it by providing the campaign ID.

To delete a campaign:

1. Add your token and campaign ID values to the following sample:

   from calypsoai import CalypsoAI
   
   # Define the URL and token for CalypsoAI
   CALYPSOAI_URL = "https://www.us1.calypsoai.app"
   CALYPSOAI_TOKEN = "ADD-YOUR-TOKEN-HERE"
   
   # Initialize the CalypsoAI client
   cai = CalypsoAI(url=CALYPSOAI_URL, token=CALYPSOAI_TOKEN)
   
   # Delete the campaign
   cai.campaigns.delete(campaign='ADD-YOUR-CAMPAIGN-ID-HERE')

2. Run the script.

---

Retry failed reports

AI Security now supports report retries for most Red Team reports.

• The maximum number of retries is 10.

You may see a failed report if the number the report includes >= 20% errored attack prompts. Errors happen for one or more of the following reasons:

• Expired API key
• 4xx HTTP error
• 5xx HTTP error
• Timeout
• DNS failure

To retry a failed report:

• Log in to the AI Security UI
• Select AI Red Team > Reports
• If retries are allowed for the report, select Retry in the Action column, and confirm.

Retry reports at the API

You can set up retries at the API. The endpoint is campaign-runs.

If you want to retry a specific report, you'll first need the report ID. You can then start a retry with the following REST call:

POST /backend/v1/campaign-runs/{campaignRunId}/retry

Test system prompt attacks

Turn a system prompt into a ready-made set of attacks. You don't need to write intents by hand.

Provide the system prompt you want to test. F5 AI Security automatically creates custom intents that target the specific rules in that prompt. Add those custom intents to your campaign, and F5 AI Security automatically creates single-turn and agentic resistance attacks using the attack vectors you select.

Because the attacks come from the real system prompt behind your model, this is true white-box testing. The attacks target your actual guardrails, not generic failure modes.

• Faster. Go from a system prompt to campaign-ready intents in one step.
• More targeted. Intents attack the exact rules your prompt lays out, surfacing the violations that matter to your deployment.
• Requires less expertise. You don't need deep red-teaming knowledge to write effective intents.

INFO

System prompts are limited to 50,000 characters.

Review report run time

Reports and the campaign runs API now show how long each report took to complete, along with the timestamp of when it last finished.

Run time is the cumulative processing time for a report across all retries. It gives you an accurate picture of the total work behind each run. Use run time to track performance and spot reports that take longer than expected.

Availability:

• Run time is available on new reports; it's not backward compatible.
• Start and end timestamps have been added to all reports, including existing ones.

References

For more information, see:

• Run all attacks campaign
• Get started with AI Red Team