Fortune 100 IT software and services company partners with Highspring to strengthen generative AI moderation through adversarial red teaming

The Challenge

A leading global IT software and services company was integrating generative AI into various internal products that led to problematic AI outputs, threatening the brand’s credibility surrounding AI generation and moderation. To protect its users and the brand from misinformation, disinformation and biases, the client sought expertise to identify AI moderation improvements through high-quality prompt testing. 

Our Solution

During a pilot with two additional vendors, Highspring prevailed as the vendor that met tight deadlines and delivered strong results. We initially deployed 6 analysts to generate high-quality, creative AI prompts and to execute a holistic testing process that included LLM prompt engineering, parity testing, and rating. All prompts and results were documented and uploaded for summarization. These reports were provided to the client through detailed project briefs, outlining observed patterns and potential policy violations for the organization to address. 

Additionally, we:

• Implemented testing methodologies grounded in current social media trends and language to simulate everyday user behaviors and communication styles that make assessments more realistic and impactful. 
• Built personas representing a wide range of viewpoints and ideologies to ensure a broad and thorough testing perspective, including that of younger users. 
• Instituted expert incubation workflows, including defining quality rubrics, policy governance frameworks, and knowledge base documentation. 
• Established a prioritization queue for efficient collaboration with product stakeholders. 
• Met or exceeded service level agreements, including adhering to a 2–3-day turnaround on new intel. 
• Provided Red Team Intel Briefs to advise on geopolitical events. 
• Conducted adversarial testing in over 14 languages to generate keywords and sentiment analysis to detect potential risks and to protect the client’s brand. 

Our Impact

Highspring has executed more than 110,000 prompts, averaging over 12,000 per month across 300 projects since the initial pilot phase, identifying and documenting potentially violative content. Analytics detailed in our team’s project briefs alerted the company to gaps in their generative AI content moderation efforts and identified an immediate need for a dedicated team to support their ongoing moderation efforts. As a result, Highspring was awarded an ongoing contract to provide moderation managed services. 

Within 12 months, the client expanded Highspring’s team to over 100 professionals to meet growing business needs. The expanding responsibilities further demonstrate the company’s confidence in Highspring’s ability to plan for the future and deliver high-quality services.