Optimizing machine learning models through data labeling for a B2B software company

The Problem

Experiencing a 33% increase in quarter-over-quarter content reviews related to phishing and other scams targeting its CRM platform, a multinational software company needed to address its backlog of policy violations. Highspring and three other firms engaged in a competitive data “bake-off” to showcase their expertise in data labeling, vying for the chance to become the company’s outsourced business partner. Highspring prevailed and was selected as the partner to deliver high-quality data labeling results while minimizing the onboarding of resources and upfront time investment.

The Solution

Highspring quickly engaged a dedicated team, including quality assurance specialists to establish quality calibration layers. Initial training sessions and ongoing syncs were led by Highspring to ensure alignment on policy guidelines, documentation requirements and quality expectations. The team dedicated 55 hours to the review process and 12 hours to quality assurance efforts. Equipped with proficiency in five languages, the review team handled nearly all reviews. For unsupported languages, Google Translate was leveraged.

Our Impact

Within four days, the team successfully labeled 2,000 items in 13 native languages and various formats. Highspring delivered all reviewed items in a ZIP file as images and developed a new review process and interface. Furthermore, 22% of the items underwent an additional quality assurance testing review, uncovering 2% of false positives that needed correction. This success enabled the review team to align their training with active reviews. Highspring provided the organization with a comprehensive recap, including relevant metrics, actionable insights and suggested policy clarifications, enabling the enhancement of machine learning modeling for the organization.