Human-in-the-loop validation system for AI-generated information against source documents.

verify_me Block

The verify_me block is RAPFlow's human-in-the-loop validation system that enables human reviewers to validate AI-generated information against source documents in an interactive viewer interface.

Overview

The verify_me block is a critical component for ensuring accuracy and quality in AI-powered document processing workflows. It provides a secure, user-friendly interface where human reviewers can:

View AI-generated results alongside the original source documents
Validate extracted information for accuracy
Make corrections or adjustments as needed
Approve validated data for downstream processing

How It Works

AI Processing: The block receives AI-generated results (extracted data, classifications, etc.) along with the source documents
Human Review Interface: Presents the information in a viewer where human reviewers can examine both the AI results and original documents
Validation Process: Reviewers can accept, reject, or modify the AI-generated information
Downstream Integration: Once validated, the approved data is passed to subsequent blocks in the workflow

Configuration Options

Input Requirements

msg.payload.ai_results: The AI-generated information to be validated
msg.payload.source_documents: The original documents used for AI processing
msg.payload.validation_config: (Optional) Configuration for the validation interface

Output Properties

msg.payload.validated_data: The human-approved information
msg.payload.validation_status: Status of the validation (approved, rejected, modified)
msg.payload.reviewer_notes: (Optional) Comments or notes from the human reviewer
msg.payload.validation_timestamp: When the validation was completed

Use Cases

Document Data Extraction Validation

Validate AI-extracted data from invoices, contracts, or forms:

OCR → AI Extraction → verify_me → Database Storage

Content Classification Review

Review AI-generated document classifications:

Document Processing → AI Classification → verify_me → Document Routing

Quality Assurance Workflows

Ensure accuracy in critical business processes:

AI Processing → verify_me → Manual Review → Approved Processing

Integration Patterns

Batch Validation

Process multiple documents with human oversight:

{
  "payload": {
    "ai_results": [
      {
        "document_id": "doc_001",
        "extracted_fields": {
          "invoice_number": "INV-2024-001",
          "total_amount": "$1,500.00",
          "vendor": "ABC Corp"
        }
      }
    ],
    "source_documents": ["/path/to/invoice_001.pdf"],
    "validation_config": {
      "required_fields": ["invoice_number", "total_amount"],
      "confidence_threshold": 0.8
    }
  }
}

Real-time Validation

Validate individual documents as they're processed:

{
  "payload": {
    "ai_results": {
      "classification": "invoice",
      "confidence": 0.95,
      "extracted_data": {
        "amount": "$2,500.00",
        "date": "2024-01-15"
      }
    },
    "source_documents": ["/path/to/document.pdf"],
    "validation_config": {
      "auto_approve_threshold": 0.98,
      "require_human_review": true
    }
  }
}

Validation Interface Features

Document Viewer

Side-by-side comparison of AI results and source documents
Zoom, pan, and annotation capabilities
Highlighting of extracted regions
Multi-page document support

Validation Controls

Accept/Reject buttons for quick decisions
Edit fields for direct data modification
Confidence score display
Batch approval for similar items

Audit Trail

Complete history of validation decisions
Reviewer identification and timestamps
Change tracking and version control
Compliance reporting capabilities

Best Practices

Workflow Design

Strategic Placement: Position verify_me after AI processing but before critical business logic
Confidence Thresholds: Use AI confidence scores to determine when human review is necessary
Batch Processing: Group similar validations to improve reviewer efficiency
Escalation Paths: Define clear processes for handling rejected or disputed validations

Quality Control

Training: Ensure reviewers understand the AI system's capabilities and limitations
Consistency: Use standardized validation criteria across similar document types
Feedback Loop: Incorporate reviewer feedback to improve AI model performance
Performance Monitoring: Track validation times and accuracy rates

Security and Compliance

Access Control: Implement proper authentication and authorization for reviewers
Data Privacy: Ensure sensitive documents are handled according to compliance requirements
Audit Logging: Maintain comprehensive logs of all validation activities
Retention Policies: Define how long validation data should be retained

Configuration Examples

Basic Validation Setup

{
  "validation_config": {
    "interface_type": "standard",
    "auto_approve_threshold": 0.95,
    "require_reviewer_notes": false,
    "timeout_minutes": 30
  }
}

Advanced Validation with Custom Rules

{
  "validation_config": {
    "interface_type": "advanced",
    "custom_validation_rules": [
      {
        "field": "amount",
        "validation_type": "numeric_range",
        "min_value": 0,
        "max_value": 1000000
      },
      {
        "field": "date",
        "validation_type": "date_format",
        "expected_format": "YYYY-MM-DD"
      }
    ],
    "require_reviewer_notes": true,
    "escalation_threshold": 0.7
  }
}

Tips

Start with High-Confidence Items: Begin validation with AI results that have high confidence scores
Use Batch Operations: Group similar validations to improve efficiency
Provide Context: Include relevant background information to help reviewers make informed decisions
Monitor Performance: Track validation times and accuracy to optimize the process
Continuous Improvement: Use validation feedback to improve AI model performance

LLM Query - For AI processing that may need validation
Document Understander - For document analysis that requires human review
OCR - For text extraction that may need validation
write_data - For storing validated results

verify_me

On this page