Blackening documents - manual and automatic

Many organizations work with confidential documents that contain sensitive information alongside non-sensitive content. The traditional approach of manually redacting these documents is time-consuming, error-prone and does not scale with modern document volumes.

We will show how Konfuzio IPD in combination with Konfuzio Tools provides a powerful solution for automated document redaction that ensures security while enabling efficient information exchange.

Manual blackening

Imagine you work in a company that regularly conducts confidential correspondence with authorities or business partners. A typical example is a letter to the Federal Ministry for Economic Affairs and Climate Protection regarding a secret protection order. This document contains both publicly accessible information and highly sensitive details such as telephone numbers, names, addresses and file numbers.

The manual redaction of such documents poses considerable challenges:

• Time required: For a single document with 22 different confidential areas, manual processing can easily take several hours
• Error-pronenessHuman reviewers may overlook sensitive information or apply redaction rules inconsistently
• ScalabilityIf you multiply this by hundreds of documents per month, it quickly becomes clear that manual processes are not scalable

Automated blackening

Konfuzio IPD transforms the document redaction process through a two-step approach that combines AI-powered content analysis with precise redaction functions. Using our sample document, we show how this process works in practice.

Stage 1: Document analysis

First, the document is securely uploaded to Konfuzio IPD. During processing, the AI analyzes the document and automatically identifies confidential information such as "Musterfirma GmbH", "TEL.-ZENTRALE (040) 3667-1", "Sanddornstraße 8" and "Dr.W.Kr E-230".

The results of the analysis can be accessed via the secure API:

curl -X 'GET' \
  'https://app.konfuzio.com/api/v3/documents/6953893/' \
  -H 'accept: application/json'

The API response contains detailed information about the document, including the automatically recognized annotations with precise coordinates for each sensitive area. For example, the system has recognized that sensitive information is located at position (x: 318, y: 243) with a width of 93 pixels and a height of 9 pixels.

Level 2: Automatic blackening

Once the analysis is complete, the identified areas can be automatically redacted. Konfuzio Tools offers two approaches for this task.

Coordinate-based redaction offers maximum precision. The system uses the exact coordinates of the detected confidential areas and blacks them out in a targeted manner.
Please note where the coordinates originate. Konfuzio IDP sets the origin in the PDF at the bottom left, while Konfuzio Tools sets the origin at the top left. The coordinates can be transferred by simple reformatting.

curl -X 'POST' \
    'http://tools.konfuzio.com:8080/tools/api/v1/security/redact' \
    -H 'accept: */*' \
    -H 'Content-Type: multipart/form-data' \
    -F 'fileInput=@Anlage_43_Seite_2.pdf;type=application/pdf' \
    -F 'redactions=[
      {"x": 318, "y": 243, "width": 93, "height": 9, "page": 1},
      {"x": 72, "y": 254, "width": 85, "height": 8, "page": 1},
      {"x": 313, "y": 259, "width": 117, "height": 8, "page": 1}
    ]' \
    -F 'convertPDFToImage=true' \
    -F 'pageRedactionColor=#000000' \
    -o Anlage_43_Seite_2-geschwärzt.pdf

Text-based redaction offers more flexibility when specific terms need to be redacted in different contexts.

curl -X 'POST' \
    'http://tools.konfuzio.com:8080/tools/api/v1/security/auto-redact' \
    -H 'accept: */*' \
    -H 'Content-Type: multipart/form-data' \
    -F 'fileInput=@Anlage_43_Seite_2.pdf;type=application/pdf' \
    -F 'listOfText=Musterfirma GmbH
        TEL.-ZENTRALE (040) 3667-1
        Sanddornstraße 8' \
    -F 'wholeWordSearch=true' \
    -F 'redactColor=000000' \
    -F 'customPadding=1.3' \
    -F 'convertPDFToImage=true' \
    -o Anlage_43_Seite_2-geschwärzt.pdf

The result is a professionally redacted document in which all confidential information has been replaced by black bars, while the non-sensitive content remains fully legible.

API parameters in detail

The Konfuzio Tools API offers various parameters for fine-tuning the redaction process. This flexibility allows organizations to precisely adapt the redaction to their specific requirements.

Coordinate-based redaction - parameters:

• fileInput - The PDF document to be edited
• redactions - Array with the exact coordinates of the areas to be blackened
• x, y - Position of the blackening area (top left corner)
• width, height - Dimensions of the area in pixels
• page - Page number of the document
• convertPDFToImage - Converts the PDF into an image, which is particularly important for guaranteed blackening
• pageRedactionColor - Blackening color (standard black #000000)

Text-based redaction - parameters:

• fileInput - The PDF document to be edited
• listOfText - Line-separated list of terms to be blacked out
• useRege - Activates regular expressions for more complex search patterns
• wholeWordSearch - Only blacks out whole words, prevents partial word matches
• redactColor - Color of the blackening bars
• customPadding - Additional margin around the blackened text (factor 1.3 = 30% larger)
• convertPDFToImage - Converts the PDF into an image, which is particularly important for guaranteed blackening

These parameters give developers control over every aspect of the blackening process, from the precise positioning to the visual design of the blackened areas.

Measurable advantages

The implementation of Konfuzio IPD in our example scenario shows significant improvements. While the manual redaction of the test document with 22 confidential areas took around 20 minutes, this time is reduced to one minute with Konfuzio. The system consistently identifies all relevant information and redacts it automatically.

API integration allows organizations to seamlessly integrate the redaction process into existing workflows. For example, a company could set up an automated system that monitors new documents from a specific directory, automatically analyzes them and provides redacted versions for forwarding to external partners.

Conclusion

Konfuzio IPD and tools combine to automate document redaction through intelligent content analysis with precise redaction capabilities. Our real-world example shows how organizations can dramatically increase efficiency while maintaining the highest standards of security and accuracy.