Python string parsing tutorial for string to sequence

Python offers a variety of options for working with strings. In this blog post, we will first introduce the basics of Python string parsing and explain the difference between string, indexing and string sequence.

I have already used my knowledge in 2025 to implement a tool that we called Quick Extract. It also helps beginners to create a simple Document API with a few sample documents in order to parse new documents.

String indexing

In Python, a String a sequence of characters. These characters can be accessed via an index, which is used for 0 begins. This way you can access the first, second and third character of a string:

text = "Python"
print(text[0])  # Gibt 'P' aus
print(text[1])  # Gibt 'y' aus
print(text[2])  # Gibt 't' aus

You can also use negative indexes to access characters from the end of the string:

print(text[-1])  # Gibt 'n' aus
print(text[-2])  # Gibt 'o' aus
print(text[-3])  # Gibt 'h' aus

String sequence

A string sequence is a part of a string specified by a start and end index. In Python, you can use the so-called slicing syntax to access parts of a string:

pythonCopy codetext = "Python"
print(text[0:3])  # Gibt 'Pyt' aus
print(text[1:4])  # Gibt 'yth' aus
print(text[:2])   # Gibt 'Py' aus (ohne Anfangsindex entspricht es Index 0)
print(text[3:])   # Gibt 'hon' aus (ohne Endindex entspricht es Index + 1)

The slicing syntax also works with negative indices:

print(text[-3:-1])  # Gibt 'ho' aus
print(text[:-2])    # Gibt 'Pyth' aus

To summarize the differences between string indexing and string sequence: String indexing refers to accessing a single character in a string, while string sequence provides access to a portion of the string.

Use case for split method in Python

Version numbers and commit hashes

In many situations, such as working with software projects, we often need to extract information from combined strings. As an example, let's take a string that contains a version number and a commit hash: "2.7.0_bf4fda703454". To separate this information, we can use the split()-method in Python.

version_hash = "2.7.0_bf4fda703454"
version_hash_list = version_hash.split("_")

The result is a list of strings that looks like this:

['2.7.0', 'bf4fda703454']

Here the string is separated at each underscore. If you want the separation to occur only after the first underscore, you can use the method split() with the optional parameter 1 use

version_hash.split("_", 1)

If you are sure that the string contains an underscore, you can even unwrap the left and right parts into separate variables:

lhs, rhs = version_hash.split("_", 1)
print(lhs)  # Gibt '2.7.0' aus
print(rhs)  # Gibt 'bf4fda703454' aus

An alternative to the split()-method is the use of partition(). The usage is similar to the last example, except that here three components are returned instead of two. The main advantage of this method is that it does not fail if the string does not contain the separator.

In general, the split()-method provides a convenient way to split and process combined information in Python. In our example, we could easily separate the version number and the commit hash.

Python string parsing with regular expressions (regex)

Decompose text with named regex groups in Python and weigh advantages and disadvantages against traditional functions such as Python string indexing, sequencing or split method

When we have complex strings that need to be decomposed due to patterns or difficult punctuation, regular expressions can help recognize and decompose these patterns.

Here is a code example of string parsing with named regex groups in Python:

import re
# Beispieltext
text = "Max Mustermann, Alter: 30 Jahre, Stadt: Berlin"
# Regulärer Ausdruck mit benannten Gruppen
pattern_text = r'(?P<name>[A-Za-zäöüÄÖÜß\s]+),\s+Alter:\s+(?P<age>\d+)\s+Jahre,\s+Stadt:\s+(?P<city>[A-Za-zäöüÄÖÜß\s]+)'
# Kompilieren des regulären Ausdrucks
pattern = re.compile(pattern_text)
# Suche nach Übereinstimmungen im Text
match = pattern.match(text)
# Überprüfen, ob eine Übereinstimmung gefunden wurde
if match:
    # Extrahieren der benannten Gruppen
    name = match.group('name')
    age = match.group('age')
    city = match.group('city')
    # Ausgabe der extrahierten Informationen
    print(f"Name: {name}")
    print(f"Alter: {age}")
    print(f"Stadt: {city}")
else:
    print("Keine Übereinstimmung gefunden.")

In this example, we have a sample text with a person's name, age, and city. We use a regular expression with named groups to extract the relevant information from the text. The groups are named as follows:

• name - for the name of the person
• age - for the age of the person
• city - for the city in which the person lives

After the regular expression is compiled, we look for matches in the text. If a match is found, we extract the named groups and output the extracted information.

Benefits

1. Flexibility - Regular expressions offer a high degree of flexibility in text analysis and can recognize complex patterns.
2. Compact - Regular expressions are often shorter and more concise than conventional methods.
3. Named groups - With named groups we can easily identify and extract parts of the pattern.

Disadvantages

1. Learning curve - Regular expressions have a steeper learning curve and are often more difficult to read and understand than conventional methods.
2. Limited applicability - Regular expressions are not suitable for deeply nested documents such as HTML, XML or JSON.

Python string indexing, sequencing or split method: Conventional functions such as Python string indexing, sequencing or split method are well suited for simple text analysis tasks where the structure of the string is clear and simple.

Benefits

1. Simplicity - Traditional methods are easy to understand and implement.
2. Readability - The code is easier to read and understand for developers who are familiar with Python.

Disadvantages

1. Limited flexibility - These methods offer less flexibility when analyzing complex text patterns.
2. Longer code - Code can become longer and more confusing when trying to recognize and decompose complex patterns.

Automation Python Regex

In this example, we will show how to use the Konfuzio SDK to automate the creation of regex. We use the SDK to create a custom regex tokenizer for the label "ContractDate" to create. More information is available in the technical documentation and in a more blog articles to find.

from konfuzio_sdk.data import Project
from konfuzio_sdk.tokenizer.regex import RegexTokenizer
from konfuzio_sdk.tokenizer.base import ListTokenizer
# Ersetzen Sie YOUR_PROJECT_ID und YOUR_CATEGORY_ID durch die entsprechenden IDs Ihres Projekts und Ihrer Kategorie.
my_project = Project(id_=YOUR_PROJECT_ID)
category = my_project.get_category_by_id(id_=YOUR_CATEGORY_ID)
tokenizer = ListTokenizer(tokenizers=[])
label = my_project.get_label_by_name("ContractDate")
# Finden von Regex-Ausdrücken, die mit Vorkommen des Labels "ContractDate"match
for regex in label.find_regex(category=category):
    regex_tokenizer = RegexTokenizer(regex=regex)
    tokenizer.tokenizers.append(regex_tokenizer)
# Use the created regex tokenizer to create annotations for each matching string in a document.
# Replace YOUR_DOCUMENT_ID with the corresponding ID of your document.
document = my_project.get_document_by_id(YOUR_DOCUMENT_ID)
tokenizer.tokenize(document)

In this code snippet:

1. Let's import the required classes and functions from the Konfuzio SDK.
2. Let's create a Project-object by specifying the project ID.
3. Do we get a Category-object by specifying the category ID.
4. Let's create an empty ListTokenizer.
5. Do we get the Label-object for the "ContractDate„.
6. Let's look for regex expressions that start with occurrences of the label "ContractDate" in the training data category match.
7. Let's add each regex expression found as a RegexTokenizer the ListTokenizer added.
8. Let's use the created regex tokenizer to create annotations for each matching string in a document. To do this, we specify the document ID and call the tokenize()-method of the tokenizer.

After the tokenizer is created, you can use it to automatically generate regex expressions for the "ContractDate" in your documents and create annotations for them.