Python: Parse Log File For Pairs Of Lines

I have a log file that I need to parse for specific events. The problem is the data I need comes from pairs of event entries that each hold pieces of the data needed. For instance

Solution 1:

Reading the entire file twice is absolutely excessive. Instead, keep track of what you have done previously while traversing the file.

seen_test = False# state variable for keeping track of what you have done
init_person = None# note snake_case variable convention pro headlessCamelCasewithopen('data.log', 'r') as f:
    for lineno, line inenumerate(f, start=1):
        if'event:type=test,'in line:
            if seen_test:
                raise ValueError(
                    'line %i: type=test without test2: %s' % (
                        lineno, line))
            init_person = line.split('initiator=')[1].split(',')[0]
            seen_test = Trueelif'event:type=test2'in line:
            if seen_test:
                # ... do whatever you want with init_person# maybe something like
                result = line.rstrip('\n').split(',')
                print('Test by %s got results %s' % (init_person, result[1:]))
            else:
                raise ValueError(
                    'line %i: type=test2 without test: %s' % (
                        lineno, line))
            seen_test = False

The enumerate is just to get a useful line number into the error message when there is a failure; if you are sure that the file is always well-formatted, maybe take that out.

This will still fail if the type=test line doesn't contain initiator= but we have no idea what would be useful to do in that scenario so I'm not trying to tackle that.

Demo: https://repl.it/repls/OverdueFruitfulComputergames#main.py

Solution 2:

This should do what you want :

import re

f = open('data.log', 'r')
lines = f.readlines()
results = {}
for line in lines:
    if"Scripting:"in line.strip():
        res = dict(re.findall(r"([^= ]+)=(.+?),", line.strip(), re.DOTALL))
        # if the eventtype ends with `2` and match an existing key in results, update dataif res['event:type'][-1] == '2'and res['event:type'][:-1] in results:
            results[res['event:type'][:-1]].update(res)
        else:
            results[res['event:type']] = res

print(results)

# {'test': {'event:type': 'test2', 'initiator': 'Abe Lincoln', 'place': 'Washinton', 't': '25277.04', 'type': 'comment'}}