Python and phyphox

Introduction

Phyphox XML and VIM

Activate the matchit.vim macro :runtime macros/matchit.vim and set the filetype :set filetype=html (or xml), and now % swaps between the tags.

The data is located in <data-containers> blocks and <container> tag contains the data of each measurements, but is in init tag, as shown below: <container size="0" init="1.009771156E1,1.012672424E1,1.004746246E1,1.009292603E1,1.024636555E1,1.024875832E1,1.015065289E1,1.007617569E1,1.012193871E1,1.011954594E1,9.999606133E0,9.978070259E0,1.0059426. Thus, we need to read the init part only.

Load the acceleration data

The simplest

import pandas as pd
df = pd.read_xml( filename )

does not work if the XML file is too large. Use lxml from etree instead:

from lxml import etree

def parse_lxml_large_xml(source, a):
    for event, elem in etree.iterparse(source, events=('end',)):
        if elem.tag == "container" and elem.text == a:
            #Split the string and convert to float
            A = elem.text
            B = elem.attrib
            BB = B.get('init').split(",")
            data = np.empty( (len( BB  ), 1 ) )
            for i, d in enumerate( BB ):
                data[i,0] = float(d)
            elem.clear()  # Free memory
    return data

and use it as

accX = parse_lxml_large_xml( "nousu_waw_tll_edited.phyphox", "accX" )
accY = parse_lxml_large_xml( "nousu_waw_tll_edited.phyphox", "accY" )
accZ = parse_lxml_large_xml( "nousu_waw_tll_edited.phyphox", "accZ" )
acc = parse_lxml_large_xml( "nousu_waw_tll_edited.phyphox", "acc" )
time = parse_lxml_large_xml( "nousu_waw_tll_edited.phyphox", "acc_time" )

A = np.column_stack( [time, accX, accY, accZ, acc]  )
df = pd.DataFrame( A, columns=['time', 'ax', 'ay', 'az', 'a'] )
df.set_index('time', inplace=True)

The last line changes the time column to be the index column, and the inplace keyword replaces that data directly into df variable.

We can load any other data similarly, too. Just change the the names in function calls.