Python and phyphox

Introduction

Phyphox XML and VIM

Activate the matchit.vim macro :runtime macros/matchit.vim and set the filetype :set filetype=html (or xml), and now % swaps between the tags.

The data is located in <data-containers> blocks and <container> tag contains the data of each measurements, but is in init tag, as shown below: <container size="0" init="1.009771156E1,1.012672424E1,1.004746246E1,1.009292603E1,1.024636555E1,1.024875832E1,1.015065289E1,1.007617569E1,1.012193871E1,1.011954594E1,9.999606133E0,9.978070259E0,1.0059426. Thus, we need to read the init part only.

Load the acceleration data

The simplest

import pandas as pd
df = pd.read_xml( filename )

does not work if the XML file is too large. Use lxml from etree instead:

from lxml import etree

def parse_lxml_large_xml(source, a):
    for event, elem in etree.iterparse(source, events=('end',)):
        if elem.tag == "container" and elem.text == a:
            #Split the string and convert to float
            A = elem.text
            B = elem.attrib
            BB = B.get('init').split(",")
            data = np.empty( (len( BB  ), 1 ) )
            for i, d in enumerate( BB ):
                data[i,0] = float(d)
            elem.clear()  # Free memory
    return data

and use it as

accX = parse_lxml_large_xml( "nousu_waw_tll_edited.phyphox", "accX" )
accY = parse_lxml_large_xml( "nousu_waw_tll_edited.phyphox", "accY" )
accZ = parse_lxml_large_xml( "nousu_waw_tll_edited.phyphox", "accZ" )
acc = parse_lxml_large_xml( "nousu_waw_tll_edited.phyphox", "acc" )
time = parse_lxml_large_xml( "nousu_waw_tll_edited.phyphox", "acc_time" )

A = np.column_stack( [time, accX, accY, accZ, acc]  )
df = pd.DataFrame( A, columns=['time', 'ax', 'ay', 'az', 'a'] )

We can load any other data similarly, too. Just change the the names in function calls.