Discussion about development tools and environments, such as STLink, OpenOCD, arm-gcc, Eclipse, Qt Creator and Visual Studio.
no avatar
User

husky

Posts

13

Joined

Sat Sep 05, 2015 1:02 pm

How to decode / parse the tll logfile format ?

by husky » Sat Oct 10, 2015 4:15 pm

digging deeper into the code with my new Sparky2 and ublox M8-N GPS I try to leverage the cool logging
features of taulabs.
I was able to write out tll files and plot data with the interactive python shell, according the the helpful
Wiki page https://github.com/TauLabs/TauLabs/wiki/Python-Logfile-Analysis from peabody124.
Now I'd need a script or program which does the same parsing as the python thing, but writes out the data plus header row
in a flat file manner like an Excel sheet for use with Octave, Excel etc.
Did somebody do this before or is at least the format of the tll file and how this is parsed according to the UAVO xmls documented somewhere ?
User avatar
User

peabody124

Posts

1339

Joined

Tue Feb 18, 2014 12:33 pm

Re: How to decode / parse the tll logfile format ?

by peabody124 » Sun Oct 11, 2015 3:16 am

There isn't an easy way yet. You can actually directly import the data into Octave though using the matlab parser.

make matlab

then run

build/matlab/LogConvert.m path/to/log.tll
no avatar
User

husky

Posts

13

Joined

Sat Sep 05, 2015 1:02 pm

Re: How to decode / parse the tll logfile format ?

by husky » Sun Oct 11, 2015 10:46 am

trying that with Octave 4.0.0.1 on Mac OS 10.10.5 gives me 3 errors:

$ octave LogConvert.m ../../TauLabs-2015-10-10_17-11-31.tll

parse error near line 4319 of file /Users/nocciolina/TauLabs/build/matlab/LogConvert.m

nested functions not implemented in this context

>>> function OPLog2csv(structIn, structName, logfile)
^

parse error near line 4347 of file /Users/nocciolina/TauLabs/build/matlab/LogConvert.m

nested functions not implemented in this context

>>> function crc = compute_crc(data)
^

parse error near line 4354 of file /Users/nocciolina/TauLabs/build/matlab/LogConvert.m

nested functions not implemented in this context

>>> function out=mcolon(inStart, inFinish)
^

error: source: error sourcing file '/Users/nocciolina/TauLabs/build/matlab/LogConvert.m'


According to http://octave.1599824.n4.nabble.com/nested-functions-td4662028.html there are issues with nested functions in Octave despite it should support that since version 3.8. One can somehow make a work around, but you need Octave know-how.
So LogConvert.m only works with Matlab - which I don't have...

What about python/dumplog.py ? Can this be transformed into a standalone app ?
no avatar
User

jihlein

Posts

97

Joined

Thu Nov 13, 2014 11:01 pm

Re: How to decode / parse the tll logfile format ?

by jihlein » Mon Oct 12, 2015 12:19 pm

FWIW, on Windows 7, Octave 4.0.0 correctly parses the tll files with LogConvert.m.
no avatar
User

husky

Posts

13

Joined

Sat Sep 05, 2015 1:02 pm

Re: How to decode / parse the tll logfile format ?

by husky » Tue Oct 13, 2015 9:32 am

@jihlein: Just tried that on Win7 with a fresh, MSI-installed Octave 4.0.0. Same result.

How did you install Octave and how did you run the script ?

i did:
d:\Octave-4.0.0\bin\octave-cli d:\code\TauLabs\build\matlab\LogConvert.m d:\code\TauLabs\TauLabs-2015-10-09_12-06-56.tll
no avatar
User

husky

Posts

13

Joined

Sat Sep 05, 2015 1:02 pm

[solved] Re: How to decode / parse the tll logfile format ?

by husky » Tue Oct 13, 2015 10:18 am

I got it. The way I tried to run the .m file is wrong. It is a function file which first needs to be loaded into Ocatve and then
executed. When double clicking the .m file, Octave opens the GUI and compiles the .m file. On the Command tab you can then invoke it with LogConvert. It then asks for the .tll file name.

Maybe somebody could add here how to run this from command line, if possible.
no avatar
User

jihlein

Posts

97

Joined

Thu Nov 13, 2014 11:01 pm

Re: How to decode / parse the tll logfile format ?

by jihlein » Tue Oct 13, 2015 12:12 pm

I'm running the gui version of Octave.

After navigating to the proper folder, type "LogConvert myData.tll" and it parses out the file.

I should add that I modify line 32 of LogConvert.m to "overo = true" so it parses in the correct format as I was having trouble with command line arguments, both in Octave and Matlab.
no avatar
User

husky

Posts

13

Joined

Sat Sep 05, 2015 1:02 pm

Re: How to decode / parse the tll logfile format ?

by husky » Tue Oct 13, 2015 4:47 pm

for me it is vice versa - setting overo = true I get one error, with overo=false I get none.
For Sparky2 I'm fine, but...

1. I get warnings that git hashes and UAVO hashes do not match. I thought that "make matlab" does patch the right values into LogConvert.m but that seems not to be the case. Do I have to patch those values myself ?

warning: Git hashes do not match. Incorrect file importer.
warning: called from
LogConvert at line 1857 column 2
warning: UAVO hashes do not match. Incorrect file importer.
warning: implicit conversion from numeric to char
wrongSyncByte instances: 3
wrongMessageByte instances: 0

Completed bytes: 46 of 25050
Est. time remaining, 00h:00m:02s
25050 records in 0.43 seconds.



2. I discovered that LogConvert.m can create .csv files, when you set outputType='csv' in line 20. This is the format I'm heading for.
But doing so throws an error:

error: OPLog2csv: A(I,J,...) = X: dimensions mismatch
error: called from
LogConvert>OPLog2csv at line 4337 column 15
LogConvert at line 4194 column 2

I'm trying to debug that in Octave, but any hint what could be wrong is welcomed.
no avatar
User

tracernz

Posts

119

Joined

Mon Apr 06, 2015 7:46 am

Re: How to decode / parse the tll logfile format ?

by tracernz » Wed Oct 14, 2015 12:31 am

Do you have firmware from the same Git commit loaded on your board? If you're using a release version on the board I'd suggest checking out the corresponding tag from the repository and running `make matlab` on that. e.g. For the latest release `git checkout tags/20150922` should get you the Endomorphism sources.
no avatar
User

husky

Posts

13

Joined

Sat Sep 05, 2015 1:02 pm

Re: How to decode / parse the tll logfile format ?

by husky » Wed Oct 14, 2015 2:35 pm

@tracernz: Done. Result: UAVO warning gone, git hash warning stays.

And the reason is this - see first lines of logfile:

:Tau Labs git hash:
20150922:f069ece4 20150922
a43a4f77431b8baca2a594463e0b7b452e74e621

And in line 17 of LogConvert.m you read:

uavoImporterHash= '20150922:f069ece4 20150922 22:25';

So the additional time is the problem. Removing that from line 17 makes the warning go away:
----
>> LogConvert
***Tau Labs log parser***

warning: implicit conversion from numeric to char
warning: called from
LogConvert at line 2990 column 3
wrongSyncByte instances: 3
wrongMessageByte instances: 0

Completed bytes: 31 of 13106
Est. time remaining, 00h:00m:01s
13106 records in 0.56 seconds.
----
So Question #1 is solved, thanks.

Question #2 regarding csv output is still open.
Next

Who is online

Users browsing this forum: No registered users and 1 guest

Powered by phpBB ® | phpBB3 Style by KomiDesign
cron