Building a data log file for downloading using FTP from a remote client

Hi-
I am trying to work out the best way of creating an historical log of 110 data values to be stored every 10 seconds. so ~5 Meg every 24 hours. Ideally, would produce a single timestamped CSV file with pre-programmed headers. Guess I will need some sort of basic script to do this? I’m sure I am not the first trying to do this, is there a BASIC program available somewhere?

Hello Jon,

This could be handled in a few different ways depending on exactly what you are looking for. Excluding String logging log parameters take up 16bytes.( eWON binary data files format) Depending on how you calculated the amount of data you may be able to store up to 24 hours of data with the default log format on the ewon. Then export the data via email or possibly use a script to write a log file to the usr partition or an SD card if configured to extend the usr directory. You can also alter the recording size but this will shrink the usr partition.

There is an example here that generates and saves a report to the SD card. SD card - #5 by Angela_Telerski

Many thanks for doing this. And thanks for the tip on calculating file size (I was using 4 bytes/data point, so my files will be 4 times larger than previously calculated). I will investigate how to modify the script to include tag headings etc. In fact, I have a large 16 GB SD card, so your tip to extend to the SD will allow me to make rather larger files.

Best regards

Jonathon

To make sure there isn’t any confusion, keep in mind the SD card is not directly used with the ewons logging. The SD card will be utilized by generating an file and storing it on the SD card. This is where the scripting come into play.

Regards,
Deryck

Thanks for the clarification. I see the difference, so I think I will have to modify the script such that if there is no file on the SD drive, it creates one with the column headers. If there is a file there, it will periodically append the data from the historic log treating it as a RAM “buffer”. This file can then grow as much as required and not be limited by the RAM allocation size. This file can then either be emailed, or simply uploaded via FTP.

Jonathon McGuire

An EBD creates a new file on request. This does not allow you to append to another file.

You could do something with a custom log file but I am not sure how well this will work with the number of tags you are trying to log.
We do have an example here.
HOW TO: Create a custom historical log file in BASIC - Ewon Support / Basic and Java - hms.how (hms-networks.com)

110 values every 10 seconds is a good amount of data. You may need to take into account the load of other processes on the Flexy. This could also wear down the flash more then intended on the ewon or the SD card if one is used.

Hi

If it will help reduce storage needs, I can significantly reduce the tags that I am storing by exporting the alarms as words instead of individual bits. In overview, I will aim to store the logs in RAM and then periodically append them to a file on the SD card. Some experimentation will tell me how often I need to do this (am I right that there is only 5 Meg available for data storage in the RAM)?

Regards

Jonathon

Hi Jon,

You can see the amount of ram available Diagnostics > status > System counters > memory information. If you are creating your log file in the basic IDE you will most likely be interested in the PrgFreeMem and PrgAvailMem.

A post was split to a new topic: Flexy With Lantronix Software

Hi

Just having a quick look at the file structure with ftp. I cannot see the SD card mounted anywhere. What am I missing? Is there a config to mount the SD card somewhere?

Also, am using a system at our factory to develop the interface, logging etc and then want to clone the active parts to other systems in the field. I tried to use backup and restore but I think that is a bit brutal and may overwrite parameters that shouldn’t be. Is there a document that describes the file structure and which files can be uploaded independently to change it. For example, I think I can change individual pages relatively easily. To update tag list, do I just upload var_lst.txt?

Regards

Jonathon

Hi Jon,

You should see the SD card added as a folder in the USR directory. You may need to make sure the SD card is formatted correctly. Here is a document that covers the process EUM Card - Extend eWON User directory memory

I am not sure if we have a dedicated guide for all the files. They all can be uploaded independently. The var_list files contain the tags, the ewon processes it when uploaded adding new tags. Most of what you will be looking to configure is in the com.cfg file, This can be edited and only contain the parameters needed. One example of this might be to get a copy of the com.cfg but remove the talk2m registration info and then configure a second ewon with the same settings but not connect it to talk2m.

Here are a few documents that can help.
Configure your eWON using FTP (windows.net)

com.cfg detais.
ssh-0050-00-en-ewon-comcfg-txt.pdf (windows.net)

The files covered in the SD card setup can also be used via FTP. aug-0062-00-en-easy-commisionning-via-sd-card-or-usb-drive.pdf (windows.net)

You can also create a com.cfg files with the talk2m registration and WAN/LAN setup using the ecatcher wizard in m2web, under the device properties > talk2m > usb/SD wizards.

Hi

Yes, the card was not formatted in ext3, now it is and works fine.

I’ll be experimenting with configuring the EWON via FTP tomorrow.

Best regards

Jonathon

Glad to hear it is working. Let me know if you have any other questions.