Historical LOG for 170 tags to SD card

Ewon Support Basic and Java
#1

Hello,

I want to write a script in BASIC to create a Historical Log for around 170 tags and save the file to the SD card as a CSV. I want it to update at an interval of one second (if the processor is fast enough) and at the end of the day create a new file. I would also like to lay the log out with the timestamp on the leftmost column and the following columns with each tag.

I have two issues I have been encountering:

Issue 1

I have been using the “HOW TO: Create a custom historical log file in BASIC” as reference and have been able to obtain a file logging for one tag that saves to the /usr directory but not to the SD card. Is there a way to create a blank file in the SD card and then append the file from the /usr location to the end of the file in the SD card?

Issue 2

Since I want to print all the tags on one line I initially used the PUT function in the following way:

OPEN fileName$ FOR BINARY APPEND AS 1

PUT 1, Time$ + STR$ tag1@ + STR$ tag2@ … + STR$ tag170@

However for 170 tags this is too long and does not fit on a line.

An idea I had was to write a for loop iterating and inserting the values of each tag with the GETIO function into an array which I can just insert into the PUT command after, but I had trouble adding these values to the array, (probably due to syntax). An array will also need to be created for the tag names to put in the title row of the file.

Any ideas and help will be much appreciated, my knowledge of BASIC is limited

#2

I believe you would need to open and read the file in the USR then write each line over to the file on the SD card. There is not built in function to append a file to a file that I can think of.

The example here iterates though all the tags. You might be able to use that part of the scrip in your log script. https://forum.hms-networks.com/uploads/short-url/lw1zdnTFSprXHbqnzqNYA8xUsSB.pdf

#3

Hello Deryck,
Sorry for the late reply, we have tried some other methods including using the export block descriptor method:
Tset 1, 1800
Ontimer 1, “GOTO CreateRecord”

CreateRecord:

T$ = TIME$
FILENAME$ = “/sdext/Historical_” + T$(1 To 2)+ T$(4 To 5) + T$(7 To 8) + “.txt”
FileCheck = FS “touch”, “/usr” + FILENAME$

IF FileCheck = -1 THEN
WriteData:
Close 1
Close 2
OPEN FILENAME$ FOR BINARY APPEND AS 2
OPEN “exp:$dtHT $st_m30 $et_0 $ftT” FOR BINARY INPUT AS 1
Onerror “goto EndCount2”
A$=GET 1 // This stops the column headers from being inputted into the file

Loop2:
A$=GET 1
PUT 2,A$
If A$ <>"" Then GOTO Loop2

EndCount2:
Print “end”
Close 1
Close 2

ELSE
WRITEEBD “exp:$dtHT $st_m30 $et_0 $ftT”, “/usr”+FILENAME$
PRINT "New File Created called: " + FILENAME$

but it find that it is causing a watchdog (timer) error after around half a day. What confuses me is that it doesn’t specifically happen when then append (loop2) function is called, but sometime inbetween when it is just collecting data and storing it in the RAM. I am wondering if there is another process happening that when put together with the historical logging is overloading the CPU i think.

I was wondering whether taking values directly from the IOs and putting it in a historical log like i was doing before and making 4 different files (due to the limitiation of only being able to fit 40 GETIOs on one line and so altogether getting 160 tags between the files) would overload the CPU as well. But i still ended up with the same result.

My next idea to try, stems from the fact that each IO has a file logging in the directory before /usr, for instance /irc_IONumberX.txt, and it is to move this file every half hour to an hour into the SD card and using FTP to move the files after to a PC to post process and merge in python. But the FS mv function only works in the /usr directory.

Do you have any suggestions? I want to avoid if possible constantly writing to the SD card in effort to prevent burn out

#4

The error would suggest an issue with the timer. Though I’m not sure what. I have seen issue in the past where the function is called faster then then function can be completed but I’m doubtful that is the issue here. It may be still worth stopping the timer at the start of the function call then restarting it at the end.

#5

Sounds like a good practice to stop the timer at the start of a function call, but doubt it would solve the problem.
I am currently trying the GETIO method again:

TSET 3, 3600 //create timer for every hour
TSET 1, 1 //create a timer every seconds
ONTIMER 3, “GOTO MoveData”
ONTIMER 1, “GOTO WriteRecordA” //every two seconds go to the first write command which then leads onto the seconds write command.
END

//Funcion used to shorten the GETIO function to fit more IO value calls on the line
FUNCTION G($T)
$G = GETIO $T
ENDFN

WriteRecordA:
T$ = TIME$ //grab current time
$Date$ = T$ (1 TO 2) + T$ (4 To 5) + T$ (7 TO 10) // Get the current date
fileNameA$ = "/Datalog Part A " + $Date$ // create the filenames of the split data
fileNameB$ = "/Datalog Part B " + $Date$
fileNameC$ = "/Datalog Part C " + $Date$
fileNameD$ = "/Datalog Part D " + $Date$
// Now below we will verify if the file exists on first cycle. If it doesn’t we will create the file and write the header. If it does exist we will just append the tag data.
SETSYS PRG, “RESUMENEXT”, 1 //this command means to continue even if an error occures, this is needed so that the program continues if there is no file already created (which produces an error)
CLOSE 1
OPEN fileNameA$ FOR TEXT INPUT AS 1 //attempt to open the file
ErrorReturned = GETSYS PRG, “LSTERR” //record if there is an error or not
IF(ErrorReturned = 33) THEN //if there is an error, create a new file and add headers to each column
CLOSE 1
OPEN fileNameA$ FOR TEXT OUTPUT AS 1
PUT 1, “Time” “LS-B10” //csv headers, to change later
CLOSE 1
Print "New File Created Called " + fileNameA$
ELSE //if there is no error then the file already exists and can just be appended with new data points
CLOSE 1
OPEN fileNameA$ FOR TEXT APPEND AS 1
PUT 1,T$@G(1)@G(2)@G(3)@G(4)@G(5)@G(6)@G(7)@G(8)@G(9)@G(10)@G(11)@G(12)@G(13)@G(14)@G(15)@G(16)@G(17)@G(18)@G(19)@G(20)@G(21)@G(22)@G(23)@G(24)@G(25)@G(26)@G(27)@G(28)@G(29)@G(30)@G(31)@G(32)@G(33)@G(34)@G(35)@G(36)@G(37)@G(38)@G(39)@G(40)
CLOSE 1
Print “New Data Inputted in A”
ENDIF
WriteRecordB:
SETSYS PRG, “RESUMENEXT”, 3
CLOSE 3
OPEN fileNameB$ FOR TEXT INPUT AS 3
ErrorReturned = GETSYS PRG, “LSTERR”
IF(ErrorReturned = 33) THEN
CLOSE 3
OPEN fileNameB$ FOR TEXT OUTPUT AS 3
PUT 3, “Time” “LS-B10”
CLOSE 3
Print "New File Created Called " + fileNameB$
ELSE
CLOSE 3
OPEN fileNameB$ FOR TEXT APPEND AS 3
PUT 3,T$@G(1)@G(2)@G(3)@G(4)@G(5)@G(6)@G(7)@G(8)@G(9)@G(10)@G(11)@G(12)@G(13)@G(14)@G(15)@G(16)@G(17)@G(18)@G(19)@G(20)@G(21)@G(22)@G(23)@G(24)@G(25)@G(26)@G(27)@G(28)@G(29)@G(30)@G(31)@G(32)@G(33)@G(34)@G(35)@G(36)@G(37)@G(38)@G(39)@G(40)
CLOSE 3
Print “New Data Inputted in B”
ENDIF
WriteRecordC:
SETSYS PRG, “RESUMENEXT”, 5
CLOSE 5
OPEN fileNameC$ FOR TEXT INPUT AS 5
ErrorReturned = GETSYS PRG, “LSTERR”
IF(ErrorReturned = 33) THEN
CLOSE 5
OPEN fileNameC$ FOR TEXT OUTPUT AS 5
PUT 5, “Time” “LS-B10”
CLOSE 5
Print "New File Created Called " + fileNameC$
ELSE
CLOSE 5
OPEN fileNameC$ FOR TEXT APPEND AS 5
PUT 5,T$@G(1)@G(2)@G(3)@G(4)@G(5)@G(6)@G(7)@G(8)@G(9)@G(10)@G(11)@G(12)@G(13)@G(14)@G(15)@G(16)@G(17)@G(18)@G(19)@G(20)@G(21)@G(22)@G(23)@G(24)@G(25)@G(26)@G(27)@G(28)@G(29)@G(30)@G(31)@G(32)@G(33)@G(34)@G(35)@G(36)@G(37)@G(38)@G(39)@G(40)
CLOSE 5
Print “New Data Inputted in C”
ENDIF
WriteRecordD:
SETSYS PRG, “RESUMENEXT”, 7
CLOSE 7
OPEN fileNameD$ FOR TEXT INPUT AS 7
ErrorReturned = GETSYS PRG, “LSTERR”
IF(ErrorReturned = 33) THEN
CLOSE 7
OPEN fileNameD$ FOR TEXT OUTPUT AS 7
PUT 7, “Time” “LS-B10”
CLOSE 7
Print "New File Created Called " + fileNameD$
ELSE
CLOSE 7
OPEN fileNameD$ FOR TEXT APPEND AS 7
PUT 7,T$@G(1)@G(2)@G(3)@G(4)@G(5)@G(6)@G(7)@G(8)@G(9)@G(10)@G(11)@G(12)@G(13)@G(14)@G(15)@G(16)@G(17)@G(18)@G(19)@G(20)@G(21)@G(22)@G(23)@G(24)@G(25)@G(26)@G(27)@G(28)@G(29)@G(30)@G(31)@G(32)@G(33)@G(34)@G(35)@G(36)@G(37)@G(38)@G(39)@G(40)
CLOSE 7
Print “New Data Inputted in D”
ENDIF
END
MoveData:
T$ = TIME$
$Date$ = T$ (1 TO 2) + T$ (4 To 5) + T$ (7 TO 10) // Get the current date
SETSYS PRG, “RESUMENEXT”, 1
FS “mv”, "/usr/Datalog Part A " + $Date$, "/usr/sdext/Datalog set/Datalog Part A " + $Date$ + T$(12 To 13)
FS “mv”, "/usr/Datalog Part B " + $Date$, "/usr/sdext/Datalog set/Datalog Part B " + $Date$ + T$(12 To 13)
FS “mv”, "/usr/Datalog Part C " + $Date$, "/usr/sdext/Datalog set/Datalog Part C " + $Date$ + T$(12 To 13)
FS “mv”, "/usr/Datalog Part D " + $Date$, "/usr/sdext/Datalog set/Datalog Part D " + $Date$ + T$(12 To 13)
END

where i would change the IO numbers for the actual IOs i want to record. Just trying it again and it has lasted 24 hours with no reboot, but I still do not trust it. Anything you would change in this?

#6

One thought is that you can use an ondate 0 * * * * to trigger an event at the start of every hour.

#7

Thats a good idea, thanks.
I could shorten it all by just creating functions as opposed to just copying and pasting the code several times for the different groups

#8

hello Deryck,

sadly, the Ewon rebooted again after about 5 days of running continuously. Is there a way we can get the Ewon to restart the script whenever it reboots? because then we only lose a small chunk of data and do not have to manually turn it on whenever we happen to not see it running.

#9

Yes, from the IDE check the Autorun in the run menu. This will start the basic script automatically.
image

#10

ahh thanks