We have a Flexy unit that has lost it's configuration twice now. The most recent loss occurred the day of the latest Talk2M outage. I could see in the eCatcher logs that the unit was connecting and disconnecting and then it went offline and never came back.
We have someone at the site try to cycle the power on it too see if it would connect and it never did. We then had to sent a technician to the customers site and replaced the Flexy base unit.
I connected to the unit here in the shop and it no longer has any tags, users, basic script, internet connection or vpn connection information.
Attached is latest backup from the device.MOVED TO STAFF NOTE (39.5 KB)
i have reviewed the backups that you supplied and I have noticed a few things:
Switch Monitoring Events: The switch monitoring checks for Electromagnetic interefernce, which can cause a myriad of issues both software and hardware. The best practice for resolving this issue is to use shileded USB cables, and to properly ground the unit. Please see below:
Freqeunt Power loss: The device has had several reboots in the event logs due to power loss. This points to the first issued is discussed regarding EM interference.
GSM Signal Strength: You current GSM level is at a 19. OUr required GSM Strength for connection to these units are 18. While being barely above minimum, changes in the signal strength can cause the device to lose connectivity to the internet/VPN.
The flash memory may have been compromised due to EM interference and the consistent rebooting of the device.
I would advise updated the ethernet cabling to shielded ports, grounding the device, check for any signal noise that is occuring around the unit. Once that is done, ensure the device has a stable connection, and monitor for any configuration resets.
So consistent rebooting of a Flexy will cause issues with the flash memory is what I am reading from your reply.
The current GSM signal is with the unit sitting on my desk. When the unit was installed at the customer site the signal strength was between 21 and 30.
There are no switch monitoring errors in the log file of the replacement unit. That would indicate there is not any issues with interference. Have you tried testing with a Flexy unit that has no configuration loaded and a plc plugged into the LAN ports of the Flexy to see if there are switch monitoring errors?
If you look into the eCatcher logs the unit that lost it’s configuration was not online any more after 08/20/19 08:55AM which was the day of the outage. The day of the Talk2M and VPN23 outage there were a large number of connects and disconnects that occurred.
When I looked at the logs of other Flexy units that we have they were constantly rebooting on the 20th as they tried to find a VPN server to connect to.
So multiple reboots can cause flash memory issues.
The outage caused multiple reboots.
The Flexy unit to lost it’s configuration due to compromised Flash memory.
Consistent rebooting due to powerloss, with EM interference can cause the flexy to lose it’s configuration.
In the event logs, I did not see any indication that the the device of the device rebooting due to losing connection to the VPN. The only reboots that occurred on the unit is due to power loss on the device.
When the reboots occur on the device due to the lack of VPN connection, it isn’t an instantaneous reboot. The VPN reboots do follow the reboot protocols to ensure integrity of the unit. Powerloss reboots do not follow such protocols. The device immediately loses power and the information stored is loss.
eCatcher logs aren’t an accurate method to determine device functionality. eCatcher logs only log events for VPN connectivity. The device may lose connectivity for many different reasons such network functionality, cabling becoming unplugged, server stablitly, different types of reboots. The only reliable method to determine device health is to review the event logs of the unit itself.
I have tested my unit with Switch Monitoring enabled, and the unit itself has not experienced power loss nor events tied to switch monitoring.
As for the next steps, I would reconfigure the unit and monitor for further issues. While you are monitoring, you can begin the RMA process as highlighted below:
You would not see anything in the device’s event logs indicating the device rebooting due to the outage on the 20th since the event logs only went back to the 21st. That was the date the customer could get to the device and tried a power cycle to see if it would connect to the VPN. All other log information was lost when the unit lost it’s configuration on the 20th.
Did you test with a plc plugged into a Flexy unit that did not have any configuration loaded?
Have you performed the steps that I have outlined above?
Before we can replace the unit, we need to verify if the hardware is comprimised. If the hardware has been comprimised, you can reach out to your distributor for the RMA process.
We can reconfigure the unit but would not be able to replicate the customer’s setup with out placing the unit back at the customers site.
Placing this unit back into service is not something that we are willing to do based on the history of the device and the location of the customer site.
If it were me, I would set up a configuration and monitor the unit for any issues that can occur. If the hardware is comprimised, then it will show regardless of the configuration.
You can reach out to the distributor and begin an RMA process, but I would still reconfigure the unit an monitor for failures of the hardware.
