I/A Series G3 station platform Application Director output window displays an Engine Watchdog Timeout by issuing a thread dump and displaying App Failed.

Issue

The Application Director output window displays an Engine Watchdog Timeout by issuing a thread dump and displaying App Failed.  This condition can occur when installing an upgraded station, after a reboot of the existing station, or during normal station operation.

Product Line

TAC I/A Series

Environment

I/A Series G3, Niagara

Cause

The software watchdog mechanism is designed to periodically report the station's engine cycle count to the platform daemon.  The main purpose is to detect and restart a "stalled" station when the control engine is believed to be in a deadlocked condition.  If the station’s engine cycle count stops changing and/or the station does not report a cycle count to the platform daemon within this defined period, the platform daemon causes the VM to generate a stack dump for diagnostic purposes, then takes the action defined by the Engine Watchdog Policy.

Depending upon station operation, the watchdog timeout may need to be increased to compensate for transient conditions where the NRE engine thread is temporarily busy during a single execution cycle.  These conditions include simultaneous execution of multiple objects or point extensions that occur on the same time interval, archiving of log history, custom objects, etc. during normal operation.  The default timeout is 1 minute.

Resolution

Tuning may be required (increase of the default watchdog timeout value). Open the running station, expand Config, expand Services, right click PlatformServices - View - Platform Service Container Plugin, locate the Engine Watchdog Timeout property.  Increase the value by 30 second or 1 minute increments and monitor controller operation (adjust as required).  If the condition continues, contact Schneider Electric Product Support for further assistance.

Watchdog policy selections include:

  • Log Only — Generates stack dump and logs an error message in the system log.
     
  • Terminate — (Default) Kills the VM process. If “restart on failure” is enabled for the station (typical), the station is restarted.
     
  • Reboot — Automatically reboots the host JACE platform. If “auto-start” is enabled for the station, the station is restarted after the system reboots.
     
  • Engine Watchdog Timeout — Default is 1 minute, and range is from 0 ms to infinity.