DigitalSENTRY maintenance and best practices guide.

Issue

Long term maintenance requirements for best product experience.

Product Line

Pelco Video Management

Environment

  • DigitalSENTRY system
  • Lagacy system (DS Xpress, DS Enterprise, DS RealVue, DS1000)
  • DS NVs
  • DSSRV
  • DSSRV2

Cause

Long term system utilization best practices.

Resolution

In order to ensure the best possible user experience, the following information is provided for system integrators, installers, and end users.  Much of this information is a collection of various Pelco knowledgebase articles organized into a cohesive outline format for ease of use in order to enhance, but not replace, published system documentation.  This is a living document, which means that it may be modified as new inforamtion comes out, so check back here often for additional information.

I. System Overview:

  1. The DigitalSENTRY video recording system is comprised of various software components, running as services, that perform specific functions.
  2. The following list of services are the core requirements for a functioning system, ie record, view live, PTZ, search, and export:
    • SQL server + Integral server agent
    • SQL Server Browser
    • DigitalSENTRY video server service
    • DigitalSENTRY storage service
  3. Ancillary services (watchdog, video server utility, front panel)
    • Watchdog: used to monitor the above services.  We don't usually run this service.
    • Video Server utility: provides a communication interface for DS ControlPoint to view the status of DS Video Server Service
    • Front Panel: provides system performance metrics to the Front Panel Notifier program (DSSI)
    • VSAN Watch: used to monitor legacy direct attached SCSI storage arrays.  We don't run this service.
  4. API Bundle.  Used for Pelco Mobile App, Video Xpert integration, and third party applications such as access control:
    • Not installed by default
    • Applies to DigitalSENTRY version 7.8.90 and later
    • DS Frame Server Service
    • DS NSM Rest Service
    • DS Serenity
    • DS API
  5. DS Web Server.  Used for Pelco Mobile app.
    • Not installed by default
    • Applies to DigitalSENTRY versions 7.3.54 thru 7.7.309
  6. Client Applications (controlpoint, admin, pelco mobile)
    • DS ControlPoint: full-featured thick client application to be run on a networked PC.
      • This software is installed on the recording server as part of the DigitalSENTRY software suite
    • Pelco Mobile App: Apple- and Android-compatible app for viewing video, PTZ and playback.
      • Available through Google Play Store or Apple's App Store
    • DS Admin: configuration client
      • This software is installed on the recording server as part of the DigitalSENTRY software suite
  7. System architecture
    1. Distributed architecture, legacy architecture that is no longer supported in version 7.8.90 and higher
      • The original software design was that each of the following components could run on separate hardware, thus be distributed across various points within the network:
      • DMS: Database Management System - where the SQL server and Storage services reside
        • Can and often hosted a VSAN share
      • VAU: Video Aquisition Unit - where the DS Video Server service resides
      • VSAN: Video Storage Area Network - used for network storage of video
      • Current Pelco product offer is for a single, stand alone, recorder
    2. RAID vs. JBOD
      • JBOD = "Just a Bunch Of Disks" and utilizes separate storage volumes per physical hard drive installed
      • RAID = "Redundant Array of Independant Disks"
        • Data is striped across multiple disks in the array
        • Provides performance enhancement due to multiple drives being accessed simultaneously
        • RAID level 5 provides fault tolerance in the event of hard drive failure (can tolerate 1 HDD going offline)
    3. Copy to network share
      • Requires at least one local volume to initially capture video
      • Video is copied to the network share every 15 minutes
      • Allows customers to increase their storage capacity over the Pelco product offer
      • Provide a degree of fault tolerance should the DigitalSENTRY system suffer catestophic failure
    4. Video capture hardware (internal capture cards, direct attached encoders, IP encoders, IP cameras)
    5. Remote client PC (DS ControlPoint)
      • Provides multiple users access to view live video, PTZ functions, playback, and export.
      • Provides support for multiple monitors
      • For full time use, it is recommended to use a remote client PC rather than use DS ControlPoint directly on the recorder
        • More info can be found here.
        • Provides system security, as the recorder may be located in a controlled area
        • Provides system integrity, as the recorder does not have the overhead of client operations running

II. Installation and configuration guidelines:

  1. physical installation
    1. Quick Install guides:
    2. Environmental factors (ambient temp, electrical, UPS)
      • Spec sheet states operational temperature of 10 to 35 degrees C (50-95 F)
        • The high end is a bit extreme and not recommended to install a system where the abient temperature approaches this
        • It is recommended to install the unit in an air-conditioned and temperature-controlled environment
      • It is recommended to use a steady, conditioned, and surge-suppressed mains power source
      • An 'APC by Schneider Electric' UPS battery backup system is recommended to provide enough runtime for the operating system to perfom a graceful shutdown in the event of a power failure
        • Please contact and consult with the APC sales engineering team to discuss prodcuts appropriate for your application.
      • Power loss and improper shutdown of the system can cause files system corruption and render the unit non-operational when/if the system boots back up
    3. Peripherals (keyboard, mouse, monitor)
      • DS ControlPoint should not be run on a full-time basis on the DigitalSENTRY recorder/server
      • Although the services start automatcially and no user intervention is required for day-to-day operation, it is recommended to install the system with a keyboard mouse and monitor.
      • For the purpose of system maintenance, configuration, and troubleshooting in the event of component failure, a keyboard mouse and monitor will be required.
      • If remote viewing is required, do not use monitor extenders.  Pelco Engineereing has not tested, nor approved, the use of monitor extenders.
      • If remote viewing is required, use a dedicated PC to run DS ControlPoint.
    4. Network environment
      • Make sure the network infrastucture has enough throughput for total camera bandwidth + ControlPoint client connections
      • Connect to a gigabit port on the network switch
      • Both network adapters may be used
      • Do not plug the unit directly into the Internet
        • Due to exposing the SQL server for remote client connections, using the unit direclty on the Internet makes it a big fat juicy target for hackers.
        • Using a port forwarding NAT router will also expose the SQL server to the Internet
        • It is recommended to use the Pelco mobile app, or provide a VPN connection for users that require remote access to the system
        • Firewalls, access control lists, and antivirus software may also be used to secure the system
    5. Analog cameras
      • Be aware of power requirements
        • Cameras
        • Enclosures
        • Zoom lens
        • Pan and tilt base
        • PTZ power requirements
        • Power requirements for outdoor and specialty applications
      • Coax cable overview and selection
      • PTZ cabling / settings
  2. System Configuration:
    1. Analog cameras
      1. Frame rate and resolution settings
      2. Scheduling
        • Continuous recording
        • Motion detection
        • Alarm recording
        • Zones
    2. IP cameras
    3. PTZ configuration
    4. Record schedules
    5. Motion detection sensitivity settings
  3. System Documentation:
    1. Keep system backups and documentation in secure location for future reference as needed
    2. System documentation and backup database should be created after system installation/configuration
    3. Also, backup the database after significant system configuration changes/updates
    4. The following information will be required for system maintenance or disaster recovery (helpful to have handy when calling in to Pelco Technical support!!!):
      1. host name
      2. IP settings
        • address
        • subnet mask
        • default gateway
        • DNS addresses
        • Which physical adapter is used for IP camera network and client network
      3. Product ID and IP camera license
      4. DigitalSENTRY software version
      5. Microsoft Windows activation key
      6. Configuration backup (DS Backup)
  4. Video index maintenance:
    1. Database rebuild  utility
  5. Recovery procedure:
    1. Recovery process
    2. ds backup (restore db)
      • Prior to restoring the database, the host name and IP configuration must be set back to thier original settings
    3. DS health check
    4. Database rebuild
  6. Disaster recovery:
    1. copy db files from non-bootable system
      • Please contact Pelco Technical Support for assistance regarding this
      • Pelco Technical Support representatives can refer to internal knowledge base article LL#18062
    2. manually restore db
      • Backup copies of db files may be copied back to the system
      • Depending on the situation, Microsoft SQL Server Management Studio may need to be used to attach the DigitalSENTRY database files
      • If restoring an older database and attempting to save newly recorded video, the rebuild utility must be used to re-index the data files
    3. DS health check
      • After reconfiguring the system, the DS health check should be followed to verify correct settings

III. System Maintenance

  1. DigitalSENTRY software is an application designed to run within the Microsoft Windows operating system environment, therefore, standard PC maintenance practices apply
  2. If the system is required to be shut down, use the normal Microsoft Windows shutdown method.
    • Do not use the power button to kill the system unless absolutely necessary (improper shutdown)
    • Not having a local console does not constitue absolute necessity to kill the system
    • Killing the system can cause the system to be dead, requiring system recovery
  3. Make sure Microsoft Windows critical updates are applied
    • Especially if the system is used in remote access applications
  4. Periodically reboot the system to ensure optimal performance
    • Use the Microsoft Windows restart function from the start menu
    • See caution above regarding improper system shutdown
    • See Microsoft Support article for more information
  5. If anitvirus software is used, keep the virus definition files up to date
  6. Review live video on a daily basis to make sure the cameras are functioning
  7. Play back video on a regular basis to ensure there are no storage drive problems
    • Use the search feature to scroll back through the time line to make sure stroage retention requirments are met.
  8. Disk defragmentation is not required
  9. It is recommended to periodically close DS ControlPoint and reboot the client PCs.
  10. Backup the DS ControlPoint configuration
    • This configuration information contains system connection information as well as:
      • Groups
      • Tours
      • Alarm popup settings
      • Bandwidth throttle settings
      • etc...

IV. Parting shots

  1. 9 times out of 10, Pelco's Technical Support team receives calls due to unexpected system performance as a result of mis-configuration or configuration changes
  2. Troubleshooting a sick system
    • First, use the DS Health Check list to verify correct settings
    • Use chkdsk and Microsoft disk management for troubleshooting file system and disk-related problems
    • Use the database rebuild command (rebuild -go) to check for inaccessible video files
      • After running rebuild -go, check c:\digitalsentry\rebuild.log file for inaccessible .XPA files
      • The rebuild.log file will instruct you to run 'cleanfiles.bat' to delete the inaccessible files
      • Chcek 'cleanfiles.bat' to make sure it isn't about to delete all video on the system
    • Use Microsoft Windows event viewer to check for software problems
    • Review system operator "use case" to make sure the system isn't being overly taxed simply by too much video being displayed at once
      • The DS ControlPoint bandwidth throttle can be used to alleviate over subscription of video resources
      • DS ControlPoint has a "remote Mode" feature that can be used to limit the number of cameras the client can view
  3. Troubleshooting remote client connection issues
    • Make sure the DSSRV/DSSRV2 is healthy
    • Identify the network topology being used and check for port blockage
      • Internal LAN
      • WAN/Internet via port forwarding through Internet gateway router
      • VPN
      • Internal firewalls, port/protocol filtering, or access control lists within the internal LAN
      • Software firewall or antivirus software running on the DigitalSENTRY server
    • Video is captured first by DigitalSENTRY, then sent to DS ControlPoint
      • If there is an offline camera, ControlPoint will show "Configuring Camera", then "No Video"
        • Troubleshoot this at the server, not from the client
      • HDD problems at the server will cause DS Video Server Service to slow down and become non-responsive
        • Symptoms of this can be:
          • "Unknown Error" in live or playback
          • DS ControlPoint fail to connect to the system
            • Error connecting to one or more of your systems. RV_CreateConnection failed: RET_ERROR
          • DS ControlPoint connection drops
          • DS ControlPoint hangs with "(not responding)" notice or white haze over the user interface
          • Choppy video
      • Too much database activity can cause some of the symptoms listed above

Related assets:
DS ControlPoint articles:

Controlpoint installation/upgrade
Software revision history
ONVIF information
Adding Optera
Adding EVO