How to Troubleshoot a SEB5100 Storage Expansion Box

NOTICE

DISCONTINUED PELCO PRODUCT.
This product has been discontinued. For further information please visit pelco.com/support/discontinued-products

Issue

Performing the Linux df command from NVR5100 or DVR5300 root prompt (via local console monitor or SSH Connection) reveals that SEB5100 Volumes are not mounted.

Details: If a SEB5100 Storage Expansion Box is attached via the NVR5100/DVR5300 secondary ethernet port, between 2 to 4 (depending on amount of Hard Disk Drives in SEB5100) volumes (/dev/sde1 and above) should be shown in the df output.
If there are 12 disks in your SEB5100, then four volumes, each with one partition, will appear in df. They will be labeled like this: sde, sde1, sdf, sdf1, and so on. If those volumes or partitions are missing from the df output, then the unit is not functioning properly, follow this article to troubleshoot.

Product Line

Pelco Video Management

Environment

Endura NVR5100 or DVR5300 Recorder with attached SEB5100 Storage Expansion Box.

Cause

Possible causes include (1) a break in network connectivity, (2) filesystem corruption, (3) incorrect file ownership, (4) an incorrect operating system, (5) incorrect link speed, (6) a corrupt database, or (7) mismatched software versions on the units.

Resolution

  1. Verify network connectivity
    1. Log into your NVR using steps 1 and 2 in Lessons Learned Article #10873.
    2. Attempt to ping the SEB5100 using the instructions found in step 3 in the Lessons Learned Article #10873.
    3. If the unit responds to ping on the default ip address of 192.168.1.100, follow the steps in LL#11751 to resolve NVR/DVR host DHCP issue.
    4. If the unit responds to the ping messages on any other address, skip to section IIstep 1 further below. Otherwise, continue to step 4.
    5. Verify that the SEB5100 and NVR5100 are connected to the correct ports. A patch cable must be connected to the port shown in Figure 1, below on both the NVR and the SEB:
      FIGURE 1
      The secondary network port icon
      To connect multiple SEB5100s to a single NVR or DVR, connect the secondary network port on all the devices in question to a gigabit switch.
    6. If the cable is connected to the correct port, verify that the lights on the port are illuminated. If they are not illuminated, try a different, known good category 5e or faster cable. It can be either a crossover or straight-through cable as the port can automatically sense the connection type. If the port is illuminated, proceed to step 6. If the port lights are not illuminated, then the port is likely damaged physically. Send the unit in for service and repair: http://www.pelco.com/sites/global/en/sales-and-support/ra/repair-authorization-request-landing.page
    7. Connect a monitor directly to the SEB5100.
    8. Log into the unit using the username of root and password of pel2899100.
    9. Type ifconfig eth1 and verify the SEB5100 has an IP address. If it does not, continue to step 9. Otherwise, skip to step 10.
    10. Type the following commands on the NVR:
      cd /var/lib/dhcp
      ls
      If there is a file named dhcp.leases, rename it to dhcpd.leases:
      mv dhcp.leases dhcpd.leases
      service dhcpd restart
    11. Type vi /proc/sys/net/ipv4/icmp_echo_ignore_broadcasts
    12. If the value is 1, highlight the value, press r, then type 0, and then press ESC. Then, press SHIFT and the colon key. Finally, type wq and then press ENTER.

       
  2. Verify the SEB5100's file system
    1. Log into your SEB5100 using the instructions found under step 4 of the Lessons Learned Article #10873.
    2. Type the following command:
      cat /proc/partitions
    3. If the unit has six hard drives, sda, sda1, sdb, and sdb1 should be shown. If the unit has 12 drives, sda, sda1, sdb, sdb1, sdc, sdc1, sdd, and sdd1 will all be visible.
      FIGURE 2
      A SEB5100 with the correct partition configuration.
    4. Type file -s /dev/sda, then repeat for sda1, sdb, sdb1, and so on. The file system type for the volumes (e.g., sda) should be x86 boot sector, extended partition table. The file system type for the partitions (e.g., sda1) should be SGI XFS filesystem data (blksz 4096, inosz 256, v2 dirs). If they are not, skip to Section III, otherwise continue to step 5 directly below.
    5. Type xfs_repair /dev/sda1. After the process has completed, repeat it for sdb1, and — if present — sdc1 and sdd1. Then, type reboot and press enter/return.
    6. Wait 15 minutes and check to see if the SEB volumes have mounted on the NVR. If they have not, reboot the NVR as well and wait an additional 15 minutes. If still not mounted, proceed to Section IV.

       
  3. Repair the SEB5100's file system
    1. Complete section 3 of Lessons Learned Article #10738 on the SEB.
    2. Complete section 8 of Lessons Learned Article #10738 on the SEB.
    3. Wait 15 minutes. If the SEB has not mounted, proceed to section IV, below.

       
  4. Verify file ownership
    1. From the command line of the SEB and the host NVR/DVR, type these commands:
      cd /usr/local/Pelco/NSD
      ls -l
    2. The third column in the output should say root, as shown in Figure 3, below. Note that the red text color was added for emphasis:
      FIGURE 3
      File ownership
    3. If the column does not say root for every entry, proceed to step 4. Otherwise, proceed to Section V.
    4. Type the following commands:
      chown root:root ./* -R
      rm Logs
      ln -s /var/log/nsdlogs Logs
      reboot
    5. After the SEB has finished restarting, repeat step 4 on the NVR.
    6. Wait 15 minutes after the SEB and NVR finish rebooting. If the SEB's volumes have not mounted on the NVR, proceed to section V.

       
  5. Verify symbolic link to Logs
    1. From the command line of the SEB and the host NVR/DVR, type these commands:
      cd /usr/local/Pelco/NSD
      ls -l
    2. Below in Figure 4 notice that the link to Logs is missing.  Also notice that the file size for Logs without the symbolic link is 155750400 which can prevent services from starting.  See Figure 3 above for the correct path.  If the symbolic link does not exist then proceed to step 3.  If the symbolic link exists and is correct then proceed to Section VI
      FIGURE 4
      Missing - Symbolic link to Logs
    3. Type the following commands:
      rm Logs
      ln -s /var/log/nsdlogs Logs
      reboot
    4. After the SEB has finished restarting, repeat step 4 on the NVR.
    5. Wait 15 minutes after the SEB and NVR finish rebooting. If the SEB's volumes have not mounted on the NVR, proceed to section V.

       
  6. Verify link speed
    1. On both the SEB and NVR, type ethtool eth1. In the output, it should say, Speed: 1000Mb/s. If it does not, verify proper CAT5E or greater Ethernet Cables/Connectors  and consider temporarily swapping to new/different CAT5E for troubleshooting purposes.
    2. If the speed still does not show as 1000 mbps or higher, execute this command on both the SEB and NVR to force it to operate at 1000 mbps: ethtool -s eth1 speed 1000 duplex full.

       
  7. Verify the mode of the SEB5100
    1. From SEB5100 Linux root, type nsdterm.
    2. From nsdterm, type v. The output should appear as shown in Figure 4, below:
      FIGURE 4
      A SEB5100 operating in the correct mode
    3. If the mode of the SEB5100 shows as NVR, then the SEB5100 must be re-imaged using a proper SEB image. Click here to download the image. Please see LL#11534 for the reimage procedure.
    4. If the SEB5100 still has not mounted, proceed to Section VII, below.

       
  8. Delete the database on the NVR
    1. Warning: This results in video not being searchable from WS5000 until after all volumes are mounted and syncdb is performed (16-65 minute runtime).
      Type the following commands:
      service hald stop
      service nsdd stop
      service mced stop [perform this step on DVRs only]
      service postgresql stop
      cd /usr/local/Pelco/Database/
      mv data data.old
      reboot
    2. Wait 15 minutes. If the SEB volumes mount finally, execute syncdb per LL#11569, otherwise (if the volumes did not mount) skip to section VIII.
      note: Make sure that no matter how far you go in this article, if you have done step 1 in this section (VII), return to LL#11569 to syncdb after the SEB5100 volumes are mounted.


       
  9. Update SEB5100 Software (or resend current version to fix corruption).
    1. Type the following command from the command line of the SEB5100 (note: The version running on the SEB5100 will be shown).
      grep -i version /usr/local/Pelco/NSD/Public/Manifest.xml
       
    2. If the NVR or DVR host is running version 1.4, but the SEB is running version 1.3 or lower, it will fail to mount and must be updated.
       
    3. If your SEB is connected to a DVR5300, visit this page to download the latest update for the SEB5100. If it is connected to an NVR5100, visit this page for the latest update.
       
    4. Since SEB5100s are not visible in Endura Utilities, the update must be applied to the NVR or DVR it is connected to. See Lessons Learned Entry 11871 for instructions on doing so.
       
    5. After the update of the SEB has completed, reboot the DVR or NVR.

       
  10. Workaround to manually mount the SEB5100 volumes.

This section is included for troubleshooting purposes, and is not meant for use as a permanent solution, as it would need repeating any time the host NVR5100 or DVR5300 reboots in the future for any reason.

  1. Type the following commands on the command line for the SEB:
    cd /usr/local/Pelco/NSD/Public
    cat SEBDevice.xml | grep uuid

    FIGURE 5
    The UUID of a SEB5100

    Use the third UUID to mount the SEB in step 3 below.
  2. Write down the third UUID from the output shown in Figure 5, above.
  3. Type exit to return to the NVR or DVR.
  4. Type cd /usr/local/Pelco/NSD/Scripts.
     
  5. Type ./addiscsi followed by the SEB's IP address, then uuid: followed by the UUID found using step 1 above.
    For example...
    ./addiscsi 192.168.1.253 uuid:b4b78038-d989-4bf0-9847-01c2cc1ed0d7-2