DAQ Failure with Many Open Files

[marcodeltutto]

After running for many days, the DAQ fails with

Traceback (most recent call last):
  File "/home/nfs/sbndprm/SBNDPurityMonitorDAQ/sbndprmdaq/mainwindow.py", line 631, in _check_status
    status = self._prm_manager.check_hv_range(control.get_id())
  File "/home/nfs/sbndprm/SBNDPurityMonitorDAQ/sbndprmdaq/manager.py", line 274, in check_hv_range
    ret = self._hv_control.check_hv_range(prm_id)
  File "/home/nfs/sbndprm/SBNDPurityMonitorDAQ/sbndprmdaq/high_voltage/hv_control_base.py", line 138, in check_hv_range
    if not hv_range[0] <= self.get_hv_sense_value(item, 'voltage', prm_id) <= hv_range[1]:
  File "/home/nfs/sbndprm/SBNDPurityMonitorDAQ/sbndprmdaq/high_voltage/hv_control_mpod.py", line 242, in get_hv_sense_value
    ret = self._get_cmd(ip=ip, name=cmd_name, ch=str(channel), extra_flags='-OUvq')
  File "/home/nfs/sbndprm/SBNDPurityMonitorDAQ/sbndprmdaq/high_voltage/hv_control_mpod.py", line 103, in _get_cmd
    with subprocess.Popen(cmd.split(), stdout=subprocess.PIPE) as proc:
  File "/usr/lib64/python3.6/subprocess.py", line 729, in __init__
    restore_signals, start_new_session)
  File "/usr/lib64/python3.6/subprocess.py", line 1254, in _execute_child
    errpipe_read, errpipe_write = os.pipe()
OSError: [Errno 24] Too many open files