Security reported problems viewing the Lenel security cameras at approximately 11:30am. The camera windows would appear to open, but there was a problem with connecting to the server. The video server was rebooted at approximately 12:45pm. This corrected the problem.
Category Archives: Unscheduled Outages
AVG rebooted with new DAT file update
The Anti-Virus Gateway was reboot after updating to the 4359 DAT file. This DAT file allows the AVG to detect the W32/Bagle.ab virus. The server was out of service for about ten minutes as it rebooted.
AVG failed to update
The Anti-Virus Gateway failed to update to the 4357 DAT file. The update was run manually and the DAT files were updated. We are now able to detect the W32/Bagle.aa virus.
Whidbey reboot
Whidbey was experiencing problems with the secure shell daemon. During the troubleshooting process we were unable to get the daemon to restart correctly, and the system rebooted at 12:15 pm.
Upon further investigation of the last log, it became clear that the shutdown command issued on April 14 during preventative maintenance froze. In working with the sshd2 daemon, the shutdown completed.
The secure shell daemon is now running correctly.
Imap issue
The University’s mail server handling imap requests this afternoon between 3:30 pm and 4:00 pm. The testing was done with the secure imap daemon which confused the server. All imap connections were closed and reopened.
Calendar and mail services preventative maintenance runs late
The application of operating system patches to the directory server ran late. The application of the patches completed at 9:45 am, thus delaying the use of calendar and mail services for 50 minutes. All services have now been restored.
contentDM renamed
The contentDM server was renamed from digitalassets to digitalcollections. This change was made so the name of the server better reflects the services on the server.
Mail server slowness I
The University mail server began to timeout clients and slow down at approximately noon today. The cause of the problem is unclear.
The number of process running on the server appearred to be at a normal load average. The servers response from secure shell was slow. It appearred as though the system was having a problem allocating resources for the processes running, but memory and disk space were both available.
We shut down process to try and identify the cause of the slowness. Imapd and Ipop3d were disabled with no luck. The web server was shutdown, no luck. Then mailman and sendmail. At this point things improved.
I disabled the webmail interface and brought sendmail and mailman backup. I then brought the web server backup. The server still appeared to be responding in a slower than normal, but somewhat timely manner. I restarted the ipop3d daemon and about five minutes later the imapd daemon. The server still looked good–slower than normal, but somewhat timely. After restarting the webmail interface things when into the tank.
We tried restarting the webmail server between 3:00 and 3:30 pm, but the unbareable slowness remained.
We then rebooted the mail server at about 3:40pm.
The server was back to normal after the reboot.
Mail service reconfiguration
The University e-mail service has been reconfigured to route as many messages as possible through the anti-virus gateway before they are sent off-campus or recieved by the University’s mail servers.
This additional stop for messages increases the delivery time, but in necessary to reduce the propogation of e-mail viruses and worms.
Cbord Odyssey unresponsive to registers
The Cbord server reported to not be responding to inquiries from the cash registers at approximately 12:30 PST. Upon inspection of the server, it was discovered that pcAnywhere was hung waiting for a connection. We received a VFEP error when tring to access the Odyssey Control panel.
It is our belief that C-Bord support encountered errors when doing routine maintenance that hung the server.
RESOLUTION: We rebooted the Odyssey server and had the cash registers re-inquire.
Corporate Time slow
Serveral users have recently reported slow response from the Corporate Time server. During the course of our investigation of the problem, several orphaned ssh process were found on Big Ben and the log files were found to be quite large.
The orphaned processes were dealt with at 16:00 PST. The Corporate Time service was stopped at approximately 20:30 PST. The log files were rotated and the service restarted. The University web server was restarted to allow users to access the Corporate Time web interface.