Email sent to liaisons:
Thursday April 17, Information Services will be performing critical changes on our database systems to improve recent performance concerns.
Access to all database systems including Cascade, Cascade Web, Banner, Famis, Basis and Millennium will be unavailable from 5:30 AM until 6 AM Thursday.
If you have questions or concerns regarding this outage, please contact Jenifer Brauen by telephone at ext. 2725, 253.879.2725, or via email at firstname.lastname@example.org. Thank you for understanding our need to conduct critical systems work.
– Jenifer Brauen
The Cbord server reported to not be responding to inquiries from the cash registers at approximately 12:30 PST. Upon inspection of the server, it was discovered that pcAnywhere was hung waiting for a connection. We received a VFEP error when tring to access the Odyssey Control panel.
It is our belief that C-Bord support encountered errors when doing routine maintenance that hung the server.
RESOLUTION: We rebooted the Odyssey server and had the cash registers re-inquire.
The Micros server, which controls the cash registers in the CBORD One-card system, ran out of disk space on the C: drive, causing serveral services to crash. This in turn caused the cash registers to go offline.
The pagefile was split among the C: and D: drives in the folowing manner: 500 MB on C: and 1 GB on D:, and serveral temp files and an old log file were deleted.
The NT backup was moved from 3:30 AM to 2:30 AM, because it was not finishing before opening time for the cafe and diner.
A corrupted dabase trigger in the basis Oracle instance caused the B.A.S.I.S. system to stop polling and recording events.
The faulty trigger was LENEL.EVENTS_SPEEDY_TI1, and the fault was the insertion of carriage returns within two AND operators.
Paul corrected and revalidated the trigger, which cleared the problem.
Ketron was rebooted to attempt to clear a “Polling Stopped” error with the B.A.S.I.S. software. It did not clear the problem.
The B.A.S.I.S. software on KETRON hung. A symptom of this was that none of the B.A.S.I.S. applications on workstations would launch. This included the System Administration and the Alarm Monitoring apps. KETRON was rebooted to clear the problem. Conversations with Lance Holloway indicated that the system should be rebooted more frequently than it is:
“I’m not as familiar with the reboot requirements of Win 2k as I was with NT. I know that NT could go maximum about two weeks before memory issues took over. Depending on the application. We’ve seen other sites lock up after a while if they have screens open for event stacks etc, and not be rebooted.”
The Micros (cash register) server was restarted