The Cascade web interface was down for a several hours on the morning of October 8, 2003 following the regularly scheduled Information Services preventative maintenance period.
This outage was partially the result of some miscommunication within Information Services, and the improper shutdown of the Oracle Application Server on the effected system, Camano.
The backup server was brought on line around 11:00 am and remained up until late in the afternoon. The issues surrounding the failure are being addressed both within Information Services and with Oracle.
The production instance of the Oracle Application Server issue was resolved, and brought back on-line by the 5:00 pm.
The specific problem was the result of the improper shutdown of the Oracle Application Server (or IAS), which resulted in orphaned lock files and process identification files. Once all services were shutdown on the system and these files were removed, Information Services was able to successfully restart the Oracle Application Server.
Procedures for the shutdown and startup of IAS have been documented, and new procudures for preventative maintenance on this system have been developed.