WebCT Sporadic Errors

We received the following late last night from our WebCT Administrators:

We’ve received confirmed reports of sporadic system exceptions within WebCT, not confined to specific tools (quizzes, file download, etc). We believe this may be related to database problems, which are currently being investigated.

We have determined the source of the WebCT database problems, and are taking steps to repair it. A data recovery process will be running overnight, and should be completed in the morning. We’ll have further status updates then.

Please email som-elearning@utdallas.edu if you are still having problems this morning. Thank you for your patience.

Update (11:25am): We are hearing from students and faculty that problems still exist. We just received an update from our admins a few minutes ago letting us know that they are still working on the issue.

Update (4:17pm): Our administrators are working directly with Blackboard/WebCT to fix the problem. They are hoping to have this issue resolved today; however, they say it could be as late as tomorrow early morning before the server is running normally. We apologize for any inconvenience this causes. We will post more updates as we receive them.

Update (9:55am, Sept.18): We do not yet have any update from our WebCT Admins; however, we do know that the problem still exists. We will post any information we receive. Thank you for your patience.

Update (10:41am): Just received this from our admins:

WebCT is still experiencing the System Exception Errors that manifested on Sunday. We are still in the process of repairing the database error. We are in contact with WebCT’s technical support to expedite solution discovery. We are working to resolve this issue as quickly as possible with no further data loss.

Update (5:54pm): We just received an email from our administrators. The email was sent out to all university faculty, staff, and students. To read the complete email, which includes issues that have been verified as directly related to this problem, please check your @utdallas.edu email account.

We are near the end of a second major data recovery effort towards resolving the system errors affecting WebCT. This evening, we should conclude the course of action WebCT Technical Support believes will repair the damage WebCT has suffered.

Update (8:21am, Sept.19): WebCT is currently down due to emergency maintenance. The current WebCT site has a link to the Server Outage website for updates. We will continue posting updates on this blog as we get information.

Update (9:31am): Received the following from our WebCT admins:

The data recovery effort completed last night has not brought a solution to the ongoing problems with WebCT. We are now attempting an expanded data recovery effort, which is currently requiring the database to be down. Based on the progress of the last three hours, this recovery will take at least another six hours to run. We are attempting to resolve this issue with speed as well as caution, in order to prevent or minimize data loss.

Update (6:44pm): Our administrators are now working with Blackboard/WebCT and Oracle on the database issues that are the cause of our server problems. They have determined the reason why previous attempts to restore WebCT to normal functionality have failed. Unfortunately, we are told, the recovery process they have just started is to take another eight hours to completed; however, they are very confident that this will fix our server issue and restore WebCT to full functionality. As always, we will post any further information we receive.

Update (6:56pm): I just noticed on the Server Outage website that they have posted the following:

WebCT outage is expected to last through 8:00am, 20 Sep 2007.

Update (7:23pm): Just received the following from our admins:

We’ve been working with the top tiers of support at both WebCT and Oracle today, and after hours of collaborative investigation, we have discovered the root cause of errors we have been running into with our data recovery efforts. Those errors have been rectified, and we are now cleared to perform another data recovery. This will run overnight, and WebCT will continue to be unavailable while this process runs. At this point, we are expecting to bring WebCT back online by 9am tomorrow morning, with no loss to data within the system. We appreciate your continued patience through this extremely frustrating situation.

Update (8:45am, Sept.20): Just received the following from our admins:

I am relieved to report that WebCT is up and functioning again as of 8:00am this morning. Initial testing shows that the errors encountered since Sunday have been resolved. There does not appear to have been any data loss in the system recovery, but we do encourage faculty and students to verify any changes or submission made recently. We thank you for your patience shown during this ordeal.

Advertisements

4 Responses to WebCT Sporadic Errors

  1. Tracey says:

    Any new updates on the progress? It was supposed to be up around 5:00, but is still down at 6:45. Not trying to bug you guys, but I will send out individual emails if it is anticipated to continue into tomorrow.

    Thanks!

    Tracey

  2. Katrina says:

    Hi Tracey. I just posted an update we just received from our admins. Unfortunately, the outage will last another estimated eight hours, we are told; however, they are confident this latest recovery attempt will restore WebCT to full functionality. We’re very sorry for this inconvenience. ~Katrina

  3. James says:

    How does this outage affect exams/homeworks/quizzes? I am in the middle of a 6-day exam window and have lost extremely valuable study time.

  4. Katrina says:

    We will be working with all of our professors in extending deadlines for assignments and assessments that have been affected by this outage.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: