Thursday, November 4, 2010

Dear Users,

once again we have to apologize for BibSonomy's downtime during Tuesday night from 11:47pm until 2:45pm (CET). We spend a lot of time, designing BibSonomy's server setup fail-safe. I will summarize our setup for short - on the one hand to explain to you what happened, on the other hand to present you our means for ensuring your data's integrity.

Whenever your browser communicates with BibSonomy, several requests are sent to our servers. Each request arrives at our so-called proxy-server which just delegates the request to our main server. For security reasons, the proxy-server is located in a separated network segment (a demilitarized zone).

Our main server hosts BibSonomy's actual web application as well as the main database, where all your posts are stored (as well as your document and profile pictures). The database is replicated on a separate server. Additionally, each night the database is stored using our university's backup facilities which are located in a different building (in a different city actually). Thus each data entry is stored at three different places and could be restored even in worst case scenarios. Of course are our servers backed up by according batteries which allow to bridge power-failures for up to 30 minutes. Furthermore, each system is monitored and our administrators (one of which I am) are informed using short text messages services.

The last downtime was caused by the proxy-server which stopped working from one moment to the other. We already prepared a replacement for the server and now we are testing to run two proxy-servers in parallel such that one server may crash without causing further downtime.

But our current 'big project' is to setup a complete BibSonomy installation in an other location (located in an other city) such that each of your browser's requests will be sent to it, if we should encounter any downtime here on our side.

I spared the bad news for the end of this post: Due to comprehensive renovations in our building, we prepare for the worst. We were assured that with a 99% probability our server room should not be affected. But we all know Murphy's law...
We were offered to the server room of an other department, which we currently prepare for hosting our servers.

I want to excuse any inconvenience which was caused by our server problems and I hope for you understanding if we should face a new problem during the two weeks. We have our next release readily waiting to be published but we postponed it until we know what the renovation work may cause.

Anyway I wish you happy posting,

.folke