Alachua Co. EMWIN Project

Alachua Co. EMWIN Project
Free weather alerts!

Thursday, March 6, 2014

AC-EMWIN Server Hiccup Last Night (Now Resolved)

Looks like the AC-EMWIN server had a hiccup, last night...

The AC-EMWIN server program locked up at about 8:45pm. All the other SUB-programs kept operating and passing on files which were apparently stuck in the ingest directory (the place where the program temporarily places new bulletins just downlinked, which are USUALLY then immediately *deleted* after they've been processed). So ZFPs and HWOs that didn't get deleted from the ingest dir (because the program locked up at just the right time) and the other subprograms just kept pulling them so they got duped to some Facebook pages a couple times overnight. As well, the radar images from 8pm last night kept getting resent all night until I noticed it early this morning after getting up.

As to how I even noticed that things were locked up... On getting up I noticed that the EMWIN test clients - here at the house, and which are connected to the main server remotely - weren't offering up the normal early morning beeps and sounds that occur when they receive the usual morning ZFPs and HWOs and paints things on the map. After years of being used this this, you notice something's not right almost immediately. This caused me to take a look around and check for problems. Indeed, the AC-SKYWARN web page hadn't refreshed the watch/warning map since 8:30pm last night. All the text bulletins on the AC-EMWIN page were "old" from late yesterday afternoon/evening. As well, no (current) bulletins were being sent to the surrounding area Emergency Management Facebook pages which were subscribed to us. (EEK!)

We're in the middle of some current ongoing weather so I didn't even bother to deal with checking logs and figuring things out. I just rebooted the machine entirely right away and that seemed to take care of it.

...So things are again working properly. I'll keep an eye on the server to make sure it doesn't happen again. The program usually worked very efficiently and I almost never had any problems with it and it's pretty stable so I don't expect another problem like that.

Coincidentally, the server hiccup started the process of me checking ALL resources for the possible problem and this included checking Alachua FreeNet. While it's not part of our server problem, I discovered that Alachua FreeNet is also apparently down, too. Something hung on THEIR server, too, last night and some of you may notice that your web page dirs are empty. (EEK!) I've notified AFN admin about it and they're taking a look. So in a way, you can thank our own server hiccup for causing a causal exploration which ended up discovering the AFN outage. :)