Outage

There was an outage this afternoon while I was with a client. I was not able to get a stable internet connection until just now, where “stable” means sitting at Honey Bear’s BBQ and dialing in over my slow cell phone. Because of my lack of access, I don’t think I’m going to be able to get a look at what happened before the logs are changed.

Would those of you with sudo access please take a look? Thanks.

Roaming Users

Roaming users was not working today for some reason. I restarted it after the reboots, but it blew it’s brains out on something.

IMAP before SMTP has been enough of a problem in the last few weeks that I’m getting really tired of it. I’ll be working on SASL and SSL for SMTP this weekend as a permanent, more stable solution. However, I wouldn’t expect to have it in production before the end of July. As with most SMTP changes, I want to test the crap out of it.

For now, if you’re having trouble sending email from your desktop client, log into the mission control page and check to see if you’re IP is listed in the roaming user table. If it is NOT, there is now an option to add it, with the IP you’re on already populated in the form. This goes in the same bin that popcheckd adds IP addresses into, so the window is open for an hour.

Another thing I’ll be adding is virus detection. As usual, I won’t delete any messages, but I will mark messages found with a virus so that they can be deleted. Personally, I’m not affected by them, but the number of viruses I’m getting today are starting to become a nuisance more than anything else.

Outage

vis.nu was down for about three hours today. I rebooted the machine this morning, and it restarted fine, but without the option for MySQL to make it not crash the entire system. That’s fixed, but I still need to get down to the roots of it and figure out a good, long term solution.

vis.nu Server Diagnostics

I have installed a set of server diagnostics on the server, and made them available all through a single interface. It was called the “admin screen”, but then clay said it was like a “vis.nu mission control”. I like that name better. :)

Right now, it’s got:

  • Nagios, a very nice network monitoring application
  • MailGraph, Which tracks Postfix SMTP performance
  • Symon, a system grapher
  • phpSysInfo, which is a good at-a-glance information page for vac
  • And a couple monitoring scripts I wrote up myself to watch POP3/IMAP and shell users

Anyway, if you’re wondering if there’s a problem with the system, this would be the place to start. To log in, you can use your shell account information. The page will not let you access it unless you’re on the SSL side of the server.

Stabilization, Fixes, MOTD System

I got used to the idea of updating vis.nu news on my journal.

The problem with the traditional MOTD system for vis.nu news is that very few of the people who use vis.nu see the MOTD on a daily basis, and that can be kind of important when there are system upgrades going on. Having all the updates on my site made it easy for people to keep up, since they could either go to the site, or see stuff in their various RSS readers.

So I’ve added some code to vis.nu that pulls the MOTD from the most recent journal entry tagged with the ‘vis.nu Networks” category. The HTML will be parsed to formatted flat text, and it’ll be converted to a usual looking /etc/motd. In addition, the front page of vis.nu now carries an RSS feed of what it says here.

There are some changes:

  • I think we have the MySQL problem licked. I had to limit the number of open files. This is suboptimal, but I’ll work on it later.
  • vis.nu Crashed over the weekend. I have no idea why, but it’s been stable since.
  • HTTP SSL now works
  • popcheckd is working again, so logging in over pop or imap will now let you SMTP out of vis.nu on port 25 or 2500.

I think I’m going to implement SMTP auth next week, if I have time.

MySQL problems

MySQL has been shutting down for no readily apparent reason. I’ve been trying to keep an eye on it, but with the moving thing going on, that’s been rough. I’ve asked clay to keep an eye on it over the weekend while I move. However, I think I have traced the issue to a login.conf thing. We’ll see if that works.

The DNS Changes are Happening Now

This are happening now. All the major services are running on the new server, and I’ve sent our old DNS servers the new address. I’ve sent .nu the new DNS servers (Thanks again, Gregory!) and I’m waiting for everything to transfer over. Once I’ve done that, I’ll copy the mail differences over to the new server, and we should be officially “done”.

Domains that use vis.nu and aren’t CNAMEs will need to update their records accordingly. Expect to hear from me soon.