/dev/sda1 is full, what can I clean?

Got problems with your B2 or B3? Share and get helped!
nobody
Posts: 226
Joined: 10 Mar 2012, 14:46

Re: /dev/sda1 is full, what can I clean?

Post by nobody » 29 Sep 2012, 06:29

Then why, according to you, is the mail stuck in the queue?

Gordon
Posts: 1384
Joined: 10 Aug 2011, 03:18

Re: /dev/sda1 is full, what can I clean?

Post by Gordon » 29 Sep 2012, 06:49

Most likely because they are bounces. This can happen if the mail server is configured for a real domain and does not include the server as a subdomain. Messages to root then become emails to root@b3.somedomain and sent out onto the ISP's smarthost. With the domain unknown the message is then returned and the server will attempt to deliver the bounce message to root@b3.somedomain. Postfix has a mechanism that will stop this from going on indefinitely, but you'll still see a multiplication by 10. This means that if you have horde alarms enabled in cron (which is the default) your mail queue will expand with at least ten emails every 5 minutes.

nobody
Posts: 226
Joined: 10 Mar 2012, 14:46

Re: /dev/sda1 is full, what can I clean?

Post by nobody » 29 Sep 2012, 09:41

Im sorry but you are making this up. Bounced messages that return are double-bounced and disappear. The ones that do not deliver because of DNS are also bounced after queue_lifetime is matched. The only messages that are not removed from queues are the from dthe delivery agent, as there is no place to return these mesages to.

Gordon
Posts: 1384
Joined: 10 Aug 2011, 03:18

Re: /dev/sda1 is full, what can I clean?

Post by Gordon » 30 Sep 2012, 04:48

I'll recheck the. Point is that I have had floodings like these for several reasons and if memory serves correct one of them was that bounces from the spam filter would fail. Obviously this can be the result from a bad server config in regards to queue lifetime.

Another one, but it is really unlikely in this case that this is the original reason, is that local mail delivery fails on account of a full disk. What I did in this case was use the Inn news server to store spam messages, using a mailpost command as alias. This would give me the chance to catch bad positives and with the retention set to one week I wouldn't have to bother clearing the box every now and then. What happened was that Innd started to refuse accepting new messages as the disk was nearing to become completely full, but then of course the mail queue started to grow and made things even worse.

I guess the full explanation for what happened here is that emails that are being send to a valid local user where mail delivery fails for some reason will in fact be kept in queue indefinitely (or until the issue is resolved). Being a production server I never bothered to verify this though, because solving the issue was definitely had preference.

Perhaps RichoDemus can check his maildir configuration. There should be a directory called "Mail" in roots home and inside you should have three other directories called "cur", "new" and "tmp". All of these directories must be accessible to root only (mode 700). You can use the maildirmake command to create this tree if it doesn't exist.

nobody
Posts: 226
Joined: 10 Mar 2012, 14:46

Re: /dev/sda1 is full, what can I clean?

Post by nobody » 30 Sep 2012, 09:13

of course, the answer is blatantly obvious if only the OP would run

Code: Select all

mailq

DanielM
Posts: 637
Joined: 28 Mar 2008, 06:37
Location: Sweden

Re: /dev/sda1 is full, what can I clean?

Post by DanielM » 30 Sep 2012, 14:12

nobody wrote:of course, the answer is blatantly obvious if only the OP would run

Code: Select all

mailq
You ought to read the whole thread before making statements about what is obvious and what isn't.

/Daniel

nobody
Posts: 226
Joined: 10 Mar 2012, 14:46

Re: /dev/sda1 is full, what can I clean?

Post by nobody » 02 Oct 2012, 02:12

ah... indeed...stupid.

should be

Code: Select all

postcat C80B4307E7
the postcat command will show the content of a specific queued mail, identified by the queue ID from the mailq command.

Gordon
Posts: 1384
Joined: 10 Aug 2011, 03:18

Re: /dev/sda1 is full, what can I clean?

Post by Gordon » 02 Oct 2012, 02:48

nobody wrote:the postcat command will show the content of a specific queued mail, identified by the queue ID from the mailq command.
True. But with a stuck mail queue you're bound to find loads and loads of headers which may be even more confusing. The first question here is whether OP has actually configured a real email domain and opened up the SMTP port for this purpose. This might make him vulnerable to either receive spam or be used as a relay for sending spam. If he didn't then the only logical explanation can be that the emails are system messages sent to root.

Now you can choose to clear the mail queue, but that obviously does not solve the problem as the queue will start to build up again with messages that are meant to warn you that something is wrong. To make things interesting, fixing roots maildir may prevent reoccurence of the mail queue build up, but since roots home is also on /dev/sda1 not solve the current issue. The smart thing would therefore be to route the emails to a user who has its homedir on /dev/mapper/bubba-storage - admin seems the logical choice here, being the systems administrator through the web interface already.

nobody
Posts: 226
Joined: 10 Mar 2012, 14:46

Re: /dev/sda1 is full, what can I clean?

Post by nobody » 02 Oct 2012, 06:14

Yes, but if you look at past threads that deal with this same issue, those loads of emails are always because some process is misbehaving and sending tons of mails. Fixing that issue also fixes the mail flood. But you need to read the mails first right?

Although admin is the logical choice for a root-mail recpipient, it is unlikely that regular user will log in as admin and check the mail on a routine basis. I'd say edit /etc/aliases to have the mails go the the users preferred address, beit internal or external (hotmal, gmail etc). This thing is, there really should not be so many mails!

Gordon
Posts: 1384
Joined: 10 Aug 2011, 03:18

Re: /dev/sda1 is full, what can I clean?

Post by Gordon » 02 Oct 2012, 09:50

Nononono!

Specifically because issues like these can occur you should never forward these messages to any web based email application. For one they will hate you for it and second it will take forever to search and clean up similar messages if you can only delete a maximum of twenty at a time.

The B3 has an IMAP server (dovecot) to which you can even connect using Outlook. Just add a new email account, provide credentials and it will start downloading the message list. Read, search, select and delete with the ease from a desktop application that you are familiar with.

nobody
Posts: 226
Joined: 10 Mar 2012, 14:46

Re: /dev/sda1 is full, what can I clean?

Post by nobody » 02 Oct 2012, 13:37

yes but not as admin because you'll forget to check.. Well maybe not you, but most people =)

Gordon
Posts: 1384
Joined: 10 Aug 2011, 03:18

Re: /dev/sda1 is full, what can I clean?

Post by Gordon » 02 Oct 2012, 14:32

nobody wrote:yes but not as admin because you'll forget to check.. Well maybe not you, but most people =)
Not if it's in the same screen as the one you're using to read your personal- and work related emails. I use thunderbird myself and the admin account is just another email folder I can select in the left hand panel and shows bold if it contains unread emails. I just changed my config 5 days ago on account of this topic and I deleted somewhere around 86.000 php warning emails from the horde alarm script, which comes down to having run the related cron job for about 9 months and 3 weeks before I changed its schedule. If I'd seen those before I would have acted on it a lot sooner.

RichoDemus
Posts: 22
Joined: 27 Mar 2012, 17:39

Re: /dev/sda1 is full, what can I clean?

Post by RichoDemus » 10 Oct 2012, 12:29

nobody wrote:ah... indeed...stupid.

should be

Code: Select all

postcat C80B4307E7
the postcat command will show the content of a specific queued mail, identified by the queue ID from the mailq command.
Hi, sorry for being absent, the forum stopped emailing me about thread updates, here is the output from that command:

Code: Select all

usr@b3:~/tons_of_emails/$ /usr/sbin/postcat CAAC637976 > ~/documents/postcat_of_mail_CAAC637976
https://www.dropbox.com/s/8jgixkf08u7mu ... 637976.txt

upon reading the contents I have to admit that I did shutdown mysql a long time ago to save RAM :oops: so that might be the reason

nobody
Posts: 226
Joined: 10 Mar 2012, 14:46

Re: /dev/sda1 is full, what can I clean?

Post by nobody » 10 Oct 2012, 13:58

So this is a cronjob for some alarms. I think i is safe to turn this cronjob off, but there are som threads on the foru. That discusses what this job does.

Gordon
Posts: 1384
Joined: 10 Aug 2011, 03:18

Re: /dev/sda1 is full, what can I clean?

Post by Gordon » 10 Oct 2012, 14:36

That's the alarm script I mentioned on the previous page. It's meant for calendar notifications. If you don't use Horde (which won't function anyway without mysql active) it's safe to disable this cronjob. However, that still does not fix the problem of the mail not being delivered - it just limits the speed of the queue build-up.

Post Reply