Fwd: Re: Message-ID

classic Classic list List threaded Threaded
4 messages Options
CP
Reply | Threaded
Open this post in threaded view
|

Fwd: Re: Message-ID

CP

on 01/27/2018 09:30 PM, Karol Augustin wrote:

> On 2018-01-27 17:24, CP wrote:
>> On 01/27/2018 02:05 PM, Matus UHLAR - fantomas wrote:
>>>>> On 27.01.18 11:16, CP wrote:
>>>>>> I'm trying to eliminate a problem with duplicate emails in alias
>>>>>> expansion and I have seen that some (local generated and SMTP also)
>>>>>> messages don't have the Message-ID header .
>>> I should add that such duplicity should not happen when using aliases. Also, it may happen that two people get different copies of the same mail
>>> (e.g. from this list).
>>>
>>> Are you sure your duplicities can't be fixed other way?
>> In what way ? check alias expansion ?
>>
>> The thing is  I have some groups say group1 group2 and group3
>>
>> there may be users  that are part of only one group but there are
>> others that maybe in all groups .
>> Until I implemented lmtp and duplicate supression from it (dovecot) a
>> user who was in all 3 groups
>> would receive the message 3 times (and if send to his own email
>> address 4 !)  The problem persists
>> even with lmtp and messages that don't have Message-ID in header  ( I
>> guess  this is the criteria for supressing
>> duplicates) .
>>
>> How can I  track down this ?
>>
>>
>> Thanks  for your time
> Dovecot documentation says: "For more complex applications, the
> duplicate test can also use the content of a specific header field or
> other parts of the message."
> https://wiki.dovecot.org/Pigeonhole/Sieve/Extensions/Duplicate
>
> I couldn't find specific example but maybe dovecot list is a better
> place to ask. It might be possible to detect duplicates using different
> headers. Anyway just Message-ID might be not a perfect solution.
>
> If I understand your issue correctly I think that I have the same
> problem that the user is a part of an alias office@ and he gets
> duplicates of all mails send to him and copied to office@.
>
> So @office -> guy1@, @guy2. If the e-mail is addressed to guy1@, office@
> then Guy1 gets two copies and Guy2 only one. Often this e-mail is sent
> in two different SMTP sessions so I don't see any possibility that
> Postfix can catch that.

With Message-ID header and dovecot lmtp (I think lda works also) you
eliminate  (a lot!!) of these duplicates so try it if you have the same
problem.
In my case I have a robot account which sends (from LAN)  emails and reports
  in two groups many times a day .  These don't get a Message-ID header
so all
users who are in both  groups get all these messages times two  a day.


George

Reply | Threaded
Open this post in threaded view
|

Re: Fwd: Re: Message-ID

Karol Augustin
On 2018-01-28 9:25, CP wrote:
> on 01/27/2018 09:30 PM, Karol Augustin wrote:

>
> With Message-ID header and dovecot lmtp (I think lda works also) you
> eliminate  (a lot!!) of these duplicates so try it if you have the same
> problem.
> In my case I have a robot account which sends (from LAN)  emails and reports
>  in two groups many times a day .  These don't get a Message-ID header
> so all
> users who are in both  groups get all these messages times two  a day.
>
>
> George

I have just configured it and it's working great.
if allof (not exists "list-id", duplicate :seconds 7200 )
{
  discard;
  stop;
}


So in your case you might configure rule that if sender is robot use
Date: or Subject: or other header instead of Message-ID. As it will only
apply to the robot e-mails you don't risk side effects on other e-mail.

Check 3.1 in https://tools.ietf.org/html/rfc7352

Also you can use unique:id somehow with other rules to mark e-mails that
are sent by bot. Let's say that the bot is sending e-mails not more that
once an hour. Than you can deduplicate all e-mails From:
bot and set the timeout to kick in before next time bot sends e-mail.

Karol


--
Karol Augustin
[hidden email]
http://karolaugustin.pl/
+353 85 775 5312
CP
Reply | Threaded
Open this post in threaded view
|

Re: Fwd: Re: Message-ID

CP
On 01/28/2018 12:04 PM, Karol Augustin wrote:

> On 2018-01-28 9:25, CP wrote:
>> on 01/27/2018 09:30 PM, Karol Augustin wrote:
>> With Message-ID header and dovecot lmtp (I think lda works also) you
>> eliminate  (a lot!!) of these duplicates so try it if you have the same
>> problem.
>> In my case I have a robot account which sends (from LAN)  emails and reports
>>   in two groups many times a day .  These don't get a Message-ID header
>> so all
>> users who are in both  groups get all these messages times two  a day.
>>
>>
>> George
> I have just configured it and it's working great.
> if allof (not exists "list-id", duplicate :seconds 7200 )
> {
>    discard;
>    stop;
> }
>
>
> So in your case you might configure rule that if sender is robot use
> Date: or Subject: or other header instead of Message-ID. As it will only
> apply to the robot e-mails you don't risk side effects on other e-mail.
>
> Check 3.1 in https://tools.ietf.org/html/rfc7352
>
> Also you can use unique:id somehow with other rules to mark e-mails that
> are sent by bot. Let's say that the bot is sending e-mails not more that
> once an hour. Than you can deduplicate all e-mails From:
> bot and set the timeout to kick in before next time bot sends e-mail.
>
> Karol
>
>
Thank you,   will try  later on and reply here, thanks again !

George
CP
Reply | Threaded
Open this post in threaded view
|

Re: Message-ID

CP
In reply to this post by Karol Augustin
On 01/28/2018 12:04 PM, Karol Augustin wrote:

> On 2018-01-28 9:25, CP wrote:
>> on 01/27/2018 09:30 PM, Karol Augustin wrote:
>> With Message-ID header and dovecot lmtp (I think lda works also) you
>> eliminate  (a lot!!) of these duplicates so try it if you have the same
>> problem.
>> In my case I have a robot account which sends (from LAN)  emails and reports
>>   in two groups many times a day .  These don't get a Message-ID header
>> so all
>> users who are in both  groups get all these messages times two  a day.
>>
>>
>> George
> I have just configured it and it's working great.
> if allof (not exists "list-id", duplicate :seconds 7200 )
> {
>    discard;
>    stop;
> }
>
>
> So in your case you might configure rule that if sender is robot use
> Date: or Subject: or other header instead of Message-ID. As it will only
> apply to the robot e-mails you don't risk side effects on other e-mail.
>
> Check 3.1 in https://tools.ietf.org/html/rfc7352
>
> Also you can use unique:id somehow with other rules to mark e-mails that
> are sent by bot. Let's say that the bot is sending e-mails not more that
> once an hour. Than you can deduplicate all e-mails From:
> bot and set the timeout to kick in before next time bot sends e-mail.
>
> Karol
>
>

Karol

yes it  works  great . In my case  forcing to add message-ID on messages
that don't carry one,  helped to track these messages without any side
effects
(until now of course !)


thanks a lot

George