Sending e-mails using postdrop - possible ?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Sending e-mails using postdrop - possible ?

Kent
Hi All,

I have a MongoDB with a set of e-mails that I want to send.   I want to be able to track their delivery / bounce / delayed status - plus link any replies back to the original e-mail.

I have already written a c++ service to handle incoming e-mails  (by piping the incoming e-mails to my app) - which is still under development, but meeting that side of my needs.


Now I'm onto the actual sending side.  I've written a separate C++ service to run on the mail server to process the table of e-mails in the MongoDB  (From, To, Subject, Body, etc).

My original plan was to dump each e-mail to a file on disk, move to the  /var/spool/postfix/maildrop/  folder, set the permissions correctly and leave Postfix to process the queue.

However, so far i've been unable to get this to work:
> Jun  1 22:05:29 mx02 postfix/pickup[30400]: warning: maildrop/tmp1: unexpected EOF in data, record type 70 length 114
> Jun  1 22:05:29 mx02 postfix/pickup[30400]: warning: uid=0: unexpected or malformed record type -2


So, I've tried to use  postdrop < tmp1,   however this keeps rejecting my file with unexpected EOF and malformed input:
> Jun  1 22:31:15 mx02 postfix/postdrop[30871]: name_mask: all
> Jun  1 22:31:15 mx02 postfix/postdrop[30871]: inet_addr_local: configured 2 IPv4 addresses
> Jun  1 22:31:15 mx02 postfix/postdrop[30871]: inet_addr_local: configured 3 IPv6 addresses
> Jun  1 22:31:15 mx02 postfix/postdrop[30871]: chdir /var/spool/postfix
> Jun  1 22:31:15 mx02 postfix/postdrop[30871]: open maildrop/E312D4108722
> Jun  1 22:31:15 mx02 postfix/postdrop[30871]: send attr queue_id = E312D4108722
> Jun  1 22:31:15 mx02 postfix/postdrop[30871]: warning: stdin: unexpected EOF in data, record type 115 length 116
> Jun  1 22:31:15 mx02 postfix/postdrop[30871]: fatal: uid=0: malformed input


The tmp file is fairly simple:
> From: test <[hidden email]>
> To: kent <[hidden email]>
> Message-Id: [hidden email]
> Subject: this is a test
>
> Thanks for reading.
> .

Is there a way to make this work - or an alternative ?


I know one alternative is I can go down the socket / port 25 path - and that's my fallback plan.

thanks

Kent.


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Sending e-mails using postdrop - possible ?

Wietse Venema
Kent:

> Hi All,
>
> I have a MongoDB with a set of e-mails that I want to send.   I want to be able to track their delivery / bounce / delayed status - plus link any replies back to the original e-mail.
>
> I have already written a c++ service to handle incoming e-mails  (by piping the incoming e-mails to my app) - which is still under development, but meeting that side of my needs.
>
>
> Now I'm onto the actual sending side.  I've written a separate C++ service to run on the mail server to process the table of e-mails in the MongoDB  (From, To, Subject, Body, etc).
>
> My original plan was to dump each e-mail to a file on disk, move to the  /var/spool/postfix/maildrop/  folder, set the permissions correctly and leave Postfix to process the queue.

You MUST use Postfix interfaces: submit over SMTP, or use the Postfix
sendmail command. WRITING TO THE QUEUE IS NOT SUPPORTED.

        Wietse
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Sending e-mails using postdrop - possible ?

Kent
Hi Wietse,

Thanks - I'll stop trying to attempt to do it this way then.
The SMTP approach does have the advantage of realtime feedback on the 'rcpt to:' being accepted  and I can spread the load across multiple threads / connections.


Next question: Is there a way to get a 'delivered' notification  (ie.  once the e-mail has left Postfix and is now at the destination mail server - or at least, the next mail server in the chain).
(or any status change for that matter)


thanks

Kent.






> On 2/06/2017, at 11:50 AM, Wietse Venema <[hidden email]> wrote:
>
> Kent:
>> Hi All,
>>
>> I have a MongoDB with a set of e-mails that I want to send.   I want to be able to track their delivery / bounce / delayed status - plus link any replies back to the original e-mail.
>>
>> I have already written a c++ service to handle incoming e-mails  (by piping the incoming e-mails to my app) - which is still under development, but meeting that side of my needs.
>>
>>
>> Now I'm onto the actual sending side.  I've written a separate C++ service to run on the mail server to process the table of e-mails in the MongoDB  (From, To, Subject, Body, etc).
>>
>> My original plan was to dump each e-mail to a file on disk, move to the  /var/spool/postfix/maildrop/  folder, set the permissions correctly and leave Postfix to process the queue.
>
> You MUST use Postfix interfaces: submit over SMTP, or use the Postfix
> sendmail command. WRITING TO THE QUEUE IS NOT SUPPORTED.
>
> Wietse

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Sending e-mails using postdrop - possible ?

Kent
Hi Wietse,

Okay - I think I've worked out the answer to my second question if I use the sendmail command line, with the -N option.

> /usr/sbin/sendmail.postfix -f [hidden email] -N 'success, delay, failure' [hidden email] < tmp



Reading through documentation / mailing lists, it looks like this has to be requested per e-mail when using the sendmail command.

Is there anything I can set in the 'main.cf' file to change the default 'delay,failure' to be 'success,delay,failure'  ?

thanks

Kent.





> On 2/06/2017, at 11:59 AM, Kent <[hidden email]> wrote:
>
> Hi Wietse,
>
> Thanks - I'll stop trying to attempt to do it this way then.
> The SMTP approach does have the advantage of realtime feedback on the 'rcpt to:' being accepted  and I can spread the load across multiple threads / connections.
>
>
> Next question: Is there a way to get a 'delivered' notification  (ie.  once the e-mail has left Postfix and is now at the destination mail server - or at least, the next mail server in the chain).
> (or any status change for that matter)
>
>
> thanks
>
> Kent.
>
>
>
>
>
>
>> On 2/06/2017, at 11:50 AM, Wietse Venema <[hidden email]> wrote:
>>
>> Kent:
>>> Hi All,
>>>
>>> I have a MongoDB with a set of e-mails that I want to send.   I want to be able to track their delivery / bounce / delayed status - plus link any replies back to the original e-mail.
>>>
>>> I have already written a c++ service to handle incoming e-mails  (by piping the incoming e-mails to my app) - which is still under development, but meeting that side of my needs.
>>>
>>>
>>> Now I'm onto the actual sending side.  I've written a separate C++ service to run on the mail server to process the table of e-mails in the MongoDB  (From, To, Subject, Body, etc).
>>>
>>> My original plan was to dump each e-mail to a file on disk, move to the  /var/spool/postfix/maildrop/  folder, set the permissions correctly and leave Postfix to process the queue.
>>
>> You MUST use Postfix interfaces: submit over SMTP, or use the Postfix
>> sendmail command. WRITING TO THE QUEUE IS NOT SUPPORTED.
>>
>> Wietse
>

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Sending e-mails using postdrop - possible ?

Wietse Venema
Kent:

> Hi Wietse,
>
> Okay - I think I've worked out the answer to my second question if I use the sendmail command line, with the -N option.
>
> > /usr/sbin/sendmail.postfix -f [hidden email] -N 'success, delay, failure' [hidden email] < tmp
>
> Reading through documentation / mailing lists, it looks like this
> has to be requested per e-mail when using the sendmail command.
>
> Is there anything I can set in the 'main.cf' file to change the
> default 'delay,failure' to be 'success,delay,failure'  ?

You can change your sendmail command line.

You can also specify the notification requirements in SMTP, as
defined in RFC 3461.

        Wietse
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Sending e-mails using postdrop - possible ?

Kent
Hi Wietse,

Thanks for your guidance so far.

I'm trying to use the postfix sendmail command line - and have this working (code is still rough).   However, I'm now trying to get any output from the command.

To simulate an error, I've intentionally added an invalid -N option - which in my manual testing outputs an error about invalid option.

Below is my c code - I can't get the error back in my buffer.  Can you have a look at let me know if there's anything obvious I'm doing.

thanks

Kent.


------------------------------------------------------------------------------------------------------
int sendMail ( std::string from, std::string to, std::string message )
{
        int ret_code = -1;
       
        // /usr/sbin/sendmail.postfix -f '[hidden email]' -N 'success, delay, failure' '[hidden email]' < tmp

        const char * exec_path = prefs_PostfixUnixPath().c_str(); // /usr/sbin/sendmail.postfix
        std::string _f; _f.assign("-f "); _f.append( from ); // -f [hidden email]

        char * child_args[] = {(char *)"-N 'success,delay,failure,invalid'", (char *)_f.c_str(), (char *)to.c_str(), NULL};

        int pid;
        int pc[2]; /* Parent to child pipe */
        int cp[2]; /* Child to parent pipe */
        char ch;

        /* Make pipes */
        if( pipe(pc) < 0) { log_write("Can't make pipe: pc"); return -1; }
        if( pipe(cp) < 0) { log_write("Can't make pipe: cp"); return -1; }


        // The 'SIGCHLD' signal is triggered when the child fork completes - we want to ignore this.
        signal(SIGCHLD,SIG_IGN); // ignore 'SIGCHLD' signal


        /* Create a child to run command. */
        switch( pid = fork() )
        {
                case -1:
                                log_write("Error: Can't fork");
                                return -1;
                case 0:
                                                                        // === Child ===
                                close(1); // Close current stdout.
                                dup2( cp[1], 1 ); // Make stdout go to write end of pipe.
                                close(2);
                                dup2( cp[1], 2 ); // Make stderr go to write end of pipe.
                                close(0); // Close current stdin.
                                dup( pc[0] ); // Make stdin come from read end of pipe.
                               
                                close( pc[1] ); // Close what we don't need.
                                close( cp[0] );
                               
                                printf("Execute: %s\n", exec_path );
                               
                                execve( exec_path, child_args, NULL);
                               
                                printf("Error: execve failed"); // this will only occur if 'execve' had an error.
                                exit(1);
                               
                default:
                                                                        // === Parent ===
                                                                                                                                                // Close what we don't need.
                               
                                // the 'pipe' buffer size can vary - let's write in 1Kb chunks.  if 'out' == 0, buffer is full - wait and write again.
                                int out = write( pc[1], message.c_str(), message.length() ); // write the message to postfix... (this is the whole from:, To:, Subject, Body, etc.)
                               
                               
                                close(pc[1]); // close once written
                                close(cp[1]);

                                int status = 0;
                                char buffer[1024]; // Space to store the response - is 1Kb enough ???
                                int L = 0;
                                do {
                                        waitpid(pid, &status, 0);
                                        printf("PID Status: %d\n", status );
                               
                                        printf("\nOutput from child:\n");
                                        while( read(cp[0], &ch, 1) == 1) // read the reply (if any)  (how do we get the errors ???)
                                        {
                                                if( L < 1022 ) buffer[L++] = ch;
                                        }
                                        buffer[L] = 0;
                                }
                                while ( !WIFEXITED(status) );
                               
                                printf("Result: %s\n", buffer);
                                ret_code = 1; // need to check if successful or not and return true / false appropriately.
                                break;
        }


        return ret_code;
}








> On 3/06/2017, at 1:32 AM, Wietse Venema <[hidden email]> wrote:
>
> Kent:
>> Hi Wietse,
>>
>> Okay - I think I've worked out the answer to my second question if I use the sendmail command line, with the -N option.
>>
>>> /usr/sbin/sendmail.postfix -f [hidden email] -N 'success, delay, failure' [hidden email] < tmp
>>
>> Reading through documentation / mailing lists, it looks like this
>> has to be requested per e-mail when using the sendmail command.
>>
>> Is there anything I can set in the 'main.cf' file to change the
>> default 'delay,failure' to be 'success,delay,failure'  ?
>
> You can change your sendmail command line.
>
> You can also specify the notification requirements in SMTP, as
> defined in RFC 3461.
>
> Wietse

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Sending e-mails using postdrop - possible ?

Kent
In reply to this post by Wietse Venema
Hi Wietse,

Further to below - after more testing I am getting errors, but not warnings.  I don't know where they are going, but I'm not getting them back at all.

eg.  If I call:
/usr/sbin/sendmail.postfix -g
sendmail.postfix: invalid option -- 'g'
sendmail.postfix: invalid option -- 'g'
sendmail.postfix: fatal: usage: sendmail.postfix [options]

Then, I do get these errors in my application buffer.

However, if I instead call:
/usr/sbin/sendmail.postfix -N 'success,delay,failure,invalid' -f [hidden email] [hidden email]
sendmail.postfix: warning: unknown DSN NOTIFY command value "invalid" in "success,delay,failure,invalid"
sendmail.postfix: warning: bad -N option value -- ignored

Then, I don't get these back in either STDOUT or STDERR

I can live with this - as ultimately it doesn't stop the e-mail from being sent.


cheers

Kent.



Hi Wietse,

Thanks for your guidance so far.

I'm trying to use the postfix sendmail command line - and have this working (code is still rough).   However, I'm now trying to get any output from the command.

To simulate an error, I've intentionally added an invalid -N option - which in my manual testing outputs an error about invalid option.

Below is my c code - I can't get the error back in my buffer.  Can you have a look at let me know if there's anything obvious I'm doing.

thanks

Kent.


------------------------------------------------------------------------------------------------------
int sendMail ( std::string from, std::string to, std::string message )
{
int ret_code = -1;

// /usr/sbin/sendmail.postfix -f '[hidden email]' -N 'success, delay, failure' '[hidden email]' < tmp

const char * exec_path = prefs_PostfixUnixPath().c_str(); // /usr/sbin/sendmail.postfix
std::string _f; _f.assign("-f "); _f.append( from ); // -f [hidden email]

char * child_args[] = {(char *)"-N 'success,delay,failure,invalid'", (char *)_f.c_str(), (char *)to.c_str(), NULL};

int pid;
int pc[2]; /* Parent to child pipe */
int cp[2]; /* Child to parent pipe */
char ch;

/* Make pipes */
if( pipe(pc) < 0) { log_write("Can't make pipe: pc"); return -1; }
if( pipe(cp) < 0) { log_write("Can't make pipe: cp"); return -1; }


// The 'SIGCHLD' signal is triggered when the child fork completes - we want to ignore this.
signal(SIGCHLD,SIG_IGN); // ignore 'SIGCHLD' signal


/* Create a child to run command. */
switch( pid = fork() )
{
case -1:
log_write("Error: Can't fork");
return -1;
case 0:
// === Child ===
close(1); // Close current stdout.
dup2( cp[1], 1 ); // Make stdout go to write end of pipe.
close(2);
dup2( cp[1], 2 ); // Make stderr go to write end of pipe.
close(0); // Close current stdin.
dup( pc[0] ); // Make stdin come from read end of pipe.

close( pc[1] ); // Close what we don't need.
close( cp[0] );

printf("Execute: %s\n", exec_path );

execve( exec_path, child_args, NULL);

printf("Error: execve failed"); // this will only occur if 'execve' had an error.
exit(1);

default:
// === Parent ===
// Close what we don't need.

// the 'pipe' buffer size can vary - let's write in 1Kb chunks.  if 'out' == 0, buffer is full - wait and write again.
int out = write( pc[1], message.c_str(), message.length() ); // write the message to postfix... (this is the whole from:, To:, Subject, Body, etc.)


close(pc[1]); // close once written
close(cp[1]);

int status = 0;
char buffer[1024]; // Space to store the response - is 1Kb enough ???
int L = 0;
do {
waitpid(pid, &status, 0);
printf("PID Status: %d\n", status );

printf("\nOutput from child:\n");
while( read(cp[0], &ch, 1) == 1) // read the reply (if any)  (how do we get the errors ???)
{
if( L < 1022 ) buffer[L++] = ch;
}
buffer[L] = 0;
}
while ( !WIFEXITED(status) );

printf("Result: %s\n", buffer);
ret_code = 1; // need to check if successful or not and return true / false appropriately.
break;
}


return ret_code;
}








On 3/06/2017, at 1:32 AM, Wietse Venema <[hidden email]> wrote:

Kent:
Hi Wietse,

Okay - I think I've worked out the answer to my second question if I use the sendmail command line, with the -N option.

/usr/sbin/sendmail.postfix -f [hidden email] -N 'success, delay, failure' [hidden email] < tmp

Reading through documentation / mailing lists, it looks like this
has to be requested per e-mail when using the sendmail command.

Is there anything I can set in the 'main.cf' file to change the
default 'delay,failure' to be 'success,delay,failure'  ?

You can change your sendmail command line.

You can also specify the notification requirements in SMTP, as
defined in RFC 3461.

Wietse

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Sending e-mails using postdrop - possible ?

Wietse Venema
In reply to this post by Kent
Kent:
> Hi Wietse,
>
> Thanks for your guidance so far.
>
> I'm trying to use the postfix sendmail command line - and have
> this working (code is still rough).   However, I'm now trying to
> get any output from the command.

In case of error it returns an exit status as defined in
/usr/include/sysexits.h. This is common UNIX practice/

        Wietse
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Sending e-mails using postdrop - possible ?

Kent
Hi Wietse,

Thanks - as per my last e-mail,  I am getting the errors back.  It's just the warnings I'm not getting back.

But, I can live with this as the warnings will be my coding errors - so apart from my test with intentional issues, it hopefully shouldn't happen.


cheers

Kent.


> On 5/06/2017, at 10:16 AM, Wietse Venema <[hidden email]> wrote:
>
> Kent:
>> Hi Wietse,
>>
>> Thanks for your guidance so far.
>>
>> I'm trying to use the postfix sendmail command line - and have
>> this working (code is still rough).   However, I'm now trying to
>> get any output from the command.
>
> In case of error it returns an exit status as defined in
> /usr/include/sysexits.h. This is common UNIX practice/
>
> Wietse

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Sending e-mails using postdrop - possible ?

Wietse Venema
Kent:
> Hi Wietse,
>
> Thanks - as per my last e-mail,  I am getting the errors back.
> It's just the warnings I'm not getting back.

If you are concerned about mistakes, sendmail command line is not
the whole story. There is a lot more that can go wrong.

The MAILLOG file has a complete(*) record of successful and
unsuccessful deliveries, including warnings that did not result
in failure.

http://www.postfix.org/DEBUG_README.html#logging

        Wietse

(*) modulo system-xxx-d rate limiting, but you still have a
choice of operating systems.
Loading...