Maillog error - warn: Unescaped left brace in regex is deprecated here (and will be fata))

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

Maillog error - warn: Unescaped left brace in regex is deprecated here (and will be fata))

Paul Schmehl-3
I'm seeing this error in the logs:

warn: Unescaped left brace in regex is deprecated here (and will be fata))

It comes right after this:

status=sent (delivered via filter service

The filter service uses filter.sh (stolen from the docs) and spamassassin.
Is it safe to assume this is a code problem in spam assassin (which I
assume they will fix at some point)?

Googling says it's a perl error, so it can't be the filter shell script.

Paul Schmehl
Independent Researcher
Reply | Threaded
Open this post in threaded view
|

Re: Maillog error - warn: Unescaped left brace in regex is deprecated here (and will be fata))

Bill Cole-3
On 14 Dec 2018, at 0:46, Paul Schmehl wrote:

> I'm seeing this error in the logs:
>
> warn: Unescaped left brace in regex is deprecated here (and will be
> fata))

This is a new warning in Perl 5.26. The use of curly-brace regex
enumeration ranges with an implied zero first term ( e.g. {,5} instead
of {0,5} ) will be a fatal error in a later version (maybe 5.28, I have
not checked...)

It would be a wise choice to update ALL of your Perl modules to the
latest releases, as this issue can arise in non-obvious places...

> It comes right after this:
>
> status=sent (delivered via filter service
>
> The filter service uses filter.sh (stolen from the docs) and
> spamassassin. Is it safe to assume this is a code problem in spam
> assassin (which I assume they will fix at some point)?

Not if you're using the current release (3.4.2) of SpamAssassin. As one
of the developers who worked on that specific issue, I am sure that we
believed all cases of that problem that were internal to the
SpamAssassin framework were fixed in 3.4.2. There are a lot of modules
which SA depends on, some of which can also cause that warning.


--
Bill Cole
[hidden email] or [hidden email]
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Available For Hire: https://linkedin.com/in/billcole
Reply | Threaded
Open this post in threaded view
|

Re: Maillog error - warn: Unescaped left brace in regex is deprecated here (and will be fata))

Paul Schmehl-3
--On December 14, 2018 at 3:32:11 PM -0500 Bill Cole
<[hidden email]> wrote:

> On 14 Dec 2018, at 0:46, Paul Schmehl wrote:
>
>> I'm seeing this error in the logs:
>>
>> warn: Unescaped left brace in regex is deprecated here (and will be
>> fata))
>
> This is a new warning in Perl 5.26. The use of curly-brace regex
> enumeration ranges with an implied zero first term ( e.g. {,5} instead of
> {0,5} ) will be a fatal error in a later version (maybe 5.28, I have not
> checked...)
>
> It would be a wise choice to update ALL of your Perl modules to the
> latest releases, as this issue can arise in non-obvious places...
>
>> It comes right after this:
>>
>> status=sent (delivered via filter service
>>
>> The filter service uses filter.sh (stolen from the docs) and
>> spamassassin. Is it safe to assume this is a code problem in spam
>> assassin (which I assume they will fix at some point)?
>
> Not if you're using the current release (3.4.2) of SpamAssassin. As one
> of the developers who worked on that specific issue, I am sure that we
> believed all cases of that problem that were internal to the SpamAssassin
> framework were fixed in 3.4.2. There are a lot of modules which SA
> depends on, some of which can also cause that warning.

# spamassassin -V
SpamAssassin version 3.4.2
running on Perl version 5.26.3

Thanks, Bill. Guess I'll start rummaging through the modules.

Paul Schmehl
Independent Researcher
Reply | Threaded
Open this post in threaded view
|

Re: Maillog error - warn: Unescaped left brace in regex is deprecated here (and will be fata))

Bill Cole-3
In reply to this post by Bill Cole-3
On 14 Dec 2018, at 15:32, Bill Cole wrote:

> On 14 Dec 2018, at 0:46, Paul Schmehl wrote:
>
>> I'm seeing this error in the logs:
>>
>> warn: Unescaped left brace in regex is deprecated here (and will be
>> fata))

NOTE: that message should specify the source of the error. If it does
not, something in your logging plumbing is mangling error messages.

Also note: there is at least one 3rd-party ruleset (http://sa.zmi.at)
which appears to have had a typo ('.' where a ',' should have been)
which also resulted in this error message in a recent version. See
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7669 for a report of
this issue (which is not a bug in SA per se, as the rules with the typo
are an independent distribution.)

--
Bill Cole
[hidden email] or [hidden email]
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Available For Hire: https://linkedin.com/in/billcole
Reply | Threaded
Open this post in threaded view
|

Re: Maillog error - warn: Unescaped left brace in regex is deprecated here (and will be fata))

Paul Schmehl-3
In reply to this post by Bill Cole-3
--On December 14, 2018 at 3:32:11 PM -0500 Bill Cole
<[hidden email]> wrote:

> On 14 Dec 2018, at 0:46, Paul Schmehl wrote:
>
>> I'm seeing this error in the logs:
>>
>> warn: Unescaped left brace in regex is deprecated here (and will be
>> fata))
>
> It would be a wise choice to update ALL of your Perl modules to the
> latest releases, as this issue can arise in non-obvious places...
>

So, I would assume it would be one of these modules?

BUILD_DEPENDS=  p5-Encode-Detect>=0:converters/p5-Encode-Detect \
                p5-HTML-Parser>=3.46:www/p5-HTML-Parser \
                p5-HTTP-Date>=0:www/p5-HTTP-Date \
                p5-Net-DNS>=0.63:dns/p5-Net-DNS \
                p5-NetAddr-IP>=4.010:net-mgmt/p5-NetAddr-IP
RUN_DEPENDS:=   ${BUILD_DEPENDS} \
                p5-Net-IDN-Encode>=0:textproc/p5-Net-IDN-Encode \
                p5-Net-LibIDN>=0:dns/p5-Net-LibIDN \
                p5-URI>=0:net/p5-URI \
                re2c>=.12.0:devel/re2c

Paul Schmehl
Independent Researcher
Reply | Threaded
Open this post in threaded view
|

Re: Maillog error - warn: Unescaped left brace in regex is deprecated here (and will be fata))

Bill Cole-3
On 14 Dec 2018, at 17:35, Paul Schmehl wrote:

> --On December 14, 2018 at 3:32:11 PM -0500 Bill Cole
> <[hidden email]> wrote:
>
>> On 14 Dec 2018, at 0:46, Paul Schmehl wrote:
>>
>>> I'm seeing this error in the logs:
>>>
>>> warn: Unescaped left brace in regex is deprecated here (and will be
>>> fata))
>>
>> It would be a wise choice to update ALL of your Perl modules to the
>> latest releases, as this issue can arise in non-obvious places...
>>
>
> So, I would assume it would be one of these modules?
>
> BUILD_DEPENDS=  p5-Encode-Detect>=0:converters/p5-Encode-Detect \
>                p5-HTML-Parser>=3.46:www/p5-HTML-Parser \
>                p5-HTTP-Date>=0:www/p5-HTTP-Date \
>                p5-Net-DNS>=0.63:dns/p5-Net-DNS \
>                p5-NetAddr-IP>=4.010:net-mgmt/p5-NetAddr-IP
> RUN_DEPENDS:=   ${BUILD_DEPENDS} \
>                p5-Net-IDN-Encode>=0:textproc/p5-Net-IDN-Encode \
>                p5-Net-LibIDN>=0:dns/p5-Net-LibIDN \
>                p5-URI>=0:net/p5-URI \
>                re2c>=.12.0:devel/re2c

Those or any of the modules upon which they depend, recursively.

There are also Perl modules that will NOT appear as hard dependencies of
SA anywhere because they are optional or only required for particular
features. When running the build of SA, these will be looked for and
noted by Makefile.PL.

It is also possible (even likely) that this particular error is the
result of problematic syntax in a SpamAssassin rules file, because many
rules include regexes.

--
Bill Cole
[hidden email] or [hidden email]
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Available For Hire: https://linkedin.com/in/billcole
Reply | Threaded
Open this post in threaded view
|

Re: Maillog error - warn: Unescaped left brace in regex is deprecated here (and will be fata))

Paul Schmehl-3
In reply to this post by Bill Cole-3
--On December 14, 2018 at 4:09:08 PM -0500 Bill Cole
<[hidden email]> wrote:

> On 14 Dec 2018, at 15:32, Bill Cole wrote:
>
>> On 14 Dec 2018, at 0:46, Paul Schmehl wrote:
>>
>>> I'm seeing this error in the logs:
>>>
>>> warn: Unescaped left brace in regex is deprecated here (and will be
>>> fata))
>
> NOTE: that message should specify the source of the error. If it does
> not, something in your logging plumbing is mangling error messages.

I tried increasing the debug level and even added -v to the pipe line in
master.cf, but the message was the same. This is the entire line.

Dec 15 18:50:32 mail postfix/pipe[33227]: A3CE22F157E:
to=<[hidden email]>, relay=filter, delay=13, delays=3.7/0/0/9.7,
dsn=2.0.0, status=sent (delivered via filter service (Dec 15 18:50:25.479
[33256] warn: Unescaped left brace in regex is deprecated here (and will be
fata))

These are the lines in master.cf that points to the pipe for filter:

# grep filter /usr/local/etc/postfix/master.cf
smtp      inet  n       -       n       -       -       smtpd  -o
content_filter=filter:dummyr

# -o content_filter=spamassassin

filter    unix  -       n       n       -      10       pipe
  flags=Rq user=filter argv=/usr/local/bin/filter.sh -f ${sender} --
${recipient}

smtpd     pass  -       -       n       -       -       smtpd -o
content_filter=filter:dummy

>
> Also note: there is at least one 3rd-party ruleset (http://sa.zmi.at)
> which appears to have had a typo ('.' where a ',' should have been) which
> also resulted in this error message in a recent version. See
> https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7669 for a report of
> this issue (which is not a bug in SA per se, as the rules with the typo
> are an independent distribution.)

I checked for this problem in the SA rules, but didn't find any problems.
I'm using the default set, and my research says these have all been fixed.
So, the problem must be in an affiliated perl module, but I have no idea
how to find it.

Everything on this server is up to date. I just upgraded to 11.2-RELEASE
last week and rebuilt all the ports. I've run portmaster -ad twice since
and updated several other ports.

If there's a way to do it, I'd like to run this problem down and notify the
developer, but I have no clue how to go about that. I don't know how to
setup a trace on the filter process.

Paul Schmehl
Independent Researcher
Reply | Threaded
Open this post in threaded view
|

Re: Maillog error - warn: Unescaped left brace in regex is deprecated here (and will be fata))

Paul Schmehl-3
In reply to this post by Bill Cole-3
--On December 14, 2018 at 4:09:08 PM -0500 Bill Cole
<[hidden email]> wrote:

> On 14 Dec 2018, at 15:32, Bill Cole wrote:
>
>> On 14 Dec 2018, at 0:46, Paul Schmehl wrote:
>>
>>> I'm seeing this error in the logs:
>>>
>>> warn: Unescaped left brace in regex is deprecated here (and will be
>>> fata))
>
> NOTE: that message should specify the source of the error. If it does
> not, something in your logging plumbing is mangling error messages.
>
> Also note: there is at least one 3rd-party ruleset (http://sa.zmi.at)
> which appears to have had a typo ('.' where a ',' should have been) which
> also resulted in this error message in a recent version. See
> https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7669 for a report of
> this issue (which is not a bug in SA per se, as the rules with the typo
> are an independent distribution.)

I did the following searches trying to locate this issue:

# grep -ro "\[,.\]" /usr/local/lib/perl5/site_perl/*
/usr/local/lib/perl5/site_perl/LWP/Authen/Digest.pm:[,;]
/usr/local/lib/perl5/site_perl/LWP/Authen/Digest.pm:[,;]
/usr/local/lib/perl5/site_perl/Mail/Header.pm:[,;]
/usr/local/lib/perl5/site_perl/Mail/SpamAssassin/ArchiveIterator.pm:[,.]

# grep -ro "{,.}" /usr/local/lib/perl5/site_perl/*
/usr/local/lib/perl5/site_perl/Text/Unidecode/x0f.pm:{, }
/usr/local/lib/perl5/site_perl/Text/Unidecode/x18.pm:{, }
{, }
/usr/local/lib/perl5/site_perl/Text/Unidecode/x20.pm:{,,}
/usr/local/lib/perl5/site_perl/Text/Unidecode/x30.pm:{, }
/usr/local/lib/perl5/site_perl/Text/Unidecode/x30.pm:{,,}
/usr/local/lib/perl5/site_perl/Text/Unidecode/x00.pm:{,,}
/usr/local/lib/perl5/site_perl/mach/5.26/opie.ph:{,;}

The same searches run on SA rules found one possible issue:

# grep -ro "\[,.\]" /usr/local/etc/mail/spamassassin/*
/usr/local/etc/mail/spamassassin/70_sare_specific.cf:[,s

I say possible because in my researching this issue I've seen both curly
braces only and also both curly braces and square braces, so I'm not
completely certain which (or if both) are a problem.

At any rate, I don't see any dependency on Text::Unicode, although I
wouldn't be surprised if it's used by SA.

If square braces are included as well, then Mail::Header and
ArchiveIterator.pm would likely be potential culprits for the error message.

If parenthetical braces are also implicated, then there is an additional
potential problem:

# grep -ro "(,.)" /usr/local/lib/perl5/site_perl/*
/usr/local/lib/perl5/site_perl/mach/5.26/sys/lock.ph:(, )
/usr/local/lib/perl5/site_perl/mach/5.26/sys/lock.ph:(, )

At this point, I need an experienced programmer to decipher all this and
let me know where the issue(s) might be.

Paul Schmehl
Independent Researcher
Reply | Threaded
Open this post in threaded view
|

Re: Maillog error - warn: Unescaped left brace in regex is deprecated here (and will be fata))

Bill Cole-3
In reply to this post by Paul Schmehl-3
On 15 Dec 2018, at 14:00, Paul Schmehl wrote:

> --On December 14, 2018 at 4:09:08 PM -0500 Bill Cole
> <[hidden email]> wrote:
>
>> On 14 Dec 2018, at 15:32, Bill Cole wrote:
>>
>>> On 14 Dec 2018, at 0:46, Paul Schmehl wrote:
>>>
>>>> I'm seeing this error in the logs:
>>>>
>>>> warn: Unescaped left brace in regex is deprecated here (and will be
>>>> fata))
>>
>> NOTE: that message should specify the source of the error. If it does
>> not, something in your logging plumbing is mangling error messages.
>
> I tried increasing the debug level and even added -v to the pipe line
> in master.cf, but the message was the same. This is the entire line.
>
> Dec 15 18:50:32 mail postfix/pipe[33227]: A3CE22F157E:
> to=<[hidden email]>, relay=filter, delay=13, delays=3.7/0/0/9.7,
> dsn=2.0.0, status=sent (delivered via filter service (Dec 15
> 18:50:25.479 [33256] warn: Unescaped left brace in regex is deprecated
> here (and will be fata))

That's broken... Something between Perl and your log is mangling and
truncating the error message. Maybe the Postfix 'pipe' plumbing? That's
a secondary issue though, as it is easy to run SA by itself to. find the
problem.

Try this:

   spamassassin -D all --lint

That should give you a large amount of debug output including the
original error message with a file &  line reference.

> These are the lines in master.cf that points to the pipe for filter:
[...]

This isn't a Postfix problem except for the incomplete log message. The
cause of the warning is what matters.

>>
>> Also note: there is at least one 3rd-party ruleset (http://sa.zmi.at)
>> which appears to have had a typo ('.' where a ',' should have been)
>> which
>> also resulted in this error message in a recent version. See
>> https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7669 for a report
>> of
>> this issue (which is not a bug in SA per se, as the rules with the
>> typo
>> are an independent distribution.)
>
> I checked for this problem in the SA rules, but didn't find any
> problems. I'm using the default set, and my research says these have
> all been fixed. So, the problem must be in an affiliated perl module,
> but I have no idea how to find it.

And yet, in your later message you say:

> The same searches run on SA rules found one possible issue:
>
> # grep -ro "\[,.\]" /usr/local/etc/mail/spamassassin/*
> /usr/local/etc/mail/spamassassin/70_sare_specific.cf:[,s

That is NOT part of the default ruleset. For a few years there was a
3rd-party collection of rules files known as SARE which was distributed
independently of the SpamAssassin project. It was last updated well over
a decade ago. No one should be using those rules with a modern SA to
catch modern spam. They are neither safe nor effective. Remove any rules
file with 'sare' in its name. It would also be wise to make sure that
you are running 'sa-update' regularly (weekly at least, ideally daily)
to be sure that you have the current default ruleset.

--
Bill Cole
[hidden email] or [hidden email]
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Available For Hire: https://linkedin.com/in/billcole
Reply | Threaded
Open this post in threaded view
|

Re: Maillog error - warn: Unescaped left brace in regex is deprecated here (and will be fata))

Bill Cole-3
In reply to this post by Paul Schmehl-3
On 15 Dec 2018, at 15:50, Paul Schmehl wrote:

> --On December 14, 2018 at 4:09:08 PM -0500 Bill Cole
> <[hidden email]> wrote:
>
>> On 14 Dec 2018, at 15:32, Bill Cole wrote:
>>
>>> On 14 Dec 2018, at 0:46, Paul Schmehl wrote:
>>>
>>>> I'm seeing this error in the logs:
>>>>
>>>> warn: Unescaped left brace in regex is deprecated here (and will be
>>>> fata))
>>
>> NOTE: that message should specify the source of the error. If it does
>> not, something in your logging plumbing is mangling error messages.
>>
>> Also note: there is at least one 3rd-party ruleset (http://sa.zmi.at)
>> which appears to have had a typo ('.' where a ',' should have been)
>> which
>> also resulted in this error message in a recent version. See
>> https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7669 for a report
>> of
>> this issue (which is not a bug in SA per se, as the rules with the
>> typo
>> are an independent distribution.)
>
> I did the following searches trying to locate this issue:
>
> # grep -ro "\[,.\]" /usr/local/lib/perl5/site_perl/*

The warning message only applies to '{' NOT '[' and NOT '('

[...]

> # grep -ro "{,.}" /usr/local/lib/perl5/site_perl/*

Try "{,.*}" instead. Also, don't use the 'o' option to grep. It
eliminates useful context, specifically whether the pattern is inside a
regex.

[...]
> The same searches run on SA rules found one possible issue:
>
> # grep -ro "\[,.\]" /usr/local/etc/mail/spamassassin/*
> /usr/local/etc/mail/spamassassin/70_sare_specific.cf:[,s

As I said in my prior message: remove that file and any others with
'sare' in their name. They are dangerously obsolete.

Also check /var/db/spamassassin, which is where anything maintained by
sa-update will be.

> I say possible because in my researching this issue I've seen both
> curly braces only and also both curly braces and square braces, so I'm
> not completely certain which (or if both) are a problem.

It's always helpful to look at actual documentation, rather than trust
research of secondary sources...

Try 'perldoc perldeprecation' for the definitive truth of deprecation
warnings.

> At any rate, I don't see any dependency on Text::Unicode, although I
> wouldn't be surprised if it's used by SA.

It absolutely is not, because Text::Unicode does not exist. :)

SA also does not use Text::Unidecode, which is a last-resort tool for
special cases where Unicode characters absolutely must be translated to
a reasonable US-ASCII representation.


--
Bill Cole
[hidden email] or [hidden email]
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Available For Hire: https://linkedin.com/in/billcole
Reply | Threaded
Open this post in threaded view
|

Re: Maillog error - warn: Unescaped left brace in regex is deprecated here (and will be fata))

Paul Schmehl-3
In reply to this post by Bill Cole-3
--On December 15, 2018 at 9:36:56 PM -0500 Bill Cole
<[hidden email]> wrote:

> Remove any rules
> file with 'sare' in its name. It would also be wise to make sure that you
> are running 'sa-update' regularly (weekly at least, ideally daily) to be
> sure that you have the current default ruleset.

Thanks, Bill. I'll do that right away.

Paul Schmehl
Independent Researcher
Reply | Threaded
Open this post in threaded view
|

Re: Maillog error - warn: Unescaped left brace in regex is deprecated here (and will be fata))

Paul Schmehl-3
In reply to this post by Bill Cole-3
--On December 15, 2018 at 9:36:56 PM -0500 Bill Cole
<[hidden email]> wrote:

> On 15 Dec 2018, at 14:00, Paul Schmehl wrote:
>
>> --On December 14, 2018 at 4:09:08 PM -0500 Bill Cole
>> <[hidden email]> wrote:
>>
>>> On 14 Dec 2018, at 15:32, Bill Cole wrote:
>>>
>>>> On 14 Dec 2018, at 0:46, Paul Schmehl wrote:
>>>>
>>>>> I'm seeing this error in the logs:
>>>>>
>>>>> warn: Unescaped left brace in regex is deprecated here (and will be
>>>>> fata))
>>>
>>> NOTE: that message should specify the source of the error. If it does
>>> not, something in your logging plumbing is mangling error messages.
>>
>> I tried increasing the debug level and even added -v to the pipe line
>> in master.cf, but the message was the same. This is the entire line.
>>
>> Dec 15 18:50:32 mail postfix/pipe[33227]: A3CE22F157E:
>> to=<[hidden email]>, relay=filter, delay=13, delays=3.7/0/0/9.7,
>> dsn=2.0.0, status=sent (delivered via filter service (Dec 15
>> 18:50:25.479 [33256] warn: Unescaped left brace in regex is deprecated
>> here (and will be fata))
>
> That's broken... Something between Perl and your log is mangling and
> truncating the error message. Maybe the Postfix 'pipe' plumbing? That's a
> secondary issue though, as it is easy to run SA by itself to. find the
> problem.
>
> Try this:
>
>    spamassassin -D all --lint
>
> That should give you a large amount of debug output including the
> original error message with a file &  line reference.
>
>> These are the lines in master.cf that points to the pipe for filter:
> [...]
>

That solved the problem. It was a single ruleset, and I removed it.

I thought I only had default rulesets, but I guess I added a few extra ones
over the years.

Thanks for your help. Errors bug me to no end, as you've probably figured
out by now.

Paul Schmehl
Independent Researcher