How to match 2a04:5200:fff4:0 in access table?

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

How to match 2a04:5200:fff4:0 in access table?

Vincent Lefevre-10
I would like to match the 2a04:5200:fff4:0 IPv6 addresses (/64 block)
in an access table (and I'd like to avoid using a cidr lookup table
for specific cases). I have

2a04:5200:fff4:0 REJECT Blacklisted

However, 2a04:5200:fff4::fe was not caught.

The access(5) man page says "The access map lookup key must be in
canonical form" but this is ambiguous as RFC 5952 does not specify
canonical form for subnetworks. For instance, if the IPv6 address
is 2a04:5200:fff4:0:1:0:0:1, then its canonical form would be
2a04:5200:fff4:0:1::1, so that the 2a04:5200:fff4:0 prefix is
necessarily valid.

--
Vincent Lefèvre <[hidden email]> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)
Reply | Threaded
Open this post in threaded view
|

Re: How to match 2a04:5200:fff4:0 in access table?

Bill Cole-3
On 12 Mar 2019, at 7:53, Vincent Lefevre wrote:

> I would like to match the 2a04:5200:fff4:0 IPv6 addresses (/64 block)
> in an access table (and I'd like to avoid using a cidr lookup table
> for specific cases). I have
>
> 2a04:5200:fff4:0 REJECT Blacklisted
>
> However, 2a04:5200:fff4::fe was not caught.
>
> The access(5) man page says "The access map lookup key must be in
> canonical form" but this is ambiguous as RFC 5952 does not specify
> canonical form for subnetworks. For instance, if the IPv6 address
> is 2a04:5200:fff4:0:1:0:0:1, then its canonical form would be
> 2a04:5200:fff4:0:1::1, so that the 2a04:5200:fff4:0 prefix is
> necessarily valid.

 From the access(5) man page:

        net    Matches  the  specified IPv6 host address or subnetwork.
An IPv6
               host address is a sequence of three to eight  hexadecimal
  octet
               pairs separated by ":".
[...]
               Subnetworks  are  matched  by  repeatedly  truncating  
the  last
               ":octetpair" from the remote IPv6 host address  string  
until  a
               match  is found in the access table, or until further
truncation
               is not possible.

               NOTE 1: the truncation and comparison are done with  the  
string
               representation  of  the IPv6 host address. Thus, not all
the ":"
               subnetworks will be tried.

"0" is not an octet pair. Demo:

# cat accessdemo
2a04:5200:fff4:0 REJECT 554 trailing zero
2a04:5200:fff4:0000 REJECT 554 trailing octet pair zeros
2a04:5200:fff4 REJECT 554 NO trailing zero

# postmap hash:accessdemo

# postmap -q 2a04:5200:fff4:0000:0001:0000:0000:0001 accessdemo

# postmap -q 2a04:5200:fff4:0000:0001:0000:0000 accessdemo

# postmap -q 2a04:5200:fff4:0000:0001:0000 accessdemo

# postmap -q 2a04:5200:fff4:0000:0001 accessdemo

# postmap -q 2a04:5200:fff4:0000 accessdemo
REJECT 554 trailing octet pair zeros

# postmap -q 2a04:5200:fff4 accessdemo
REJECT 554 NO trailing zero



--
Bill Cole
[hidden email] or [hidden email]
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Available For Hire: https://linkedin.com/in/billcole
Reply | Threaded
Open this post in threaded view
|

Re: How to match 2a04:5200:fff4:0 in access table?

Vincent Lefevre-10
On 2019-03-12 08:49:28 -0400, Bill Cole wrote:

> "0" is not an octet pair. Demo:
>
> # cat accessdemo
> 2a04:5200:fff4:0 REJECT 554 trailing zero
> 2a04:5200:fff4:0000 REJECT 554 trailing octet pair zeros
> 2a04:5200:fff4 REJECT 554 NO trailing zero
>
> # postmap hash:accessdemo
>
> # postmap -q 2a04:5200:fff4:0000:0001:0000:0000:0001 accessdemo
>
> # postmap -q 2a04:5200:fff4:0000:0001:0000:0000 accessdemo
>
> # postmap -q 2a04:5200:fff4:0000:0001:0000 accessdemo
>
> # postmap -q 2a04:5200:fff4:0000:0001 accessdemo
>
> # postmap -q 2a04:5200:fff4:0000 accessdemo
> REJECT 554 trailing octet pair zeros
>
> # postmap -q 2a04:5200:fff4 accessdemo
> REJECT 554 NO trailing zero

OK, so you mean that "0" must be written as "0000"?

Then why does the access(5) man page say "The access map lookup key
must be in canonical form" while "0000" is not the canonical form?

According to https://tools.ietf.org/html/rfc5952

------------------------------------------------------------------------
4.1.  Handling Leading Zeros in a 16-Bit Field

   Leading zeros MUST be suppressed.  For example, 2001:0db8::0001 is
   not acceptable and must be represented as 2001:db8::1.  A single 16-
   bit 0000 field MUST be represented as 0.
------------------------------------------------------------------------

--
Vincent Lefèvre <[hidden email]> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)
Reply | Threaded
Open this post in threaded view
|

Re: How to match 2a04:5200:fff4:0 in access table?

Wietse Venema
In reply to this post by Vincent Lefevre-10
Vincent Lefevre:
[ Charset ISO-8859-1 converted... ]
> I would like to match the 2a04:5200:fff4:0 IPv6 addresses (/64 block)
> in an access table (and I'd like to avoid using a cidr lookup table
> for specific cases). I have
>
> 2a04:5200:fff4:0 REJECT Blacklisted

Short answer: 2a04:5200:fff4 (strip zero octets).

Or use a cidr map fox maximal control.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: How to match 2a04:5200:fff4:0 in access table?

Bill Cole-3
In reply to this post by Vincent Lefevre-10
On 12 Mar 2019, at 9:16, Vincent Lefevre wrote:

> On 2019-03-12 08:49:28 -0400, Bill Cole wrote:
>> "0" is not an octet pair. Demo:
>>
>> # cat accessdemo
>> 2a04:5200:fff4:0 REJECT 554 trailing zero
>> 2a04:5200:fff4:0000 REJECT 554 trailing octet pair zeros
>> 2a04:5200:fff4 REJECT 554 NO trailing zero
>>
>> # postmap hash:accessdemo
>>
>> # postmap -q 2a04:5200:fff4:0000:0001:0000:0000:0001 accessdemo
>>
>> # postmap -q 2a04:5200:fff4:0000:0001:0000:0000 accessdemo
>>
>> # postmap -q 2a04:5200:fff4:0000:0001:0000 accessdemo
>>
>> # postmap -q 2a04:5200:fff4:0000:0001 accessdemo
>>
>> # postmap -q 2a04:5200:fff4:0000 accessdemo
>> REJECT 554 trailing octet pair zeros
>>
>> # postmap -q 2a04:5200:fff4 accessdemo
>> REJECT 554 NO trailing zero
>
> OK, so you mean that "0" must be written as "0000"?

Yes, if you need it to match (i.e. if it isn't just a placeholder.)

> Then why does the access(5) man page say "The access map lookup key
> must be in canonical form" while "0000" is not the canonical form?

I have no answer for that. All I know is what actually works.

The RFC definition of "canonical form" is arguably inconsistent with the
description of the required format for Postfix and its matching strategy
in the access(5) man page.

--
Bill Cole
[hidden email] or [hidden email]
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Available For Hire: https://linkedin.com/in/billcole
Reply | Threaded
Open this post in threaded view
|

Re: How to match 2a04:5200:fff4:0 in access table?

Wietse Venema
Bill, I think that an IPv6 address ending in 0000 will work only in
a cidr access map, because all other maps use string comparison.

When the Postfix SMTP server queries a table it will use a system
library function to convert the client IP address to string, and
that function will
- first strip leading zeros, leaving a single zero when an octet pair is null.
- then it will collapse repeated '0' fields.

So that is the canonical form for seearching a non-cidr map.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: How to match 2a04:5200:fff4:0 in access table?

Bill Cole-3
On 12 Mar 2019, at 10:29, Wietse Venema wrote:

> Bill, I think that an IPv6 address ending in 0000 will work only in
> a cidr access map, because all other maps use string comparison.

Did you notice the demo I included, using a hash map?


> When the Postfix SMTP server queries a table it will use a system
> library function to convert the client IP address to string, and
> that function will
> - first strip leading zeros, leaving a single zero when an octet pair
> is null.
> - then it will collapse repeated '0' fields.
>
> So that is the canonical form for searching a non-cidr map.

That does not seem to match the experience of the OP or the behavior I
documented with the posted demo, in which I *think* I reproduced the
documented matching algorithm.

I expect that my puzzlement is related to postmap and smtpd not behaving
quite identically.

--
Bill Cole
[hidden email] or [hidden email]
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Available For Hire: https://linkedin.com/in/billcole
Reply | Threaded
Open this post in threaded view
|

Re: How to match 2a04:5200:fff4:0 in access table?

Wietse Venema
Bill Cole:
> On 12 Mar 2019, at 10:29, Wietse Venema wrote:
>
> > Bill, I think that an IPv6 address ending in 0000 will work only in
> > a cidr access map, because all other maps use string comparison.
>
> Did you notice the demo I included, using a hash map?

The postmap command does not know how the Postfix SMTP daemon converts
an IPv6 address to string, which means that IPv6 address tests with
the postmap command are valid only when using cidr maps.

The Postfix SMTP daemon looks up the same address form as the form
that it logs in information about the remote SMTP client. For
example, 2604:8d00:0:1::4 or 2001:41d0:1:88ce:: (these are real
samples from my maillog files).

> > When the Postfix SMTP server queries a table it will use a system
> > library function to convert the client IP address to string, and
> > that function will
> > - first strip leading zeros, leaving a single zero when an octet pair
> > is null.
> > - then it will collapse repeated '0' fields.
> >
> > So that is the canonical form for searching a non-cidr map.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: How to match 2a04:5200:fff4:0 in access table?

Vincent Lefevre-10
On 2019-03-12 11:39:54 -0400, Wietse Venema wrote:
> Bill Cole:
> > On 12 Mar 2019, at 10:29, Wietse Venema wrote:
> >
> > > Bill, I think that an IPv6 address ending in 0000 will work only in
> > > a cidr access map, because all other maps use string comparison.

After looking at the postfix source, I think that it might work with
2 tests if they are written in the "right" way (see below).

> > Did you notice the demo I included, using a hash map?
>
> The postmap command does not know how the Postfix SMTP daemon converts
> an IPv6 address to string, which means that IPv6 address tests with
> the postmap command are valid only when using cidr maps.
>
> The Postfix SMTP daemon looks up the same address form as the form
> that it logs in information about the remote SMTP client. For
> example, 2604:8d00:0:1::4 or 2001:41d0:1:88ce:: (these are real
> samples from my maillog files).

IMHO, the access(5) man page should be clarified to mention that
and also how truncating is done. So, what is in canonical form is
not the access map lookup key, but the IPv6 address. After that,
this is just a sequence of purely string operations.

The ":octetpair" in the man page is not clear because when the
IPv6 address is in compressed form, one may wonder what this means
when reaching a "::". If I understand correctly, the code that does
the checks is check_addr_access in src/smtpd/smtpd_check.c, which
has:

#ifdef HAS_IPV6
    if (strchr(addr, ':') != 0)
        delim = ':';
    else
#endif
        delim = '.';
[...]
    do {
[...]
    } while (split_at_right(addr, delim));

So, what is removed in the truncation is the last sequence starting
with ":", whatever it means (not necessarily an octet pair).

On my example, 2a04:5200:fff4::fe would be successively transformed
into:

2a04:5200:fff4:
2a04:5200:fff4
2a04:5200
2a04

(even though the first key, which ends with ":", is not in some usual
form). Am I right? If yes, this would mean that

2a04:5200:fff4:0 REJECT Blacklisted
2a04:5200:fff4:  REJECT Blacklisted

would be equivalent to 2a04:5200:fff4::/64 in a cidr table, because
"2a04:5200:fff4:0" would match an uncompressed 0 in the 4th octet
pair and "2a04:5200:fff4:" would match a compressed 0 in the 4th
octet pair.

BTW, I'm wondering why a canonical form is used for the match
instead of a form without compressed zeros. Wouldn't the latter
be more useful in practice, while not slower?

--
Vincent Lefèvre <[hidden email]> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)
Reply | Threaded
Open this post in threaded view
|

Re: How to match 2a04:5200:fff4:0 in access table?

Bill Cole-3
In reply to this post by Wietse Venema
On 12 Mar 2019, at 11:39, Wietse Venema wrote:

> Bill Cole:
>> On 12 Mar 2019, at 10:29, Wietse Venema wrote:
>>
>>> Bill, I think that an IPv6 address ending in 0000 will work only in
>>> a cidr access map, because all other maps use string comparison.
>>
>> Did you notice the demo I included, using a hash map?
>
> The postmap command does not know how the Postfix SMTP daemon converts
> an IPv6 address to string, which means that IPv6 address tests with
> the postmap command are valid only when using cidr maps.

I suspected that might be the case...

> The Postfix SMTP daemon looks up the same address form as the form
> that it logs in information about the remote SMTP client. For
> example, 2604:8d00:0:1::4 or 2001:41d0:1:88ce:: (these are real
> samples from my maillog files).

Also (as I discovered by testing) smtpd does not canonicalize
ADDR=IPv6:* strings given to it by XCLIENT, so testing that way requires
the tester to know how to canonicalize an address.




--
Bill Cole
[hidden email] or [hidden email]
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Available For Hire: https://linkedin.com/in/billcole
Reply | Threaded
Open this post in threaded view
|

Re: How to match 2a04:5200:fff4:0 in access table?

Wietse Venema
In reply to this post by Vincent Lefevre-10
Vincent Lefevre:
> BTW, I'm wondering why a canonical form is used for the match
> instead of a form without compressed zeros. Wouldn't the latter
> be more useful in practice, while not slower?

Because the compressed form is what Postfix logs, therefore the
compressed form is what Postfix uses for access map lookups. If
Postfix were to use form X for logging and some unspecified different
form Y for access map lookups, then that would be a real WTF.
Instead, Postfix documents how it derives lookups from the from X.

Again, I recmmend using cidr if you need control over the matching
process.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: How to match 2a04:5200:fff4:0 in access table?

Wietse Venema
In reply to this post by Bill Cole-3
Bill Cole:
> Also (as I discovered by testing) smtpd does not canonicalize
> ADDR=IPv6:* strings given to it by XCLIENT, so testing that way requires
> the tester to know how to canonicalize an address.

Aha, that is a good point. Added canonicalization to the todo list.

        Wietse