Dpkg::Shlibs::Symbol/SymbolFile: introduce symver pattern, deprecate wildcards

Wildcards are still recognized. Their functionality is like of
(symver|optional). Also underline notations of all basic pattern names
in the man page.

Signed-off-by: Raphaël Hertzog <hertzog@debian.org>

man/dpkg-gensymbols.1

@@ -162,6 +162,10 @@ some low level toolchain libraries like libgcc.
 Denotes \fIc++\fR symbol pattern. See \fBUsing symbol patterns\fR subsection
 below.
 .TP
+.B symver
+Denotes \fIsymver\fR (symbol version) symbol pattern. See \fBUsing symbol
+patterns\fR subsection below.
+.TP
 .B regex
 Denotes \fIregex\fR symbol pattern. See \fBUsing symbol patterns\fR subsection
 below.
@@ -222,26 +226,29 @@ for them). However, as these collisions happen on the ABI level, they should
 not degrade quality of the symbol file.
 .RE
 .TP
-.B wildcards
-Wildcard patterns are denoted by the string of the form \fI*@version\fR in the
-symbol name field, e.g. "*@GLIBC_2.0". Well maintained libraries have
+.B symver
+This pattern is denoted by the \fIsymver\fR tag. Well maintained libraries have
 versioned symbols where each version corresponds to the upstream version where
-the symbol got added. If that's the case, you can use wildcard patterns to
-match any symbol associated to the specific version. So "*@GLIBC_2.0" would
-match all symbols associated to the version GLIBC_2.0. For example:
+the symbol got added. If that's the case, you can use a \fIsymver\fR pattern to
+match any symbol associated to the specific version. For example:
 .RS
 .PP
 libc.so.6 libc6 #MINVER#
- *@GLIBC_2.0 2.0
+ (symver)GLIBC_2.0 2.0
  [...]
- *@GLIBC_2.7 2.7
+ (symver)GLIBC_2.7 2.7
  access@GLIBC_2.0 2.2
 .PP
 All symbols associated with versions GLIBC_2.0 and GLIBC_2.7 will lead to
 minimal version of 2.0 and 2.7 respectively with the exception of the symbol
 access@GLIBC_2.0. The latter will lead to a minimal dependency on libc6 version
-2.2 despite being in the scope of the wildcard "*@GLIBC_2.0" because specific
-symbols take precedence over patterns.
+2.2 despite being in the scope of the "(symver)GLIBC_2.0" pattern because
+specific symbols take precedence over patterns.
+.P
+Please note that while old style wildcard patterns (denoted by "*@version" in
+the symbol name field) are still supported, they have been deprecated by new
+style syntax "(symver|optional)version". For example, "*@GLIBC_2.0 2.0" should
+be written as "(symver|optional)GLIBC_2.0 2.0" if the same behaviour is needed.
 .RE
 .TP
 .B regex
@@ -263,34 +270,33 @@ names and matches will inherit \fIoptional\fR tag from the pattern.
 .RE
 .P
 Basic patterns listed above can be combined where it makes sense. In that case,
-they are processed in the order in which the tags are specified. Since wildcard
-pattern does not have a specific tag, it is processed last if the symbol
-specification looks like a wildcard. For example, both
+they are processed in the order in which the tags are specified. For example,
+both
 .PP
  (c++|regex)"^NSA::ClassA::Private::privmethod\\d\\(int\\)@Base" 1.0
  (regex|c++)N3NSA6ClassA7Private11privmethod\\dEi@Base 1.0
 .P
 will match symbols "_ZN3NSA6ClassA7Private11privmethod1Ei@Base" and
 "_ZN3NSA6ClassA7Private11privmethod2Ei@Base". When matching the first pattern,
-the raw symbol is first demangled as c++ symbol, then the demangled name is
+the raw symbol is first demangled as C++ symbol, then the demangled name is
 matched against the regular expression. On the other hand, when matching the
 second pattern, regular expression is matched against the raw symbol name, then
-the symbol is tested if it is c++ one by attempting to demangle it. A failure
+the symbol is tested if it is C++ one by attempting to demangle it. A failure
 of any basic pattern will result in the failure of the whole pattern.
 Therefore, for example, "__N3NSA6ClassA7Private11privmethod\\dEi@Base" will not
-match either of the patterns because it is not a valid c++ symbol.
+match either of the patterns because it is not a valid C++ symbol.
 .P
-In general, all patterns are divided into two groups: aliases (basic c++ and
-wildcards) and generic patterns (regex, all combinations of multiple basic
-patterns). Matching of basic alias-based patterns is fast (O(1)) while generic
-patterns are O(N) (N - generic pattern count) for each symbol.  Therefore, it
-is recommended not to overuse generic patterns.
+In general, all patterns are divided into two groups: aliases (basic \fIc++\fR
+and \fIsymver\fR) and generic patterns (\fIregex\fR, all combinations of
+multiple basic patterns). Matching of basic alias-based patterns is fast (O(1))
+while generic patterns are O(N) (N - generic pattern count) for each symbol.
+Therefore, it is recommended not to overuse generic patterns.
 .P
-When multiple patterns match the same real symbol, aliases (first c++, then
-wildcards) are preferred over generic patterns. Generic patterns are matched in
-the order they are found in the symbol file template until the first success.
-Please note, however, that manual reordering of template file entries is not
-recommended because \fBdpkg-gensymbols\fR generates diffs based on the
+When multiple patterns match the same real symbol, aliases (first \fIc++\fR,
+then \fIsymver\fR) are preferred over generic patterns. Generic patterns are
+matched in the order they are found in the symbol file template until the first
+success.  Please note, however, that manual reordering of template file entries
+is not recommended because \fBdpkg-gensymbols\fR generates diffs based on the
 alphanumerical order of their names.
 .SS Using includes
 .P

scripts/Dpkg/Shlibs/Symbol.pm

@@ -25,6 +25,9 @@ use Dpkg::Version;
 use Storable qw();
 use Dpkg::Shlibs::Cppfilt;
 
+# Supported alias types in the order of matching preference
+use constant 'ALIAS_TYPES' => qw(c++ symver);
+
 sub new {
     my $this = shift;
     my $class = ref($this) || $this;
@@ -146,27 +149,44 @@ sub initialize {
 
     # Look for tags marking symbol patterns. The pattern may match multiple
     # real symbols.
+    my $type;
     if ($self->has_tag('c++')) {
 	# Raw symbol name is always demangled to the same alias while demangled
 	# symbol name cannot be reliably converted back to raw symbol name.
 	# Therefore, we can use hash for mapping.
-	$self->init_pattern('alias-c++'); # Alias subtype is c++.
+	$type = 'alias-c++';
+    }
+
+    if ($self->has_tag('symver')) {
+	# Each symbol is matched against its version rather than full
+	# name@version string.
+	$type = (defined $type) ? 'generic' : 'alias-symver';
     }
-    # Wildcard is an alias based pattern. It gets recognized here even if it is
-    # not specially tagged.
-    if (my $ver = $self->get_wildcard_version()) {
-	error(_g("you can't use wildcards on unversioned symbols: %s"), $_) if $ver eq "Base";
-	$self->init_pattern(($self->is_pattern()) ? 'generic' : 'alias-wildcard');
-	$self->{pattern}{wildcard} = 1;
+    # Support old style wildcard syntax as well. That's basically a symver
+    # with implicit optional tag.
+    if ($self->get_symbolname() =~ /^\*@(.*)$/) {
+	error(_g("you can't use wildcards on unversioned symbols: %s"), $_) if $1 eq "Base";
+	# symver pattern needs symbol name to be its version. However, keeping
+	# dumping this as old style wildcard in the output.
+	unless (defined $self->{symbol_templ}) {
+	    $self->{symbol_templ} = $self->get_symbolname();
+	}
+	$type = (defined $type) ? 'generic' : 'alias-symver';
+	$self->{symbol} = $1;
+	$self->{pattern}{old_wildcard} = 1;
     }
+
     # As soon as regex is involved, we need to match each real
     # symbol against each pattern (aka 'generic' pattern).
     if ($self->has_tag('regex')) {
-	$self->init_pattern('generic');
+	$type = 'generic';
 	# Pre-compile regular expression for better performance.
 	my $regex = $self->get_symbolname();
 	$self->{pattern}{regex} = qr/$regex/;
     }
+    if (defined $type) {
+	$self->init_pattern($type);
+    }
 }
 
 sub get_symbolname {
@@ -190,14 +210,6 @@ sub set_symbolname {
     }
 }
 
-sub get_wildcard_version {
-    my $self = shift;
-    if ($self->get_symbolname() =~ /^\*@(.*)$/) {
-	return $1;
-    }
-    return undef;
-}
-
 sub has_tags {
     my $self = shift;
     return scalar (@{$self->{tagorder}});
@@ -258,7 +270,8 @@ sub equals {
 
 sub is_optional {
     my $self = shift;
-    return $self->has_tag("optional");
+    return $self->has_tag("optional") ||
+           (exists $self->{pattern} && exists $self->{pattern}{old_wildcard});
 }
 
 sub is_arch_specific {
@@ -349,10 +362,10 @@ sub convert_to_alias {
     $type = $self->get_alias_type() unless $type;
 
     if ($type) {
-	if ($type eq 'wildcard') {
-	    # In case of wildcard, alias is like "*@SYMBOL_VERSION". Extract
-	    # symbol version from the rawname.
-	    return "*\@$1" if ($rawname =~ /\@([^@]+)$/);
+	if ($type eq 'symver') {
+	    # In case of symver, alias is symbol version. Extract it from the
+	    # rawname.
+	    return "$1" if ($rawname =~ /\@([^@]+)$/);
 	} elsif ($rawname =~ /^_Z/ && $type eq "c++") {
 	    return cppfilt_demangle($rawname, "auto");
 	}
@@ -383,9 +396,13 @@ sub get_symbolspec {
     my $spec = "";
     $spec .= "#MISSING: $self->{deprecated}#" if $self->{deprecated};
     $spec .= " ";
-    if ($template_mode && $self->has_tags()) {
-	$spec .= sprintf('%s%3$s%s%3$s', $self->get_tagspec(),
-	    $self->get_symboltempl(), $self->{symbol_quoted} || "");
+    if ($template_mode) {
+	if ($self->has_tags()) {
+	    $spec .= sprintf('%s%3$s%s%3$s', $self->get_tagspec(),
+		$self->get_symboltempl(), $self->{symbol_quoted} || "");
+	} else {
+	    $spec .= $self->get_symboltempl();
+	}
     } else {
 	$spec .= $self->get_symbolname();
     }
@@ -458,9 +475,8 @@ sub matches_rawname {
     if ($self->is_pattern()) {
 	# Process pattern tags in the order they were specified.
 	for my $tag (@{$self->{tagorder}}) {
-	    if ($tag eq "c++") {
-		# Demangle it.
-		$ok = not not ($target = $self->convert_to_alias($target, "c++"));
+	    if (grep { $tag eq $_ } ALIAS_TYPES) {
+		$ok = not not ($target = $self->convert_to_alias($target, $tag));
 	    } elsif ($tag eq "regex") {
 		# Symbol name is a regex. Match it against the target
 		$do_eq_match = 0;
@@ -468,12 +484,6 @@ sub matches_rawname {
 	    }
 	    last if not $ok;
 	}
-	if ($ok) {
-	    # Wildcards are checked last
-	    if ($self->{pattern}{wildcard}) {
-		$target = $self->convert_to_alias($target, "wildcard");
-	    }
-	}
     }
 
     # Equality match by default

scripts/Dpkg/Shlibs/SymbolFile.pm

@@ -25,10 +25,6 @@ use Dpkg::Control::Fields;
 use Dpkg::Shlibs::Symbol;
 use Dpkg::Arch qw(get_host_arch);
 
-# Supported alias types in the order of matching preference
-# See: find_matching_pattern().
-use constant 'ALIAS_TYPES' => qw(c++ wildcard);
-
 my %blacklist = (
     '__bss_end__' => 1,		# arm
     '__bss_end' => 1,		# arm
@@ -318,7 +314,7 @@ sub find_matching_pattern {
 	next unless defined $obj;
 
 	my $all_aliases = $obj->{patterns}{aliases};
-	for my $type (ALIAS_TYPES) {
+	for my $type (Dpkg::Shlibs::Symbol::ALIAS_TYPES) {
 	    if (exists $all_aliases->{$type}) {
 		my $aliases = $all_aliases->{$type};
 		if (my $alias = $aliases->{converter}->convert_to_alias($name)) {

scripts/t/200_Dpkg_Shlibs.t

@@ -13,7 +13,7 @@
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
-use Test::More tests => 97;
+use Test::More tests => 104;
 use Cwd;
 use IO::String;
 
@@ -440,20 +440,46 @@ is_deeply( \@tmp, [], "no LOST symbols if all patterns matched." );
 @tmp = map { $_->get_symbolname() } $sym_file->get_new_symbols($sym_file_dup);
 is_deeply( \@tmp, [], "no NEW symbols if all patterns matched." );
 
-# Pattern resolution order: aliases (c++, wildcard), generic
-$sym = $sym_file->lookup_symbol('VERSION_1@VERSION_1',['libpatterns.so.1']);
-is ( $sym->{minver}, '1', "specific VERSION_1 symbol" );
+# Pattern resolution order: aliases (c++, symver), generic
+$sym = $sym_file->lookup_symbol('SYMVER_1@SYMVER_1',['libpatterns.so.1']);
+is ( $sym->{minver}, '1', "specific SYMVER_1 symbol" );
 
-$sym = $sym_file->lookup_symbol('_ZN3NSB8Wildcard16wildcard_method1Ev@VERSION_1', ['libpatterns.so.1']);
+$sym = $sym_file->lookup_symbol('_ZN3NSB6Symver14symver_method1Ev@SYMVER_1', ['libpatterns.so.1']);
 is ( $sym->{minver}, '1.method1', "specific symbol prefered over pattern" );
 
-$sym = $sym_file->lookup_symbol('_ZN3NSB8Wildcard16wildcard_method2Ev@VERSION_1', ['libpatterns.so.1']);
+$sym = $sym_file->lookup_symbol('_ZN3NSB6Symver14symver_method2Ev@SYMVER_1', ['libpatterns.so.1']);
 is ( $sym->{minver}, '1.method2', "c++ alias pattern preferred over generic pattern" );
-is ( $sym->get_pattern()->get_symbolname(), 'NSB::Wildcard::wildcard_method2()@VERSION_1' );
+is ( $sym->get_pattern()->get_symbolname(), 'NSB::Symver::symver_method2()@SYMVER_1' );
 
-$sym = $sym_file->lookup_symbol('_ZN3NSB8WildcardD1Ev@VERSION_1', ['libpatterns.so.1']);
-is ( $sym->{minver}, '1.generic', 'generic (wildcard & c++) pattern covers the rest (destructor)' );
-is ( $sym->get_pattern()->get_symbolname(), '*@VERSION_1' );
+$sym = $sym_file->lookup_symbol('_ZN3NSB6SymverD1Ev@SYMVER_1', ['libpatterns.so.1']);
+is ( $sym->{minver}, '1.generic', 'generic (c++ & symver) pattern covers the rest (destructor)' );
+ok ( $sym->get_pattern()->equals($sym_file->create_symbol('(c++|symver)SYMVER_1 1.generic')) );
+
+# Test old style wildcard support
+load_patterns_symbols();
+$pat = $sym_file->lookup_pattern($sym_file->create_symbol('*@SYMVEROPT_2 2'), ['libpatterns.so.1']);
+ok ( $pat->is_optional(), "Old style wildcard is optional");
+is ( $pat->get_alias_type(), "symver", "old style wildcard is a symver pattern" );
+
+# Get rid of all SymverOptional symbols
+foreach my $tmp (keys %{$obj->{dynsyms}}) {
+    delete $obj->{dynsyms}{$tmp} if ($tmp =~ /SymverOptional/);
+}
+$sym_file->merge_symbols($obj, '100.MISSING');
+is_deeply ( [ map { $_->get_symbolname() } $pat->get_pattern_matches() ],
+    [], "old style wildcard matches nothing.");
+is ( $pat->{deprecated}, '100.MISSING', "old style wildcard gets deprecated." );
+@tmp = map { $_->get_symbolname() } $sym_file->get_lost_symbols($sym_file_dup);
+is_deeply( \@tmp, [], "but old style wildcard is not LOST." );
+
+# 'Internal' pattern covers all internal symbols
+load_patterns_obj();
+@tmp = grep { $_->get_symbolname() =~ /Internal/ } values %{$sym_file->{objects}{'libpatterns.so.1'}{syms}};
+$sym = $sym_file->create_symbol('(regex|c++)^_Z(T[ISV])?N3NSA6ClassA8Internal.*@Base$ 1.internal'),
+$pat = $sym_file->lookup_pattern($sym, ['libpatterns.so.1']);
+is_deeply ([ sort $pat->get_pattern_matches() ], [ sort @tmp ],
+    "Pattern covers all internal symbols");
+is ( $tmp[0]->{minver}, '1.internal' );
 
 # Lookup private pattern
 my @private_symnames = sort qw(

scripts/t/200_Dpkg_Shlibs/patterns.cpp

@@ -82,17 +82,26 @@ namespace NSB
     EXPORT(ClassD::~ClassD());
     EXPORT(void ClassD::generate_vt(const char*) const);
 
-    class Wildcard {
+    class Symver {
 	public:
-	    Wildcard();
-	    ~Wildcard();
+	    Symver();
+	    ~Symver();
 
-	    void wildcard_method1();
-	    void wildcard_method2();
+	    void symver_method1();
+	    void symver_method2();
     };
 
-    EXPORT(Wildcard::Wildcard());
-    EXPORT(Wildcard::~Wildcard());
-    EXPORT(void Wildcard::wildcard_method1());
-    EXPORT(void Wildcard::wildcard_method2());
+    EXPORT(Symver::Symver());
+    EXPORT(Symver::~Symver());
+    EXPORT(void Symver::symver_method1());
+    EXPORT(void Symver::symver_method2());
+
+    class SymverOptional {
+	public:
+	    SymverOptional();
+	    ~SymverOptional();
+    };
+
+    EXPORT(SymverOptional::SymverOptional());
+    EXPORT(SymverOptional::~SymverOptional());
 };

scripts/t/200_Dpkg_Shlibs/patterns.map

@@ -1,4 +1,9 @@
-VERSION_1 {
+SYMVER_1 {
 	global:
-		_ZN3NSB8Wildcard*;
+		_ZN3NSB6Symver*;
+};
+
+SYMVEROPT_2 {
+	global:
+		_ZN3NSB14SymverOptional*;
 };

scripts/t/200_Dpkg_Shlibs/patterns.symbols

@@ -1,8 +1,10 @@
 libpatterns.so.1 libpatterns1 #MINVER#
- (c++)*@VERSION_1 1.generic
+ *@SYMVEROPT_2 2
  (c++|regex|optional)NSA::ClassA::Private(::.*)?@Base 1.private
- (c++)NSB::Wildcard::wildcard_method2()@VERSION_1 1.method2
- VERSION_1@VERSION_1 1
+ (c++)NSB::Symver::symver_method2()@SYMVER_1 1.method2
+ SYMVEROPT_2@SYMVEROPT_2 2
+ (c++|symver)SYMVER_1 1.generic
+ SYMVER_1@SYMVER_1 1
  (regex|c++)^_Z(T[ISV])?N3NSA6ClassA8Internal.*@Base$ 1.internal
  _ZN3NSA6ClassAC1Ev@Base 1
  _ZN3NSA6ClassAC2Ev@Base 1
@@ -14,7 +16,7 @@ libpatterns.so.1 libpatterns1 #MINVER#
  _ZN3NSB6ClassDD0Ev@Base 1
  _ZN3NSB6ClassDD1Ev@Base 1
  _ZN3NSB6ClassDD2Ev@Base 1
- _ZN3NSB8Wildcard16wildcard_method1Ev@VERSION_1 1.method1
+ _ZN3NSB6Symver14symver_method1Ev@SYMVER_1 1.method1
  _ZN6ClassBC1Ev@Base 1
  _ZN6ClassBC2Ev@Base 1
  _ZN6ClassBD0Ev@Base 1