Add support of wildcard entries in symbols files

* scripts/Dpkg/Shlibs/SymbolFile.pm (load): Parse *@<version> symbols names as
wildcards and not as regular symbols.
* scripts/Dpkg/Shlibs/SymbolFile.pm (merge_symbols): Use information
provided by wildcards entries when new symbols are added.
* scripts/dpkg-gensymbols.pl: Disable some checks which can't be done when
symbols files have been generated with the help of wildcard entries.
* man/dpkg-gensymbols.1: Update the manual page to explain how to use
wildcard entries.

ChangeLog

@@ -1,3 +1,18 @@
+2008-01-18  Raphael Hertzog  <hertzog@debian.org>
+
+	* scripts/Dpkg/Shlibs/SymbolFile.pm (load): Parse *@<version>
+	symbols names as wildcards and not as regular symbols. They
+	provide the same dependency information for all symbols that share
+	the same version.
+	* scripts/Dpkg/Shlibs/SymbolFile.pm (merge_symbols): Use
+	information provided by wildcards entries when new symbols are
+	added.
+	* scripts/dpkg-gensymbols.pl: Disable some checks which can't be
+	done when symbols files have been generated with the help of
+	wildcard entries.
+	* man/dpkg-gensymbols.1: Update the manual page to explain how to
+	use wildcard entries.
+
 2008-01-18  Raphael Hertzog  <hertzog@debian.org>
 
 	* scripts/Dpkg/Changelog.pm (parse_changelog): Rewrite it completely

debian/changelog

@@ -61,6 +61,10 @@ dpkg (1.14.16) UNRELEASED; urgency=low
   * Some code refactoring on dpkg-genchanges and bug fixes in the generation
     of the Description: field. As a result, source only uploads will no more
     have Description fields.
+  * Add support of wildcard entries in symbols files. This makes it much
+    simpler to write symbols files for well managed libraries but in that case
+    dpkg-gensymbols can't check any more if symbols have disappeared.
+    Closes: #459359
 
   [ Updated manpages translations ]
   * Fix typo in French. Closes: #460021

man/dpkg-gensymbols.1

@@ -44,8 +44,14 @@ The symbols files are really useful only if they reflect the evolution of
 the package through several releases. Thus the maintainer has to update
 them every time that a new symbol is added so that its associated minimal
 version matches reality. To do this properly he can use the diffs contained
-in the build logs. In most cases, the diff can be directly applied to his
-debian/\fIpackage\fR.symbols file.
+in the build logs. In most cases, the diff applies directly to his
+debian/\fIpackage\fR.symbols file. That said, further tweaks are usually
+needed: it's recommended for example to drop the Debian revision
+from the minimal version so that backports with a lower version number
+but the same upstream version still satisfy the generated dependencies.
+If the Debian revision can't be dropped because the symbol really got
+added by the Debian specific change, then one should suffix the version
+with "~".
 .P
 Before applying any patch to the symbols file, the maintainer should
 double-check that it's sane. Public symbols are not supposed to disappear,
@@ -74,6 +80,29 @@ to do it is the following:
 .PP
 #include "libsomething1.symbols.common"
  arch_specific_symbol@Base 1.0
+.SS Using wildcards with versioned symbols
+.P
+Well maintained libraries have versioned symbols where each version
+corresponds to the upstream version where the symbol got added. If that's
+the case, it's possible to write a symbols file with wildcard entries like
+"*@GLIBC_2.0" that would match any symbol associated to the version
+GLIBC_2.0. It's still possible to include specific symbols in the file,
+they'll take precedence over any matching wildcard entry. An example:
+.PP
+libc.so.6 libc6 #MINVER#
+ *@GLIBC_2.0 2.0
+ [...]
+ *@GLIBC_2.7 2.7
+ access@GLIBC_2.0 2.2
+.P
+The symbol access@GLIBC_2.0 will lead to a minimal dependency on libc6
+version 2.2 despite the wildcard entry *@GLIBC_2.0 which associates
+symbols versioned as GLIBC_2.0 with the minimal version 2.0.
+.p
+Note that using wildcards means that \fBdpkg\-gensymbols\fR can't check
+for symbols that might have disappeared and can't generate a diff between
+the maintainer-supplied symbols file and the generated one in the binary
+package.
 .SS Good library management
 .P
 A well-maintained library has the following features:

scripts/Dpkg/Shlibs/SymbolFile.pm

@@ -66,6 +66,7 @@ sub new {
     my $class = ref($this) || $this;
     my $self = { };
     bless $self, $class;
+    $self->clear();
     if (defined($file) ) {
 	$self->{file} = $file;
 	$self->load($file) if -e $file;
@@ -76,6 +77,7 @@ sub new {
 sub clear {
     my ($self) = @_;
     $self->{objects} = {};
+    $self->{used_wildcards} = 0;
 }
 
 sub clear_except {
@@ -108,13 +110,19 @@ sub load {
 	    if (not defined ($object)) {
 		error(sprintf(_g("Symbol information must be preceded by a header (file %s, line %s).", $file, $.)));
 	    }
+	    my $name = $1;
 	    # New symbol
 	    my $sym = {
 		minver => $2,
 		dep_id => defined($3) ? $3 : 0,
 		deprecated => 0
 	    };
-	    $self->{objects}{$object}{syms}{$1} = $sym;
+	    if ($name =~ /^\*@(.*)$/) {
+		error(_g("you can't use wildcards on unversioned symbols: %s"), $_) if $1 eq "Base";
+		$self->{objects}{$object}{wildcards}{$1} = $sym;
+	    } else {
+		$self->{objects}{$object}{syms}{$name} = $sym;
+	    }
 	} elsif (/^#include\s+"([^"]+)"/) {
 	    my $filename = $1;
 	    my $dir = $file;
@@ -209,9 +217,10 @@ sub merge_symbols {
     }
     # Scan all symbols provided by the objects
     foreach my $sym (keys %dynsyms) {
-	if (exists $self->{objects}{$soname}{syms}{$sym}) {
+	my $obj = $self->{objects}{$soname};
+	if (exists $obj->{syms}{$sym}) {
 	    # If the symbol is already listed in the file
-	    my $info = $self->{objects}{$soname}{syms}{$sym};
+	    my $info = $obj->{syms}{$sym};
 	    if ($info->{deprecated}) {
 		# Symbol reappeared somehow
 		$info->{deprecated} = 0;
@@ -225,12 +234,21 @@ sub merge_symbols {
 	    }
 	} else {
 	    # The symbol is new and not present in the file
-	    my $info = {
-		minver => $minver,
-		deprecated => 0,
-		dep_id => 0
-	    };
-	    $self->{objects}{$soname}{syms}{$sym} = $info;
+	    my $info;
+	    my $symobj = $dynsyms{$sym};
+	    if ($symobj->{version} and exists $obj->{wildcards}{$symobj->{version}}) {
+		# Get the info from wildcards
+		$info = $obj->{wildcards}{$symobj->{version}};
+		$self->{used_wildcards}++;
+	    } else {
+		# New symbol provided by the current release
+		$info = {
+		    minver => $minver,
+		    deprecated => 0,
+		    dep_id => 0
+		};
+	    }
+	    $obj->{syms}{$sym} = $info;
 	}
     }
 
@@ -265,6 +283,7 @@ sub create_object {
     $self->{objects}{$soname} = {
 	syms => {},
 	fields => {},
+	wildcards => {},
 	deps => [ @deps ]
     };
 }
@@ -288,6 +307,22 @@ sub get_field {
     return undef;
 }
 
+sub contains_wildcards {
+    my ($self) = @_;
+    my $res = 0;
+    foreach my $soname (sort keys %{$self->{objects}}) {
+	if (scalar keys %{$self->{objects}{$soname}{wildcards}}) {
+	    $res = 1;
+	}
+    }
+    return $res;
+}
+
+sub used_wildcards {
+    my ($self) = @_;
+    return $self->{used_wildcards};
+}
+
 sub lookup_symbol {
     my ($self, $name, $sonames, $inc_deprecated) = @_;
     $inc_deprecated = 0 unless defined($inc_deprecated);

scripts/dpkg-gensymbols.pl

@@ -207,31 +207,40 @@ if ($compare) {
 	$exitcode = 3 if ($compare >= 3);
     }
     if ($symfile->has_new_symbols($ref_symfile)) {
-	warning(_g("some new symbols appeared in the symbols file."));
-	$exitcode = 2 if ($compare >= 2);
+	unless ($symfile->used_wildcards()) {
+	    # Wildcards are used to replace many additional symbols, so we
+	    # have no idea if this is really true, so don't say it and
+	    # don't check it
+	    warning(_g("some new symbols appeared in the symbols file."));
+	    $exitcode = 2 if ($compare >= 2);
+	}
     }
     if ($symfile->has_lost_symbols($ref_symfile)) {
 	warning(_g("some symbols disappeared in the symbols file."));
 	$exitcode = 1 if ($compare >= 1);
     }
-    # Output diffs between symbols files if needed
-    my $before = File::Temp->new(TEMPLATE=>'dpkg-gensymbolsXXXXXX');
-    my $after = File::Temp->new(TEMPLATE=>'dpkg-gensymbolsXXXXXX');
-    $ref_symfile->dump($before); $symfile->dump($after);
-    seek($before, 0, 0); seek($after, 0, 0);
-    my ($md5_before, $md5_after) = (Digest::MD5->new(), Digest::MD5->new());
-    $md5_before->addfile($before);
-    $md5_after->addfile($after);
-    if ($md5_before->hexdigest() ne $md5_after->hexdigest()) {
-	if (defined($ref_symfile->{file})) {
-	    warning(_g("%s doesn't match completely %s\n"),
-		    $output, $ref_symfile->{file});
-	} else {
-	    warning(_g("no debian/symbols file used as basis for generating %s\n"),
-	            $output);
+    unless ($symfile->used_wildcards()) {
+	# If wildcards are not used, we can compare symbols files before
+	# and after
+	my $before = File::Temp->new(TEMPLATE=>'dpkg-gensymbolsXXXXXX');
+	my $after = File::Temp->new(TEMPLATE=>'dpkg-gensymbolsXXXXXX');
+	$ref_symfile->dump($before); $symfile->dump($after);
+	seek($before, 0, 0); seek($after, 0, 0);
+	my ($md5_before, $md5_after) = (Digest::MD5->new(), Digest::MD5->new());
+	$md5_before->addfile($before);
+	$md5_after->addfile($after);
+	# Output diffs between symbols files if any
+	if ($md5_before->hexdigest() ne $md5_after->hexdigest()) {
+	    if (defined($ref_symfile->{file})) {
+		warning(_g("%s doesn't match completely %s\n"),
+			$output, $ref_symfile->{file});
+	    } else {
+		warning(_g("no debian/symbols file used as basis for generating %s\n"),
+			$output);
+	    }
+	    my ($a, $b) = ($before->filename, $after->filename);
+	    system("diff", "-u", $a, $b) if -x "/usr/bin/diff";
 	}
-	my ($a, $b) = ($before->filename, $after->filename);
-	system("diff", "-u", $a, $b) if -x "/usr/bin/diff";
     }
 }
 exit($exitcode);