Symbol tagging support and implementation of optional and arch tags

Symbols might be tagged with arbitary number of tags which are separated by '|'
(aka pipe) character. Tags (only if there are any) must be enclosed in the ()
brackets right before a symbol name. Each tag can appear only once in the tag
specification for each symbol. A tag can optionally have a value. The tag value
is separated from the tag name by character '='. Tag names and values can be
arbitrary strings except they cannot contain any of the special ()|=
characters. A name of the tagged symbol can optionally be quoted with either '
or " characters (e.g. to allow whitespaces in the symbol name). If there are no
tags specified, symbol name cannot be quoted and symbol name continues up until
the first whitespace.

Example of the quoted symbol with 2 tags (the first tag has a value while the
second does not):

 (tag1=value of the first tag|tag2)"Foo::Bar::foobar()"@Base 1.0 1

Tags serve two purposes:

* they can be used for symbol marking (e.g. when a symbol must be treated
  specially under certain circumstances);
* they tell the parser that the specified symbol name must be processed in
  certain way to become a raw symbol name as found in the object files.
  Such tags are processed in the order they are specified.

Symbol files with tags are not backwards compatible, so a new parameter
'template_mode' has been added to SymbolFile::dump(). If template_mode is not
set (default), dump() emits a fully post-processed symbol specifications as
directed by their tags. Tags themselves are ommited from output and unknown
tags are ignored. If template_mode is set, dump() keeps tags in the output and
emits symbol specifications as they were read from the symbol file.

As a result, a new parameter -t was added to dpkg-gensymbols to support writing
the symbol file in template mode. Default is to write in non-template mode.

(tag|...|tagN)#include "file_to_include"

All symbols in the "file_to_include" will be tagged with #include tags by
default. A symbol in "file_to_include" can specify additional tags in its tag
specification or even override the tag value inherited from #include tags by
specifying another one in its own tag specifaction. There is no way for the
symbol to untag itself, even if the tag was inherited from #include.  #include
tags are assumed to go always before (in tagorder) symbol specific tags even if
the symbol overrides the tag. Recursive tagged #includes are supported.

The patch also adds support for 2 tags:

* optional - a symbol marked as optional can disappear from the object file at
  any time and that will never cause dpkg-gensymbols to fail (just emit the
  diff).  If optional symbol disappears, it is dumped as MISSING with
  deprecated string set to the current package version (hence it always appears
  in the dpkg-gensymbols diff). If optional symbol reappears, it gets
  undeprecated but its minver is kept unchanged (contrary to reappearing
  MISSING symbols).

  Example: C++ template instantiation which disappearance is not ABI breaking
  (i.e. basically it is a private symbol).

  (optional)_ZN6Phonon22ObjectDescriptionModelILNS_21ObjectDescriptionTypeE0EE11qt_metacastEPKc@Base 4:4.2.0

* arch=<list of architectures as in Build-Depend dep> - allows to
  mark a symbol as arch-specific. When dumping in non-templace mode, only
  symbols concerned for the current host architecture are dumped. When dumping
  in template mode, all arch-specific symbols are always dumped (including
  proper tags). If arch-specific symbol appears on the arch that it is not
  supposed to appear, it is made arch neutral (hence it appears in the
  dpkg-gensymbols diff). If arch-specific symbol disappears from its arch, it
  gets declared as MISSING.

  Example: armel specific symbol due to qreal mangling as float on arm(el)
  and double on other arches.

  (arch=arm armel)_ZN6Phonon11AudioOutput13volumeChangedEd@Base 4:4.2.0
  (arch=!armel !arm)_ZN6Phonon11AudioOutput13volumeChangedEf@Base 4:4.2.0

man/dpkg-gensymbols.1

@@ -1,5 +1,5 @@
 .\" Author: Raphael Hertzog
-.TH dpkg\-gensymbols 1 "2008-08-18" "Debian Project" "dpkg utilities"
+.TH dpkg\-gensymbols 1 "2009-06-19" "Debian Project" "dpkg utilities"
 .SH NAME
 dpkg\-gensymbols \- generate symbols files (shared library dependency information)
 .
@@ -64,22 +64,118 @@ use the marker \fI#PACKAGE#\fR. It will be replaced by the real package
 name during installation of the symbols files. Contrary to the
 \fI#MINVER#\fR marker, \fI#PACKAGE#\fR will never appear in a symbols file
 inside a binary package.
+.SS Using symbol tags
+.P
+Symbol tagging is useful for marking symbols that are special in some way.  Any
+symbol can have arbitrary number of tags associated with it. While all tags are
+parsed and stored, only a couple of them are understood by
+\fBdpkg-gensymbols\fR and trigger special handling of the symbols. See
+subsection \fBStandard symbol tags\fR for reference of these tags.
+.P
+Tag specification comes right before the symbol name (no whitespace is allowed
+in between). It always starts with an opening bracket \fB(\fR, ends with a
+closing bracket \fB)\fR and must contain at least one tag. Multiple tags are
+separated by the \fB|\fR character. Each tag can optionally have a value which
+is separated form the tag name by the \fB=\fR character. Tag names and values
+can be arbitrary strings except they cannot contain any of the special \fB)\fR
+\fB|\fR \fB=\fR characters. Symbol names following a tag specification can
+optionally be quoted with either \fB'\fR or \fB"\fR characters to allow
+whitespaces in them. However, if there are no tags specified for the symbol,
+quotes are treated as part of the symbol name which continues up until the
+first space.
+.P
+ (tag1=i am marked|tag name with space)"tagged quoted symbol"@Base 1.0
+ (optional)tagged_unquoted_symbol@Base 1.0 1
+ untagged_symbol@Base 1.0
+.P
+The first symbol in the example is named \fItagged quoted symbol\fR and has two
+tags: \fItag1\fR with value \fIi am marked\fR and \fItag name with space\fR
+that is valueless. Then second symbol named \fItagged_unquoted_symbol\fR is
+tagged with the only valueless tag named \fIoptional\fR. The last symbol is an
+example of the normal untagged symbol.
+.P
+Since symbol tags are an extension to the \fIdeb-symbols(5)\fR format, they can
+only be part of the symbol file template. When \fBdpkg-gensymbols\fR writes a
+symbol file in \fIdeb-symbols(5)\fR compatible non-template mode (default), it
+fully processes symbols according to the requirements of their standard tags
+and strips all tags from the output. On the contrary, in template mode all
+symbols and their tags (both standard and unknown ones) are written in their
+orignal form as they were loaded.
+.SS Standard symbol tags
+.TP
+.B optional
+A symbol marked as optional can disappear from the library at any time and that
+will never cause \fBdpkg-gensymbols\fR to fail. However, disappeared optional
+symbols will continuously appear as MISSING in the diff in each new package
+revision.  This behaviour serves as a reminder for the maintainer that such a
+symbol needs to be removed from the symbol file or readded to the library. When
+the optional symbol, which was previously declared as MISSING, suddently
+reappears in the next revision, it will be upgraded back to the "existing"
+status with its minimum version unchanged.
+
+This tag is useful for symbols which are private or their disappearance does
+not cause ABI breakage. For example, most of C++ template instantiations fall
+into this category. Like any other tag, this one may also have an arbitrary
+value which could indicate why the symbol is considered optional.
+.TP
+.B arch=\fIarchitecture list\fR
+This tag allows to mark a symbol as arch-specific. When matching the symbol
+file against the library, all arch-specific symbols which do not concern
+current host architecture are treated like they do not exist. If an
+arch-specific symbol disappears from the arch it is concerned about, normal
+procedures for missing symbols apply and it may cause \fBdpkg-gensymbols\fR to
+fail. On the other hand, if the arch-specific symbol is found on the arch that
+it is not supposed to appear on, it is made arch neutral (i.e. the arch tag is
+dropped which triggers the symbol to appear in the diff), but it is not
+considered as new.
+
+When operating in non-templace mode, only symbols concerned for the current
+host architecture are written to the symbol file. On the contrary, all
+arch-specific symbols (including those from foreign arches) are always written
+to the symbol file template.
+
+\fIarchitecture list\fR format is the same as the one used in the
+\fIBuild-Depends\fR field of \fIdebian/control\fR (excluding enclosing []
+brackets). For example, the first symbol from the list below will be considered
+only on alpha, amd64 and ia64 architectures while the second one anywhere but
+armel.
+
+ (arch=alpha amd64 ia64)a_64bit_specific_symbol@Base 1.0
+ (arch=!armel)symbol_armel_does_not_have@Base 1.0
 .SS Using includes
 .P 
-When the set of exported symbols differ between architectures, it's no
-more possible to use a common symbols file. Using one file per
-architecture works, but it can also lead to duplication of information.
-In those cases, you can factorize the common part in some external file
+When the set of exported symbols differ between architectures, it may become
+inefficient to use a single symbol file. In those cases, an include directive
+may prove to be useful in a couple of ways:
+.IP \(bu
+You can factorize the common part in some external file
 and include that file in your \fIpackage\fR.symbols.\fIarch\fR file by
 using an include directive like this:
-.PP
+
 #include "\fIpackages\fR.symbols.common"
+.IP \(bu
+The include directive may also be tagged like any symbol:
+
+(tag|..|tagN)#include "file_to_include"
+
+As a result, all symbols included from \fIfile_to_include\fR will be considered
+to be tagged with \fItag\fR .. \fItagN\fR by default. You can use this feature
+to create a common \fIpackage\fR.symbols file which includes architecture
+specific symbol files:
+
+  common_symbol1@Base 1.0
+ (arch=amd64 ia64 alpha)#include "package.symbols.64bit"
+ (arch=!amd64 !ia64 !alpha)#include "package.symbols.32bit"
+  common_symbol2@Base 1.0
 .P
-The symbols files are read line by line, and include directives
-are processed as soon as they are encountered. This means that
-the content of the included file can override any content that appeared
-before the include directive and that any content after the
-directive can override anything contained in the included file.
+The symbols files are read line by line, and include directives are processed
+as soon as they are encountered. This means that the content of the included
+file can override any content that appeared before the include directive and
+that any content after the directive can override anything contained in the
+included file. Any symbol (or even another #include directive) in the included
+file can specify additional tags or override values of the inherited tags in
+its tag specification. However, there is no way for the symbol to untag itself
+from the inherited tags.
 .P
 An included file can repeat the header line containing the SONAME of the
 library. In that case, it overrides any header line previously read.
@@ -122,7 +218,8 @@ changes;
 ideally, it uses symbol versioning to achieve ABI stability despite
 internal changes and API extension;
 .IP \(bu 4
-it doesn't export private symbols.
+it doesn't export private symbols (such symbols can be tagged optional as
+workaround).
 .P
 While maintaining the symbols file, it's easy to notice appearance and
 disappearance of symbols. But it's more difficult to catch incompatible
@@ -164,6 +261,15 @@ pre-existing, its content is used as basis for the generated symbols file.
 You can use this feature to update a symbols file so that it matches a
 newer upstream version of your library.
 .TP
+.BI \-t
+Write the symbol file in template mode rather than the format compatible with
+\fIdeb-symbols(5)\fR.The main difference is that in the template mode symbol
+names and tags are written in their original form contrary to the
+post-processed symbol names with tags stripped in the compatibility mode.
+Moreover, some symbols might be omitted when writing a standard
+\fIdeb-symbols(5)\fR file (according to the tag processing rules) while all
+symbols are always written to the symbol file template.
+.TP
 .BI \-c [0-4]
 Define the checks to do when comparing the generated symbols file
 with the file used as starting point. By default the level is 1.

scripts/Dpkg/Shlibs/Symbol.pm

@@ -0,0 +1,239 @@
+# Copyright (C) 2007  Raphael Hertzog
+#           (C) 2009  Modestas Vainius
+
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+package Dpkg::Shlibs::Symbol;
+
+use strict;
+use warnings;
+use Dpkg::Gettext;
+use Dpkg::Deps;
+use Dpkg::ErrorHandling;
+use Storable qw( dclone );
+
+my $arches64bit = qr/amd64|ia64|alpha/;
+
+sub new {
+    my $this = shift;
+    my $class = ref($this) || $this;
+    my %args = @_;
+    my $self = bless {
+	symbol => undef,
+	symbol_templ => undef,
+	minver => undef,
+	dep_id => 0,
+	deprecated => 0,
+	tags => {},
+	tagorder => [],
+    }, $class;
+    $self->{$_} = $args{$_} foreach keys %args;
+    return $self;
+}
+
+sub clone {
+    my $self = shift;
+    my $clone = dclone($self);
+    if (@_) {
+	my %args=@_;
+	$clone->{$_} = $args{$_} foreach keys %args;
+    }
+    return $clone;
+};
+
+sub parse_tagspec {
+    my ($self, $tagspec) = @_;
+
+    if ($tagspec =~ /^\((.*?)\)(.*)$/ && $1) {
+	# (tag1=t1 value|tag2|...|tagN=tNp)
+	# Symbols ()|= cannot appear in the tag names and values
+	my $tagspec = $1;
+	my $rest = ($2) ? $2 : "";
+	my @tags = split(/\|/, $tagspec);
+
+	# Parse each tag
+	for my $tag (@tags) {
+	    if ($tag =~ /^(.*)=(.*)$/) {
+		# Tag with value
+		$self->add_tag($1, $2);
+	    } else {
+		# Tag without value
+		$self->add_tag($tag, undef);
+	    }
+	}
+	return $rest;
+    }
+    return undef;
+}
+
+sub parse {
+    my ($self, $symbolspec) = @_;
+    my $symbol;
+    my $symbol_templ;
+    my $symbol_quoted;
+    my $rest;
+
+    if (defined($symbol = $self->parse_tagspec($symbolspec))) {
+	# (tag1=t1 value|tag2|...|tagN=tNp)"Foo::Bar::foobar()"@Base 1.0 1
+	# Symbols ()|= cannot appear in the tag names and values
+
+	# If the tag specification exists symbol name template might be quoted too
+	if ($symbol =~ /^(['"])/ && $symbol =~ /^($1)(.*?)$1(.*)$/) {
+	    $symbol_quoted = $1;
+	    $symbol_templ = $2;
+	    $symbol = $2;
+	    $rest = $3;
+	} else {
+	    if ($symbol =~ m/^(\S+)(.*)$/) {
+		$symbol_templ = $1;
+		$symbol = $1;
+		$rest = $2;
+	    }
+	}
+	error(_g("symbol name unspecified: %s"), $symbolspec) if (!$symbol);
+    } else {
+	# No tag specification. Symbol name is up to the first space
+	# foobarsymbol@Base 1.0 1
+	if ($symbolspec =~ m/^(\S+)(.*)$/) {
+	    $symbol = $1;
+	    $rest = $2;
+	} else {
+	    return 0;
+	}
+    }
+    $self->{symbol} = $symbol;
+    $self->{symbol_templ} = $symbol_templ;
+    $self->{symbol_quoted} = $symbol_quoted if ($symbol_quoted);
+
+    # Now parse "the rest" (minver and dep_id)
+    if ($rest =~ /^\s(\S+)(?:\s(\d+))?/) {
+	$self->{minver} = $1;
+	$self->{dep_id} = defined($2) ? $2 : 0;
+    } else {
+	return 0;
+    }
+    return 1;
+}
+
+# A hook for processing of tags which may change symbol name.
+# Called from Dpkg::Shlibs::SymbolFile::load(). Empty for now.
+sub process_tags {
+    my $self = shift;
+}
+
+sub get_symbolname {
+    return $_[0]->{symbol};
+}
+
+sub get_symboltempl {
+    return $_[0]->{symbol_templ} || $_[0]->{symbol};
+}
+
+sub get_wildcard_version {
+    my $self = shift;
+    if ($self->get_symbolname() =~ /^\*@(.*)$/) {
+	return $1;
+    }
+    return undef;
+}
+
+sub has_tags {
+    my $self = shift;
+    return scalar (@{$self->{tagorder}});
+}
+
+sub add_tag {
+    my ($self, $tagname, $tagval) = @_;
+    if (exists $self->{tags}{$tagname}) {
+	$self->{tags}{$tagname} = $tagval;
+	return 0;
+    } else {
+	$self->{tags}{$tagname} = $tagval;
+	push @{$self->{tagorder}}, $tagname;
+    }
+    return 1;
+}
+
+sub delete_tag {
+    my ($self, $tagname) = @_;
+    if (exists $self->{tags}{$tagname}) {
+	delete $self->{tags}{$tagname};
+	for (my $i = 0; $i <= $#{$self->{tagorder}}; $i++) {
+	    delete $self->{tagorder}->[$i];
+	    last;
+	}
+	return 1;
+    }
+    return 0;
+}
+
+sub is_optional {
+    my $self = shift;
+    return exists $self->{tags}{optional};
+}
+
+sub is_arch_specific {
+    my $self = shift;
+    return exists $self->{tags}{arch};
+}
+
+sub arch_is_concerned {
+    my ($self, $arch) = @_;
+    my $arches = $self->{tags}{arch};
+
+    if (defined $arch && defined $arches) {
+	my $dep = Dpkg::Deps::Simple->new();
+	my @arches = split(/[\s,]+/, $arches);
+	$dep->{package} = "dummy";
+	$dep->{arches} = \@arches;
+	return $dep->arch_is_concerned($arch);
+    }
+
+    return 1;
+}
+
+sub get_tagspec {
+    my ($self) = @_;
+    if ($self->has_tags()) {
+	my @tags;
+	for my $tagname (@{$self->{tagorder}}) {
+	    my $tagval = $self->{tags}{$tagname};
+	    if (defined $tagval) {
+		push @tags, $tagname . "="  . $tagval;
+	    } else {
+		push @tags, $tagname;
+	    }
+	}
+	return "(". join("|", @tags) . ")";
+    }
+    return "";
+}
+
+sub get_symbolspec {
+    my $self = shift;
+    my $template_mode = shift;
+    my $spec = "";
+    $spec .= "#MISSING: $self->{deprecated}#" if $self->{deprecated};
+    $spec .= " ";
+    if ($template_mode && $self->has_tags()) {
+	$spec .= sprintf('%s%3$s%s%3$s', $self->get_tagspec(),
+	    $self->get_symboltempl(), $self->{symbol_quoted} || "");
+    } else {
+	$spec .= $self->get_symbolname();
+    }
+    $spec .= " $self->{minver}";
+    $spec .= " $self->{dep_id}" if $self->{dep_id};
+    return $spec;
+}

scripts/Dpkg/Shlibs/SymbolFile.pm

@@ -1,4 +1,5 @@
 # Copyright (C) 2007  Raphael Hertzog
+#           (C) 2009  Modestas Vainius
 
 # This program is free software; you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
@@ -20,6 +21,8 @@ use Dpkg::Gettext;
 use Dpkg::ErrorHandling;
 use Dpkg::Version qw(vercmp);
 use Dpkg::Fields qw(capit);
+use Dpkg::Shlibs::Symbol;
+use Dpkg::Arch qw(get_host_arch);
 
 my %blacklist = (
     '__bss_end__' => 1,		# arm
@@ -72,14 +75,14 @@ unwind_cpp_pr2 uread4 uread8 uwrite4 uwrite8));
 
 sub new {
     my $this = shift;
-    my $file = shift;
+    my %opts=@_;
     my $class = ref($this) || $this;
-    my $self = { };
+    my $self = \%opts;
     bless $self, $class;
+    $self->{arch} = get_host_arch() unless exists $self->{arch};
     $self->clear();
-    if (defined($file) ) {
-	$self->{file} = $file;
-	$self->load($file) if -e $file;
+    if (defined($self->{file})) {
+	$self->load($self->{file}) if -e $self->{file};
     }
     return $self;
 }
@@ -99,9 +102,30 @@ sub clear_except {
     }
 }
 
+sub add_symbol {
+    my ($self, $soname, $symbol) = @_;
+    my $object = (ref $soname) ? $soname : $self->{objects}{$soname};
+
+    if (!$symbol->{deprecated} && (my $ver = $symbol->get_wildcard_version())) {
+	error(_g("you can't use wildcards on unversioned symbols: %s"), $_) if $1 eq "Base";
+	$object->{wildcards}{$ver} = $symbol;
+	return 'wildcards';
+    } else {
+	$object->{syms}{$symbol->get_symbolname()} = $symbol;
+	return 'syms';
+    }
+}
+
 # Parameter seen is only used for recursive calls
 sub load {
-    my ($self, $file, $seen, $current_object_ref) = @_;
+    my ($self, $file, $seen, $current_object_ref, $base_symbol) = @_;
+
+    my $_new_symbol = sub {
+	if (defined $base_symbol) {
+	    return (ref $base_symbol) ? $base_symbol->clone(@_) : $base_symbol->new(@_);
+	}
+	return Dpkg::Shlibs::Symbol->new(@_);
+    };
 
     if (defined($seen)) {
 	return if exists $seen->{$file}; # Avoid include loops
@@ -120,35 +144,31 @@ sub load {
     local *object = $current_object_ref;
     while (defined($_ = <$sym_file>)) {
 	chomp($_);
-	if (/^\s+(\S+)\s(\S+)(?:\s(\d+))?/) {
+
+	if (/^(?:\s+|#(?:DEPRECATED|MISSING): ([^#]+)#\s*)(.*)/) {
 	    if (not defined ($object)) {
 		error(_g("Symbol information must be preceded by a header (file %s, line %s)."), $file, $.);
 	    }
-	    my $name = $1;
-	    # New symbol
-	    my $sym = {
-		minver => $2,
-		dep_id => defined($3) ? $3 : 0,
-		deprecated => 0
-	    };
-	    if ($name =~ /^\*@(.*)$/) {
-		error(_g("you can't use wildcards on unversioned symbols: %s"), $_) if $1 eq "Base";
-		$self->{objects}{$object}{wildcards}{$1} = $sym;
+	    # Symbol specification
+	    my $deprecated = ($1) ? $1 : 0;
+	    my $sym = &$_new_symbol(deprecated => $deprecated);
+	    if ($sym->parse($2)) {
+		$sym->process_tags(arch => $self->{arch});
+		$self->add_symbol($object, $sym);
 	    } else {
-		$self->{objects}{$object}{syms}{$name} = $sym;
+		warning(_g("Failed to parse line in %s: %s"), $file, $_);
 	    }
-	} elsif (/^#include\s+"([^"]+)"/) {
-	    my $filename = $1;
+	} elsif (/^(\(.*\))?#include\s+"([^"]+)"/) {
+	    my $tagspec = $1;
+	    my $filename = $2;
 	    my $dir = $file;
+	    my $new_base_symbol;
+	    if (defined $tagspec) {
+		$new_base_symbol = &$_new_symbol();
+		$new_base_symbol->parse_tagspec($tagspec);
+	    }
 	    $dir =~ s{[^/]+$}{}; # Strip filename
-	    $self->load("$dir$filename", $seen, $current_object_ref);
-	} elsif (/^#(?:DEPRECATED|MISSING): ([^#]+)#\s*(\S+)\s(\S+)(?:\s(\d+))?/) {
-	    my $sym = {
-		minver => $3,
-		dep_id => defined($4) ? $4 : 0,
-		deprecated => $1
-	    };
-	    $self->{objects}{$object}{syms}{$2} = $sym;
+	    $self->load("$dir$filename", $seen, $current_object_ref, $new_base_symbol);
 	} elsif (/^#/) {
 	    # Skip possible comments
 	} elsif (/^\|\s*(.*)$/) {
@@ -203,6 +223,7 @@ sub save {
 
 sub dump {
     my ($self, $fh, %opts) = @_;
+    $opts{template_mode} = 0 unless exists $opts{template_mode};
     $opts{with_deprecated} = 1 unless exists $opts{with_deprecated};
     foreach my $soname (sort keys %{$self->{objects}}) {
 	my @deps = @{$self->{objects}{$soname}{deps}};
@@ -219,13 +240,15 @@ sub dump {
             $value =~ s/#PACKAGE#/$opts{package}/g if exists $opts{package};
 	    print $fh "* $field: $value\n";
         }
-	foreach my $sym (sort keys %{$self->{objects}{$soname}{syms}}) {
+	my $syms = $self->{objects}{$soname}{syms};
+	foreach my $sym (sort { $syms->{$a}->get_symboltempl() cmp $syms->{$b}->get_symboltempl() }
+	                      keys %$syms) {
 	    my $info = $self->{objects}{$soname}{syms}{$sym};
 	    next if $info->{deprecated} and not $opts{with_deprecated};
-	    print $fh "#MISSING: $info->{deprecated}#" if $info->{deprecated};
-	    print $fh " $sym $info->{minver}";
-	    print $fh " $info->{dep_id}" if $info->{dep_id};
-	    print $fh "\n";
+	    # Do not dump symbols from foreign arch unless dumping a template.
+	    next if !$opts{template_mode} && !$info->arch_is_concerned($self->{arch});
+	    # Dump symbol specification. Dump symbol tags only in template mode.
+	    print $fh $info->get_symbolspec($opts{template_mode}), "\n";
 	}
     }
 }
@@ -259,31 +282,33 @@ sub merge_symbols {
 	    if ($info->{deprecated}) {
 		# Symbol reappeared somehow
 		$info->{deprecated} = 0;
-		$info->{minver} = $minver;
-		next;
+		$info->{minver} = $minver if (!$info->is_optional());
+	    } else {
+		# We assume that the right dependency information is already
+		# there.
+		if (vercmp($minver, $info->{minver}) < 0) {
+		    $info->{minver} = $minver;
+		}
 	    }
-	    # We assume that the right dependency information is already
-	    # there.
-	    if (vercmp($minver, $info->{minver}) < 0) {
-		$info->{minver} = $minver;
+	    if (!$info->arch_is_concerned($self->{arch})) {
+		# Remove arch tag because it is incorrect.
+		$info->delete_tag('arch');
 	    }
 	} else {
 	    # The symbol is new and not present in the file
 	    my $info;
 	    my $symobj = $dynsyms{$sym};
-	    if ($symobj->{version} and exists $obj->{wildcards}{$symobj->{version}}) {
+	    my $w_obj = $obj->{wildcards}{$symobj->{version}};
+	    if ($symobj->{version} and defined $w_obj
+	        and $w_obj->arch_is_concerned($self->{arch})) {
 		# Get the info from wildcards
 		$info = $obj->{wildcards}{$symobj->{version}};
 		$self->{used_wildcards}++;
 	    } else {
 		# New symbol provided by the current release
-		$info = {
-		    minver => $minver,
-		    deprecated => 0,
-		    dep_id => 0
-		};
+		$info = Dpkg::Shlibs::Symbol->new(symbol => $sym, minver => $minver);
 	    }
-	    $obj->{syms}{$sym} = $info;
+	    $self->add_symbol($obj, $info);
 	}
     }
 
@@ -292,12 +317,16 @@ sub merge_symbols {
     # the symbol was introduced)
     foreach my $sym (keys %{$self->{objects}{$soname}{syms}}) {
 	if (! exists $dynsyms{$sym}) {
-	    # Do nothing if already deprecated
-	    next if $self->{objects}{$soname}{syms}{$sym}{deprecated};
-
 	    my $info = $self->{objects}{$soname}{syms}{$sym};
-	    if (vercmp($minver, $info->{minver}) > 0) {
-		$self->{objects}{$soname}{syms}{$sym}{deprecated} = $minver;
+
+	    # Ignore symbols from foreign arch
+	    next if (!$info->arch_is_concerned($self->{arch}));
+
+	    if ($info->{deprecated}) {
+		# Bump deprecated if the symbol is optional.
+		$info->{deprecated} = $minver if ($info->is_optional());
+	    } elsif (vercmp($minver, $info->{minver}) > 0) {
+		$info->{deprecated} = $minver;
 	    }
 	}
     }
@@ -382,11 +411,10 @@ sub lookup_symbol {
 	    $self->{objects}{$so}{syms}{$name}{deprecated}))
 	{
 	    my $dep_id = $self->{objects}{$so}{syms}{$name}{dep_id};
-	    return {
-		'depends' => $self->{objects}{$so}{deps}[$dep_id],
-		'soname' => $so,
-		%{$self->{objects}{$so}{syms}{$name}}
-	    };
+	    my $clone = $self->{objects}{$so}{syms}{$name}->clone();
+	    $clone->{depends} = $self->{objects}{$so}{deps}[$dep_id];
+	    $clone->{soname} = $so;
+	    return $clone;
 	}
     }
     return undef;
@@ -399,17 +427,16 @@ sub get_new_symbols {
 	my $mysyms = $self->{objects}{$soname}{syms};
 	next if not exists $ref->{objects}{$soname};
 	my $refsyms = $ref->{objects}{$soname}{syms};
-	foreach my $sym (grep { not $mysyms->{$_}{deprecated} }
+	foreach my $sym (grep {  not $mysyms->{$_}{deprecated} and
+	                         not $mysyms->{$_}->is_optional() and
+	                         $mysyms->{$_}->arch_is_concerned($self->{arch}) }
 	    keys %{$mysyms})
 	{
 	    if ((not exists $refsyms->{$sym}) or
-		$refsyms->{$sym}{deprecated})
+		$refsyms->{$sym}{deprecated} or
+		not $refsyms->{$sym}->arch_is_concerned($self->{arch}) )
 	    {
-		push @res, {
-		    'soname' => $soname,
-		    'name' => $sym,
-		    %{$mysyms->{$sym}}
-		};
+		push @res, $mysyms->{$sym}->clone(soname => $soname);
 	    }
 	}
     }

scripts/Makefile.am

@@ -53,6 +53,9 @@ EXTRA_DIST = \
 	t/200_Dpkg_Shlibs/symbols.fake-2 \
 	t/200_Dpkg_Shlibs/symbols.include-1 \
 	t/200_Dpkg_Shlibs/symbols.include-2 \
+	t/200_Dpkg_Shlibs/symbols.include-3 \
+	t/200_Dpkg_Shlibs/symbols.tags.in \
+	t/200_Dpkg_Shlibs/symboltags.c \
 	t/200_Dpkg_Shlibs/ld.so.conf \
 	t/200_Dpkg_Shlibs/ld.so.conf_2 \
 	t/200_Dpkg_Shlibs/ld.so.conf.d/normal.conf \
@@ -63,6 +66,8 @@ EXTRA_DIST = \
 	t/200_Dpkg_Shlibs/objdump.libc6-2.6 \
 	t/200_Dpkg_Shlibs/objdump.dbd-pg \
 	t/200_Dpkg_Shlibs/objdump.ls \
+	t/200_Dpkg_Shlibs/objdump.tags-amd64 \
+	t/200_Dpkg_Shlibs/objdump.tags-i386 \
 	t/300_Dpkg_BuildOptions.t \
 	t/400_Dpkg_Deps.t \
 	t/500_Dpkg_Path.t \
@@ -100,6 +105,7 @@ nobase_dist_perllib_DATA = \
 	Dpkg/Path.pm \
 	Dpkg/Shlibs.pm \
 	Dpkg/Shlibs/Objdump.pm \
+	Dpkg/Shlibs/Symbol.pm \
 	Dpkg/Shlibs/SymbolFile.pm \
 	Dpkg/Source/Archive.pm \
 	Dpkg/Source/CompressedFile.pm \
@@ -170,8 +176,20 @@ clean-local:
 	[ ! -d t.tmp ] || chmod -R +w t.tmp
 	rm -fr t.tmp
 
-check: $(TEST_FILES) $(wildcard $(srcdir)/t/200_Dpkg_Shlibs/ld.so.conf*)
-# a bit hacky...
+mkdir_t_tmp:
 	$(mkdir_p) t.tmp
+
+check: $(TEST_FILES) $(wildcard $(srcdir)/t/200_Dpkg_Shlibs/ld.so.conf*) mkdir_t_tmp
+# a bit hacky...
 	cp -dRl $(srcdir)/t/200_Dpkg_Shlibs/ld.so.conf* t.tmp/
 	PATH="$(top_builddir)/src:$(top_builddir)/scripts:$(PATH)" srcdir=$(srcdir) PERL5LIB=$(srcdir) DPKG_DATADIR=$(srcdir)/.. PERL_DL_NONLAZY=1 $(PERL) -I$(srcdir) "-MExtUtils::Command::MM" "-e" "test_harness($(TEST_VERBOSE), '.')" $(TEST_FILES)
+
+objdump_lib_name = lib$(shell basename $@).so
+$(srcdir)/t/200_Dpkg_Shlibs/objdump.tags-amd64 $(srcdir)/t/200_Dpkg_Shlibs/objdump.tags-i386: $(srcdir)/t/200_Dpkg_Shlibs/symboltags.c
+	$(CC) $(CFLAGS) -shared -fPIC -Wl,-soname -Wl,libsymboltags.so.1 $< \
+	  $(if $(findstring amd64,$@),-DAMD64,) -o $(objdump_lib_name)
+	objdump -w -f -p -T -R $(objdump_lib_name) | sed 's/$(objdump_lib_name)/libsymboltags.so.1/g' > $@
+	rm -f $(objdump_lib_name)
+
+update-objdumps .PHONY: $(srcdir)/t/200_Dpkg_Shlibs/objdump.tags-amd64
+update-objdumps .PHONY: $(srcdir)/t/200_Dpkg_Shlibs/objdump.tags-i386

scripts/dpkg-gensymbols.pl

@@ -24,6 +24,7 @@ my $oppackage;
 my $compare = 1; # Bail on missing symbols by default
 my $input;
 my $output;
+my $template_mode = 0; # non-template mode by default
 my $debug = 0;
 my $host_arch = get_host_arch();
 
@@ -60,6 +61,8 @@ Options:
                            file instead of the default file.
   -O<file>                 write to <file>, not .../DEBIAN/symbols.
   -O                       write to stdout, not .../DEBIAN/symbols.
+  -t                       write in template mode (tags are not
+                           processed and included in output).
   -d                       display debug information during work.
   -h, --help               show this help message.
       --version            show the version.
@@ -95,6 +98,8 @@ while (@ARGV) {
 	$input = $1;
     } elsif (m/^-O(.+)$/) {
 	$output = $1;
+    } elsif (m/^-t$/) {
+	$template_mode = 1;
     } elsif (m/^-(h|-help)$/) {
 	usage();
 	exit(0);
@@ -125,8 +130,8 @@ if (not defined($oppackage)) {
     $oppackage = $packages[0];
 }
 
-my $symfile = Dpkg::Shlibs::SymbolFile->new();
-my $ref_symfile = Dpkg::Shlibs::SymbolFile->new();
+my $symfile = Dpkg::Shlibs::SymbolFile->new(arch => $host_arch);
+my $ref_symfile = Dpkg::Shlibs::SymbolFile->new(arch => $host_arch);
 # Load source-provided symbol information
 foreach my $file ($input, $output, "debian/$oppackage.symbols.$host_arch",
     "debian/symbols.$host_arch", "debian/$oppackage.symbols",
@@ -189,7 +194,7 @@ $symfile->clear_except(keys %{$od->{objects}});
 # Write out symbols files
 if ($stdout) {
     $output = "standard output";
-    $symfile->save("-", package => $oppackage);
+    $symfile->save("-", package => $oppackage, template_mode => $template_mode);
 } else {
     unless (defined($output)) {
 	unless($symfile->is_empty()) {
@@ -199,7 +204,7 @@ if ($stdout) {
     }
     if (defined($output)) {
 	print "Storing symbols in $output.\n" if $debug;
-	$symfile->save($output, package => $oppackage);
+	$symfile->save($output, package => $oppackage, template_mode => $template_mode);
     } else {
 	print "No symbol information to store.\n" if $debug;
     }
@@ -237,12 +242,12 @@ if ($compare) {
 	    $list = "\n";
 	    my $cur_soname = "";
 	    foreach my $sym (sort { $a->{soname} cmp $b->{soname} or
-				    $a->{name} cmp $b->{name} } @syms) {
+				    $a->get_symboltempl() cmp $b->get_symboltempl() } @syms) {
 		if ($cur_soname ne $sym->{soname}) {
 		    $list .= $sym->{soname} . "\n";
 		    $cur_soname = $sym->{soname};
 		}
-		$list .= " " . $sym->{name} . "\n";
+		$list .= " " . $sym->get_symbolname() . "\n";
 	    }
 	}
 	warning(_g("some symbols disappeared in the symbols file: %s"), $list);
@@ -253,8 +258,8 @@ if ($compare) {
 	# and after
 	my $before = File::Temp->new(TEMPLATE=>'dpkg-gensymbolsXXXXXX');
 	my $after = File::Temp->new(TEMPLATE=>'dpkg-gensymbolsXXXXXX');
-	$ref_symfile->dump($before, package => $oppackage);
-        $symfile->dump($after, package => $oppackage);
+	$ref_symfile->dump($before, package => $oppackage, template_mode => 1);
+	$symfile->dump($after, package => $oppackage, template_mode => 1);
 	seek($before, 0, 0); seek($after, 0, 0);
 	my ($md5_before, $md5_after) = (Digest::MD5->new(), Digest::MD5->new());
 	$md5_before->addfile($before);
@@ -269,7 +274,7 @@ if ($compare) {
 			$output);
 	    }
 	    my ($a, $b) = ($before->filename, $after->filename);
-	    system("diff", "-u", $a, $b) if -x "/usr/bin/diff";
+	    system("diff", "-u", "-L", "dpkg-gensymbols_${oppackage}_${host_arch}", $a, $b) if -x "/usr/bin/diff";
 	}
     }
 }

scripts/dpkg-shlibdeps.pl

@@ -215,7 +215,7 @@ foreach my $file (keys %exec) {
                 print "Using symbols file $symfile_path for $soname\n" if $debug;
                 unless (exists $symfile_cache{$symfile_path}) {
                     $symfile_cache{$symfile_path} =
-                        Dpkg::Shlibs::SymbolFile->new($symfile_path);
+                        Dpkg::Shlibs::SymbolFile->new(file => $symfile_path);
                 }
                 $symfile->merge_object_from_symfile($symfile_cache{$symfile_path}, $soname);
             }

scripts/t/200_Dpkg_Shlibs.t

@@ -1,6 +1,6 @@
 # -*- mode: cperl;-*-
 
-use Test::More tests => 39;
+use Test::More tests => 61;
 use IO::String;
 
 use strict;
@@ -8,6 +8,8 @@ use warnings;
 
 use_ok('Dpkg::Shlibs');
 
+my @tmp;
+
 my @save_paths = @Dpkg::Shlibs::librarypaths;
 @Dpkg::Shlibs::librarypaths = ();
 
@@ -77,7 +79,7 @@ is_deeply( $sym, { name => '_IO_stdin_used', version => '',
 		   soname => 'libc.so.6', objid => 'libc.so.6',
 		   section => '*UND*', dynamic => 1,
 		   debug => '', type => ' ', weak => 1,
-		   local => '', global => '', visibility => '',   
+		   local => '', global => '', visibility => '',
 		   hidden => '', defined => '' }, 'Symbol 2' );
 
 my @syms = $obj->get_exported_dynamic_symbols;
@@ -95,10 +97,11 @@ close $objdump;
 
 
 use_ok('Dpkg::Shlibs::SymbolFile');
+use_ok('Dpkg::Shlibs::Symbol');
 
-my $sym_file = Dpkg::Shlibs::SymbolFile->new("$srcdir/symbol_file.tmp");
-my $sym_file_dup = Dpkg::Shlibs::SymbolFile->new("$srcdir/symbol_file.tmp");
-my $sym_file_old = Dpkg::Shlibs::SymbolFile->new("$srcdir/symbol_file.tmp");
+my $sym_file = Dpkg::Shlibs::SymbolFile->new(file => "$srcdir/symbol_file.tmp");
+my $sym_file_dup = Dpkg::Shlibs::SymbolFile->new(file => "$srcdir/symbol_file.tmp");
+my $sym_file_old = Dpkg::Shlibs::SymbolFile->new(file => "$srcdir/symbol_file.tmp");
 
 $sym_file->merge_symbols($obj_old, "2.3.6.ds1-13");
 $sym_file_old->merge_symbols($obj_old, "2.3.6.ds1-13");
@@ -114,9 +117,11 @@ is( $sym_file_old->lookup_symbol('__bss_start@Base', ['libc.so.6']),
     undef, 'internal symbols are blacklisted');
 
 $sym = $sym_file->lookup_symbol('_errno@GLIBC_2.0', ['libc.so.6'], 1);
-is_deeply($sym, { 'minver' => '2.3.6.ds1-13', 'dep_id' => 0, 
-		  'deprecated' => '2.6-1', 'depends' => '', 
-		  'soname' => 'libc.so.6' }, 'deprecated symbol');
+isa_ok($sym, 'Dpkg::Shlibs::Symbol');
+is_deeply($sym, Dpkg::Shlibs::Symbol->new( 'symbol' => '_errno@GLIBC_2.0',
+		  'minver' => '2.3.6.ds1-13', 'dep_id' => 0,
+		  'deprecated' => '2.6-1', 'depends' => '',
+		  'soname' => 'libc.so.6' ), 'deprecated symbol');
 
 use File::Temp;
 
@@ -130,36 +135,41 @@ $sym_file_dup->{file} = "$srcdir/symbol_file.tmp";
 is_deeply($sym_file_dup, $sym_file, 'save -> load' );
 
 # Test include mechanism of SymbolFile
-$sym_file = Dpkg::Shlibs::SymbolFile->new("$srcdir/symbols.include-1");
+$sym_file = Dpkg::Shlibs::SymbolFile->new(file => "$srcdir/symbols.include-1");
 
 $sym = $sym_file->lookup_symbol('symbol_before@Base', ['libfake.so.1']);
-is_deeply($sym, { 'minver' => '0.9', 'dep_id' => 0, 'deprecated' => 0,
-		  'depends' => 'libfake1 #MINVER#', 'soname' => 'libfake.so.1' }, 
+is_deeply($sym, Dpkg::Shlibs::Symbol->new( 'symbol' => 'symbol_before@Base',
+		  'minver' => '0.9', 'dep_id' => 0, 'deprecated' => 0,
+		  'depends' => 'libfake1 #MINVER#', 'soname' => 'libfake.so.1' ),
 	    'symbol before include not lost');
 
 $sym = $sym_file->lookup_symbol('symbol_after@Base', ['libfake.so.1']);
-is_deeply($sym, {'minver' => '1.1', 'dep_id' => 0, 'deprecated' => 0, 
-		  'depends' => 'libfake1 #MINVER#', 'soname' => 'libfake.so.1' }, 
+is_deeply($sym, Dpkg::Shlibs::Symbol->new( 'symbol' => 'symbol_after@Base',
+		  'minver' => '1.1', 'dep_id' => 0, 'deprecated' => 0,
+		  'depends' => 'libfake1 #MINVER#', 'soname' => 'libfake.so.1' ),
 	    'symbol after include not lost');
 
 $sym = $sym_file->lookup_symbol('symbol1_fake1@Base', ['libfake.so.1']);
-is_deeply($sym, {'minver' => '1.0', 'dep_id' => 0, 'deprecated' => 0, 
-		  'depends' => 'libfake1 #MINVER#', 'soname' => 'libfake.so.1' }, 
+is_deeply($sym, Dpkg::Shlibs::Symbol->new( 'symbol' => 'symbol1_fake1@Base',
+		  'minver' => '1.0', 'dep_id' => 0, 'deprecated' => 0,
+		  'depends' => 'libfake1 #MINVER#', 'soname' => 'libfake.so.1' ),
 	    'overrides order with #include');
 
 $sym = $sym_file->lookup_symbol('symbol3_fake1@Base', ['libfake.so.1']);
-is_deeply($sym, { 'minver' => '0', 'dep_id' => 0, 'deprecated' => 0,
-		  'depends' => 'libfake1 #MINVER#', 'soname' => 'libfake.so.1' }, 
+is_deeply($sym, Dpkg::Shlibs::Symbol->new ( 'symbol' => 'symbol3_fake1@Base',
+		  'minver' => '0', 'dep_id' => 0, 'deprecated' => 0,
+		  'depends' => 'libfake1 #MINVER#', 'soname' => 'libfake.so.1' ),
 	    'overrides order with #include');
 
 is($sym_file->get_smallest_version('libfake.so.1'), "0",
    'get_smallest_version with null version');
 
-$sym_file = Dpkg::Shlibs::SymbolFile->new("$srcdir/symbols.include-2");
+$sym_file = Dpkg::Shlibs::SymbolFile->new(file => "$srcdir/symbols.include-2");
 
 $sym = $sym_file->lookup_symbol('symbol1_fake2@Base', ['libfake.so.1']);
-is_deeply($sym, { 'minver' => '1.0', 'dep_id' => 1, 'deprecated' => 0,
-		  'depends' => 'libvirtualfake', 'soname' => 'libfake.so.1' }, 
+is_deeply($sym, Dpkg::Shlibs::Symbol->new ( 'symbol' => 'symbol1_fake2@Base',
+		  'minver' => '1.0', 'dep_id' => 1, 'deprecated' => 0,
+		  'depends' => 'libvirtualfake', 'soname' => 'libfake.so.1' ),
 	    'overrides order with circular #include');
 
 is($sym_file->get_smallest_version('libfake.so.1'), "1.0",
@@ -193,6 +203,153 @@ is_deeply( $sym, { name => 'IA__g_free', version => '',
 		   section => '.text', dynamic => 1,
 		   debug => '', type => 'F', weak => '',
 		   local => 1, global => '', visibility => 'hidden',
-		   hidden => '', defined => 1 }, 
+		   hidden => '', defined => 1 },
 		   'symbol with visibility without version' );
 
+####### Test symbol tagging support  ######
+
+# Parsing/dumping
+# Template mode
+$sym_file = Dpkg::Shlibs::SymbolFile->new(file => "$srcdir/symbols.tags.in", arch => 'amd64');
+$sym_file->save($save_file->filename, template_mode => 1);
+
+$sym_file_dup = Dpkg::Shlibs::SymbolFile->new(file => $save_file, arch => 'amd64');
+$sym_file_dup->{file} = "$srcdir/symbols.tags.in";
+
+is_deeply($sym_file_dup, $sym_file, 'template save -> load' );
+is( system("diff -u '$srcdir/symbols.tags.in' '" . $save_file->filename . "' >&2"), 0, "symbols.tags.in template dumped identical" );
+
+# Dumping in non-template mode (amd64) (test for arch, subst tags)
+$io = IO::String->new();
+$sym_file->dump($io);
+is(${$io->string_ref()},
+'libsymboltags.so.1 libsymboltags1 #MINVER#
+| libsymboltags1 (>= 1.1)
+ symbol11_optional@Base 1.1 1
+ symbol21_amd64@Base 2.1
+ symbol31_randomtag@Base 3.1
+ symbol51_untagged@Base 5.1
+', "template vs. non-template on amd64" );
+
+# Dumping in non-template mode (i386) (test for arch, subst tags)
+$io = IO::String->new();
+$sym_file = Dpkg::Shlibs::SymbolFile->new(file => "$srcdir/symbols.tags.in", arch => 'i386');
+$sym_file_dup = Dpkg::Shlibs::SymbolFile->new(file => "$srcdir/symbols.tags.in", arch => 'i386');
+$sym_file->dump($io);
+is(${$io->string_ref()},
+'libsymboltags.so.1 libsymboltags1 #MINVER#
+| libsymboltags1 (>= 1.1)
+ symbol11_optional@Base 1.1 1
+ symbol22_i386@Base 2.2
+ symbol31_randomtag@Base 3.1
+ symbol41_i386_and_optional@Base 4.1
+ symbol51_untagged@Base 5.1
+', "template vs. non-template on i386" );
+
+ok (defined $sym_file->{objects}{'libsymboltags.so.1'}{syms}{'symbol21_amd64@Base'},
+    "syms keys are symbol names without quotes");
+
+# Preload objdumps
+my $tags_obj_i386 = Dpkg::Shlibs::Objdump::Object->new();
+open $objdump, '<', "$srcdir/objdump.tags-i386" or die "$srcdir/objdump.tags-i386: $!";
+$tags_obj_i386->_parse($objdump);
+close $objdump;
+$sym_file->merge_symbols($tags_obj_i386, '100.MISSING');
+is_deeply($sym_file, $sym_file_dup, "is objdump.tags-i386 and symbols.tags.in in sync");
+
+my $tags_obj_amd64 = Dpkg::Shlibs::Objdump::Object->new();
+open $objdump, '<', "$srcdir/objdump.tags-amd64" or die "$srcdir/objdump.tags-amd64: $!";
+$tags_obj_amd64->_parse($objdump);
+close $objdump;
+
+# Merge/get_{new,lost} tests for optional tag:
+#  - disappeared
+my $symbol11 = $tags_obj_i386->get_symbol('symbol11_optional@Base');
+delete $tags_obj_i386->{dynsyms}{'symbol11_optional@Base'};
+$sym_file->merge_symbols($tags_obj_i386, '100.MISSING');
+
+$sym = $sym_file->lookup_symbol('symbol11_optional@Base', ['libsymboltags.so.1'], 1);
+is_deeply($sym, Dpkg::Shlibs::Symbol->new( 'symbol' => 'symbol11_optional@Base', 'symbol_templ' => 'symbol11_optional@Base',
+		  'minver' => '1.1', 'dep_id' => 1, 'deprecated' => '100.MISSING',
+		  'depends' => 'libsymboltags1 (>= 1.1)', 'soname' => 'libsymboltags.so.1',
+		  'tags' => { 'optional' => undef }, 'tagorder' => [ 'optional' ] ),
+	    'disappered optional symbol gets deprecated');
+
+$sym_file->merge_symbols($tags_obj_i386, '101.MISSING');
+$sym = $sym_file->lookup_symbol('symbol11_optional@Base', ['libsymboltags.so.1'], 1);
+is_deeply($sym, Dpkg::Shlibs::Symbol->new( 'symbol' => 'symbol11_optional@Base', 'symbol_templ' => 'symbol11_optional@Base',
+		  'minver' => '1.1', 'dep_id' => 1, 'deprecated' => '101.MISSING',
+		  'depends' => 'libsymboltags1 (>= 1.1)', 'soname' => 'libsymboltags.so.1',
+		  'tags' => { 'optional' => undef }, 'tagorder' => [ 'optional' ] ),
+	    'deprecated text of MISSING optional symbol gets rebumped each merge');
+
+is( scalar($sym_file->get_lost_symbols($sym_file_dup)), 0, "missing optional symbol is not LOST");
+
+# - reappeared (undeprecate, minver should be 1.1, not 100.MISSED)
+$tags_obj_i386->add_dynamic_symbol($symbol11);
+$sym_file->merge_symbols($tags_obj_i386, '100.MISSING');
+$sym = $sym_file->lookup_symbol('symbol11_optional@Base', ['libsymboltags.so.1']);
+is_deeply($sym, Dpkg::Shlibs::Symbol->new( 'symbol' => 'symbol11_optional@Base', 'symbol_templ' => 'symbol11_optional@Base',
+		  'minver' => '1.1', 'dep_id' => 1, 'deprecated' => 0,
+		  'depends' => 'libsymboltags1 (>= 1.1)', 'soname' => 'libsymboltags.so.1',
+		  'tags' => { 'optional' => undef }, 'tagorder' => [ 'optional' ] ),
+	    'reappered optional symbol gets undeprecated + minver');
+is( scalar($sym_file->get_lost_symbols($sym_file_dup) +
+           $sym_file->get_new_symbols($sym_file_dup)), 0, "reappeared optional symbol: neither NEW nor LOST");
+
+# Merge/get_{new,lost} tests for arch tag:
+# - arch specific appears on wrong arch: 'arch' tag should be removed
+my $symbol21 = $tags_obj_amd64->get_symbol('symbol21_amd64@Base');
+$tags_obj_i386->add_dynamic_symbol($symbol21);
+$sym_file->merge_symbols($tags_obj_i386, '100.MISSING');
+$sym = $sym_file->lookup_symbol('symbol21_amd64@Base', ['libsymboltags.so.1']);
+is_deeply($sym, Dpkg::Shlibs::Symbol->new( 'symbol' => 'symbol21_amd64@Base', 'symbol_templ' => 'symbol21_amd64@Base',
+		  'symbol_quoted' => "'", 'minver' => '2.1', 'dep_id' => 0, 'deprecated' => 0,
+		  'depends' => 'libsymboltags1 #MINVER#', 'soname' => 'libsymboltags.so.1' ),
+	    'symbol appears on foreign arch, arch tag should be removed');
+@tmp = map { $_->get_symbolname() } $sym_file->get_new_symbols($sym_file_dup);
+is_deeply( \@tmp, [ 'symbol21_amd64@Base' ], "symbol from foreign arch is NEW");
+is( $sym->get_symbolspec(1), ' symbol21_amd64@Base 2.1', 'no tags => no quotes in the symbol name' );
+
+# - arch specific symbol disappears
+delete $tags_obj_i386->{dynsyms}{'symbol22_i386@Base'};
+delete $tags_obj_i386->{dynsyms}{'symbol41_i386_and_optional@Base'};
+$sym_file->merge_symbols($tags_obj_i386, '100.MISSING');
+
+$sym = $sym_file->lookup_symbol('symbol22_i386@Base', ['libsymboltags.so.1'], 1);
+is_deeply($sym, Dpkg::Shlibs::Symbol->new( 'symbol' => 'symbol22_i386@Base', 'symbol_templ' => 'symbol22_i386@Base',
+		  'minver' => '2.2', 'dep_id' => 0, 'deprecated' => '100.MISSING',
+		  'depends' => 'libsymboltags1 #MINVER#', 'soname' => 'libsymboltags.so.1',
+		  'tags' => { 'arch' => '!amd64 !ia64 !alpha' }, 'tagorder' => [ 'arch' ] ),
+	    'disappeared arch specific symbol gets deprecated');
+$sym = $sym_file->lookup_symbol('symbol41_i386_and_optional@Base', ['libsymboltags.so.1'], 1);
+is_deeply($sym, Dpkg::Shlibs::Symbol->new( 'symbol' => 'symbol41_i386_and_optional@Base',
+		  'symbol_templ' => 'symbol41_i386_and_optional@Base', 'symbol_quoted' => '"',
+		  'minver' => '4.1', 'dep_id' => 0, 'deprecated' => '100.MISSING',
+		  'depends' => 'libsymboltags1 #MINVER#', 'soname' => 'libsymboltags.so.1',
+		  'tags' => { 'arch' => 'i386', 'optional' => 'reason' }, 'tagorder' => [ 'arch', 'optional' ] ),
+	    'disappeared optional arch specific symbol gets deprecated');
+@tmp = map { $_->get_symbolname() } $sym_file->get_lost_symbols($sym_file_dup);
+is_deeply( \@tmp, [ 'symbol22_i386@Base' ], "missing arch specific is LOST, but optional arch specific isn't");
+
+# Tests for tagged #includes
+$sym_file = Dpkg::Shlibs::SymbolFile->new(file => "$srcdir/symbols.include-3", arch => 'i386');
+$sym = $sym_file->lookup_symbol('symbol2_fake1@Base', ['libsymboltags.so.2']);
+is_deeply($sym, Dpkg::Shlibs::Symbol->new( 'symbol' => 'symbol2_fake1@Base',
+		  'minver' => '1.0', 'depends' => 'libsymboltags2', 'soname' => 'libsymboltags.so.2',
+		  'tags' => { 'optional' => undef, 'random tag' => 'random value' },
+		  'tagorder' => [ 'optional', 'random tag' ] ),
+	    'symbols from #included file inherits tags');
+$sym = $sym_file->lookup_symbol('symbol41_i386_and_optional@Base', ['libsymboltags.so.1']);
+is_deeply($sym, Dpkg::Shlibs::Symbol->new( 'symbol' => 'symbol41_i386_and_optional@Base',
+		  'symbol_templ' => 'symbol41_i386_and_optional@Base', symbol_quoted => '"',
+		  'minver' => '4.1', 'depends' => 'libsymboltags1 #MINVER#', 'soname' => 'libsymboltags.so.1',
+		  'tags' => { 'optional' => 'reason', 't' => 'v', 'arch' => 'i386' },
+		  'tagorder' => [ 'optional', 't', 'arch' ] ),
+	    'symbols in #included file can override tag values');
+$sym = $sym_file->lookup_symbol('symbol51_untagged@Base', ['libsymboltags.so.1']);
+is_deeply($sym, Dpkg::Shlibs::Symbol->new( 'symbol' => 'symbol51_untagged@Base',
+		  'minver' => '5.1', 'depends' => 'libsymboltags1 #MINVER#', 'soname' => 'libsymboltags.so.1',
+		  'tags' => { 'optional' => 'from parent', 't' => 'v' },
+		  'tagorder' => [ 'optional', 't' ] ),
+	    'symbols are properly cloned when #including');

scripts/t/200_Dpkg_Shlibs/objdump.tags-amd64

@@ -0,0 +1,70 @@
+
+libsymboltags.so.1:     file format elf64-x86-64
+architecture: i386:x86-64, flags 0x00000150:
+HAS_SYMS, DYNAMIC, D_PAGED
+start address 0x0000000000000500
+
+Program Header:
+    LOAD off    0x0000000000000000 vaddr 0x0000000000000000 paddr 0x0000000000000000 align 2**21
+         filesz 0x0000000000000714 memsz 0x0000000000000714 flags r-x
+    LOAD off    0x0000000000000718 vaddr 0x0000000000200718 paddr 0x0000000000200718 align 2**21
+         filesz 0x0000000000000208 memsz 0x0000000000000218 flags rw-
+ DYNAMIC off    0x0000000000000740 vaddr 0x0000000000200740 paddr 0x0000000000200740 align 2**3
+         filesz 0x00000000000001a0 memsz 0x00000000000001a0 flags rw-
+EH_FRAME off    0x0000000000000668 vaddr 0x0000000000000668 paddr 0x0000000000000668 align 2**2
+         filesz 0x000000000000002c memsz 0x000000000000002c flags r--
+   STACK off    0x0000000000000000 vaddr 0x0000000000000000 paddr 0x0000000000000000 align 2**3
+         filesz 0x0000000000000000 memsz 0x0000000000000000 flags rw-
+
+Dynamic Section:
+  NEEDED               libc.so.6
+  SONAME               libsymboltags.so.1
+  INIT                 0x00000000000004c0
+  FINI                 0x0000000000000658
+  HASH                 0x0000000000000158
+  GNU_HASH             0x00000000000001a8
+  STRTAB               0x0000000000000340
+  SYMTAB               0x00000000000001f0
+  STRSZ                0x00000000000000c6
+  SYMENT               0x0000000000000018
+  PLTGOT               0x00000000002008f8
+  PLTRELSZ             0x0000000000000018
+  PLTREL               0x0000000000000007
+  JMPREL               0x00000000000004a8
+  RELA                 0x0000000000000448
+  RELASZ               0x0000000000000060
+  RELAENT              0x0000000000000018
+  VERNEED              0x0000000000000428
+  VERNEEDNUM           0x0000000000000001
+  VERSYM               0x0000000000000406
+  RELACOUNT            0x0000000000000001
+
+Version References:
+  required from libc.so.6:
+    0x09691a75 0x00 02 GLIBC_2.2.5
+
+DYNAMIC SYMBOL TABLE:
+00000000000004c0 l    d  .init	0000000000000000              .init
+0000000000000000  w   D  *UND*	0000000000000000              __gmon_start__
+0000000000000000  w   D  *UND*	0000000000000000              _Jv_RegisterClasses
+0000000000000000  w   DF *UND*	0000000000000000  GLIBC_2.2.5 __cxa_finalize
+00000000000005f0 g    DF .text	0000000000000002  Base        symbol21_amd64
+0000000000200930 g    D  *ABS*	0000000000000000  Base        _end
+0000000000200920 g    D  *ABS*	0000000000000000  Base        _edata
+0000000000200920 g    D  *ABS*	0000000000000000  Base        __bss_start
+00000000000004c0 g    DF .init	0000000000000000  Base        _init
+0000000000000658 g    DF .fini	0000000000000000  Base        _fini
+00000000000005e0 g    DF .text	0000000000000002  Base        symbol11_optional
+0000000000000600 g    DF .text	0000000000000002  Base        symbol31_randomtag
+0000000000000610 g    DF .text	0000000000000002  Base        symbol51_untagged
+
+
+DYNAMIC RELOCATION RECORDS
+OFFSET           TYPE              VALUE 
+0000000000200918 R_X86_64_RELATIVE  *ABS*+0x0000000000200918
+00000000002008e0 R_X86_64_GLOB_DAT  __gmon_start__
+00000000002008e8 R_X86_64_GLOB_DAT  _Jv_RegisterClasses
+00000000002008f0 R_X86_64_GLOB_DAT  __cxa_finalize
+0000000000200910 R_X86_64_JUMP_SLOT  __cxa_finalize
+
+

scripts/t/200_Dpkg_Shlibs/objdump.tags-i386

@@ -0,0 +1,71 @@
+
+libsymboltags.so.1:     file format elf64-x86-64
+architecture: i386:x86-64, flags 0x00000150:
+HAS_SYMS, DYNAMIC, D_PAGED
+start address 0x0000000000000530
+
+Program Header:
+    LOAD off    0x0000000000000000 vaddr 0x0000000000000000 paddr 0x0000000000000000 align 2**21
+         filesz 0x0000000000000774 memsz 0x0000000000000774 flags r-x
+    LOAD off    0x0000000000000778 vaddr 0x0000000000200778 paddr 0x0000000000200778 align 2**21
+         filesz 0x0000000000000208 memsz 0x0000000000000218 flags rw-
+ DYNAMIC off    0x00000000000007a0 vaddr 0x00000000002007a0 paddr 0x00000000002007a0 align 2**3
+         filesz 0x00000000000001a0 memsz 0x00000000000001a0 flags rw-
+EH_FRAME off    0x00000000000006a8 vaddr 0x00000000000006a8 paddr 0x00000000000006a8 align 2**2
+         filesz 0x0000000000000034 memsz 0x0000000000000034 flags r--
+   STACK off    0x0000000000000000 vaddr 0x0000000000000000 paddr 0x0000000000000000 align 2**3
+         filesz 0x0000000000000000 memsz 0x0000000000000000 flags rw-
+
+Dynamic Section:
+  NEEDED               libc.so.6
+  SONAME               libsymboltags.so.1
+  INIT                 0x00000000000004f8
+  FINI                 0x0000000000000698
+  HASH                 0x0000000000000158
+  GNU_HASH             0x00000000000001a8
+  STRTAB               0x0000000000000360
+  SYMTAB               0x00000000000001f8
+  STRSZ                0x00000000000000e0
+  SYMENT               0x0000000000000018
+  PLTGOT               0x0000000000200958
+  PLTRELSZ             0x0000000000000018
+  PLTREL               0x0000000000000007
+  JMPREL               0x00000000000004e0
+  RELA                 0x0000000000000480
+  RELASZ               0x0000000000000060
+  RELAENT              0x0000000000000018
+  VERNEED              0x0000000000000460
+  VERNEEDNUM           0x0000000000000001
+  VERSYM               0x0000000000000440
+  RELACOUNT            0x0000000000000001
+
+Version References:
+  required from libc.so.6:
+    0x09691a75 0x00 02 GLIBC_2.2.5
+
+DYNAMIC SYMBOL TABLE:
+00000000000004f8 l    d  .init	0000000000000000              .init
+0000000000000000  w   D  *UND*	0000000000000000              __gmon_start__
+0000000000000000  w   D  *UND*	0000000000000000              _Jv_RegisterClasses
+0000000000000000  w   DF *UND*	0000000000000000  GLIBC_2.2.5 __cxa_finalize
+0000000000000640 g    DF .text	0000000000000002  Base        symbol41_i386_and_optional
+0000000000200990 g    D  *ABS*	0000000000000000  Base        _end
+0000000000200980 g    D  *ABS*	0000000000000000  Base        _edata
+0000000000000620 g    DF .text	0000000000000002  Base        symbol22_i386
+0000000000200980 g    D  *ABS*	0000000000000000  Base        __bss_start
+00000000000004f8 g    DF .init	0000000000000000  Base        _init
+0000000000000698 g    DF .fini	0000000000000000  Base        _fini
+0000000000000610 g    DF .text	0000000000000002  Base        symbol11_optional
+0000000000000630 g    DF .text	0000000000000002  Base        symbol31_randomtag
+0000000000000650 g    DF .text	0000000000000002  Base        symbol51_untagged
+
+
+DYNAMIC RELOCATION RECORDS
+OFFSET           TYPE              VALUE 
+0000000000200978 R_X86_64_RELATIVE  *ABS*+0x0000000000200978
+0000000000200940 R_X86_64_GLOB_DAT  __gmon_start__
+0000000000200948 R_X86_64_GLOB_DAT  _Jv_RegisterClasses
+0000000000200950 R_X86_64_GLOB_DAT  __cxa_finalize
+0000000000200970 R_X86_64_JUMP_SLOT  __cxa_finalize
+
+

scripts/t/200_Dpkg_Shlibs/symbols.include-3

@@ -0,0 +1,5 @@
+libsymboltags.so.2 libsymboltags2
+ symbol0_before@Base 1.0
+(optional|random tag=random value)#include "symbols.fake-1"
+ symbol9_after@Base 1.0
+(optional=from parent|t=v)#include "symbols.tags.in"

scripts/t/200_Dpkg_Shlibs/symbols.tags.in

@@ -0,0 +1,8 @@
+libsymboltags.so.1 libsymboltags1 #MINVER#
+| libsymboltags1 (>= 1.1)
+ (optional)symbol11_optional@Base 1.1 1
+ (arch=amd64 ia64 alpha)'symbol21_amd64@Base' 2.1
+ (arch=!amd64 !ia64 !alpha)symbol22_i386@Base 2.2
+ (randomtagname=randomtagvalue)symbol31_randomtag@Base 3.1
+ (arch=i386|optional=reason)"symbol41_i386_and_optional@Base" 4.1
+ symbol51_untagged@Base 5.1

scripts/t/200_Dpkg_Shlibs/symboltags.c

@@ -0,0 +1,23 @@
+// gcc -shared -fPIC -Wl,-soname -Wl,libsymboltags.so.1 -o libsymboltags.so.1 -DAMD64 symboltags.c
+// objdump -wfpTR libsymboltags.so.1 > objdump.tags-amd64
+// gcc -shared -fPIC -Wl,-soname -Wl,libsymboltags.so.1 -o libsymboltags.so.1 symboltags.c
+// objdump -wfpTR libsymboltags.so.1 > objdump.tags-i386
+
+void symbol11_optional() {}
+
+#ifdef AMD64
+void symbol21_amd64() {}
+#endif
+
+#ifndef AMD64
+void* symbol22_i386() {}
+#endif
+
+void symbol31_randomtag() {}
+
+// (arch=i386|optional)symbol41_i386_and_optional@Base
+#ifndef AMD64
+void symbol41_i386_and_optional() {}
+#endif
+
+void symbol51_untagged() {}