merged from lp:~donkult/apt/sid

.bzrignore

@@ -11,14 +11,17 @@ configure
 buildlib/config.sub
 buildlib/config.guess
 
+# abichecker related files/dir
+abicheck/apt_build.xml
+abicheck/apt_installed.xml
+abicheck/compat_reports/
+abicheck/descriptors_storage/
+abicheck/header_compile_errors/
+abicheck/test_results/
+abicheck/tests/
+
 # generated files in the progress to build all
 # apt man pages and other documentation
-doc/*.1
-doc/*.5
-doc/*.8
-doc/*/*.1
-doc/*/*.5
-doc/*/*.8
-doc/*/apt.ent
-doc/*/*.xml
-doc/*/*.sgml
+./doc/*/
+!./doc/examples/
+!./doc/po/

README.MultiArch

@@ -0,0 +1,113 @@
+Before we start with this topic: Note that MultiArch is not yet ready for
+prime time and/or for the casual user. The implementation is so far widely
+untested and only useful for developers of packagemanagment tools which
+use APT and his friends and maintainers of (upcoming) MultiArch packages.
+This README is especially NOT written for the casual user and is NOT a
+usage guide - you have been warned. It is assumed that the reader has
+at least a bit of knowledge about APT internals, dependency relations
+and the MultiArch spec [0].
+
+Note also that the toolchain isn't ready yet, e.g. while you can simulate
+the installation of MultiArch packages they will more sooner than later
+cause enormous problems if really installed as dpkg can't handle MultiArch
+yet (no, --force-{overwrite,architecture} aren't good options here).
+Other parts of the big picture are missing and/or untested too.
+You have been warned!
+
+
+The implementation is focused on NOT breaking existing singleArch-only
+applications and/or systems as this is the current status-quo for all
+systems. Also, many systems don't need (or can't make use of) MultiArch,
+so APT will proceed in thinking SingleArch as long as it is not explicitly
+told to handle MultiArch:
+To activate MultiArch handling you need to specify architectures you
+want to be considered by APT with the config list APT::Architectures
+(Insert architectures in order of preference).
+APT will download Packages files for all these architectures in the
+update step. Exception: In the sourcelist is the optionfield used:
+deb [ arch=amd64,i386 ] http://example.org/ experimental main
+(This optionfield is a NOP in previous apt versions)
+
+Internally in APT a package is represented as a PkgIterator -
+before MultiArch this PkgIterator was architecture unaware,
+only VerIterators include the architecture they came from.
+This is/was a big problem as all versions in a package are
+considered for dependency resolution, so pinning will not work in all cases.
+
+The problem is solved by a conceptional change:
+A PkgIterator is now architecture aware, so the packages
+of foobar for amd64 and for i386 are now for apt internal totally
+different packages. That is a good thing for e.g. pinning, but
+sometimes you need the information that such packages are belonging together:
+All these foobar packages therefore form a Group accessible with GrpIterators.
+Note that the GrpIterator has the same name as all the packages in this group,
+so e.g. apt-cache pkgnames iterates over GrpIterator to get the package names:
+This is compatible to SingleArch as a Group consists only of a single package
+and also to MultiArch as a Group consists of possible many packages which
+all have the same name and are therefore out of interest for pkgnames.
+
+
+Caused by the paragraph "Dependencies involving Architecture: all packages"
+in the MultiArch spec we have a second major conceptional change
+which could even break existing applications, but we hope for the best…
+An Architecture: all package is internally split into pseudo packages
+for all MultiArch Architectures and additional a package with the
+architecture "all" with no dependencies which is a dependency of all
+these architecture depending packages. While the architecture depending
+packages are mainly used for dependency resolution (a package of arch A which
+depends on an arch all package assumes that the dependencies of this package
+are also from arch A. Packages also sometimes change from any to all or v.v.)
+the arch "all" package is used for scheduling download/installation of the
+underlying "real" package. Note that the architecture depending packages can
+be detected with Pseudo() while the "all" package reports exactly this arch
+as package architecture and as pseudo architecture of the versions of this pkg.
+Beware: All versions of a "real" architecture all package will be report "all"
+as their architecture if asked with Arch() regardless if they are the "all" or
+the architecture depending packages. If you want to know the architecture this
+pseudo package was created for call Arch(true). Also, while the spec say that
+arch:all packages are not allowed to have a MultiArch flag APT assigns a
+special value to them: MultiArch: all.
+
+
+As you might guess this arch:all handling has a few problems (but we think so
+far that the problems are minor compared to the problems we would have with
+other implementations.)
+APT doesn't know which pseudo packages of such an arch all package are
+"installed" (to satisfy dependencies), so APT will generate a Cache in which
+all these pseudo packages are installed (e.g. apt-cache policy will display
+them all as installed). Later in the DepCache step it will "remove"
+all pseudo packages whose dependencies are not satisfied.
+The expense is that if the package state is broken APT could come to the
+conclusion to "remove" too many pseudo packages, but in a stable environment
+APT should never end up in a broken system state…
+
+
+Given all these internal changes it is quite interesting that the actual
+implementation of MultiArch is trivial: Some implicit dependencies and a few
+more provides are all changes needed to get it working. Especially noteworthy
+is that it wasn't needed to change the resolver in any way and other parts only
+need to be told about ignoring pseudo packages or using GrpIterator instead of
+PkgIterator, so chances are good that libapt-applications will proceed to work
+without or at least only require minor changes, but your mileage may vary…
+
+
+Known Issues and/or noteworthy stuff:
+* The implementation is mostly untested, so it is very likely that APT will
+  eat your kids if you aren't as lucky as the author of these patches.
+* the (install)size of a pseudo package is always NULL - if you want to know
+  the (install)size you need to get the info from the arch "all" package.
+* It is maybe confusing, but the arch "all" package does have the same versions
+  and in general roughly the same information with one subtil difference:
+  It doesn't have any dependency, regardless of the type. The pseudo packages
+  depend on this package.
+* apt-cache policy foobar on installed architecture all package foobar will
+  report all architecture depending packages as installed. Displaying here the
+  correct information would require to build the complete DepCache…
+* [BUG] An installed package which changes the architecture from any to all
+  (and v.v.) shows up in the NEW packages section instead of UPGRADE.
+* [TODO] Investigate the DepCache pseudo-package killer heuristic:
+  e.g. add more safety guards…
+* [FIXME] a few corner cases/missing features marked as FIXME in the code
+
+
+[0] https://wiki.ubuntu.com/MultiarchSpec

README.arch

@@ -1,12 +0,0 @@
-
-You can build apt from arch, but this needs the following additional
-packages (in addtion to the usual build-depends):
-autoconf automake xmlto perlsgml sgml2x sgmlspl docbook doxygen
-
-then run:
-
-$ debian/rules arch-build
-
-that will build packages in the "debian/arch-build" directory. It will
-honor "DEB_BUILD_PROG_OPTS" as options that are passed to debuild (e.g. -S 
-may be usefull).

apt-inst/contrib/arfile.h

@@ -39,6 +39,7 @@ class ARArchive
 
    // Locate a member by name
    const Member *FindMember(const char *Name) const;
+   inline Member *Members() { return List; }
    
    ARArchive(FileFd &File);
    ~ARArchive();

apt-inst/deb/dpkgdb.cc

@@ -142,7 +142,7 @@ bool debDpkgDB::ReadyPkgCache(OpProgress &Progress)
       CacheMap = 0;
    }
    
-   if (pkgMakeOnlyStatusCache(Progress,&CacheMap) == false)
+   if (pkgCacheGenerator::MakeOnlyStatusCache(&Progress,&CacheMap) == false)
       return false;
    Cache->DropProgress();
    

apt-inst/makefile

@@ -14,7 +14,7 @@ include ../buildlib/libversion.mak
 
 # The library name
 LIBRARY=apt-inst
-MAJOR=1.1
+MAJOR=1.2
 MINOR=0
 SLIBS=$(PTHREADLIB) -lapt-pkg
 APT_DOMAIN:=libapt-inst$(MAJOR)

apt-pkg/acquire-item.cc

@@ -33,6 +33,7 @@
 #include <string>
 #include <sstream>
 #include <stdio.h>
+#include <ctime>
 									/*}}}*/
 
 using namespace std;
@@ -64,6 +65,7 @@ void pkgAcquire::Item::Failed(string Message,pkgAcquire::MethodConfig *Cnf)
 {
    Status = StatIdle;
    ErrorText = LookupTag(Message,"Message");
+   UsedMirror =  LookupTag(Message,"UsedMirror");
    if (QueueCounter <= 1)
    {
       /* This indicates that the file is not available right now but might
@@ -76,10 +78,17 @@ void pkgAcquire::Item::Failed(string Message,pkgAcquire::MethodConfig *Cnf)
 	 Dequeue();
 	 return;
       }
-      
+
       Status = StatError;
       Dequeue();
    }   
+
+   // report mirror failure back to LP if we actually use a mirror
+   string FailReason = LookupTag(Message, "FailReason");
+   if(FailReason.size() != 0)
+      ReportMirrorFailure(FailReason);
+   else
+      ReportMirrorFailure(ErrorText);
 }
 									/*}}}*/
 // Acquire::Item::Start - Item has begun to download			/*{{{*/
@@ -101,7 +110,7 @@ void pkgAcquire::Item::Done(string Message,unsigned long Size,string Hash,
 {
    // We just downloaded something..
    string FileName = LookupTag(Message,"Filename");
-   // we only inform the Log class if it was actually not a local thing
+   UsedMirror =  LookupTag(Message,"UsedMirror");
    if (Complete == false && !Local && FileName == DestFile)
    {
       if (Owner->Log != 0)
@@ -110,7 +119,6 @@ void pkgAcquire::Item::Done(string Message,unsigned long Size,string Hash,
 
    if (FileSize == 0)
       FileSize= Size;
-   
    Status = StatDone;
    ErrorText = string();
    Owner->Dequeue(this);
@@ -132,6 +140,50 @@ void pkgAcquire::Item::Rename(string From,string To)
    }   
 }
 									/*}}}*/
+// Acquire::Item::ReportMirrorFailure					/*{{{*/
+// ---------------------------------------------------------------------
+void pkgAcquire::Item::ReportMirrorFailure(string FailCode)
+{
+   // we only act if a mirror was used at all
+   if(UsedMirror.empty())
+      return;
+#if 0
+   std::cerr << "\nReportMirrorFailure: " 
+	     << UsedMirror
+	     << " Uri: " << DescURI()
+	     << " FailCode: " 
+	     << FailCode << std::endl;
+#endif
+   const char *Args[40];
+   unsigned int i = 0;
+   string report = _config->Find("Methods::Mirror::ProblemReporting", 
+				 "/usr/lib/apt/apt-report-mirror-failure");
+   if(!FileExists(report))
+      return;
+   Args[i++] = report.c_str();
+   Args[i++] = UsedMirror.c_str();
+   Args[i++] = DescURI().c_str();
+   Args[i++] = FailCode.c_str();
+   Args[i++] = NULL;
+   pid_t pid = ExecFork();
+   if(pid < 0) 
+   {
+      _error->Error("ReportMirrorFailure Fork failed");
+      return;
+   }
+   else if(pid == 0) 
+   {
+      execvp(Args[0], (char**)Args);
+      std::cerr << "Could not exec " << Args[0] << std::endl;
+      _exit(100);
+   }
+   if(!ExecWait(pid, "report-mirror-failure")) 
+   {
+      _error->Warning("Couldn't report problem to '%s'",
+		      _config->Find("Methods::Mirror::ProblemReporting").c_str());
+   }
+}
+									/*}}}*/
 // AcqDiffIndex::AcqDiffIndex - Constructor				/*{{{*/
 // ---------------------------------------------------------------------
 /* Get the DiffIndex file first and see if there are patches availabe 
@@ -245,7 +297,7 @@ bool pkgAcqDiffIndex::ParseDiffIndex(string IndexDiffFile)		/*{{{*/
       else 
       {
 	 if(Debug)
-	    std::clog << "SHA1-Current: " << ServerSha1 << std::endl;
+	    std::clog << "SHA1-Current: " << ServerSha1 << " and we start at "<< fd.Name() << " " << fd.Size() << " " << local_sha1 << std::endl;
 
 	 // check the historie and see what patches we need
 	 string const history = Tags.FindS("SHA1-History");
@@ -310,7 +362,7 @@ bool pkgAcqDiffIndex::ParseDiffIndex(string IndexDiffFile)		/*{{{*/
 	 if(last_space != string::npos)
 	    Description.erase(last_space, Description.size()-last_space);
 	 new pkgAcqIndexDiffs(Owner, RealURI, Description, Desc.ShortDesc,
-			      ExpectedHash, available_patches);
+			      ExpectedHash, ServerSha1, available_patches);
 	 Complete = false;
 	 Status = StatDone;
 	 Dequeue();
@@ -378,9 +430,10 @@ void pkgAcqDiffIndex::Done(string Message,unsigned long Size,string Md5Hash,	/*{
 pkgAcqIndexDiffs::pkgAcqIndexDiffs(pkgAcquire *Owner,
 				   string URI,string URIDesc,string ShortDesc,
 				   HashString ExpectedHash, 
+				   string ServerSha1,
 				   vector<DiffInfo> diffs)
    : Item(Owner), RealURI(URI), ExpectedHash(ExpectedHash), 
-     available_patches(diffs)
+     available_patches(diffs), ServerSha1(ServerSha1)
 {
    
    DestFile = _config->FindDir("Dir::State::lists") + "partial/";
@@ -466,6 +519,13 @@ bool pkgAcqIndexDiffs::QueueNextDiff()					/*{{{*/
       std::clog << "QueueNextDiff: " 
 		<< FinalFile << " (" << local_sha1 << ")"<<std::endl;
 
+   // final file reached before all patches are applied
+   if(local_sha1 == ServerSha1)
+   {
+      Finish(true);
+      return true;
+   }
+
    // remove all patches until the next matching patch is found
    // this requires the Index file to be ordered
    for(vector<DiffInfo>::iterator I=available_patches.begin();
@@ -563,7 +623,7 @@ void pkgAcqIndexDiffs::Done(string Message,unsigned long Size,string Md5Hash,	/*
       // see if there is more to download
       if(available_patches.size() > 0) {
 	 new pkgAcqIndexDiffs(Owner, RealURI, Description, Desc.ShortDesc,
-			      ExpectedHash, available_patches);
+			      ExpectedHash, ServerSha1, available_patches);
 	 return Finish();
       } else 
 	 return Finish(true);
@@ -612,11 +672,12 @@ string pkgAcqIndex::Custom600Headers()
 {
    string Final = _config->FindDir("Dir::State::lists");
    Final += URItoFileName(RealURI);
+   if (_config->FindB("Acquire::GzipIndexes",false))
+      Final += ".gz";
    
    struct stat Buf;
    if (stat(Final.c_str(),&Buf) != 0)
       return "\nIndex-File: true";
-   
    return "\nIndex-File: true\nLast-Modified: " + TimeRFC1123(Buf.st_mtime);
 }
 									/*}}}*/
@@ -684,6 +745,7 @@ void pkgAcqIndex::Done(string Message,unsigned long Size,string Hash,
          Status = StatAuthError;
          ErrorText = _("Hash Sum mismatch");
          Rename(DestFile,DestFile + ".FAILED");
+	 ReportMirrorFailure("HashChecksumFailure");
          return;
       }
       // Done, move it into position
@@ -729,18 +791,36 @@ void pkgAcqIndex::Done(string Message,unsigned long Size,string Hash,
       ErrorText = "Method gave a blank filename";
    }
    
+   string compExt = flExtension(flNotDir(URI(Desc.URI).Path));
+
    // The files timestamp matches
-   if (StringToBool(LookupTag(Message,"IMS-Hit"),false) == true)
+   if (StringToBool(LookupTag(Message,"IMS-Hit"),false) == true) {
+       if (_config->FindB("Acquire::GzipIndexes",false) && compExt == "gz")
+	  // Update DestFile for .gz suffix so that the clean operation keeps it
+	  DestFile += ".gz";
       return;
+    }
 
    if (FileName == DestFile)
       Erase = true;
    else
       Local = true;
    
-   string compExt = flExtension(flNotDir(URI(Desc.URI).Path));
    string decompProg;
 
+   // If we enable compressed indexes and already have gzip, keep it
+   if (_config->FindB("Acquire::GzipIndexes",false) && compExt == "gz" && !Local) {
+      string FinalFile = _config->FindDir("Dir::State::lists");
+      FinalFile += URItoFileName(RealURI) + ".gz";
+      Rename(DestFile,FinalFile);
+      chmod(FinalFile.c_str(),0644);
+      
+      // Update DestFile for .gz suffix so that the clean operation keeps it
+      DestFile = _config->FindDir("Dir::State::lists") + "partial/";
+      DestFile += URItoFileName(RealURI) + ".gz";
+      return;
+    }
+
    // get the binary name for your used compression type
    decompProg = _config->Find(string("Acquire::CompressionTypes::").append(compExt),"");
    if(decompProg.empty() == false);
@@ -770,6 +850,19 @@ pkgAcqIndexTrans::pkgAcqIndexTrans(pkgAcquire *Owner,
 			    string URI,string URIDesc,string ShortDesc) 
   : pkgAcqIndex(Owner, URI, URIDesc, ShortDesc, HashString(), "")
 {
+}
+									/*}}}*/
+// AcqIndexTrans::Custom600Headers - Insert custom request headers	/*{{{*/
+// ---------------------------------------------------------------------
+string pkgAcqIndexTrans::Custom600Headers()
+{
+   string Final = _config->FindDir("Dir::State::lists");
+   Final += URItoFileName(RealURI);
+
+   struct stat Buf;
+   if (stat(Final.c_str(),&Buf) != 0)
+      return "\nFail-Ignore: true";
+   return "\nFail-Ignore: true\nLast-Modified: " + TimeRFC1123(Buf.st_mtime);
 }
 									/*}}}*/
 // AcqIndexTrans::Failed - Silence failure messages for missing files	/*{{{*/
@@ -874,8 +967,9 @@ void pkgAcqMetaSig::Done(string Message,unsigned long Size,string MD5,
       Rename(LastGoodSig, DestFile);
 
    // queue a pkgAcqMetaIndex to be verified against the sig we just retrieved
-   new pkgAcqMetaIndex(Owner, MetaIndexURI, MetaIndexURIDesc, MetaIndexShortDesc,
-		       DestFile, IndexTargets, MetaIndexParser);
+   new pkgAcqMetaIndex(Owner, MetaIndexURI, MetaIndexURIDesc, 
+		       MetaIndexShortDesc,  DestFile, IndexTargets, 
+		       MetaIndexParser);
 
 }
 									/*}}}*/
@@ -888,7 +982,7 @@ void pkgAcqMetaSig::Failed(string Message,pkgAcquire::MethodConfig *Cnf)/*{{{*/
    {
       Item::Failed(Message,Cnf);
       // move the sigfile back on transient network failures 
-      if(FileExists(DestFile))
+      if(FileExists(LastGoodSig))
  	 Rename(LastGoodSig,Final);
 
       // set the status back to , Item::Failed likes to reset it
@@ -963,6 +1057,15 @@ void pkgAcqMetaIndex::Done(string Message,unsigned long Size,string Hash,	/*{{{*
    if (AuthPass == true)
    {
       AuthDone(Message);
+
+      // all cool, move Release file into place
+      Complete = true;
+
+      string FinalFile = _config->FindDir("Dir::State::lists");
+      FinalFile += URItoFileName(RealURI);
+      Rename(DestFile,FinalFile);
+      chmod(FinalFile.c_str(),0644);
+      DestFile = FinalFile;
    }
    else
    {
@@ -1014,22 +1117,15 @@ void pkgAcqMetaIndex::RetrievalDone(string Message)			/*{{{*/
       return;
    }
 
-   // see if the download was a IMSHit
+   // make sure to verify against the right file on I-M-S hit
    IMSHit = StringToBool(LookupTag(Message,"IMS-Hit"),false);
+   if(IMSHit)
+   {
+      string FinalFile = _config->FindDir("Dir::State::lists");
+      FinalFile += URItoFileName(RealURI);
+      DestFile = FinalFile;
+   }
    Complete = true;
-
-   string FinalFile = _config->FindDir("Dir::State::lists");
-   FinalFile += URItoFileName(RealURI);
-
-   // If we get a IMS hit we can remove the empty file in partial
-   // othersie we move the file in place
-   if (IMSHit)
-      unlink(DestFile.c_str());
-   else
-      Rename(DestFile,FinalFile);
-
-   chmod(FinalFile.c_str(),0644);
-   DestFile = FinalFile;
 }
 									/*}}}*/
 void pkgAcqMetaIndex::AuthDone(string Message)				/*{{{*/
@@ -1059,7 +1155,6 @@ void pkgAcqMetaIndex::AuthDone(string Message)				/*{{{*/
    QueueIndexes(true);
 
    // Done, move signature file into position
-
    string VerifiedSigFile = _config->FindDir("Dir::State::lists") +
       URItoFileName(RealURI) + ".gpg";
    Rename(SigFile,VerifiedSigFile);
@@ -1097,13 +1192,16 @@ void pkgAcqMetaIndex::QueueIndexes(bool verify)				/*{{{*/
             return;
          }
       }
-      
-      // Queue Packages file (either diff or full packages files, depending
-      // on the users option)
-      if(_config->FindB("Acquire::PDiffs",true) == true) 
+
+      /* Queue Packages file (either diff or full packages files, depending
+         on the users option) - we also check if the PDiff Index file is listed
+         in the Meta-Index file. Ideal would be if pkgAcqDiffIndex would test this
+         instead, but passing the required info to it is to much hassle */
+      if(_config->FindB("Acquire::PDiffs",true) == true && (verify == false ||
+	  MetaIndexParser->Exists(string((*Target)->MetaKey).append(".diff/Index")) == true))
 	 new pkgAcqDiffIndex(Owner, (*Target)->URI, (*Target)->Description,
 			     (*Target)->ShortDesc, ExpectedIndexHash);
-      else 
+      else
 	 new pkgAcqIndex(Owner, (*Target)->URI, (*Target)->Description,
 			    (*Target)->ShortDesc, ExpectedIndexHash);
    }
@@ -1169,6 +1267,17 @@ bool pkgAcqMetaIndex::VerifyVendor(string Message)			/*{{{*/
       Transformed = "";
    }
 
+   if (_config->FindB("Acquire::Check-Valid-Until", true) == true &&
+       MetaIndexParser->GetValidUntil() > 0) {
+      time_t const invalid_since = time(NULL) - MetaIndexParser->GetValidUntil();
+      if (invalid_since > 0)
+	 // TRANSLATOR: The first %s is the URL of the bad Release file, the second is
+	 // the time since then the file is invalid - formated in the same way as in
+	 // the download progress display (e.g. 7d 3h 42min 1s)
+	 return _error->Error(_("Release file expired, ignoring %s (invalid since %s)"),
+	                      RealURI.c_str(), TimeToStr(invalid_since).c_str());
+   }
+
    if (_config->FindB("Debug::pkgAcquire::Auth", false)) 
    {
       std::cerr << "Got Codename: " << MetaIndexParser->GetDist() << std::endl;
@@ -1186,7 +1295,7 @@ bool pkgAcqMetaIndex::VerifyVendor(string Message)			/*{{{*/
 //       return false;
       if (!Transformed.empty())
       {
-         _error->Warning("Conflicting distribution: %s (expected %s but got %s)",
+         _error->Warning(_("Conflicting distribution: %s (expected %s but got %s)"),
                          Desc.Description.c_str(),
                          Transformed.c_str(),
                          MetaIndexParser->GetDist().c_str());
@@ -1203,30 +1312,30 @@ void pkgAcqMetaIndex::Failed(string Message,pkgAcquire::MethodConfig *Cnf)
 {
    if (AuthPass == true)
    {
-      // if we fail the authentication but got the file via a IMS-Hit 
-      // this means that the file wasn't downloaded and that it might be
-      // just stale (server problem, proxy etc). we delete what we have
-      // queue it again without i-m-s 
-      // alternatively we could just unlink the file and let the user try again
-      if (IMSHit)
+      // gpgv method failed, if we have a good signature 
+      string LastGoodSigFile = _config->FindDir("Dir::State::lists") +
+	 "partial/" + URItoFileName(RealURI) + ".gpg.reverify";
+      if(FileExists(LastGoodSigFile))
       {
-	 Complete = false;
-	 Local = false;
-	 AuthPass = false;
-	 unlink(DestFile.c_str());
-
-	 DestFile = _config->FindDir("Dir::State::lists") + "partial/";
-	 DestFile += URItoFileName(RealURI);
-	 Desc.URI = RealURI;
-	 QueueURI(Desc);
+	 string VerifiedSigFile = _config->FindDir("Dir::State::lists") +
+	    URItoFileName(RealURI) + ".gpg";
+	 Rename(LastGoodSigFile,VerifiedSigFile);
+	 Status = StatTransientNetworkError;
+	 _error->Warning(_("A error occurred during the signature "
+			   "verification. The repository is not updated "
+			   "and the previous index files will be used."
+			   "GPG error: %s: %s\n"),
+			 Desc.Description.c_str(),
+			 LookupTag(Message,"Message").c_str());
+	 RunScripts("APT::Update::Auth-Failure");
 	 return;
+      } else {
+	 _error->Warning(_("GPG error: %s: %s"),
+			 Desc.Description.c_str(),
+			 LookupTag(Message,"Message").c_str());
       }
-
       // gpgv method failed 
-      _error->Warning("GPG error: %s: %s",
-                      Desc.Description.c_str(),
-                      LookupTag(Message,"Message").c_str());
-
+      ReportMirrorFailure("GPGFailure");
    }
 
    // No Release file was present, or verification failed, so fall
@@ -1319,7 +1428,8 @@ pkgAcqArchive::pkgAcqArchive(pkgAcquire *Owner,pkgSourceList *Sources,
    the archive is already available in the cache and stashs the MD5 for
    checking later. */
 bool pkgAcqArchive::QueueNext()
-{   
+{
+   string const ForceHash = _config->Find("Acquire::ForceHash");
    for (; Vf.end() == false; Vf++)
    {
       // Ignore not source sources
@@ -1342,12 +1452,25 @@ bool pkgAcqArchive::QueueNext()
 	 return false;
       
       string PkgFile = Parse.FileName();
-      if(Parse.SHA256Hash() != "")
-	 ExpectedHash = HashString("SHA256", Parse.SHA256Hash());
-      else if (Parse.SHA1Hash() != "")
-	 ExpectedHash = HashString("SHA1", Parse.SHA1Hash());
-      else 
-	 ExpectedHash = HashString("MD5Sum", Parse.MD5Hash());
+      if (ForceHash.empty() == false)
+      {
+	 if(stringcasecmp(ForceHash, "sha256") == 0)
+	    ExpectedHash = HashString("SHA256", Parse.SHA256Hash());
+	 else if (stringcasecmp(ForceHash, "sha1") == 0)
+	    ExpectedHash = HashString("SHA1", Parse.SHA1Hash());
+	 else
+	    ExpectedHash = HashString("MD5Sum", Parse.MD5Hash());
+      }
+      else
+      {
+	 string Hash;
+	 if ((Hash = Parse.SHA256Hash()).empty() == false)
+	    ExpectedHash = HashString("SHA256", Hash);
+	 else if ((Hash = Parse.SHA1Hash()).empty() == false)
+	    ExpectedHash = HashString("SHA1", Hash);
+	 else
+	    ExpectedHash = HashString("MD5Sum", Parse.MD5Hash());
+      }
       if (PkgFile.empty() == true)
 	 return _error->Error(_("The package index files are corrupted. No Filename: "
 			      "field for package %s."),
@@ -1587,7 +1710,7 @@ void pkgAcqFile::Done(string Message,unsigned long Size,string CalcHash,
    if(!ExpectedHash.empty() && ExpectedHash.toStr() != CalcHash)
    {
       Status = StatError;
-      ErrorText = "Hash Sum mismatch";
+      ErrorText = _("Hash Sum mismatch");
       Rename(DestFile,DestFile + ".FAILED");
       return;
    }
@@ -1663,5 +1786,6 @@ string pkgAcqFile::Custom600Headers()
 {
    if (IsIndexFile)
       return "\nIndex-File: true";
+   return "";
 }
 									/*}}}*/

apt-pkg/acquire-item.h

@@ -27,6 +27,7 @@
 #include <apt-pkg/pkgrecords.h>
 #include <apt-pkg/indexrecords.h>
 #include <apt-pkg/hashes.h>
+#include <apt-pkg/weakptr.h>
 
 /** \addtogroup acquire
  *  @{
@@ -46,7 +47,7 @@
  *
  *  \see pkgAcquire
  */
-class pkgAcquire::Item
+class pkgAcquire::Item : public WeakPointable
 {  
    protected:
    
@@ -111,10 +112,10 @@ class pkgAcquire::Item
    string ErrorText;
 
    /** \brief The size of the object to fetch. */
-   unsigned long FileSize;
+   unsigned long long FileSize;
 
    /** \brief How much of the object was already fetched. */
-   unsigned long PartialSize;
+   unsigned long long PartialSize;
 
    /** \brief If not \b NULL, contains the name of a subprocess that
     *  is operating on this object (for instance, "gzip" or "gpgv").
@@ -142,6 +143,7 @@ class pkgAcquire::Item
     *  download progress indicator's overall statistics.
     */
    bool Local;
+   string UsedMirror;
 
    /** \brief The number of fetch queues into which this item has been
     *  inserted.
@@ -242,6 +244,17 @@ class pkgAcquire::Item
 
    /** \return \b true if this object is being fetched from a trusted source. */
    virtual bool IsTrusted() {return false;};
+   
+   // report mirror problems
+   /** \brief Report mirror problem
+    * 
+    *  This allows reporting mirror failures back to a centralized
+    *  server. The apt-report-mirror-failure script is called for this
+    * 
+    *  \param FailCode A short failure string that is send
+    */
+   void ReportMirrorFailure(string FailCode);
+
 
    /** \brief Initialize an item.
     *
@@ -422,6 +435,10 @@ class pkgAcqIndexDiffs : public pkgAcquire::Item
     *  off the front?
     */
    vector<DiffInfo> available_patches;
+
+   /** Stop applying patches when reaching that sha1 */
+   string ServerSha1;
+
    /** The current status of this patch. */
    enum DiffState
      {
@@ -475,6 +492,7 @@ class pkgAcqIndexDiffs : public pkgAcquire::Item
     */
    pkgAcqIndexDiffs(pkgAcquire *Owner,string URI,string URIDesc,
 		    string ShortDesc, HashString ExpectedHash,
+		    string ServerSha1,
 		    vector<DiffInfo> diffs=vector<DiffInfo>());
 };
 									/*}}}*/
@@ -545,7 +563,8 @@ class pkgAcqIndex : public pkgAcquire::Item
     *  fallback is ".gz" or none.
     */
    pkgAcqIndex(pkgAcquire *Owner,string URI,string URIDesc,
-	       string ShortDesc, HashString ExpectedHash, string compressExt="");
+	       string ShortDesc, HashString ExpectedHash, 
+	       string compressExt="");
 };
 									/*}}}*/
 /** \brief An acquire item that is responsible for fetching a		{{{
@@ -560,6 +579,7 @@ class pkgAcqIndexTrans : public pkgAcqIndex
    public:
   
    virtual void Failed(string Message,pkgAcquire::MethodConfig *Cnf);
+   virtual string Custom600Headers();
 
    /** \brief Create a pkgAcqIndexTrans.
     *
@@ -608,7 +628,6 @@ class pkgAcqMetaSig : public pkgAcquire::Item
    /** \brief The last good signature file */
    string LastGoodSig;
 
-
    /** \brief The fetch request that is currently being processed. */
    pkgAcquire::ItemDesc Desc;
 

apt-pkg/acquire-method.cc

@@ -96,12 +96,11 @@ void pkgAcqMethod::Fail(string Err,bool Transient)
    }
    
    char S[1024];
+   char *End = S;
    if (Queue != 0)
    {
-      snprintf(S,sizeof(S)-50,"400 URI Failure\nURI: %s\n"
-	       "Message: %s %s\n",Queue->Uri.c_str(),Err.c_str(),
-	       FailExtra.c_str());
-
+      End += snprintf(S,sizeof(S)-50,"400 URI Failure\nURI: %s\n"
+		      "Message: %s %s\n",Queue->Uri.c_str(), Err.c_str(), IP.c_str());
       // Dequeue
       FetchItem *Tmp = Queue;
       Queue = Queue->Next;
@@ -110,10 +109,14 @@ void pkgAcqMethod::Fail(string Err,bool Transient)
 	 QueueBack = Queue;
    }
    else
-      snprintf(S,sizeof(S)-50,"400 URI Failure\nURI: <UNKNOWN>\n"
-	       "Message: %s %s\n",Err.c_str(),
-	       FailExtra.c_str());
-      
+   {
+      End += snprintf(S,sizeof(S)-50,"400 URI Failure\nURI: <UNKNOWN>\n"
+		      "Message: %s\n",Err.c_str());
+   }
+   if(FailReason.empty() == false)
+      End += snprintf(End,sizeof(S)-50 - (End - S),"FailReason: %s\n",FailReason.c_str());
+   if (UsedMirror.empty() == false)
+      End += snprintf(End,sizeof(S)-50 - (End - S),"UsedMirror: %s\n",UsedMirror.c_str());
    // Set the transient flag 
    if (Transient == true)
       strcat(S,"Transient-Failure: true\n\n");
@@ -184,6 +187,8 @@ void pkgAcqMethod::URIDone(FetchResult &Res, FetchResult *Alt)
       End += snprintf(End,sizeof(S)-50 - (End - S),"SHA1-Hash: %s\n",Res.SHA1Sum.c_str());
    if (Res.SHA256Sum.empty() == false)
       End += snprintf(End,sizeof(S)-50 - (End - S),"SHA256-Hash: %s\n",Res.SHA256Sum.c_str());
+   if (UsedMirror.empty() == false)
+      End += snprintf(End,sizeof(S)-50 - (End - S),"UsedMirror: %s\n",UsedMirror.c_str());
    if (Res.GPGVOutput.size() > 0)
       End += snprintf(End,sizeof(S)-50 - (End - S),"GPGVOutput:\n");     
    for (vector<string>::iterator I = Res.GPGVOutput.begin();
@@ -373,9 +378,10 @@ int pkgAcqMethod::Run(bool Single)
 	    
 	    Tmp->Uri = LookupTag(Message,"URI");
 	    Tmp->DestFile = LookupTag(Message,"FileName");
-	    if (StrToTime(LookupTag(Message,"Last-Modified"),Tmp->LastModified) == false)
+	    if (RFC1123StrToTime(LookupTag(Message,"Last-Modified").c_str(),Tmp->LastModified) == false)
 	       Tmp->LastModified = 0;
 	    Tmp->IndexFile = StringToBool(LookupTag(Message,"Index-File"),false);
+	    Tmp->FailIgnore = StringToBool(LookupTag(Message,"Fail-Ignore"),false);
 	    Tmp->Next = 0;
 	    
 	    // Append it to the list

apt-pkg/acquire-method.h

@@ -37,6 +37,7 @@ class pkgAcqMethod
       string DestFile;
       time_t LastModified;
       bool IndexFile;
+      bool FailIgnore;
    };
    
    struct FetchResult
@@ -59,7 +60,9 @@ class pkgAcqMethod
    vector<string> Messages;
    FetchItem *Queue;
    FetchItem *QueueBack;
-   string FailExtra;
+   string FailReason;
+   string UsedMirror;
+   string IP;
    
    // Handlers for messages
    virtual bool Configuration(string Message);
@@ -68,14 +71,14 @@ class pkgAcqMethod
    // Outgoing messages
    void Fail(bool Transient = false);
    inline void Fail(const char *Why, bool Transient = false) {Fail(string(Why),Transient);};
-   void Fail(string Why, bool Transient = false);
-   void URIStart(FetchResult &Res);
-   void URIDone(FetchResult &Res,FetchResult *Alt = 0);
+   virtual void Fail(string Why, bool Transient = false);
+   virtual void URIStart(FetchResult &Res);
+   virtual void URIDone(FetchResult &Res,FetchResult *Alt = 0);
+
    bool MediaFail(string Required,string Drive);
    virtual void Exit() {};
 
    public:
-
    enum CnfFlags {SingleInstance = (1<<0),
                   Pipeline = (1<<1), SendConfig = (1<<2),
                   LocalOnly = (1<<3), NeedsCleanup = (1<<4), 
@@ -87,7 +90,8 @@ class pkgAcqMethod
    void Redirect(const string &NewURI);
  
    int Run(bool Single = false);
-   inline void SetFailExtraMsg(string Msg) {FailExtra = Msg;};
+   inline void SetFailReason(string Msg) {FailReason = Msg;};
+   inline void SetIP(string aIP) {IP = aIP;};
    
    pkgAcqMethod(const char *Ver,unsigned long Flags = 0);
    virtual ~pkgAcqMethod() {};

apt-pkg/acquire-worker.h

@@ -20,6 +20,7 @@
 #define PKGLIB_ACQUIRE_WORKER_H
 
 #include <apt-pkg/acquire.h>
+#include <apt-pkg/weakptr.h>
 
 
 /** \brief A fetch subprocess.
@@ -41,7 +42,7 @@
  *
  *  \sa pkgAcqMethod, pkgAcquire::Item, pkgAcquire
  */
-class pkgAcquire::Worker
+class pkgAcquire::Worker : public WeakPointable
 {
    friend class pkgAcquire;
    

apt-pkg/acquire.cc

@@ -19,6 +19,7 @@
 #include <apt-pkg/configuration.h>
 #include <apt-pkg/error.h>
 #include <apt-pkg/strutl.h>
+#include <apt-pkg/fileutl.h>
 
 #include <apti18n.h>
 
@@ -29,7 +30,6 @@
 #include <dirent.h>
 #include <sys/time.h>
 #include <errno.h>
-#include <sys/stat.h>
 									/*}}}*/
 
 using namespace std;
@@ -37,32 +37,60 @@ using namespace std;
 // Acquire::pkgAcquire - Constructor					/*{{{*/
 // ---------------------------------------------------------------------
 /* We grab some runtime state from the configuration space */
-pkgAcquire::pkgAcquire(pkgAcquireStatus *Log) : Log(Log)
+pkgAcquire::pkgAcquire() : Queues(0), Workers(0), Configs(0), Log(NULL), ToFetch(0),
+			   Debug(_config->FindB("Debug::pkgAcquire",false)),
+			   Running(false), LockFD(-1)
 {
-   Queues = 0;
-   Configs = 0;
-   Workers = 0;
-   ToFetch = 0;
-   Running = false;
-   
-   string Mode = _config->Find("Acquire::Queue-Mode","host");
+   string const Mode = _config->Find("Acquire::Queue-Mode","host");
+   if (strcasecmp(Mode.c_str(),"host") == 0)
+      QueueMode = QueueHost;
+   if (strcasecmp(Mode.c_str(),"access") == 0)
+      QueueMode = QueueAccess;
+}
+pkgAcquire::pkgAcquire(pkgAcquireStatus *Progress) : Queues(0), Workers(0),
+			   Configs(0), Log(Progress), ToFetch(0),
+			   Debug(_config->FindB("Debug::pkgAcquire",false)),
+			   Running(false), LockFD(-1)
+{
+   string const Mode = _config->Find("Acquire::Queue-Mode","host");
    if (strcasecmp(Mode.c_str(),"host") == 0)
       QueueMode = QueueHost;
    if (strcasecmp(Mode.c_str(),"access") == 0)
-      QueueMode = QueueAccess;   
+      QueueMode = QueueAccess;
+   Setup(Progress, "");
+}
+									/*}}}*/
+// Acquire::Setup - Delayed Constructor					/*{{{*/
+// ---------------------------------------------------------------------
+/* Do everything needed to be a complete Acquire object and report the
+   success (or failure) back so the user knows that something is wrong… */
+bool pkgAcquire::Setup(pkgAcquireStatus *Progress, string const &Lock)
+{
+   Log = Progress;
 
-   Debug = _config->FindB("Debug::pkgAcquire",false);
-   
-   // This is really a stupid place for this
-   struct stat St;
-   if (stat((_config->FindDir("Dir::State::lists") + "partial/").c_str(),&St) != 0 ||
-       S_ISDIR(St.st_mode) == 0)
-      _error->Error(_("Lists directory %spartial is missing."),
-		    _config->FindDir("Dir::State::lists").c_str());
-   if (stat((_config->FindDir("Dir::Cache::Archives") + "partial/").c_str(),&St) != 0 ||
-       S_ISDIR(St.st_mode) == 0)
-      _error->Error(_("Archive directory %spartial is missing."),
-		    _config->FindDir("Dir::Cache::Archives").c_str());
+   // check for existence and possibly create auxiliary directories
+   string const listDir = _config->FindDir("Dir::State::lists");
+   string const partialListDir = listDir + "partial/";
+   string const archivesDir = _config->FindDir("Dir::Cache::Archives");
+   string const partialArchivesDir = archivesDir + "partial/";
+
+   if (CheckDirectory(_config->FindDir("Dir::State"), partialListDir) == false &&
+       CheckDirectory(listDir, partialListDir) == false)
+      return _error->Errno("Acquire", _("List directory %spartial is missing."), listDir.c_str());
+
+   if (CheckDirectory(_config->FindDir("Dir::Cache"), partialArchivesDir) == false &&
+       CheckDirectory(archivesDir, partialArchivesDir) == false)
+      return _error->Errno("Acquire", _("Archives directory %spartial is missing."), archivesDir.c_str());
+
+   if (Lock.empty() == true || _config->FindB("Debug::NoLocking", false) == true)
+      return true;
+
+   // Lock the directory this acquire object will work in
+   LockFD = GetLock(flCombine(Lock, "lock"));
+   if (LockFD == -1)
+      return _error->Error(_("Unable to lock directory %s"), Lock.c_str());
+
+   return true;
 }
 									/*}}}*/
 // Acquire::~pkgAcquire	- Destructor					/*{{{*/
@@ -71,7 +99,10 @@ pkgAcquire::pkgAcquire(pkgAcquireStatus *Log) : Log(Log)
 pkgAcquire::~pkgAcquire()
 {
    Shutdown();
-   
+
+   if (LockFD != -1)
+      close(LockFD);
+
    while (Configs != 0)
    {
       MethodConfig *Jnk = Configs;
@@ -454,9 +485,9 @@ bool pkgAcquire::Clean(string Dir)
 // Acquire::TotalNeeded - Number of bytes to fetch			/*{{{*/
 // ---------------------------------------------------------------------
 /* This is the total number of bytes needed */
-double pkgAcquire::TotalNeeded()
+unsigned long long pkgAcquire::TotalNeeded()
 {
-   double Total = 0;
+   unsigned long long Total = 0;
    for (ItemCIterator I = ItemsBegin(); I != ItemsEnd(); I++)
       Total += (*I)->FileSize;
    return Total;
@@ -465,9 +496,9 @@ double pkgAcquire::TotalNeeded()
 // Acquire::FetchNeeded - Number of bytes needed to get			/*{{{*/
 // ---------------------------------------------------------------------
 /* This is the number of bytes that is not local */
-double pkgAcquire::FetchNeeded()
+unsigned long long pkgAcquire::FetchNeeded()
 {
-   double Total = 0;
+   unsigned long long Total = 0;
    for (ItemCIterator I = ItemsBegin(); I != ItemsEnd(); I++)
       if ((*I)->Local == false)
 	 Total += (*I)->FileSize;
@@ -477,9 +508,9 @@ double pkgAcquire::FetchNeeded()
 // Acquire::PartialPresent - Number of partial bytes we already have	/*{{{*/
 // ---------------------------------------------------------------------
 /* This is the number of bytes that is not local */
-double pkgAcquire::PartialPresent()
+unsigned long long pkgAcquire::PartialPresent()
 {
-  double Total = 0;
+  unsigned long long Total = 0;
    for (ItemCIterator I = ItemsBegin(); I != ItemsEnd(); I++)
       if ((*I)->Local == false)
 	 Total += (*I)->PartialSize;

apt-pkg/acquire.h

@@ -66,6 +66,9 @@
 #ifndef PKGLIB_ACQUIRE_H
 #define PKGLIB_ACQUIRE_H
 
+#include <apt-pkg/macros.h>
+#include <apt-pkg/weakptr.h>
+
 #include <vector>
 #include <string>
 
@@ -161,7 +164,7 @@ class pkgAcquire
      QueueAccess} QueueMode;
 
    /** \brief If \b true, debugging information will be dumped to std::clog. */
-   bool Debug;
+   bool const Debug;
    /** \brief If \b true, a download is currently in progress. */
    bool Running;
 
@@ -320,27 +323,34 @@ class pkgAcquire
    /** \return the total size in bytes of all the items included in
     *  this download.
     */
-   double TotalNeeded();
+   unsigned long long TotalNeeded();
 
    /** \return the size in bytes of all non-local items included in
     *  this download.
     */
-   double FetchNeeded();
+   unsigned long long FetchNeeded();
 
    /** \return the amount of data to be fetched that is already
     *  present on the filesystem.
     */
-   double PartialPresent();
+   unsigned long long PartialPresent();
 
-   /** \brief Construct a new pkgAcquire.
+   /** \brief Delayed constructor
     *
-    *  \param Log The progress indicator associated with this
-    *  download, or \b NULL for none.  This object is not owned by the
+    *  \param Progress indicator associated with this download or
+    *  \b NULL for none.  This object is not owned by the
     *  download process and will not be deleted when the pkgAcquire
     *  object is destroyed.  Naturally, it should live for at least as
     *  long as the pkgAcquire object does.
+    *  \param Lock defines a lock file that should be acquired to ensure
+    *  only one Acquire class is in action at the time or an empty string
+    *  if no lock file should be used.
     */
-   pkgAcquire(pkgAcquireStatus *Log = 0);
+   bool Setup(pkgAcquireStatus *Progress = NULL, string const &Lock = "");
+
+   /** \brief Construct a new pkgAcquire. */
+   pkgAcquire(pkgAcquireStatus *Log) __deprecated;
+   pkgAcquire();
 
    /** \brief Destroy this pkgAcquire object.
     *
@@ -348,6 +358,10 @@ class pkgAcquire
     *  this download.
     */
    virtual ~pkgAcquire();
+
+   private:
+   /** \brief FD of the Lock file we acquire in Setup (if any) */
+   int LockFD;
 };
 
 /** \brief Represents a single download source from which an item
@@ -355,7 +369,7 @@ class pkgAcquire
  *
  *  An item may have several assocated ItemDescs over its lifetime.
  */
-struct pkgAcquire::ItemDesc
+struct pkgAcquire::ItemDesc : public WeakPointable
 {
    /** \brief The URI from which to download this item. */
    string URI;

apt-pkg/algorithms.cc

@@ -58,7 +58,7 @@ void pkgSimulate::Describe(PkgIterator Pkg,ostream &out,bool Current,bool Candid
 {
    VerIterator Ver(Sim);
  
-   out << Pkg.Name();
+   out << Pkg.FullName(true);
 
    if (Current == true)
    {
@@ -83,13 +83,28 @@ void pkgSimulate::Describe(PkgIterator Pkg,ostream &out,bool Current,bool Candid
 bool pkgSimulate::Install(PkgIterator iPkg,string /*File*/)
 {
    // Adapt the iterator
-   PkgIterator Pkg = Sim.FindPkg(iPkg.Name());
+   PkgIterator Pkg = Sim.FindPkg(iPkg.Name(), iPkg.Arch());
    Flags[Pkg->ID] = 1;
    
    cout << "Inst ";
    Describe(Pkg,cout,true,true);
    Sim.MarkInstall(Pkg,false);
-   
+
+   if (strcmp(Pkg.Arch(),"all") == 0)
+   {
+      pkgCache::GrpIterator G = Pkg.Group();
+      pkgCache::GrpIterator iG = iPkg.Group();
+      for (pkgCache::PkgIterator P = G.FindPkg("any"); P.end() != true; P = G.NextPkg(P))
+      {
+	 if (strcmp(P.Arch(), "all") == 0)
+	    continue;
+	 if (iG.FindPkg(P.Arch())->CurrentVer == 0)
+	    continue;
+	 Flags[P->ID] = 1;
+	 Sim.MarkInstall(P, false);
+      }
+   }
+
    // Look for broken conflicts+predepends.
    for (PkgIterator I = Sim.PkgBegin(); I.end() == false; I++)
    {
@@ -108,9 +123,9 @@ bool pkgSimulate::Install(PkgIterator iPkg,string /*File*/)
          {
 	    if ((Sim[End] & pkgDepCache::DepGInstall) == 0)
 	    {
-	       cout << " [" << I.Name() << " on " << Start.TargetPkg().Name() << ']';
+	       cout << " [" << I.FullName(false) << " on " << Start.TargetPkg().FullName(false) << ']';
 	       if (Start->Type == pkgCache::Dep::Conflicts)
-		  _error->Error("Fatal, conflicts violated %s",I.Name());
+		  _error->Error("Fatal, conflicts violated %s",I.FullName(false).c_str());
 	    }	    
 	 }
       }      
@@ -131,13 +146,26 @@ bool pkgSimulate::Install(PkgIterator iPkg,string /*File*/)
 bool pkgSimulate::Configure(PkgIterator iPkg)
 {
    // Adapt the iterator
-   PkgIterator Pkg = Sim.FindPkg(iPkg.Name());
+   PkgIterator Pkg = Sim.FindPkg(iPkg.Name(), iPkg.Arch());
    
    Flags[Pkg->ID] = 2;
+
+   if (strcmp(Pkg.Arch(),"all") == 0)
+   {
+      pkgCache::GrpIterator G = Pkg.Group();
+      for (pkgCache::PkgIterator P = G.FindPkg("any"); P.end() != true; P = G.NextPkg(P))
+      {
+	 if (strcmp(P.Arch(), "all") == 0)
+	    continue;
+	 if (Flags[P->ID] == 1)
+	    Flags[P->ID] = 2;
+      }
+   }
+
 //   Sim.MarkInstall(Pkg,false);
    if (Sim[Pkg].InstBroken() == true)
    {
-      cout << "Conf " << Pkg.Name() << " broken" << endl;
+      cout << "Conf " << Pkg.FullName(false) << " broken" << endl;
 
       Sim.Update();
       
@@ -149,17 +177,17 @@ bool pkgSimulate::Configure(PkgIterator iPkg)
 	    continue;
 	 
 	 if (D->Type == pkgCache::Dep::Obsoletes)
-	    cout << " Obsoletes:" << D.TargetPkg().Name();
+	    cout << " Obsoletes:" << D.TargetPkg().FullName(false);
 	 else if (D->Type == pkgCache::Dep::Conflicts)
-	    cout << " Conflicts:" << D.TargetPkg().Name();
+	    cout << " Conflicts:" << D.TargetPkg().FullName(false);
 	 else if (D->Type == pkgCache::Dep::DpkgBreaks)
-	    cout << " Breaks:" << D.TargetPkg().Name();
+	    cout << " Breaks:" << D.TargetPkg().FullName(false);
 	 else
-	    cout << " Depends:" << D.TargetPkg().Name();
+	    cout << " Depends:" << D.TargetPkg().FullName(false);
       }	    
       cout << endl;
 
-      _error->Error("Conf Broken %s",Pkg.Name());
+      _error->Error("Conf Broken %s",Pkg.FullName(false).c_str());
    }   
    else
    {
@@ -181,10 +209,26 @@ bool pkgSimulate::Configure(PkgIterator iPkg)
 bool pkgSimulate::Remove(PkgIterator iPkg,bool Purge)
 {
    // Adapt the iterator
-   PkgIterator Pkg = Sim.FindPkg(iPkg.Name());
+   PkgIterator Pkg = Sim.FindPkg(iPkg.Name(), iPkg.Arch());
 
    Flags[Pkg->ID] = 3;
    Sim.MarkDelete(Pkg);
+
+   if (strcmp(Pkg.Arch(),"all") == 0)
+   {
+      pkgCache::GrpIterator G = Pkg.Group();
+      pkgCache::GrpIterator iG = iPkg.Group();
+      for (pkgCache::PkgIterator P = G.FindPkg("any"); P.end() != true; P = G.NextPkg(P))
+      {
+	 if (strcmp(P.Arch(), "all") == 0)
+	    continue;
+	 if (iG.FindPkg(P.Arch())->CurrentVer == 0)
+	    continue;
+	 Flags[P->ID] = 3;
+	 Sim.MarkDelete(P);
+      }
+   }
+
    if (Purge == true)
       cout << "Purg ";
    else
@@ -210,7 +254,7 @@ void pkgSimulate::ShortBreaks()
       if (Sim[I].InstBroken() == true)
       {
 	 if (Flags[I->ID] == 0)
-	    cout << I.Name() << ' ';
+	    cout << I.FullName(false) << ' ';
 /*	 else
 	    cout << I.Name() << "! ";*/
       }      
@@ -246,7 +290,7 @@ bool pkgApplyStatus(pkgDepCache &Cache)
 	       Cache.MarkInstall(I, false, 0, false);
 	    else
 	       return _error->Error(_("The package %s needs to be reinstalled, "
-				    "but I can't find an archive for it."),I.Name());
+				    "but I can't find an archive for it."),I.FullName(true).c_str());
 	 }
 	 
 	 continue;
@@ -281,7 +325,7 @@ bool pkgApplyStatus(pkgDepCache &Cache)
 	 default:
 	 if (I->InstState != pkgCache::State::Ok)
 	    return _error->Error("The package %s is not ok and I "
-				 "don't know how to fix it!",I.Name());
+				 "don't know how to fix it!",I.FullName(false).c_str());
       }
    }
    return true;
@@ -491,11 +535,11 @@ void pkgProblemResolver::MakeScores()
    // Important Required Standard Optional Extra
    signed short PrioMap[] = {
       0,
-      _config->FindI("pkgProblemResolver::Scores::Important",3),
-      _config->FindI("pkgProblemResolver::Scores::Required",2),
-      _config->FindI("pkgProblemResolver::Scores::Standard",1),
-      _config->FindI("pkgProblemResolver::Scores::Optional",-1),
-      _config->FindI("pkgProblemResolver::Scores::Extra",-2)
+      (signed short) _config->FindI("pkgProblemResolver::Scores::Important",3),
+      (signed short) _config->FindI("pkgProblemResolver::Scores::Required",2),
+      (signed short) _config->FindI("pkgProblemResolver::Scores::Standard",1),
+      (signed short) _config->FindI("pkgProblemResolver::Scores::Optional",-1),
+      (signed short) _config->FindI("pkgProblemResolver::Scores::Extra",-2)
    };
    signed short PrioEssentials = _config->FindI("pkgProblemResolver::Scores::Essentials",100);
    signed short PrioInstalledAndNotObsolete = _config->FindI("pkgProblemResolver::Scores::NotObsolete",1);
@@ -665,7 +709,7 @@ bool pkgProblemResolver::DoUpgrade(pkgCache::PkgIterator Pkg)
 	 if ((Flags[P->ID] & Protected) == Protected)
 	 {
 	    if (Debug == true)
-	       clog << "    Reinst Failed because of protected " << P.Name() << endl;
+	       clog << "    Reinst Failed because of protected " << P.FullName(false) << endl;
 	    Fail = true;
 	 }      
 	 else
@@ -676,7 +720,7 @@ bool pkgProblemResolver::DoUpgrade(pkgCache::PkgIterator Pkg)
 	       if (DoUpgrade(P) == false)
 	       {
 		  if (Debug == true)
-		     clog << "    Reinst Failed because of " << P.Name() << endl;
+		     clog << "    Reinst Failed because of " << P.FullName(false) << endl;
 		  Fail = true;
 	       }
 	       else
@@ -695,7 +739,7 @@ bool pkgProblemResolver::DoUpgrade(pkgCache::PkgIterator Pkg)
 		   break;
 	       
 	       if (Debug == true)
-		  clog << "    Reinst Failed early because of " << Start.TargetPkg().Name() << endl;
+		  clog << "    Reinst Failed early because of " << Start.TargetPkg().FullName(false) << endl;
 	       Fail = true;
 	    }     
 	 }
@@ -719,7 +763,7 @@ bool pkgProblemResolver::DoUpgrade(pkgCache::PkgIterator Pkg)
    }	 
    
    if (Debug == true)
-      clog << "  Re-Instated " << Pkg.Name() << endl;
+      clog << "  Re-Instated " << Pkg.FullName(false) << endl;
    return true;
 }
 									/*}}}*/
@@ -803,6 +847,7 @@ bool pkgProblemResolver::Resolve(bool BrokenFix)
       not be possible for a loop to form (that is a < b < c and fixing b by
       changing a breaks c) */
    bool Change = true;
+   bool const TryFixByInstall = _config->FindB("pkgProblemResolver::FixByInstall", true);
    for (int Counter = 0; Counter != 10 && Change == true; Counter++)
    {
       Change = false;
@@ -819,7 +864,7 @@ bool pkgProblemResolver::Resolve(bool BrokenFix)
 	     (Flags[I->ID] & ReInstateTried) == 0)
 	 {
 	    if (Debug == true)
-	       clog << " Try to Re-Instate " << I.Name() << endl;
+	       clog << " Try to Re-Instate (" << Counter << ") " << I.FullName(false) << endl;
 	    unsigned long OldBreaks = Cache.BrokenCount();
 	    pkgCache::Version *OldVer = Cache[I].InstallVer;
 	    Flags[I->ID] &= ReInstateTried;
@@ -835,14 +880,14 @@ bool pkgProblemResolver::Resolve(bool BrokenFix)
 	    }	    
 	    else
 	       if (Debug == true)
-		  clog << "Re-Instated " << I.Name() << " (" << OldBreaks << " vs " << Cache.BrokenCount() << ')' << endl;
+		  clog << "Re-Instated " << I.FullName(false) << " (" << OldBreaks << " vs " << Cache.BrokenCount() << ')' << endl;
 	 }
 	    
 	 if (Cache[I].InstallVer == 0 || Cache[I].InstBroken() == false)
 	    continue;
 	 
 	 if (Debug == true)
-	    clog << "Investigating " << I.Name() << endl;
+	    clog << "Investigating (" << Counter << ") " << I << endl;
 	 
 	 // Isolate the problem dependency
 	 PackageKill KillList[100];
@@ -860,22 +905,22 @@ bool pkgProblemResolver::Resolve(bool BrokenFix)
 	    if (Start == End)
 	    {
 	       // Decide what to do
-	       if (InOr == true)
+	       if (InOr == true && OldEnd == LEnd)
 	       {
-		  if (OldEnd == LEnd && OrOp == OrRemove)
+		  if (OrOp == OrRemove)
 		  {
 		     if ((Flags[I->ID] & Protected) != Protected)
 		     {
 			if (Debug == true)
-			   clog << "  Or group remove for " << I.Name() << endl;
+			   clog << "  Or group remove for " << I.FullName(false) << endl;
 			Cache.MarkDelete(I);
 			Change = true;
-		     }		     
-		  }		  
-		  if (OldEnd == LEnd && OrOp == OrKeep)
+		     }
+		  }
+		  else if (OrOp == OrKeep)
 		  {
 		     if (Debug == true)
-			clog << "  Or group keep for " << I.Name() << endl;
+			clog << "  Or group keep for " << I.FullName(false) << endl;
 		     Cache.MarkKeep(I, false, false);
 		     Change = true;
 		  }
@@ -912,7 +957,7 @@ bool pkgProblemResolver::Resolve(bool BrokenFix)
 	    }
 	    
 	    if (Debug == true)
-	       clog << "Package " << I.Name() << " has broken " << Start.DepType() << " on " << Start.TargetPkg().Name() << endl;
+	       clog << "Broken " << Start << endl;
 
 	    /* Look across the version list. If there are no possible
 	       targets then we keep the package and bail. This is necessary
@@ -943,8 +988,8 @@ bool pkgProblemResolver::Resolve(bool BrokenFix)
 	       pkgCache::PkgIterator Pkg = Ver.ParentPkg();
 
 	       if (Debug == true)
-		  clog << "  Considering " << Pkg.Name() << ' ' << (int)Scores[Pkg->ID] <<
-		  " as a solution to " << I.Name() << ' ' << (int)Scores[I->ID] << endl;
+		  clog << "  Considering " << Pkg.FullName(false) << ' ' << (int)Scores[Pkg->ID] <<
+		  " as a solution to " << I.FullName(false) << ' ' << (int)Scores[I->ID] << endl;
 
 	       /* Try to fix the package under consideration rather than
 	          fiddle with the VList package */
@@ -982,7 +1027,7 @@ bool pkgProblemResolver::Resolve(bool BrokenFix)
 			Cache.MarkInstall(I, false, 0, false);
 		     
 		     if (Debug == true)
-			clog << "  Holding Back " << I.Name() << " rather than change " << Start.TargetPkg().Name() << endl;
+			clog << "  Holding Back " << I.FullName(false) << " rather than change " << Start.TargetPkg().FullName(false) << endl;
 		  }		  
 		  else
 		  {		     
@@ -992,11 +1037,29 @@ bool pkgProblemResolver::Resolve(bool BrokenFix)
 			if (InOr == false)
 			{
 			   if (Debug == true)
-			      clog << "  Removing " << I.Name() << " rather than change " << Start.TargetPkg().Name() << endl;
+			      clog << "  Removing " << I.FullName(false) << " rather than change " << Start.TargetPkg().FullName(false) << endl;
 			   Cache.MarkDelete(I);
 			   if (Counter > 1 && Scores[Pkg->ID] > Scores[I->ID])
 			      Scores[I->ID] = Scores[Pkg->ID];
 			}
+			else if (TryFixByInstall == true &&
+				 Start.TargetPkg()->CurrentVer == 0 &&
+				 Cache[Start.TargetPkg()].Delete() == false &&
+				 Cache.GetCandidateVer(Start.TargetPkg()).end() == false)
+			{
+			   /* Before removing or keeping the package with the broken dependency
+			      try instead to install the first not previously installed package
+			      solving this dependency. This helps every time a previous solver
+			      is removed by the resolver because of a conflict or alike but it is
+			      dangerous as it could trigger new breaks/conflicts… */
+			   if (Debug == true)
+			      clog << "  Try Installing " << Start.TargetPkg() << " before changing " << I.FullName(false) << std::endl;
+			   unsigned long const OldBroken = Cache.BrokenCount();
+			   Cache.MarkInstall(Start.TargetPkg(), true, 1, false);
+			   // FIXME: we should undo the complete MarkInstall process here
+			   if (Cache[Start.TargetPkg()].InstBroken() == true || Cache.BrokenCount() > OldBroken)
+			      Cache.MarkDelete(Start.TargetPkg(), false, 1, false);
+			}
 		     }
 		  }
 		  		  
@@ -1024,7 +1087,7 @@ bool pkgProblemResolver::Resolve(bool BrokenFix)
 		     if (Cache[End] & pkgDepCache::DepGCVer) 
 		     {
 			if (Debug)
-			   clog << "  Upgrading " << Pkg.Name() << " due to Breaks field in " << I.Name() << endl;
+			   clog << "  Upgrading " << Pkg.FullName(false) << " due to Breaks field in " << I.FullName(false) << endl;
 			Cache.MarkInstall(Pkg, false, 0, false);
 			continue;
 		     }
@@ -1035,7 +1098,7 @@ bool pkgProblemResolver::Resolve(bool BrokenFix)
 		     continue;
 		
 		  if (Debug == true)
-		     clog << "  Added " << Pkg.Name() << " to the remove list" << endl;
+		     clog << "  Added " << Pkg.FullName(false) << " to the remove list" << endl;
 		  
 		  LEnd->Pkg = Pkg;
 		  LEnd->Dep = End;
@@ -1067,12 +1130,12 @@ bool pkgProblemResolver::Resolve(bool BrokenFix)
 		     Cache.MarkInstall(I, false, 0, false);
 		  
 		  if (Debug == true)
-		     clog << "  Holding Back " << I.Name() << " because I can't find " << Start.TargetPkg().Name() << endl;
+		     clog << "  Holding Back " << I.FullName(false) << " because I can't find " << Start.TargetPkg().FullName(false) << endl;
 	       }	       
 	       else
 	       {
 		  if (Debug == true)
-		     clog << "  Removing " << I.Name() << " because I can't find " << Start.TargetPkg().Name() << endl;
+		     clog << "  Removing " << I.FullName(false) << " because I can't find " << Start.TargetPkg().FullName(false) << endl;
 		  if (InOr == false)
 		     Cache.MarkDelete(I);
 	       }
@@ -1102,14 +1165,14 @@ bool pkgProblemResolver::Resolve(bool BrokenFix)
 		      J->Dep->Type == pkgCache::Dep::Obsoletes)
 		  {
 		     if (Debug == true)
-			clog << "  Fixing " << I.Name() << " via remove of " << J->Pkg.Name() << endl;
+			clog << "  Fixing " << I.FullName(false) << " via remove of " << J->Pkg.FullName(false) << endl;
 		     Cache.MarkDelete(J->Pkg);
 		  }
 	       }
 	       else
 	       {
 		  if (Debug == true)
-		     clog << "  Fixing " << I.Name() << " via keep of " << J->Pkg.Name() << endl;
+		     clog << "  Fixing " << I.FullName(false) << " via keep of " << J->Pkg.FullName(false) << endl;
 		  Cache.MarkKeep(J->Pkg, false, false);
 	       }
 
@@ -1140,13 +1203,12 @@ bool pkgProblemResolver::Resolve(bool BrokenFix)
       return _error->Error(_("Unable to correct problems, you have held broken packages."));
    }
    
-   // set the auto-flags (mvo: I'm not sure if we _really_ need this, but
-   // I didn't managed 
+   // set the auto-flags (mvo: I'm not sure if we _really_ need this)
    pkgCache::PkgIterator I = Cache.PkgBegin();
    for (;I.end() != true; I++) {
       if (Cache[I].NewInstall() && !(Flags[I->ID] & PreInstalled)) {
 	 if(_config->FindI("Debug::pkgAutoRemove",false)) {
-	    std::clog << "Resolve installed new pkg: " << I.Name() 
+	    std::clog << "Resolve installed new pkg: " << I.FullName(false) 
 		      << " (now marking it as auto)" << std::endl;
 	 }
 	 Cache[I].Flags |= pkgCache::Flag::Auto;
@@ -1209,7 +1271,7 @@ bool pkgProblemResolver::ResolveByKeep()
       if ((Flags[I->ID] & Protected) == 0)
       {
 	 if (Debug == true)
-	    clog << "Keeping package " << I.Name() << endl;
+	    clog << "Keeping package " << I.FullName(false) << endl;
 	 Cache.MarkKeep(I, false, false);
 	 if (Cache[I].InstBroken() == false)
 	 {
@@ -1240,7 +1302,7 @@ bool pkgProblemResolver::ResolveByKeep()
 	 while (true)
 	 {
 	    if (Debug == true)
-	       clog << "Package " << I.Name() << " has broken " << Start.DepType() << " on " << Start.TargetPkg().Name() << endl;
+	       clog << "Package " << I.FullName(false) << " " << Start << endl;
 
 	    // Look at all the possible provides on this package
 	    SPtrArray<pkgCache::Version *> VList = Start.AllTargets();
@@ -1257,7 +1319,7 @@ bool pkgProblemResolver::ResolveByKeep()
 	       if ((Flags[I->ID] & Protected) == 0)
 	       {
 		  if (Debug == true)
-		     clog << "  Keeping Package " << Pkg.Name() << " due to " << Start.DepType() << endl;
+		     clog << "  Keeping Package " << Pkg.FullName(false) << " due to " << Start.DepType() << endl;
 		  Cache.MarkKeep(Pkg, false, false);
 	       }
 	       
@@ -1282,7 +1344,7 @@ bool pkgProblemResolver::ResolveByKeep()
       
       // Restart again.
       if (K == LastStop)
-	 return _error->Error("Internal Error, pkgProblemResolver::ResolveByKeep is looping on package %s.",I.Name());
+	 return _error->Error("Internal Error, pkgProblemResolver::ResolveByKeep is looping on package %s.",I.FullName(false).c_str());
       LastStop = K;
       K = PList - 1;
    }   
@@ -1354,7 +1416,9 @@ bool ListUpdate(pkgAcquireStatus &Stat,
 		int PulseInterval)
 {
    pkgAcquire::RunResult res;
-   pkgAcquire Fetcher(&Stat);
+   pkgAcquire Fetcher;
+   if (Fetcher.Setup(&Stat, _config->FindDir("Dir::State::Lists")) == false)
+      return false;
 
    // Populate it with the source selection
    if (List.GetIndexes(&Fetcher) == false)

apt-pkg/algorithms.h

@@ -48,7 +48,7 @@ class pkgSimulate : public pkgPackageManager				/*{{{*/
       pkgDepCache *Cache;
       public:
       
-      virtual VerIterator GetCandidateVer(PkgIterator Pkg)
+      virtual VerIterator GetCandidateVer(PkgIterator const &Pkg)
       {
 	 return (*Cache)[Pkg].CandidateVerIter(*Cache);
       }

apt-pkg/aptconfiguration.cc

@@ -8,13 +8,19 @@
    ##################################################################### */
 									/*}}}*/
 // Include Files							/*{{{*/
-#include <apt-pkg/fileutl.h>
 #include <apt-pkg/aptconfiguration.h>
 #include <apt-pkg/configuration.h>
+#include <apt-pkg/error.h>
+#include <apt-pkg/fileutl.h>
+#include <apt-pkg/macros.h>
+#include <apt-pkg/strutl.h>
+
+#include <sys/types.h>
+#include <dirent.h>
 
-#include <vector>
-#include <string>
 #include <algorithm>
+#include <string>
+#include <vector>
 									/*}}}*/
 namespace APT {
 // getCompressionTypes - Return Vector of usbale compressiontypes	/*{{{*/
@@ -87,4 +93,271 @@ const Configuration::getCompressionTypes(bool const &Cached) {
 	return types;
 }
 									/*}}}*/
+// GetLanguages - Return Vector of Language Codes			/*{{{*/
+// ---------------------------------------------------------------------
+/* return a vector of language codes in the prefered order.
+   the special word "environment" will be replaced with the long and the short
+   code of the local settings and it will be insured that this will not add
+   duplicates. So in an german local the setting "environment, de_DE, en, de"
+   will result in "de_DE, de, en".
+   The special word "none" is the stopcode for the not-All code vector */
+std::vector<std::string> const Configuration::getLanguages(bool const &All,
+				bool const &Cached, char const ** const Locale) {
+	using std::string;
+
+	// The detection is boring and has a lot of cornercases,
+	// so we cache the results to calculated it only once.
+	std::vector<string> static allCodes;
+	std::vector<string> static codes;
+
+	// we have something in the cache
+	if (codes.empty() == false || allCodes.empty() == false) {
+		if (Cached == true) {
+			if(All == true && allCodes.empty() == false)
+				return allCodes;
+			else
+				return codes;
+		} else {
+			allCodes.clear();
+			codes.clear();
+		}
+	}
+
+	// Include all Language codes we have a Translation file for in /var/lib/apt/lists
+	// so they will be all included in the Cache.
+	std::vector<string> builtin;
+	DIR *D = opendir(_config->FindDir("Dir::State::lists").c_str());
+	if (D != 0) {
+		builtin.push_back("none");
+		for (struct dirent *Ent = readdir(D); Ent != 0; Ent = readdir(D)) {
+			string const name = Ent->d_name;
+			size_t const foundDash = name.rfind("-");
+			size_t const foundUnderscore = name.rfind("_");
+			if (foundDash == string::npos || foundUnderscore == string::npos ||
+			    foundDash <= foundUnderscore ||
+			    name.substr(foundUnderscore+1, foundDash-(foundUnderscore+1)) != "Translation")
+				continue;
+			string const c = name.substr(foundDash+1);
+			if (unlikely(c.empty() == true) || c == "en")
+				continue;
+			// Skip unusual files, like backups or that alike
+			string::const_iterator s = c.begin();
+			for (;s != c.end(); ++s) {
+				if (isalpha(*s) == 0)
+					break;
+			}
+			if (s != c.end())
+				continue;
+			if (std::find(builtin.begin(), builtin.end(), c) != builtin.end())
+				continue;
+			builtin.push_back(c);
+		}
+	}
+	closedir(D);
+
+	// get the environment language codes: LC_MESSAGES (and later LANGUAGE)
+	// we extract both, a long and a short code and then we will
+	// check if we actually need both (rare) or if the short is enough
+	string const envMsg = string(Locale == 0 ? std::setlocale(LC_MESSAGES, NULL) : *Locale);
+	size_t const lenShort = (envMsg.find('_') != string::npos) ? envMsg.find('_') : 2;
+	size_t const lenLong = (envMsg.find_first_of(".@") != string::npos) ? envMsg.find_first_of(".@") : (lenShort + 3);
+
+	string envLong = envMsg.substr(0,lenLong);
+	string const envShort = envLong.substr(0,lenShort);
+	bool envLongIncluded = true;
+
+	// first cornercase: LANG=C, so we use only "en" Translation
+	if (envLong == "C") {
+		codes.push_back("en");
+		allCodes = codes;
+		allCodes.insert(allCodes.end(), builtin.begin(), builtin.end());
+		if (All == true)
+			return allCodes;
+		else
+			return codes;
+	}
+
+	// to save the servers from unneeded queries, we only try also long codes
+	// for languages it is realistic to have a long code translation file…
+	// TODO: Improve translation acquire system to drop them dynamic
+	char const *needLong[] = { "cs", "en", "pt", "sv", "zh", NULL };
+	if (envLong != envShort) {
+		for (char const **l = needLong; *l != NULL; l++)
+			if (envShort.compare(*l) == 0) {
+				envLongIncluded = false;
+				break;
+			}
+	}
+
+	// we don't add the long code, but we allow the user to do so
+	if (envLongIncluded == true)
+		envLong.clear();
+
+	// FIXME: Remove support for the old APT::Acquire::Translation
+	// it was undocumented and so it should be not very widthly used
+	string const oldAcquire = _config->Find("APT::Acquire::Translation","");
+	if (oldAcquire.empty() == false && oldAcquire != "environment") {
+		// TRANSLATORS: the two %s are APT configuration options
+		_error->Notice("Option '%s' is deprecated. Please use '%s' instead, see 'man 5 apt.conf' for details.",
+				"APT::Acquire::Translation", "Acquire::Languages");
+		if (oldAcquire != "none")
+			codes.push_back(oldAcquire);
+		codes.push_back("en");
+		allCodes = codes;
+		for (std::vector<string>::const_iterator b = builtin.begin();
+		     b != builtin.end(); ++b)
+			if (std::find(allCodes.begin(), allCodes.end(), *b) == allCodes.end())
+				allCodes.push_back(*b);
+		if (All == true)
+			return allCodes;
+		else
+			return codes;
+	}
+
+	// It is very likely we will need to environment codes later,
+	// so let us generate them now from LC_MESSAGES and LANGUAGE
+	std::vector<string> environment;
+	// take care of LC_MESSAGES
+	if (envLongIncluded == false)
+		environment.push_back(envLong);
+	environment.push_back(envShort);
+	// take care of LANGUAGE
+	const char *language_env = getenv("LANGUAGE") == 0 ? "" : getenv("LANGUAGE");
+	string envLang = Locale == 0 ? language_env : *(Locale+1);
+	if (envLang.empty() == false) {
+		std::vector<string> env = VectorizeString(envLang,':');
+		short addedLangs = 0; // add a maximum of 3 fallbacks from the environment
+		for (std::vector<string>::const_iterator e = env.begin();
+		     e != env.end() && addedLangs < 3; ++e) {
+			if (unlikely(e->empty() == true) || *e == "en")
+				continue;
+			if (*e == envLong || *e == envShort)
+				continue;
+			if (std::find(environment.begin(), environment.end(), *e) != environment.end())
+				continue;
+			if (e->find('_') != string::npos) {
+				// Drop LongCodes here - ShortCodes are also included
+				string const shorty = e->substr(0, e->find('_'));
+				char const **n = needLong;
+				for (; *n != NULL; ++n)
+					if (shorty == *n)
+						break;
+				if (*n == NULL)
+					continue;
+			}
+			++addedLangs;
+			environment.push_back(*e);
+		}
+	}
+
+	// Support settings like Acquire::Translation=none on the command line to
+	// override the configuration settings vector of languages.
+	string const forceLang = _config->Find("Acquire::Languages","");
+	if (forceLang.empty() == false) {
+		if (forceLang == "environment") {
+			codes = environment;
+		} else if (forceLang != "none")
+			codes.push_back(forceLang);
+		allCodes = codes;
+		for (std::vector<string>::const_iterator b = builtin.begin();
+		     b != builtin.end(); ++b)
+			if (std::find(allCodes.begin(), allCodes.end(), *b) == allCodes.end())
+				allCodes.push_back(*b);
+		if (All == true)
+			return allCodes;
+		else
+			return codes;
+	}
+
+	std::vector<string> const lang = _config->FindVector("Acquire::Languages");
+	// the default setting -> "environment, en"
+	if (lang.empty() == true) {
+		codes = environment;
+		if (envShort != "en")
+			codes.push_back("en");
+		allCodes = codes;
+		for (std::vector<string>::const_iterator b = builtin.begin();
+		     b != builtin.end(); ++b)
+			if (std::find(allCodes.begin(), allCodes.end(), *b) == allCodes.end())
+				allCodes.push_back(*b);
+		if (All == true)
+			return allCodes;
+		else
+			return codes;
+	}
+
+	// the configs define the order, so add the environment
+	// then needed and ensure the codes are not listed twice.
+	bool noneSeen = false;
+	for (std::vector<string>::const_iterator l = lang.begin();
+	     l != lang.end(); l++) {
+		if (*l == "environment") {
+			for (std::vector<string>::const_iterator e = environment.begin();
+			     e != environment.end(); ++e) {
+				if (std::find(allCodes.begin(), allCodes.end(), *e) != allCodes.end())
+					continue;
+				if (noneSeen == false)
+					codes.push_back(*e);
+				allCodes.push_back(*e);
+			}
+			continue;
+		} else if (*l == "none") {
+			noneSeen = true;
+			continue;
+		} else if (std::find(allCodes.begin(), allCodes.end(), *l) != allCodes.end())
+			continue;
+
+		if (noneSeen == false)
+			codes.push_back(*l);
+		allCodes.push_back(*l);
+	}
+
+	for (std::vector<string>::const_iterator b = builtin.begin();
+	     b != builtin.end(); ++b)
+		if (std::find(allCodes.begin(), allCodes.end(), *b) == allCodes.end())
+			allCodes.push_back(*b);
+
+	if (All == true)
+		return allCodes;
+	else
+		return codes;
+}
+									/*}}}*/
+// getArchitectures - Return Vector of prefered Architectures		/*{{{*/
+std::vector<std::string> const Configuration::getArchitectures(bool const &Cached) {
+	using std::string;
+
+	std::vector<string> static archs;
+	if (likely(Cached == true) && archs.empty() == false)
+		return archs;
+
+	archs = _config->FindVector("APT::Architectures");
+	string const arch = _config->Find("APT::Architecture");
+	if (unlikely(arch.empty() == true))
+		return archs;
+
+	if (archs.empty() == true ||
+	    std::find(archs.begin(), archs.end(), arch) == archs.end())
+		archs.push_back(arch);
+
+	// erase duplicates and empty strings
+	for (std::vector<string>::reverse_iterator a = archs.rbegin();
+	     a != archs.rend(); ++a) {
+		if (a->empty() == true || std::find(a + 1, archs.rend(), *a) != archs.rend())
+			archs.erase(a.base()-1);
+		if (a == archs.rend())
+			break;
+	}
+
+	return archs;
+}
+									/*}}}*/
+// checkArchitecture - are we interested in the given Architecture?	/*{{{*/
+bool const Configuration::checkArchitecture(std::string const &Arch) {
+	if (Arch == "all")
+		return true;
+	std::vector<std::string> const archs = getArchitectures(true);
+	return (std::find(archs.begin(), archs.end(), Arch) != archs.end());
+}
+									/*}}}*/
 }

apt-pkg/aptconfiguration.h

@@ -39,6 +39,49 @@ public:									/*{{{*/
 	 *  \return a vector of (all) Language Codes in the prefered usage order
 	 */
 	std::vector<std::string> static const getCompressionTypes(bool const &Cached = true);
+
+	/** \brief Returns a vector of Language Codes
+	 *
+	 *  Languages can be defined with their two or five chars long code.
+	 *  This methods handles the various ways to set the prefered codes,
+	 *  honors the environment and ensures that the codes are not listed twice.
+	 *
+	 *  The special word "environment" will be replaced with the long and the short
+	 *  code of the local settings and it will be insured that this will not add
+	 *  duplicates. So in an german local the setting "environment, de_DE, en, de"
+	 *  will result in "de_DE, de, en".
+	 *
+	 *  Another special word is "none" which separates the prefered from all codes
+	 *  in this setting. So setting and method can be used to get codes the user want
+	 *  to see or to get all language codes APT (should) have Translations available.
+	 *
+	 *  \param All return all codes or only codes for languages we want to use
+	 *  \param Cached saves the result so we need to calculated it only once
+	 *                this parameter should ony be used for testing purposes.
+	 *  \param Locale don't get the locale from the system but use this one instead
+	 *                this parameter should ony be used for testing purposes.
+	 *
+	 *  \return a vector of (all) Language Codes in the prefered usage order
+	 */
+	std::vector<std::string> static const getLanguages(bool const &All = false,
+			bool const &Cached = true, char const ** const Locale = 0);
+
+	/** \brief Returns a vector of Architectures we support
+	 *
+	 *  \param Cached saves the result so we need to calculated it only once
+	 *                this parameter should ony be used for testing purposes.
+	 *
+	 *  \return a vector of Architectures in prefered order
+	 */
+	std::vector<std::string> static const getArchitectures(bool const &Cached = true);
+
+	/** \brief Are we interested in the given Architecture?
+	 *
+	 *  \param Arch we want to check
+	 *  \return true if we are interested, false otherwise
+	 */
+	bool static const checkArchitecture(std::string const &Arch);
+
 									/*}}}*/
 };
 									/*}}}*/

apt-pkg/cachefile.cc

@@ -27,7 +27,8 @@
 // CacheFile::CacheFile - Constructor					/*{{{*/
 // ---------------------------------------------------------------------
 /* */
-pkgCacheFile::pkgCacheFile() : Map(0), Cache(0), DCache(0), Policy(0)
+pkgCacheFile::pkgCacheFile() : Map(NULL), Cache(NULL), DCache(NULL),
+				SrcList(NULL), Policy(NULL)
 {
 }
 									/*}}}*/
@@ -38,16 +39,30 @@ pkgCacheFile::~pkgCacheFile()
 {
    delete DCache;
    delete Policy;
+   delete SrcList;
    delete Cache;
    delete Map;
    _system->UnLock(true);
-}   
+}
 									/*}}}*/
 // CacheFile::BuildCaches - Open and build the cache files		/*{{{*/
 // ---------------------------------------------------------------------
 /* */
-bool pkgCacheFile::BuildCaches(OpProgress &Progress,bool WithLock)
+bool pkgCacheFile::BuildCaches(OpProgress *Progress, bool WithLock)
 {
+   if (Cache != NULL)
+      return true;
+
+   if (_config->FindB("pkgCacheFile::Generate", true) == false)
+   {
+      Map = new MMap(*new FileFd(_config->FindFile("Dir::Cache::pkgcache"),
+		     FileFd::ReadOnly),MMap::Public|MMap::ReadOnly);
+      Cache = new pkgCache(Map);
+      if (_error->PendingError() == true)
+         return false;
+      return true;
+   }
+
    const bool ErrorWasEmpty = _error->empty();
    if (WithLock == true)
       if (_system->Lock() == false)
@@ -58,15 +73,13 @@ bool pkgCacheFile::BuildCaches(OpProgress &Progress,bool WithLock)
       
    if (_error->PendingError() == true)
       return false;
-   
-   // Read the source list
-   pkgSourceList List;
-   if (List.ReadMainList() == false)
-      return _error->Error(_("The list of sources could not be read."));
+
+   BuildSourceList(Progress);
 
    // Read the caches
-   bool Res = pkgMakeStatusCache(List,Progress,&Map,!WithLock);
-   Progress.Done();
+   bool Res = pkgCacheGenerator::MakeStatusCache(*SrcList,Progress,&Map,!WithLock);
+   if (Progress != NULL)
+      Progress->Done();
    if (Res == false)
       return _error->Error(_("The package lists or status file could not be parsed or opened."));
 
@@ -80,29 +93,70 @@ bool pkgCacheFile::BuildCaches(OpProgress &Progress,bool WithLock)
    return true;
 }
 									/*}}}*/
-// CacheFile::Open - Open the cache files, creating if necessary	/*{{{*/
+// CacheFile::BuildSourceList - Open and build all relevant sources.list/*{{{*/
 // ---------------------------------------------------------------------
 /* */
-bool pkgCacheFile::Open(OpProgress &Progress,bool WithLock)
+bool pkgCacheFile::BuildSourceList(OpProgress *Progress)
 {
-   if (BuildCaches(Progress,WithLock) == false)
-      return false;
-   
-   // The policy engine
+   if (SrcList != NULL)
+      return true;
+
+   SrcList = new pkgSourceList();
+   if (SrcList->ReadMainList() == false)
+      return _error->Error(_("The list of sources could not be read."));
+   return true;
+}
+									/*}}}*/
+// CacheFile::BuildPolicy - Open and build all relevant preferences	/*{{{*/
+// ---------------------------------------------------------------------
+/* */
+bool pkgCacheFile::BuildPolicy(OpProgress *Progress)
+{
+   if (Policy != NULL)
+      return true;
+
    Policy = new pkgPolicy(Cache);
    if (_error->PendingError() == true)
       return false;
 
    if (ReadPinFile(*Policy) == false || ReadPinDir(*Policy) == false)
       return false;
-   
-   // Create the dependency cache
+
+   return true;
+}
+									/*}}}*/
+// CacheFile::BuildDepCache - Open and build the dependency cache	/*{{{*/
+// ---------------------------------------------------------------------
+/* */
+bool pkgCacheFile::BuildDepCache(OpProgress *Progress)
+{
+   if (DCache != NULL)
+      return true;
+
    DCache = new pkgDepCache(Cache,Policy);
    if (_error->PendingError() == true)
       return false;
-   
-   DCache->Init(&Progress);
-   Progress.Done();
+
+   DCache->Init(Progress);
+   return true;
+}
+									/*}}}*/
+// CacheFile::Open - Open the cache files, creating if necessary	/*{{{*/
+// ---------------------------------------------------------------------
+/* */
+bool pkgCacheFile::Open(OpProgress *Progress, bool WithLock)
+{
+   if (BuildCaches(Progress,WithLock) == false)
+      return false;
+
+   if (BuildPolicy(Progress) == false)
+      return false;
+
+   if (BuildDepCache(Progress) == false)
+      return false;
+
+   if (Progress != NULL)
+      Progress->Done();
    if (_error->PendingError() == true)
       return false;
    
@@ -117,12 +171,14 @@ void pkgCacheFile::Close()
    delete DCache;
    delete Policy;
    delete Cache;
+   delete SrcList;
    delete Map;
    _system->UnLock(true);
 
-   Map = 0;
-   DCache = 0;
-   Policy = 0;
-   Cache = 0;
+   Map = NULL;
+   DCache = NULL;
+   Policy = NULL;
+   Cache = NULL;
+   SrcList = NULL;
 }
 									/*}}}*/

apt-pkg/cachefile.h

@@ -20,9 +20,9 @@
 
 #include <apt-pkg/depcache.h>
 #include <apt-pkg/acquire.h>
+#include <apt-pkg/policy.h>
 #include <apt-pkg/sourcelist.h>
 
-class pkgPolicy;
 class pkgCacheFile
 {
    protected:
@@ -30,27 +30,47 @@ class pkgCacheFile
    MMap *Map;
    pkgCache *Cache;
    pkgDepCache *DCache;
-   
-   public:
+   pkgSourceList *SrcList;
 
+   public:
    pkgPolicy *Policy;
-      
+
    // We look pretty much exactly like a pointer to a dep cache
    inline operator pkgCache &() {return *Cache;};
    inline operator pkgCache *() {return Cache;};
    inline operator pkgDepCache &() {return *DCache;};
    inline operator pkgDepCache *() {return DCache;};
+   inline operator pkgPolicy &() {return *Policy;};
+   inline operator pkgPolicy *() {return Policy;};
+   inline operator pkgSourceList &() {return *SrcList;};
+   inline operator pkgSourceList *() {return SrcList;};
    inline pkgDepCache *operator ->() {return DCache;};
    inline pkgDepCache &operator *() {return *DCache;};
    inline pkgDepCache::StateCache &operator [](pkgCache::PkgIterator const &I) {return (*DCache)[I];};
    inline unsigned char &operator [](pkgCache::DepIterator const &I) {return (*DCache)[I];};
 
-   bool BuildCaches(OpProgress &Progress,bool WithLock = true);
-   bool Open(OpProgress &Progress,bool WithLock = true);
+   bool BuildCaches(OpProgress *Progress = NULL,bool WithLock = true);
+   __deprecated bool BuildCaches(OpProgress &Progress,bool const &WithLock = true) { return BuildCaches(&Progress, WithLock); };
+   bool BuildSourceList(OpProgress *Progress = NULL);
+   bool BuildPolicy(OpProgress *Progress = NULL);
+   bool BuildDepCache(OpProgress *Progress = NULL);
+   bool Open(OpProgress *Progress = NULL, bool WithLock = true);
+   inline bool ReadOnlyOpen(OpProgress *Progress = NULL) { return Open(Progress, false); };
+   __deprecated bool Open(OpProgress &Progress,bool const &WithLock = true) { return Open(&Progress, WithLock); };
    void Close();
-   
+
+   inline pkgCache* GetPkgCache() { BuildCaches(NULL, false); return Cache; };
+   inline pkgDepCache* GetDepCache() { BuildDepCache(); return DCache; };
+   inline pkgPolicy* GetPolicy() { BuildPolicy(); return Policy; };
+   inline pkgSourceList* GetSourceList() { BuildSourceList(); return SrcList; };
+
+   inline bool IsPkgCacheBuilt() const { return (Cache != NULL); };
+   inline bool IsDepCacheBuilt() const { return (DCache != NULL); };
+   inline bool IsPolicyBuilt() const { return (Policy != NULL); };
+   inline bool IsSrcListBuilt() const { return (SrcList != NULL); };
+
    pkgCacheFile();
-   ~pkgCacheFile();
+   virtual ~pkgCacheFile();
 };
 
 #endif

apt-pkg/cachefilter.cc

@@ -0,0 +1,54 @@
+// -*- mode: cpp; mode: fold -*-
+// Description								/*{{{*/
+/** \file cachefilter.h
+    Collection of functor classes */
+									/*}}}*/
+// Include Files							/*{{{*/
+#include <apt-pkg/cachefilter.h>
+#include <apt-pkg/error.h>
+#include <apt-pkg/pkgcache.h>
+
+#include <apti18n.h>
+
+#include <string>
+
+#include <regex.h>
+									/*}}}*/
+namespace APT {
+namespace CacheFilter {
+PackageNameMatchesRegEx::PackageNameMatchesRegEx(std::string const &Pattern) {/*{{{*/
+	pattern = new regex_t;
+	int const Res = regcomp(pattern, Pattern.c_str(), REG_EXTENDED | REG_ICASE | REG_NOSUB);
+	if (Res == 0)
+		return;
+
+	delete pattern;
+	pattern = NULL;
+	char Error[300];
+	regerror(Res, pattern, Error, sizeof(Error));
+	_error->Error(_("Regex compilation error - %s"), Error);
+}
+									/*}}}*/
+bool PackageNameMatchesRegEx::operator() (pkgCache::PkgIterator const &Pkg) {/*{{{*/
+	if (unlikely(pattern == NULL))
+		return false;
+	else
+		return regexec(pattern, Pkg.Name(), 0, 0, 0) == 0;
+}
+									/*}}}*/
+bool PackageNameMatchesRegEx::operator() (pkgCache::GrpIterator const &Grp) {/*{{{*/
+	if (unlikely(pattern == NULL))
+		return false;
+	else
+		return regexec(pattern, Grp.Name(), 0, 0, 0) == 0;
+}
+									/*}}}*/
+PackageNameMatchesRegEx::~PackageNameMatchesRegEx() {			/*{{{*/
+	if (pattern == NULL)
+		return;
+	regfree(pattern);
+	delete pattern;
+}
+									/*}}}*/
+}
+}

apt-pkg/cachefilter.h

@@ -0,0 +1,29 @@
+// -*- mode: cpp; mode: fold -*-
+// Description								/*{{{*/
+/** \file cachefilter.h
+   Collection of functor classes */
+									/*}}}*/
+#ifndef APT_CACHEFILTER_H
+#define APT_CACHEFILTER_H
+// Include Files							/*{{{*/
+#include <apt-pkg/pkgcache.h>
+
+#include <string>
+
+#include <regex.h>
+									/*}}}*/
+namespace APT {
+namespace CacheFilter {
+// PackageNameMatchesRegEx						/*{{{*/
+class PackageNameMatchesRegEx {
+	regex_t* pattern;
+public:
+	PackageNameMatchesRegEx(std::string const &Pattern);
+	bool operator() (pkgCache::PkgIterator const &Pkg);
+	bool operator() (pkgCache::GrpIterator const &Grp);
+	~PackageNameMatchesRegEx();
+};
+									/*}}}*/
+}
+}
+#endif

apt-pkg/cacheiterators.h

@@ -1,6 +1,5 @@
 // -*- mode: cpp; mode: fold -*-
 // Description								/*{{{*/
-// $Id: cacheiterators.h,v 1.18.2.1 2004/05/08 22:44:27 mdz Exp $
 /* ######################################################################
    
    Cache Iterators - Iterators for navigating the cache structure
@@ -30,417 +29,413 @@
 									/*}}}*/
 #ifndef PKGLIB_CACHEITERATORS_H
 #define PKGLIB_CACHEITERATORS_H
+#include<iterator>
+
+#include<string.h>
+// abstract Iterator template						/*{{{*/
+/* This template provides the very basic iterator methods we
+   need to have for doing some walk-over-the-cache magic */
+template<typename Str, typename Itr> class pkgCache::Iterator :
+			public std::iterator<std::forward_iterator_tag, Str> {
+	protected:
+	Str *S;
+	pkgCache *Owner;
+
+	/** \brief Returns the Pointer for this struct in the owner
+	 *  The implementation of this method should be pretty short
+	 *  as it will only return the Pointer into the mmap stored
+	 *  in the owner but the name of this pointer is different for
+	 *  each stucture and we want to abstract here at least for the
+	 *  basic methods from the actual structure.
+	 *  \return Pointer to the first structure of this type
+	 */
+	virtual Str* OwnerPointer() const = 0;
+
+	public:
+	// Iteration
+	virtual void operator ++(int) = 0;
+	virtual void operator ++() = 0; // Should be {operator ++(0);};
+	inline bool end() const {return Owner == 0 || S == OwnerPointer();};
+
+	// Comparison
+	inline bool operator ==(const Itr &B) const {return S == B.S;};
+	inline bool operator !=(const Itr &B) const {return S != B.S;};
+
+	// Accessors
+	inline Str *operator ->() {return S;};
+	inline Str const *operator ->() const {return S;};
+	inline operator Str *() {return S == OwnerPointer() ? 0 : S;};
+	inline operator Str const *() const {return S == OwnerPointer() ? 0 : S;};
+	inline Str &operator *() {return *S;};
+	inline Str const &operator *() const {return *S;};
+	inline pkgCache *Cache() const {return Owner;};
+
+	// Mixed stuff
+	inline void operator =(const Itr &B) {S = B.S; Owner = B.Owner;};
+	inline bool IsGood() const { return S && Owner && ! end();};
+	inline unsigned long Index() const {return S - OwnerPointer();};
+
+	void ReMap(void const * const oldMap, void const * const newMap) {
+		if (Owner == 0 || S == 0)
+			return;
+		S += (Str*)(newMap) - (Str*)(oldMap);
+	}
+
+	// Constructors - look out for the variable assigning
+	inline Iterator() : S(0), Owner(0) {};
+	inline Iterator(pkgCache &Owner,Str *T = 0) : S(T), Owner(&Owner) {};
+};
+									/*}}}*/
+// Group Iterator							/*{{{*/
+/* Packages with the same name are collected in a Group so someone only
+   interest in package names can iterate easily over the names, so the
+   different architectures can be treated as of the "same" package
+   (apt internally treat them as totally different packages) */
+class pkgCache::GrpIterator: public Iterator<Group, GrpIterator> {
+	long HashIndex;
+
+	protected:
+	inline Group* OwnerPointer() const {
+		return Owner->GrpP;
+	};
+
+	public:
+	// This constructor is the 'begin' constructor, never use it.
+	inline GrpIterator(pkgCache &Owner) : Iterator<Group, GrpIterator>(Owner), HashIndex(-1) {
+		S = OwnerPointer();
+		operator ++(0);
+	};
+
+	virtual void operator ++(int);
+	virtual void operator ++() {operator ++(0);};
+
+	inline const char *Name() const {return S->Name == 0?0:Owner->StrP + S->Name;};
+	inline PkgIterator PackageList() const;
+	PkgIterator FindPkg(string Arch = "any") const;
+	/** \brief find the package with the "best" architecture
+
+	    The best architecture is either the "native" or the first
+	    in the list of Architectures which is not an end-Pointer
+
+	    \param PreferNonVirtual tries to respond with a non-virtual package
+		   and only if this fails returns the best virtual package */
+	PkgIterator FindPreferredPkg(bool const &PreferNonVirtual = true) const;
+	PkgIterator NextPkg(PkgIterator const &Pkg) const;
+
+	// Constructors
+	inline GrpIterator(pkgCache &Owner, Group *Trg) : Iterator<Group, GrpIterator>(Owner, Trg), HashIndex(0) {
+		if (S == 0)
+			S = OwnerPointer();
+	};
+	inline GrpIterator() : Iterator<Group, GrpIterator>(), HashIndex(0) {};
 
-
+};
+									/*}}}*/
 // Package Iterator							/*{{{*/
-class pkgCache::PkgIterator
-{
-   friend class pkgCache;
-   Package *Pkg;
-   pkgCache *Owner;
-   long HashIndex;
-
-   protected:
-   
-   // This constructor is the 'begin' constructor, never use it.
-   inline PkgIterator(pkgCache &Owner) : Owner(&Owner), HashIndex(-1)
-   {
-      Pkg = Owner.PkgP;
-      operator ++(0);
-   };
-   
-   public:
-
-   enum OkState {NeedsNothing,NeedsUnpack,NeedsConfigure};
-      
-   // Iteration
-   void operator ++(int);
-   inline void operator ++() {operator ++(0);};
-   inline bool end() const {return Owner == 0 || Pkg == Owner->PkgP?true:false;};
-
-   // Comparison
-   inline bool operator ==(const PkgIterator &B) const {return Pkg == B.Pkg;};
-   inline bool operator !=(const PkgIterator &B) const {return Pkg != B.Pkg;};
-			   
-   // Accessors
-   inline Package *operator ->() {return Pkg;};
-   inline Package const *operator ->() const {return Pkg;};
-   inline Package const &operator *() const {return *Pkg;};
-   inline operator Package *() {return Pkg == Owner->PkgP?0:Pkg;};
-   inline operator Package const *() const {return Pkg == Owner->PkgP?0:Pkg;};
-   inline pkgCache *Cache() {return Owner;};
-   
-   inline const char *Name() const {return Pkg->Name == 0?0:Owner->StrP + Pkg->Name;};
-   inline const char *Section() const {return Pkg->Section == 0?0:Owner->StrP + Pkg->Section;};
-   inline bool Purge() const {return Pkg->CurrentState == pkgCache::State::Purge ||
-	 (Pkg->CurrentVer == 0 && Pkg->CurrentState == pkgCache::State::NotInstalled);};
-   inline VerIterator VersionList() const;
-   inline VerIterator CurrentVer() const;
-   inline DepIterator RevDependsList() const;
-   inline PrvIterator ProvidesList() const;
-   inline unsigned long Index() const {return Pkg - Owner->PkgP;};
-   OkState State() const;
-
-   //Nice printable representation
-   friend std::ostream& operator<<(std::ostream& out, pkgCache::PkgIterator Pkg);
-
-   const char *CandVersion() const;
-   const char *CurVersion() const;
-
-   // Constructors
-   inline PkgIterator(pkgCache &Owner,Package *Trg) : Pkg(Trg), Owner(&Owner),
-          HashIndex(0) 
-   { 
-      if (Pkg == 0)
-	 Pkg = Owner.PkgP;
-   };
-   inline PkgIterator() : Pkg(0), Owner(0), HashIndex(0) {};
+class pkgCache::PkgIterator: public Iterator<Package, PkgIterator> {
+	long HashIndex;
+
+	protected:
+	inline Package* OwnerPointer() const {
+		return Owner->PkgP;
+	};
+
+	public:
+	// This constructor is the 'begin' constructor, never use it.
+	inline PkgIterator(pkgCache &Owner) : Iterator<Package, PkgIterator>(Owner), HashIndex(-1) {
+		S = OwnerPointer();
+		operator ++(0);
+	};
+
+	virtual void operator ++(int);
+	virtual void operator ++() {operator ++(0);};
+
+	enum OkState {NeedsNothing,NeedsUnpack,NeedsConfigure};
+
+	// Accessors
+	inline const char *Name() const {return S->Name == 0?0:Owner->StrP + S->Name;};
+	inline const char *Section() const {return S->Section == 0?0:Owner->StrP + S->Section;};
+	inline bool Purge() const {return S->CurrentState == pkgCache::State::Purge ||
+		(S->CurrentVer == 0 && S->CurrentState == pkgCache::State::NotInstalled);};
+	inline const char *Arch() const {return S->Arch == 0?0:Owner->StrP + S->Arch;};
+	inline GrpIterator Group() const { return GrpIterator(*Owner, Owner->GrpP + S->Group);};
+
+	inline VerIterator VersionList() const;
+	inline VerIterator CurrentVer() const;
+	inline DepIterator RevDependsList() const;
+	inline PrvIterator ProvidesList() const;
+	OkState State() const;
+	const char *CandVersion() const;
+	const char *CurVersion() const;
+
+	//Nice printable representation
+	friend std::ostream& operator <<(std::ostream& out, PkgIterator i);
+	std::string FullName(bool const &Pretty = false) const;
+
+	// Constructors
+	inline PkgIterator(pkgCache &Owner,Package *Trg) : Iterator<Package, PkgIterator>(Owner, Trg), HashIndex(0) {
+		if (S == 0)
+			S = OwnerPointer();
+	};
+	inline PkgIterator() : Iterator<Package, PkgIterator>(), HashIndex(0) {};
 };
 									/*}}}*/
 // Version Iterator							/*{{{*/
-class pkgCache::VerIterator
-{
-   Version *Ver;
-   pkgCache *Owner;
-
-   void _dummy();
-   
-   public:
-
-   // Iteration
-   void operator ++(int) {if (Ver != Owner->VerP) Ver = Owner->VerP + Ver->NextVer;};
-   inline void operator ++() {operator ++(0);};
-   inline bool end() const {return Owner == 0 || (Ver == Owner->VerP?true:false);};
-   inline void operator =(const VerIterator &B) {Ver = B.Ver; Owner = B.Owner;};
-   
-   // Comparison
-   inline bool operator ==(const VerIterator &B) const {return Ver == B.Ver;};
-   inline bool operator !=(const VerIterator &B) const {return Ver != B.Ver;};
-   int CompareVer(const VerIterator &B) const;
-
-   // Testing
-   inline bool IsGood() const { return Ver && Owner && ! end();};
-
-   // Accessors
-   inline Version *operator ->() {return Ver;};
-   inline Version const *operator ->() const {return Ver;};
-   inline Version &operator *() {return *Ver;};
-   inline Version const &operator *() const {return *Ver;};
-   inline operator Version *() {return Ver == Owner->VerP?0:Ver;};
-   inline operator Version const *() const {return Ver == Owner->VerP?0:Ver;};
-   inline pkgCache *Cache() {return Owner;};
-      
-   inline const char *VerStr() const {return Ver->VerStr == 0?0:Owner->StrP + Ver->VerStr;};
-   inline const char *Section() const {return Ver->Section == 0?0:Owner->StrP + Ver->Section;};
-   inline const char *Arch() const {return Ver->Arch == 0?0:Owner->StrP + Ver->Arch;};
-   inline PkgIterator ParentPkg() const {return PkgIterator(*Owner,Owner->PkgP + Ver->ParentPkg);};
-   inline DescIterator DescriptionList() const;
-   DescIterator TranslatedDescription() const;
-   inline DepIterator DependsList() const;
-   inline PrvIterator ProvidesList() const;
-   inline VerFileIterator FileList() const;
-   inline unsigned long Index() const {return Ver - Owner->VerP;};
-   bool Downloadable() const;
-   inline const char *PriorityType() {return Owner->Priority(Ver->Priority);};
-   string RelStr();
-   
-   bool Automatic() const;
-   VerFileIterator NewestFile() const;
-
-   inline VerIterator() : Ver(0), Owner(0) {};   
-   inline VerIterator(pkgCache &Owner,Version *Trg = 0) : Ver(Trg), 
-              Owner(&Owner) 
-   { 
-      if (Ver == 0)
-	 Ver = Owner.VerP;
-   };
+class pkgCache::VerIterator : public Iterator<Version, VerIterator> {
+	protected:
+	inline Version* OwnerPointer() const {
+		return Owner->VerP;
+	};
+
+	public:
+	// Iteration
+	void operator ++(int) {if (S != Owner->VerP) S = Owner->VerP + S->NextVer;};
+	inline void operator ++() {operator ++(0);};
+
+	// Comparison
+	int CompareVer(const VerIterator &B) const;
+	/** \brief compares two version and returns if they are similar
+
+	    This method should be used to identify if two pseudo versions are
+	    refering to the same "real" version */
+	inline bool SimilarVer(const VerIterator &B) const {
+		return (B.end() == false && S->Hash == B->Hash && strcmp(VerStr(), B.VerStr()) == 0);
+	};
+
+	// Accessors
+	inline const char *VerStr() const {return S->VerStr == 0?0:Owner->StrP + S->VerStr;};
+	inline const char *Section() const {return S->Section == 0?0:Owner->StrP + S->Section;};
+	inline const char *Arch() const {
+		if(S->MultiArch == pkgCache::Version::All)
+			return "all";
+		return S->ParentPkg == 0?0:Owner->StrP + ParentPkg()->Arch;
+	};
+	inline const char *Arch(bool const pseudo) const {
+		if(pseudo == false)
+			return Arch();
+		return S->ParentPkg == 0?0:Owner->StrP + ParentPkg()->Arch;
+	};
+	inline PkgIterator ParentPkg() const {return PkgIterator(*Owner,Owner->PkgP + S->ParentPkg);};
+
+	inline DescIterator DescriptionList() const;
+	DescIterator TranslatedDescription() const;
+	inline DepIterator DependsList() const;
+	inline PrvIterator ProvidesList() const;
+	inline VerFileIterator FileList() const;
+	bool Downloadable() const;
+	inline const char *PriorityType() const {return Owner->Priority(S->Priority);};
+	string RelStr() const;
+
+	bool Automatic() const;
+	bool Pseudo() const;
+	VerFileIterator NewestFile() const;
+
+	inline VerIterator(pkgCache &Owner,Version *Trg = 0) : Iterator<Version, VerIterator>(Owner, Trg) {
+		if (S == 0)
+			S = OwnerPointer();
+	};
+	inline VerIterator() : Iterator<Version, VerIterator>() {};
 };
 									/*}}}*/
 // Description Iterator							/*{{{*/
-class pkgCache::DescIterator
-{
-   Description *Desc;
-   pkgCache *Owner;
-   
-   void _dummy();
-   
-   public:
-
-   // Iteration
-   void operator ++(int) {if (Desc != Owner->DescP) Desc = Owner->DescP + Desc->NextDesc;};
-   inline void operator ++() {operator ++(0);};
-   inline bool end() const {return Owner == 0 || Desc == Owner->DescP?true:false;};
-   inline void operator =(const DescIterator &B) {Desc = B.Desc; Owner = B.Owner;};
-   
-   // Comparison
-   inline bool operator ==(const DescIterator &B) const {return Desc == B.Desc;};
-   inline bool operator !=(const DescIterator &B) const {return Desc != B.Desc;};
-   int CompareDesc(const DescIterator &B) const;
-   
-   // Accessors
-   inline Description *operator ->() {return Desc;};
-   inline Description const *operator ->() const {return Desc;};
-   inline Description &operator *() {return *Desc;};
-   inline Description const &operator *() const {return *Desc;};
-   inline operator Description *() {return Desc == Owner->DescP?0:Desc;};
-   inline operator Description const *() const {return Desc == Owner->DescP?0:Desc;};
-   inline pkgCache *Cache() {return Owner;};
-      
-   inline const char *LanguageCode() const {return Owner->StrP + Desc->language_code;};
-   inline const char *md5() const {return Owner->StrP + Desc->md5sum;};
-   inline DescFileIterator FileList() const;
-   inline unsigned long Index() const {return Desc - Owner->DescP;};
-
-   inline DescIterator() : Desc(0), Owner(0) {};   
-   inline DescIterator(pkgCache &Owner,Description *Trg = 0) : Desc(Trg), 
-              Owner(&Owner) 
-   { 
-      if (Desc == 0)
-	 Desc = Owner.DescP;
-   };
+class pkgCache::DescIterator : public Iterator<Description, DescIterator> {
+	protected:
+	inline Description* OwnerPointer() const {
+		return Owner->DescP;
+	};
+
+	public:
+	// Iteration
+	void operator ++(int) {if (S != Owner->DescP) S = Owner->DescP + S->NextDesc;};
+	inline void operator ++() {operator ++(0);};
+
+	// Comparison
+	int CompareDesc(const DescIterator &B) const;
+
+	// Accessors
+	inline const char *LanguageCode() const {return Owner->StrP + S->language_code;};
+	inline const char *md5() const {return Owner->StrP + S->md5sum;};
+	inline DescFileIterator FileList() const;
+
+	inline DescIterator() : Iterator<Description, DescIterator>() {};
+	inline DescIterator(pkgCache &Owner,Description *Trg = 0) : Iterator<Description, DescIterator>(Owner, Trg) {
+		if (S == 0)
+			S = Owner.DescP;
+	};
 };
 									/*}}}*/
 // Dependency iterator							/*{{{*/
-class pkgCache::DepIterator
-{
-   Dependency *Dep;
-   enum {DepVer, DepRev} Type;
-   pkgCache *Owner;
-   
-   void _dummy();
-   
-   public:
-
-   // Iteration
-   void operator ++(int) {if (Dep != Owner->DepP) Dep = Owner->DepP +
-	(Type == DepVer?Dep->NextDepends:Dep->NextRevDepends);};
-   inline void operator ++() {operator ++(0);};
-   inline bool end() const {return Owner == 0 || Dep == Owner->DepP?true:false;};
-   
-   // Comparison
-   inline bool operator ==(const DepIterator &B) const {return Dep == B.Dep;};
-   inline bool operator !=(const DepIterator &B) const {return Dep != B.Dep;};
-
-   // Accessors
-   inline Dependency *operator ->() {return Dep;};
-   inline Dependency const *operator ->() const {return Dep;};
-   inline Dependency &operator *() {return *Dep;};
-   inline Dependency const &operator *() const {return *Dep;};
-   inline operator Dependency *() {return Dep == Owner->DepP?0:Dep;};
-   inline operator Dependency const *() const {return Dep == Owner->DepP?0:Dep;};
-   inline pkgCache *Cache() {return Owner;};
-   
-   inline const char *TargetVer() const {return Dep->Version == 0?0:Owner->StrP + Dep->Version;};
-   inline PkgIterator TargetPkg() {return PkgIterator(*Owner,Owner->PkgP + Dep->Package);};
-   inline PkgIterator SmartTargetPkg() {PkgIterator R(*Owner,0);SmartTargetPkg(R);return R;};
-   inline VerIterator ParentVer() {return VerIterator(*Owner,Owner->VerP + Dep->ParentVer);};
-   inline PkgIterator ParentPkg() {return PkgIterator(*Owner,Owner->PkgP + Owner->VerP[Dep->ParentVer].ParentPkg);};
-   inline bool Reverse() {return Type == DepRev;};
-   inline unsigned long Index() const {return Dep - Owner->DepP;};
-   bool IsCritical();
-   void GlobOr(DepIterator &Start,DepIterator &End);
-   Version **AllTargets();   
-   bool SmartTargetPkg(PkgIterator &Result);
-   inline const char *CompType() {return Owner->CompType(Dep->CompareOp);};
-   inline const char *DepType() {return Owner->DepType(Dep->Type);};
-   
-   inline DepIterator(pkgCache &Owner,Dependency *Trg,Version * = 0) :
-          Dep(Trg), Type(DepVer), Owner(&Owner) 
-   {
-      if (Dep == 0)
-	 Dep = Owner.DepP;
-   };
-   inline DepIterator(pkgCache &Owner,Dependency *Trg,Package *) :
-          Dep(Trg), Type(DepRev), Owner(&Owner)
-   {
-      if (Dep == 0)
-	 Dep = Owner.DepP;
-   };
-   inline DepIterator() : Dep(0), Type(DepVer), Owner(0) {};
+class pkgCache::DepIterator : public Iterator<Dependency, DepIterator> {
+	enum {DepVer, DepRev} Type;
+
+	protected:
+	inline Dependency* OwnerPointer() const {
+		return Owner->DepP;
+	};
+
+	public:
+	// Iteration
+	void operator ++(int) {if (S != Owner->DepP) S = Owner->DepP +
+		(Type == DepVer ? S->NextDepends : S->NextRevDepends);};
+	inline void operator ++() {operator ++(0);};
+
+	// Accessors
+	inline const char *TargetVer() const {return S->Version == 0?0:Owner->StrP + S->Version;};
+	inline PkgIterator TargetPkg() const {return PkgIterator(*Owner,Owner->PkgP + S->Package);};
+	inline PkgIterator SmartTargetPkg() const {PkgIterator R(*Owner,0);SmartTargetPkg(R);return R;};
+	inline VerIterator ParentVer() const {return VerIterator(*Owner,Owner->VerP + S->ParentVer);};
+	inline PkgIterator ParentPkg() const {return PkgIterator(*Owner,Owner->PkgP + Owner->VerP[S->ParentVer].ParentPkg);};
+	inline bool Reverse() const {return Type == DepRev;};
+	bool IsCritical() const;
+	void GlobOr(DepIterator &Start,DepIterator &End);
+	Version **AllTargets() const;
+	bool SmartTargetPkg(PkgIterator &Result) const;
+	inline const char *CompType() const {return Owner->CompType(S->CompareOp);};
+	inline const char *DepType() const {return Owner->DepType(S->Type);};
+
+	//Nice printable representation
+	friend std::ostream& operator <<(std::ostream& out, DepIterator D);
+
+	inline DepIterator(pkgCache &Owner, Dependency *Trg, Version* = 0) :
+		Iterator<Dependency, DepIterator>(Owner, Trg), Type(DepVer) {
+		if (S == 0)
+			S = Owner.DepP;
+	};
+	inline DepIterator(pkgCache &Owner, Dependency *Trg, Package*) :
+		Iterator<Dependency, DepIterator>(Owner, Trg), Type(DepRev) {
+		if (S == 0)
+			S = Owner.DepP;
+	};
+	inline DepIterator() : Iterator<Dependency, DepIterator>(), Type(DepVer) {};
 };
 									/*}}}*/
 // Provides iterator							/*{{{*/
-class pkgCache::PrvIterator
-{
-   Provides *Prv;
-   enum {PrvVer, PrvPkg} Type;
-   pkgCache *Owner;
-   
-   void _dummy();
-   
-   public:
-
-   // Iteration
-   void operator ++(int) {if (Prv != Owner->ProvideP) Prv = Owner->ProvideP +
-	(Type == PrvVer?Prv->NextPkgProv:Prv->NextProvides);};
-   inline void operator ++() {operator ++(0);};
-   inline bool end() const {return Owner == 0 || Prv == Owner->ProvideP?true:false;};
-   
-   // Comparison
-   inline bool operator ==(const PrvIterator &B) const {return Prv == B.Prv;};
-   inline bool operator !=(const PrvIterator &B) const {return Prv != B.Prv;};
-
-   // Accessors
-   inline Provides *operator ->() {return Prv;};
-   inline Provides const *operator ->() const {return Prv;};
-   inline Provides &operator *() {return *Prv;};
-   inline Provides const &operator *() const {return *Prv;};
-   inline operator Provides *() {return Prv == Owner->ProvideP?0:Prv;};
-   inline operator Provides const *() const {return Prv == Owner->ProvideP?0:Prv;};
-   inline pkgCache *Cache() {return Owner;};
-
-   inline const char *Name() const {return Owner->StrP + Owner->PkgP[Prv->ParentPkg].Name;};
-   inline const char *ProvideVersion() const {return Prv->ProvideVersion == 0?0:Owner->StrP + Prv->ProvideVersion;};
-   inline PkgIterator ParentPkg() {return PkgIterator(*Owner,Owner->PkgP + Prv->ParentPkg);};
-   inline VerIterator OwnerVer() {return VerIterator(*Owner,Owner->VerP + Prv->Version);};
-   inline PkgIterator OwnerPkg() {return PkgIterator(*Owner,Owner->PkgP + Owner->VerP[Prv->Version].ParentPkg);};
-   inline unsigned long Index() const {return Prv - Owner->ProvideP;};
-
-   inline PrvIterator() : Prv(0), Type(PrvVer), Owner(0)  {};
-
-   inline PrvIterator(pkgCache &Owner,Provides *Trg,Version *) :
-          Prv(Trg), Type(PrvVer), Owner(&Owner) 
-   {
-      if (Prv == 0)
-	 Prv = Owner.ProvideP;
-   };
-   inline PrvIterator(pkgCache &Owner,Provides *Trg,Package *) : 
-          Prv(Trg), Type(PrvPkg), Owner(&Owner)
-   {
-      if (Prv == 0)
-	 Prv = Owner.ProvideP;
-   };
+class pkgCache::PrvIterator : public Iterator<Provides, PrvIterator> {
+	enum {PrvVer, PrvPkg} Type;
+
+	protected:
+	inline Provides* OwnerPointer() const {
+		return Owner->ProvideP;
+	};
+
+	public:
+	// Iteration
+	void operator ++(int) {if (S != Owner->ProvideP) S = Owner->ProvideP +
+		(Type == PrvVer?S->NextPkgProv:S->NextProvides);};
+	inline void operator ++() {operator ++(0);};
+
+	// Accessors
+	inline const char *Name() const {return Owner->StrP + Owner->PkgP[S->ParentPkg].Name;};
+	inline const char *ProvideVersion() const {return S->ProvideVersion == 0?0:Owner->StrP + S->ProvideVersion;};
+	inline PkgIterator ParentPkg() const {return PkgIterator(*Owner,Owner->PkgP + S->ParentPkg);};
+	inline VerIterator OwnerVer() const {return VerIterator(*Owner,Owner->VerP + S->Version);};
+	inline PkgIterator OwnerPkg() const {return PkgIterator(*Owner,Owner->PkgP + Owner->VerP[S->Version].ParentPkg);};
+
+	inline PrvIterator() : Iterator<Provides, PrvIterator>(), Type(PrvVer) {};
+
+	inline PrvIterator(pkgCache &Owner, Provides *Trg, Version*) :
+		Iterator<Provides, PrvIterator>(Owner, Trg), Type(PrvVer) {
+		if (S == 0)
+			S = Owner.ProvideP;
+	};
+	inline PrvIterator(pkgCache &Owner, Provides *Trg, Package*) :
+		Iterator<Provides, PrvIterator>(Owner, Trg), Type(PrvPkg) {
+		if (S == 0)
+			S = Owner.ProvideP;
+	};
 };
 									/*}}}*/
 // Package file								/*{{{*/
-class pkgCache::PkgFileIterator
-{
-   pkgCache *Owner;
-   PackageFile *File;
-
-   public:
-
-   // Iteration
-   void operator ++(int) {if (File!= Owner->PkgFileP) File = Owner->PkgFileP + File->NextFile;};
-   inline void operator ++() {operator ++(0);};
-   inline bool end() const {return Owner == 0 || File == Owner->PkgFileP?true:false;};
-
-   // Comparison
-   inline bool operator ==(const PkgFileIterator &B) const {return File == B.File;};
-   inline bool operator !=(const PkgFileIterator &B) const {return File != B.File;};
-			   
-   // Accessors
-   inline PackageFile *operator ->() {return File;};
-   inline PackageFile const *operator ->() const {return File;};
-   inline PackageFile const &operator *() const {return *File;};
-   inline operator PackageFile *() {return File == Owner->PkgFileP?0:File;};
-   inline operator PackageFile const *() const {return File == Owner->PkgFileP?0:File;};
-   inline pkgCache *Cache() {return Owner;};
-
-   inline const char *FileName() const {return File->FileName == 0?0:Owner->StrP + File->FileName;};
-   inline const char *Archive() const {return File->Archive == 0?0:Owner->StrP + File->Archive;};
-   inline const char *Component() const {return File->Component == 0?0:Owner->StrP + File->Component;};
-   inline const char *Version() const {return File->Version == 0?0:Owner->StrP + File->Version;};
-   inline const char *Origin() const {return File->Origin == 0?0:Owner->StrP + File->Origin;};
-   inline const char *Codename() const {return File->Codename ==0?0:Owner->StrP + File->Codename;};
-   inline const char *Label() const {return File->Label == 0?0:Owner->StrP + File->Label;};
-   inline const char *Site() const {return File->Site == 0?0:Owner->StrP + File->Site;};
-   inline const char *Architecture() const {return File->Architecture == 0?0:Owner->StrP + File->Architecture;};
-   inline const char *IndexType() const {return File->IndexType == 0?0:Owner->StrP + File->IndexType;};
-
-   inline unsigned long Index() const {return File - Owner->PkgFileP;};
-
-   bool IsOk();
-   string RelStr();
-   
-   // Constructors
-   inline PkgFileIterator() : Owner(0), File(0) {};
-   inline PkgFileIterator(pkgCache &Owner) : Owner(&Owner), File(Owner.PkgFileP) {};
-   inline PkgFileIterator(pkgCache &Owner,PackageFile *Trg) : Owner(&Owner), File(Trg) {};
+class pkgCache::PkgFileIterator : public Iterator<PackageFile, PkgFileIterator> {
+	protected:
+	inline PackageFile* OwnerPointer() const {
+		return Owner->PkgFileP;
+	};
+
+	public:
+	// Iteration
+	void operator ++(int) {if (S != Owner->PkgFileP) S = Owner->PkgFileP + S->NextFile;};
+	inline void operator ++() {operator ++(0);};
+
+	// Accessors
+	inline const char *FileName() const {return S->FileName == 0?0:Owner->StrP + S->FileName;};
+	inline const char *Archive() const {return S->Archive == 0?0:Owner->StrP + S->Archive;};
+	inline const char *Component() const {return S->Component == 0?0:Owner->StrP + S->Component;};
+	inline const char *Version() const {return S->Version == 0?0:Owner->StrP + S->Version;};
+	inline const char *Origin() const {return S->Origin == 0?0:Owner->StrP + S->Origin;};
+	inline const char *Codename() const {return S->Codename ==0?0:Owner->StrP + S->Codename;};
+	inline const char *Label() const {return S->Label == 0?0:Owner->StrP + S->Label;};
+	inline const char *Site() const {return S->Site == 0?0:Owner->StrP + S->Site;};
+	inline const char *Architecture() const {return S->Architecture == 0?0:Owner->StrP + S->Architecture;};
+	inline const char *IndexType() const {return S->IndexType == 0?0:Owner->StrP + S->IndexType;};
+
+	bool IsOk();
+	string RelStr();
+
+	// Constructors
+	inline PkgFileIterator() : Iterator<PackageFile, PkgFileIterator>() {};
+	inline PkgFileIterator(pkgCache &Owner) : Iterator<PackageFile, PkgFileIterator>(Owner, Owner.PkgFileP) {};
+	inline PkgFileIterator(pkgCache &Owner,PackageFile *Trg) : Iterator<PackageFile, PkgFileIterator>(Owner, Trg) {};
 };
 									/*}}}*/
 // Version File								/*{{{*/
-class pkgCache::VerFileIterator
-{
-   pkgCache *Owner;
-   VerFile *FileP;
-
-   public:
-
-   // Iteration
-   void operator ++(int) {if (FileP != Owner->VerFileP) FileP = Owner->VerFileP + FileP->NextFile;};
-   inline void operator ++() {operator ++(0);};
-   inline bool end() const {return Owner == 0 || FileP == Owner->VerFileP?true:false;};
-
-   // Comparison
-   inline bool operator ==(const VerFileIterator &B) const {return FileP == B.FileP;};
-   inline bool operator !=(const VerFileIterator &B) const {return FileP != B.FileP;};
-			   
-   // Accessors
-   inline VerFile *operator ->() {return FileP;};
-   inline VerFile const *operator ->() const {return FileP;};
-   inline VerFile const &operator *() const {return *FileP;};
-   inline operator VerFile *() {return FileP == Owner->VerFileP?0:FileP;};
-   inline operator VerFile const *() const {return FileP == Owner->VerFileP?0:FileP;};
-   inline pkgCache *Cache() {return Owner;};
-  
-   inline PkgFileIterator File() const {return PkgFileIterator(*Owner,FileP->File + Owner->PkgFileP);};
-   inline unsigned long Index() const {return FileP - Owner->VerFileP;};
-      
-   inline VerFileIterator() : Owner(0), FileP(0) {};
-   inline VerFileIterator(pkgCache &Owner,VerFile *Trg) : Owner(&Owner), FileP(Trg) {};
+class pkgCache::VerFileIterator : public pkgCache::Iterator<VerFile, VerFileIterator> {
+	protected:
+	inline VerFile* OwnerPointer() const {
+		return Owner->VerFileP;
+	};
+
+	public:
+	// Iteration
+	void operator ++(int) {if (S != Owner->VerFileP) S = Owner->VerFileP + S->NextFile;};
+	inline void operator ++() {operator ++(0);};
+
+	// Accessors
+	inline PkgFileIterator File() const {return PkgFileIterator(*Owner,S->File + Owner->PkgFileP);};
+
+	inline VerFileIterator() : Iterator<VerFile, VerFileIterator>() {};
+	inline VerFileIterator(pkgCache &Owner,VerFile *Trg) : Iterator<VerFile, VerFileIterator>(Owner, Trg) {};
 };
 									/*}}}*/
 // Description File							/*{{{*/
-class pkgCache::DescFileIterator
-{
-   pkgCache *Owner;
-   DescFile *FileP;
-
-   public:
-
-   // Iteration
-   void operator ++(int) {if (FileP != Owner->DescFileP) FileP = Owner->DescFileP + FileP->NextFile;};
-   inline void operator ++() {operator ++(0);};
-   inline bool end() const {return Owner == 0 ||  FileP == Owner->DescFileP?true:false;};
-
-   // Comparison
-   inline bool operator ==(const DescFileIterator &B) const {return FileP == B.FileP;};
-   inline bool operator !=(const DescFileIterator &B) const {return FileP != B.FileP;};
-			   
-   // Accessors
-   inline DescFile *operator ->() {return FileP;};
-   inline DescFile const *operator ->() const {return FileP;};
-   inline DescFile const &operator *() const {return *FileP;};
-   inline operator DescFile *() {return FileP == Owner->DescFileP?0:FileP;};
-   inline operator DescFile const *() const {return FileP == Owner->DescFileP?0:FileP;};
-   inline pkgCache *Cache() {return Owner;};
-  
-   inline PkgFileIterator File() const {return PkgFileIterator(*Owner,FileP->File + Owner->PkgFileP);};
-   inline unsigned long Index() const {return FileP - Owner->DescFileP;};
-      
-   inline DescFileIterator() : Owner(0), FileP(0) {};
-   inline DescFileIterator(pkgCache &Owner,DescFile *Trg) : Owner(&Owner), FileP(Trg) {};
+class pkgCache::DescFileIterator : public Iterator<DescFile, DescFileIterator> {
+	protected:
+	inline DescFile* OwnerPointer() const {
+		return Owner->DescFileP;
+	};
+
+	public:
+	// Iteration
+	void operator ++(int) {if (S != Owner->DescFileP) S = Owner->DescFileP + S->NextFile;};
+	inline void operator ++() {operator ++(0);};
+
+	// Accessors
+	inline PkgFileIterator File() const {return PkgFileIterator(*Owner,S->File + Owner->PkgFileP);};
+
+	inline DescFileIterator() : Iterator<DescFile, DescFileIterator>() {};
+	inline DescFileIterator(pkgCache &Owner,DescFile *Trg) : Iterator<DescFile, DescFileIterator>(Owner, Trg) {};
 };
 									/*}}}*/
 // Inlined Begin functions cant be in the class because of order problems /*{{{*/
+inline pkgCache::PkgIterator pkgCache::GrpIterator::PackageList() const
+       {return PkgIterator(*Owner,Owner->PkgP + S->FirstPackage);};
 inline pkgCache::VerIterator pkgCache::PkgIterator::VersionList() const
-       {return VerIterator(*Owner,Owner->VerP + Pkg->VersionList);};
+       {return VerIterator(*Owner,Owner->VerP + S->VersionList);};
 inline pkgCache::VerIterator pkgCache::PkgIterator::CurrentVer() const
-       {return VerIterator(*Owner,Owner->VerP + Pkg->CurrentVer);};
+       {return VerIterator(*Owner,Owner->VerP + S->CurrentVer);};
 inline pkgCache::DepIterator pkgCache::PkgIterator::RevDependsList() const
-       {return DepIterator(*Owner,Owner->DepP + Pkg->RevDepends,Pkg);};
+       {return DepIterator(*Owner,Owner->DepP + S->RevDepends,S);};
 inline pkgCache::PrvIterator pkgCache::PkgIterator::ProvidesList() const
-       {return PrvIterator(*Owner,Owner->ProvideP + Pkg->ProvidesList,Pkg);};
+       {return PrvIterator(*Owner,Owner->ProvideP + S->ProvidesList,S);};
 inline pkgCache::DescIterator pkgCache::VerIterator::DescriptionList() const
-       {return DescIterator(*Owner,Owner->DescP + Ver->DescriptionList);};
+       {return DescIterator(*Owner,Owner->DescP + S->DescriptionList);};
 inline pkgCache::PrvIterator pkgCache::VerIterator::ProvidesList() const
-       {return PrvIterator(*Owner,Owner->ProvideP + Ver->ProvidesList,Ver);};
+       {return PrvIterator(*Owner,Owner->ProvideP + S->ProvidesList,S);};
 inline pkgCache::DepIterator pkgCache::VerIterator::DependsList() const
-       {return DepIterator(*Owner,Owner->DepP + Ver->DependsList,Ver);};
+       {return DepIterator(*Owner,Owner->DepP + S->DependsList,S);};
 inline pkgCache::VerFileIterator pkgCache::VerIterator::FileList() const
-       {return VerFileIterator(*Owner,Owner->VerFileP + Ver->FileList);};
+       {return VerFileIterator(*Owner,Owner->VerFileP + S->FileList);};
 inline pkgCache::DescFileIterator pkgCache::DescIterator::FileList() const
-       {return DescFileIterator(*Owner,Owner->DescFileP + Desc->FileList);};
+       {return DescFileIterator(*Owner,Owner->DescFileP + S->FileList);};
 									/*}}}*/
 #endif

apt-pkg/cacheset.cc

@@ -0,0 +1,511 @@
+// -*- mode: cpp; mode: fold -*-
+// Description								/*{{{*/
+/* ######################################################################
+
+   Simple wrapper around a std::set to provide a similar interface to
+   a set of cache structures as to the complete set of all structures
+   in the pkgCache. Currently only Package is supported.
+
+   ##################################################################### */
+									/*}}}*/
+// Include Files							/*{{{*/
+#include <apt-pkg/aptconfiguration.h>
+#include <apt-pkg/cachefilter.h>
+#include <apt-pkg/cacheset.h>
+#include <apt-pkg/error.h>
+#include <apt-pkg/strutl.h>
+#include <apt-pkg/versionmatch.h>
+
+#include <apti18n.h>
+
+#include <vector>
+
+#include <regex.h>
+									/*}}}*/
+namespace APT {
+// FromTask - Return all packages in the cache from a specific task	/*{{{*/
+PackageSet PackageSet::FromTask(pkgCacheFile &Cache, std::string pattern, CacheSetHelper &helper) {
+	size_t const archfound = pattern.find_last_of(':');
+	std::string arch = "native";
+	if (archfound != std::string::npos) {
+		arch = pattern.substr(archfound+1);
+		pattern.erase(archfound);
+	}
+
+	if (pattern[pattern.length() -1] != '^')
+		return APT::PackageSet(TASK);
+	pattern.erase(pattern.length()-1);
+
+	if (unlikely(Cache.GetPkgCache() == 0 || Cache.GetDepCache() == 0))
+		return APT::PackageSet(TASK);
+
+	PackageSet pkgset(TASK);
+	// get the records
+	pkgRecords Recs(Cache);
+
+	// build regexp for the task
+	regex_t Pattern;
+	char S[300];
+	snprintf(S, sizeof(S), "^Task:.*[, ]%s([, ]|$)", pattern.c_str());
+	if(regcomp(&Pattern,S, REG_EXTENDED | REG_NOSUB | REG_NEWLINE) != 0) {
+		_error->Error("Failed to compile task regexp");
+		return pkgset;
+	}
+
+	for (pkgCache::GrpIterator Grp = Cache->GrpBegin(); Grp.end() == false; ++Grp) {
+		pkgCache::PkgIterator Pkg = Grp.FindPkg(arch);
+		if (Pkg.end() == true)
+			continue;
+		pkgCache::VerIterator ver = Cache[Pkg].CandidateVerIter(Cache);
+		if(ver.end() == true)
+			continue;
+
+		pkgRecords::Parser &parser = Recs.Lookup(ver.FileList());
+		const char *start, *end;
+		parser.GetRec(start,end);
+		unsigned int const length = end - start;
+		char buf[length];
+		strncpy(buf, start, length);
+		buf[length-1] = '\0';
+		if (regexec(&Pattern, buf, 0, 0, 0) != 0)
+			continue;
+
+		pkgset.insert(Pkg);
+	}
+	regfree(&Pattern);
+
+	if (pkgset.empty() == true)
+		return helper.canNotFindTask(Cache, pattern);
+
+	helper.showTaskSelection(pkgset, pattern);
+	return pkgset;
+}
+									/*}}}*/
+// FromRegEx - Return all packages in the cache matching a pattern	/*{{{*/
+PackageSet PackageSet::FromRegEx(pkgCacheFile &Cache, std::string pattern, CacheSetHelper &helper) {
+	static const char * const isregex = ".?+*|[^$";
+	if (pattern.find_first_of(isregex) == std::string::npos)
+		return PackageSet(REGEX);
+
+	size_t archfound = pattern.find_last_of(':');
+	std::string arch = "native";
+	if (archfound != std::string::npos) {
+		arch = pattern.substr(archfound+1);
+		if (arch.find_first_of(isregex) == std::string::npos)
+			pattern.erase(archfound);
+		else
+			arch = "native";
+	}
+
+	if (unlikely(Cache.GetPkgCache() == 0))
+		return PackageSet(REGEX);
+
+	APT::CacheFilter::PackageNameMatchesRegEx regexfilter(pattern);
+
+	PackageSet pkgset(REGEX);
+	for (pkgCache::GrpIterator Grp = Cache.GetPkgCache()->GrpBegin(); Grp.end() == false; ++Grp) {
+		if (regexfilter(Grp) == false)
+			continue;
+		pkgCache::PkgIterator Pkg = Grp.FindPkg(arch);
+		if (Pkg.end() == true) {
+			if (archfound == std::string::npos) {
+				std::vector<std::string> archs = APT::Configuration::getArchitectures();
+				for (std::vector<std::string>::const_iterator a = archs.begin();
+				     a != archs.end() && Pkg.end() != true; ++a)
+					Pkg = Grp.FindPkg(*a);
+			}
+			if (Pkg.end() == true)
+				continue;
+		}
+
+		pkgset.insert(Pkg);
+	}
+
+	if (pkgset.empty() == true)
+		return helper.canNotFindRegEx(Cache, pattern);
+
+	helper.showRegExSelection(pkgset, pattern);
+	return pkgset;
+}
+									/*}}}*/
+// FromName - Returns the package defined  by this string		/*{{{*/
+pkgCache::PkgIterator PackageSet::FromName(pkgCacheFile &Cache,
+			std::string const &str, CacheSetHelper &helper) {
+	std::string pkg = str;
+	size_t archfound = pkg.find_last_of(':');
+	std::string arch;
+	if (archfound != std::string::npos) {
+		arch = pkg.substr(archfound+1);
+		pkg.erase(archfound);
+	}
+
+	if (Cache.GetPkgCache() == 0)
+		return pkgCache::PkgIterator(Cache, 0);
+
+	pkgCache::PkgIterator Pkg(Cache, 0);
+	if (arch.empty() == true) {
+		pkgCache::GrpIterator Grp = Cache.GetPkgCache()->FindGrp(pkg);
+		if (Grp.end() == false)
+			Pkg = Grp.FindPreferredPkg();
+	} else
+		Pkg = Cache.GetPkgCache()->FindPkg(pkg, arch);
+
+	if (Pkg.end() == true)
+		return helper.canNotFindPkgName(Cache, str);
+	return Pkg;
+}
+									/*}}}*/
+// GroupedFromCommandLine - Return all versions specified on commandline/*{{{*/
+std::map<unsigned short, PackageSet> PackageSet::GroupedFromCommandLine(
+		pkgCacheFile &Cache, const char **cmdline,
+		std::list<PackageSet::Modifier> const &mods,
+		unsigned short const &fallback, CacheSetHelper &helper) {
+	std::map<unsigned short, PackageSet> pkgsets;
+	for (const char **I = cmdline; *I != 0; ++I) {
+		unsigned short modID = fallback;
+		std::string str = *I;
+		bool modifierPresent = false;
+		for (std::list<PackageSet::Modifier>::const_iterator mod = mods.begin();
+		     mod != mods.end(); ++mod) {
+			size_t const alength = strlen(mod->Alias);
+			switch(mod->Pos) {
+			case PackageSet::Modifier::POSTFIX:
+				if (str.compare(str.length() - alength, alength,
+						mod->Alias, 0, alength) != 0)
+					continue;
+				str.erase(str.length() - alength);
+				modID = mod->ID;
+				break;
+			case PackageSet::Modifier::PREFIX:
+				continue;
+			case PackageSet::Modifier::NONE:
+				continue;
+			}
+			modifierPresent = true;
+			break;
+		}
+		if (modifierPresent == true) {
+			bool const errors = helper.showErrors(false);
+			pkgCache::PkgIterator Pkg = FromName(Cache, *I, helper);
+			helper.showErrors(errors);
+			if (Pkg.end() == false) {
+				pkgsets[fallback].insert(Pkg);
+				continue;
+			}
+		}
+		pkgsets[modID].insert(PackageSet::FromString(Cache, str, helper));
+	}
+	return pkgsets;
+}
+									/*}}}*/
+// FromCommandLine - Return all packages specified on commandline	/*{{{*/
+PackageSet PackageSet::FromCommandLine(pkgCacheFile &Cache, const char **cmdline, CacheSetHelper &helper) {
+	PackageSet pkgset;
+	for (const char **I = cmdline; *I != 0; ++I) {
+		PackageSet pset = FromString(Cache, *I, helper);
+		pkgset.insert(pset.begin(), pset.end());
+	}
+	return pkgset;
+}
+									/*}}}*/
+// FromString - Return all packages matching a specific string		/*{{{*/
+PackageSet PackageSet::FromString(pkgCacheFile &Cache, std::string const &str, CacheSetHelper &helper) {
+	_error->PushToStack();
+
+	PackageSet pkgset;
+	pkgCache::PkgIterator Pkg = FromName(Cache, str, helper);
+	if (Pkg.end() == false)
+		pkgset.insert(Pkg);
+	else {
+		pkgset = FromTask(Cache, str, helper);
+		if (pkgset.empty() == true) {
+			pkgset = FromRegEx(Cache, str, helper);
+			if (pkgset.empty() == true)
+				pkgset = helper.canNotFindPackage(Cache, str);
+		}
+	}
+
+	if (pkgset.empty() == false)
+		_error->RevertToStack();
+	else
+		_error->MergeWithStack();
+	return pkgset;
+}
+									/*}}}*/
+// GroupedFromCommandLine - Return all versions specified on commandline/*{{{*/
+std::map<unsigned short, VersionSet> VersionSet::GroupedFromCommandLine(
+		pkgCacheFile &Cache, const char **cmdline,
+		std::list<VersionSet::Modifier> const &mods,
+		unsigned short const &fallback, CacheSetHelper &helper) {
+	std::map<unsigned short, VersionSet> versets;
+	for (const char **I = cmdline; *I != 0; ++I) {
+		unsigned short modID = fallback;
+		VersionSet::Version select = VersionSet::NEWEST;
+		std::string str = *I;
+		bool modifierPresent = false;
+		for (std::list<VersionSet::Modifier>::const_iterator mod = mods.begin();
+		     mod != mods.end(); ++mod) {
+			if (modID == fallback && mod->ID == fallback)
+				select = mod->SelectVersion;
+			size_t const alength = strlen(mod->Alias);
+			switch(mod->Pos) {
+			case VersionSet::Modifier::POSTFIX:
+				if (str.compare(str.length() - alength, alength,
+						mod->Alias, 0, alength) != 0)
+					continue;
+				str.erase(str.length() - alength);
+				modID = mod->ID;
+				select = mod->SelectVersion;
+				break;
+			case VersionSet::Modifier::PREFIX:
+				continue;
+			case VersionSet::Modifier::NONE:
+				continue;
+			}
+			modifierPresent = true;
+			break;
+		}
+
+		if (modifierPresent == true) {
+			bool const errors = helper.showErrors(false);
+			VersionSet const vset = VersionSet::FromString(Cache, std::string(*I), select, helper, true);
+			helper.showErrors(errors);
+			if (vset.empty() == false) {
+				versets[fallback].insert(vset);
+				continue;
+			}
+		}
+		versets[modID].insert(VersionSet::FromString(Cache, str, select , helper));
+	}
+	return versets;
+}
+									/*}}}*/
+// FromCommandLine - Return all versions specified on commandline	/*{{{*/
+APT::VersionSet VersionSet::FromCommandLine(pkgCacheFile &Cache, const char **cmdline,
+		APT::VersionSet::Version const &fallback, CacheSetHelper &helper) {
+	VersionSet verset;
+	for (const char **I = cmdline; *I != 0; ++I)
+		verset.insert(VersionSet::FromString(Cache, *I, fallback, helper));
+	return verset;
+}
+									/*}}}*/
+// FromString - Returns all versions spedcified by a string		/*{{{*/
+APT::VersionSet VersionSet::FromString(pkgCacheFile &Cache, std::string pkg,
+		APT::VersionSet::Version const &fallback, CacheSetHelper &helper,
+		bool const &onlyFromName) {
+	std::string ver;
+	bool verIsRel = false;
+	size_t const vertag = pkg.find_last_of("/=");
+	if (vertag != string::npos) {
+		ver = pkg.substr(vertag+1);
+		verIsRel = (pkg[vertag] == '/');
+		pkg.erase(vertag);
+	}
+	PackageSet pkgset;
+	if (onlyFromName == false)
+		pkgset = PackageSet::FromString(Cache, pkg, helper);
+	else {
+		pkgset.insert(PackageSet::FromName(Cache, pkg, helper));
+	}
+
+	VersionSet verset;
+	bool errors = true;
+	if (pkgset.getConstructor() != PackageSet::UNKNOWN)
+		errors = helper.showErrors(false);
+	for (PackageSet::const_iterator P = pkgset.begin();
+	     P != pkgset.end(); ++P) {
+		if (vertag == string::npos) {
+			verset.insert(VersionSet::FromPackage(Cache, P, fallback, helper));
+			continue;
+		}
+		pkgCache::VerIterator V;
+		if (ver == "installed")
+			V = getInstalledVer(Cache, P, helper);
+		else if (ver == "candidate")
+			V = getCandidateVer(Cache, P, helper);
+		else if (ver == "newest") {
+			if (P->VersionList != 0)
+				V = P.VersionList();
+			else
+				V = helper.canNotFindNewestVer(Cache, P);
+		} else {
+			pkgVersionMatch Match(ver, (verIsRel == true ? pkgVersionMatch::Release :
+					pkgVersionMatch::Version));
+			V = Match.Find(P);
+			if (V.end() == true) {
+				if (verIsRel == true)
+					_error->Error(_("Release '%s' for '%s' was not found"),
+							ver.c_str(), P.FullName(true).c_str());
+				else
+					_error->Error(_("Version '%s' for '%s' was not found"),
+							ver.c_str(), P.FullName(true).c_str());
+				continue;
+			}
+		}
+		if (V.end() == true)
+			continue;
+		helper.showSelectedVersion(P, V, ver, verIsRel);
+		verset.insert(V);
+	}
+	if (pkgset.getConstructor() != PackageSet::UNKNOWN)
+		helper.showErrors(errors);
+	return verset;
+}
+									/*}}}*/
+// FromPackage - versions from package based on fallback		/*{{{*/
+VersionSet VersionSet::FromPackage(pkgCacheFile &Cache, pkgCache::PkgIterator const &P,
+		VersionSet::Version const &fallback, CacheSetHelper &helper) {
+	VersionSet verset;
+	pkgCache::VerIterator V;
+	bool showErrors;
+	switch(fallback) {
+	case VersionSet::ALL:
+		if (P->VersionList != 0)
+			for (V = P.VersionList(); V.end() != true; ++V)
+				verset.insert(V);
+		else
+			verset.insert(helper.canNotFindAllVer(Cache, P));
+		break;
+	case VersionSet::CANDANDINST:
+		verset.insert(getInstalledVer(Cache, P, helper));
+		verset.insert(getCandidateVer(Cache, P, helper));
+		break;
+	case VersionSet::CANDIDATE:
+		verset.insert(getCandidateVer(Cache, P, helper));
+		break;
+	case VersionSet::INSTALLED:
+		verset.insert(getInstalledVer(Cache, P, helper));
+		break;
+	case VersionSet::CANDINST:
+		showErrors = helper.showErrors(false);
+		V = getCandidateVer(Cache, P, helper);
+		if (V.end() == true)
+			V = getInstalledVer(Cache, P, helper);
+		helper.showErrors(showErrors);
+		if (V.end() == false)
+			verset.insert(V);
+		else
+			verset.insert(helper.canNotFindInstCandVer(Cache, P));
+		break;
+	case VersionSet::INSTCAND:
+		showErrors = helper.showErrors(false);
+		V = getInstalledVer(Cache, P, helper);
+		if (V.end() == true)
+			V = getCandidateVer(Cache, P, helper);
+		helper.showErrors(showErrors);
+		if (V.end() == false)
+			verset.insert(V);
+		else
+			verset.insert(helper.canNotFindInstCandVer(Cache, P));
+		break;
+	case VersionSet::NEWEST:
+		if (P->VersionList != 0)
+			verset.insert(P.VersionList());
+		else
+			verset.insert(helper.canNotFindNewestVer(Cache, P));
+		break;
+	}
+	return verset;
+}
+									/*}}}*/
+// getCandidateVer - Returns the candidate version of the given package	/*{{{*/
+pkgCache::VerIterator VersionSet::getCandidateVer(pkgCacheFile &Cache,
+		pkgCache::PkgIterator const &Pkg, CacheSetHelper &helper) {
+	pkgCache::VerIterator Cand;
+	if (Cache.IsPolicyBuilt() == true || Cache.IsDepCacheBuilt() == false)
+	{
+		if (unlikely(Cache.GetPolicy() == 0))
+			return pkgCache::VerIterator(Cache);
+		Cand = Cache.GetPolicy()->GetCandidateVer(Pkg);
+	} else {
+		Cand = Cache[Pkg].CandidateVerIter(Cache);
+	}
+	if (Cand.end() == true)
+		return helper.canNotFindCandidateVer(Cache, Pkg);
+	return Cand;
+}
+									/*}}}*/
+// getInstalledVer - Returns the installed version of the given package	/*{{{*/
+pkgCache::VerIterator VersionSet::getInstalledVer(pkgCacheFile &Cache,
+		pkgCache::PkgIterator const &Pkg, CacheSetHelper &helper) {
+	if (Pkg->CurrentVer == 0)
+		return helper.canNotFindInstalledVer(Cache, Pkg);
+	return Pkg.CurrentVer();
+}
+									/*}}}*/
+// canNotFindPkgName - handle the case no package has this name		/*{{{*/
+pkgCache::PkgIterator CacheSetHelper::canNotFindPkgName(pkgCacheFile &Cache,
+			std::string const &str) {
+	if (ShowError == true)
+		_error->Insert(ErrorType, _("Unable to locate package %s"), str.c_str());
+	return pkgCache::PkgIterator(Cache, 0);
+}
+									/*}}}*/
+// canNotFindTask - handle the case no package is found for a task	/*{{{*/
+PackageSet CacheSetHelper::canNotFindTask(pkgCacheFile &Cache, std::string pattern) {
+	if (ShowError == true)
+		_error->Insert(ErrorType, _("Couldn't find task '%s'"), pattern.c_str());
+	return PackageSet();
+}
+									/*}}}*/
+// canNotFindRegEx - handle the case no package is found by a regex	/*{{{*/
+PackageSet CacheSetHelper::canNotFindRegEx(pkgCacheFile &Cache, std::string pattern) {
+	if (ShowError == true)
+		_error->Insert(ErrorType, _("Couldn't find any package by regex '%s'"), pattern.c_str());
+	return PackageSet();
+}
+									/*}}}*/
+// canNotFindPackage - handle the case no package is found from a string/*{{{*/
+PackageSet CacheSetHelper::canNotFindPackage(pkgCacheFile &Cache, std::string const &str) {
+	return PackageSet();
+}
+									/*}}}*/
+// canNotFindAllVer							/*{{{*/
+VersionSet CacheSetHelper::canNotFindAllVer(pkgCacheFile &Cache,
+		pkgCache::PkgIterator const &Pkg) {
+	if (ShowError == true)
+		_error->Insert(ErrorType, _("Can't select versions from package '%s' as it purely virtual"), Pkg.FullName(true).c_str());
+	return VersionSet();
+}
+									/*}}}*/
+// canNotFindInstCandVer						/*{{{*/
+VersionSet CacheSetHelper::canNotFindInstCandVer(pkgCacheFile &Cache,
+		pkgCache::PkgIterator const &Pkg) {
+	if (ShowError == true)
+		_error->Insert(ErrorType, _("Can't select installed nor candidate version from package '%s' as it has neither of them"), Pkg.FullName(true).c_str());
+	return VersionSet();
+}
+									/*}}}*/
+// canNotFindInstCandVer						/*{{{*/
+VersionSet CacheSetHelper::canNotFindCandInstVer(pkgCacheFile &Cache,
+		pkgCache::PkgIterator const &Pkg) {
+	if (ShowError == true)
+		_error->Insert(ErrorType, _("Can't select installed nor candidate version from package '%s' as it has neither of them"), Pkg.FullName(true).c_str());
+	return VersionSet();
+}
+									/*}}}*/
+// canNotFindNewestVer							/*{{{*/
+pkgCache::VerIterator CacheSetHelper::canNotFindNewestVer(pkgCacheFile &Cache,
+		pkgCache::PkgIterator const &Pkg) {
+	if (ShowError == true)
+		_error->Insert(ErrorType, _("Can't select newest version from package '%s' as it is purely virtual"), Pkg.FullName(true).c_str());
+	return pkgCache::VerIterator(Cache, 0);
+}
+									/*}}}*/
+// canNotFindCandidateVer						/*{{{*/
+pkgCache::VerIterator CacheSetHelper::canNotFindCandidateVer(pkgCacheFile &Cache,
+		pkgCache::PkgIterator const &Pkg) {
+	if (ShowError == true)
+		_error->Insert(ErrorType, _("Can't select candidate version from package %s as it has no candidate"), Pkg.FullName(true).c_str());
+	return pkgCache::VerIterator(Cache, 0);
+}
+									/*}}}*/
+// canNotFindInstalledVer						/*{{{*/
+pkgCache::VerIterator CacheSetHelper::canNotFindInstalledVer(pkgCacheFile &Cache,
+		pkgCache::PkgIterator const &Pkg) {
+	if (ShowError == true)
+		_error->Insert(ErrorType, _("Can't select installed version from package %s as it is not installed"), Pkg.FullName(true).c_str());
+	return pkgCache::VerIterator(Cache, 0);
+}
+									/*}}}*/
+}

apt-pkg/cacheset.h

@@ -0,0 +1,389 @@
+// -*- mode: cpp; mode: fold -*-
+// Description								/*{{{*/
+/** \file cacheset.h
+   Wrappers around std::set to have set::iterators which behave
+   similar to the Iterators of the cache structures.
+
+   Provides also a few helper methods which work with these sets */
+									/*}}}*/
+#ifndef APT_CACHESET_H
+#define APT_CACHESET_H
+// Include Files							/*{{{*/
+#include <iostream>
+#include <fstream>
+#include <list>
+#include <map>
+#include <set>
+#include <string>
+
+#include <apt-pkg/cachefile.h>
+#include <apt-pkg/pkgcache.h>
+									/*}}}*/
+namespace APT {
+class PackageSet;
+class VersionSet;
+class CacheSetHelper {							/*{{{*/
+/** \class APT::CacheSetHelper
+    Simple base class with a lot of virtual methods which can be overridden
+    to alter the behavior or the output of the CacheSets.
+
+    This helper is passed around by the static methods in the CacheSets and
+    used every time they hit an error condition or something could be
+    printed out.
+*/
+public:									/*{{{*/
+	CacheSetHelper(bool const &ShowError = true,
+		GlobalError::MsgType ErrorType = GlobalError::ERROR) :
+			ShowError(ShowError), ErrorType(ErrorType) {};
+	virtual ~CacheSetHelper() {};
+
+	virtual void showTaskSelection(PackageSet const &pkgset, string const &pattern) {};
+	virtual void showRegExSelection(PackageSet const &pkgset, string const &pattern) {};
+	virtual void showSelectedVersion(pkgCache::PkgIterator const &Pkg, pkgCache::VerIterator const Ver,
+				 string const &ver, bool const &verIsRel) {};
+
+	virtual pkgCache::PkgIterator canNotFindPkgName(pkgCacheFile &Cache, std::string const &str);
+	virtual PackageSet canNotFindTask(pkgCacheFile &Cache, std::string pattern);
+	virtual PackageSet canNotFindRegEx(pkgCacheFile &Cache, std::string pattern);
+	virtual PackageSet canNotFindPackage(pkgCacheFile &Cache, std::string const &str);
+	virtual VersionSet canNotFindAllVer(pkgCacheFile &Cache, pkgCache::PkgIterator const &Pkg);
+	virtual VersionSet canNotFindInstCandVer(pkgCacheFile &Cache,
+				pkgCache::PkgIterator const &Pkg);
+	virtual VersionSet canNotFindCandInstVer(pkgCacheFile &Cache,
+				pkgCache::PkgIterator const &Pkg);
+	virtual pkgCache::VerIterator canNotFindNewestVer(pkgCacheFile &Cache,
+				pkgCache::PkgIterator const &Pkg);
+	virtual pkgCache::VerIterator canNotFindCandidateVer(pkgCacheFile &Cache,
+				pkgCache::PkgIterator const &Pkg);
+	virtual pkgCache::VerIterator canNotFindInstalledVer(pkgCacheFile &Cache,
+				pkgCache::PkgIterator const &Pkg);
+
+	bool showErrors() const { return ShowError; };
+	bool showErrors(bool const &newValue) { if (ShowError == newValue) return ShowError; else return ((ShowError = newValue) == false); };
+	GlobalError::MsgType errorType() const { return ErrorType; };
+	GlobalError::MsgType errorType(GlobalError::MsgType const &newValue)
+	{
+		if (ErrorType == newValue) return ErrorType;
+		else {
+			GlobalError::MsgType const &oldValue = ErrorType;
+			ErrorType = newValue;
+			return oldValue;
+		}
+	};
+
+									/*}}}*/
+protected:
+	bool ShowError;
+	GlobalError::MsgType ErrorType;
+};									/*}}}*/
+class PackageSet : public std::set<pkgCache::PkgIterator> {		/*{{{*/
+/** \class APT::PackageSet
+
+    Simple wrapper around a std::set to provide a similar interface to
+    a set of packages as to the complete set of all packages in the
+    pkgCache. */
+public:									/*{{{*/
+	/** \brief smell like a pkgCache::PkgIterator */
+	class const_iterator : public std::set<pkgCache::PkgIterator>::const_iterator {/*{{{*/
+	public:
+		const_iterator(std::set<pkgCache::PkgIterator>::const_iterator x) :
+			 std::set<pkgCache::PkgIterator>::const_iterator(x) {}
+
+		operator pkgCache::PkgIterator(void) { return **this; }
+
+		inline const char *Name() const {return (**this).Name(); }
+		inline std::string FullName(bool const &Pretty) const { return (**this).FullName(Pretty); }
+		inline std::string FullName() const { return (**this).FullName(); }
+		inline const char *Section() const {return (**this).Section(); }
+		inline bool Purge() const {return (**this).Purge(); }
+		inline const char *Arch() const {return (**this).Arch(); }
+		inline pkgCache::GrpIterator Group() const { return (**this).Group(); }
+		inline pkgCache::VerIterator VersionList() const { return (**this).VersionList(); }
+		inline pkgCache::VerIterator CurrentVer() const { return (**this).CurrentVer(); }
+		inline pkgCache::DepIterator RevDependsList() const { return (**this).RevDependsList(); }
+		inline pkgCache::PrvIterator ProvidesList() const { return (**this).ProvidesList(); }
+		inline pkgCache::PkgIterator::OkState State() const { return (**this).State(); }
+		inline const char *CandVersion() const { return (**this).CandVersion(); }
+		inline const char *CurVersion() const { return (**this).CurVersion(); }
+		inline pkgCache *Cache() const { return (**this).Cache(); };
+		inline unsigned long Index() const {return (**this).Index();};
+		// we have only valid iterators here
+		inline bool end() const { return false; };
+
+		friend std::ostream& operator<<(std::ostream& out, const_iterator i) { return operator<<(out, (*i)); }
+
+		inline pkgCache::Package const * operator->() const {
+			return &***this;
+		};
+	};
+	// 103. set::iterator is required to be modifiable, but this allows modification of keys
+	typedef APT::PackageSet::const_iterator iterator;
+									/*}}}*/
+
+	using std::set<pkgCache::PkgIterator>::insert;
+	inline void insert(pkgCache::PkgIterator const &P) { if (P.end() == false) std::set<pkgCache::PkgIterator>::insert(P); };
+	inline void insert(PackageSet const &pkgset) { insert(pkgset.begin(), pkgset.end()); };
+
+	/** \brief returns all packages in the cache who belong to the given task
+
+	    A simple helper responsible for search for all members of a task
+	    in the cache. Optional it prints a a notice about the
+	    packages chosen cause of the given task.
+	    \param Cache the packages are in
+	    \param pattern name of the task
+	    \param helper responsible for error and message handling */
+	static APT::PackageSet FromTask(pkgCacheFile &Cache, std::string pattern, CacheSetHelper &helper);
+	static APT::PackageSet FromTask(pkgCacheFile &Cache, std::string const &pattern) {
+		CacheSetHelper helper;
+		return APT::PackageSet::FromTask(Cache, pattern, helper);
+	}
+
+	/** \brief returns all packages in the cache whose name matchs a given pattern
+
+	    A simple helper responsible for executing a regular expression on all
+	    package names in the cache. Optional it prints a a notice about the
+	    packages chosen cause of the given package.
+	    \param Cache the packages are in
+	    \param pattern regular expression for package names
+	    \param helper responsible for error and message handling */
+	static APT::PackageSet FromRegEx(pkgCacheFile &Cache, std::string pattern, CacheSetHelper &helper);
+	static APT::PackageSet FromRegEx(pkgCacheFile &Cache, std::string const &pattern) {
+		CacheSetHelper helper;
+		return APT::PackageSet::FromRegEx(Cache, pattern, helper);
+	}
+
+	/** \brief returns all packages specified by a string
+
+	    \param Cache the packages are in
+	    \param string String the package name(s) should be extracted from
+	    \param helper responsible for error and message handling */
+	static APT::PackageSet FromString(pkgCacheFile &Cache, std::string const &string, CacheSetHelper &helper);
+	static APT::PackageSet FromString(pkgCacheFile &Cache, std::string const &string) {
+		CacheSetHelper helper;
+		return APT::PackageSet::FromString(Cache, string, helper);
+	}
+
+	/** \brief returns a package specified by a string
+
+	    \param Cache the package is in
+	    \param string String the package name should be extracted from
+	    \param helper responsible for error and message handling */
+	static pkgCache::PkgIterator FromName(pkgCacheFile &Cache, std::string const &string, CacheSetHelper &helper);
+	static pkgCache::PkgIterator FromName(pkgCacheFile &Cache, std::string const &string) {
+		CacheSetHelper helper;
+		return APT::PackageSet::FromName(Cache, string, helper);
+	}
+
+	/** \brief returns all packages specified on the commandline
+
+	    Get all package names from the commandline and executes regex's if needed.
+	    No special package command is supported, just plain names.
+	    \param Cache the packages are in
+	    \param cmdline Command line the package names should be extracted from
+	    \param helper responsible for error and message handling */
+	static APT::PackageSet FromCommandLine(pkgCacheFile &Cache, const char **cmdline, CacheSetHelper &helper);
+	static APT::PackageSet FromCommandLine(pkgCacheFile &Cache, const char **cmdline) {
+		CacheSetHelper helper;
+		return APT::PackageSet::FromCommandLine(Cache, cmdline, helper);
+	}
+
+	struct Modifier {
+		enum Position { NONE, PREFIX, POSTFIX };
+		unsigned short ID;
+		const char * const Alias;
+		Position Pos;
+		Modifier (unsigned short const &id, const char * const alias, Position const &pos) : ID(id), Alias(alias), Pos(pos) {};
+	};
+
+	/** \brief group packages by a action modifiers
+
+	    At some point it is needed to get from the same commandline
+	    different package sets grouped by a modifier. Take
+		apt-get install apt awesome-
+	    as an example.
+	    \param Cache the packages are in
+	    \param cmdline Command line the package names should be extracted from
+	    \param mods list of modifiers the method should accept
+	    \param fallback the default modifier group for a package
+	    \param helper responsible for error and message handling */
+	static std::map<unsigned short, PackageSet> GroupedFromCommandLine(
+		pkgCacheFile &Cache, const char **cmdline,
+		std::list<PackageSet::Modifier> const &mods,
+		unsigned short const &fallback, CacheSetHelper &helper);
+	static std::map<unsigned short, PackageSet> GroupedFromCommandLine(
+		pkgCacheFile &Cache, const char **cmdline,
+		std::list<PackageSet::Modifier> const &mods,
+		unsigned short const &fallback) {
+		CacheSetHelper helper;
+		return APT::PackageSet::GroupedFromCommandLine(Cache, cmdline,
+				mods, fallback, helper);
+	}
+
+	enum Constructor { UNKNOWN, REGEX, TASK };
+	Constructor getConstructor() const { return ConstructedBy; };
+
+	PackageSet() : ConstructedBy(UNKNOWN) {};
+	PackageSet(Constructor const &by) : ConstructedBy(by) {};
+									/*}}}*/
+private:								/*{{{*/
+	Constructor ConstructedBy;
+									/*}}}*/
+};									/*}}}*/
+class VersionSet : public std::set<pkgCache::VerIterator> {		/*{{{*/
+/** \class APT::VersionSet
+
+    Simple wrapper around a std::set to provide a similar interface to
+    a set of versions as to the complete set of all versions in the
+    pkgCache. */
+public:									/*{{{*/
+	/** \brief smell like a pkgCache::VerIterator */
+	class const_iterator : public std::set<pkgCache::VerIterator>::const_iterator {/*{{{*/
+	public:
+		const_iterator(std::set<pkgCache::VerIterator>::const_iterator x) :
+			 std::set<pkgCache::VerIterator>::const_iterator(x) {}
+
+		operator pkgCache::VerIterator(void) { return **this; }
+
+		inline pkgCache *Cache() const { return (**this).Cache(); };
+		inline unsigned long Index() const {return (**this).Index();};
+		// we have only valid iterators here
+		inline bool end() const { return false; };
+
+		inline pkgCache::Version const * operator->() const {
+			return &***this;
+		};
+
+		inline int CompareVer(const pkgCache::VerIterator &B) const { return (**this).CompareVer(B); };
+		inline const char *VerStr() const { return (**this).VerStr(); };
+		inline const char *Section() const { return (**this).Section(); };
+		inline const char *Arch() const { return (**this).Arch(); };
+		inline const char *Arch(bool const pseudo) const { return (**this).Arch(pseudo); };
+		inline pkgCache::PkgIterator ParentPkg() const { return (**this).ParentPkg(); };
+		inline pkgCache::DescIterator DescriptionList() const { return (**this).DescriptionList(); };
+		inline pkgCache::DescIterator TranslatedDescription() const { return (**this).TranslatedDescription(); };
+		inline pkgCache::DepIterator DependsList() const { return (**this).DependsList(); };
+		inline pkgCache::PrvIterator ProvidesList() const { return (**this).ProvidesList(); };
+		inline pkgCache::VerFileIterator FileList() const { return (**this).FileList(); };
+		inline bool Downloadable() const { return (**this).Downloadable(); };
+		inline const char *PriorityType() const { return (**this).PriorityType(); };
+		inline string RelStr() const { return (**this).RelStr(); };
+		inline bool Automatic() const { return (**this).Automatic(); };
+		inline bool Pseudo() const { return (**this).Pseudo(); };
+		inline pkgCache::VerFileIterator NewestFile() const { return (**this).NewestFile(); };
+	};
+									/*}}}*/
+	// 103. set::iterator is required to be modifiable, but this allows modification of keys
+	typedef APT::VersionSet::const_iterator iterator;
+
+	using std::set<pkgCache::VerIterator>::insert;
+	inline void insert(pkgCache::VerIterator const &V) { if (V.end() == false) std::set<pkgCache::VerIterator>::insert(V); };
+	inline void insert(VersionSet const &verset) { insert(verset.begin(), verset.end()); };
+
+	/** \brief specifies which version(s) will be returned if non is given */
+	enum Version {
+		/** All versions */
+		ALL,
+		/** Candidate and installed version */
+		CANDANDINST,
+		/** Candidate version */
+		CANDIDATE,
+		/** Installed version */
+		INSTALLED,
+		/** Candidate or if non installed version */
+		CANDINST,
+		/** Installed or if non candidate version */
+		INSTCAND,
+		/** Newest version */
+		NEWEST
+	};
+
+	/** \brief returns all versions specified on the commandline
+
+	    Get all versions from the commandline, uses given default version if
+	    non specifically requested  and executes regex's if needed on names.
+	    \param Cache the packages and versions are in
+	    \param cmdline Command line the versions should be extracted from
+	    \param helper responsible for error and message handling */
+	static APT::VersionSet FromCommandLine(pkgCacheFile &Cache, const char **cmdline,
+			APT::VersionSet::Version const &fallback, CacheSetHelper &helper);
+	static APT::VersionSet FromCommandLine(pkgCacheFile &Cache, const char **cmdline,
+			APT::VersionSet::Version const &fallback) {
+		CacheSetHelper helper;
+		return APT::VersionSet::FromCommandLine(Cache, cmdline, fallback, helper);
+	}
+	static APT::VersionSet FromCommandLine(pkgCacheFile &Cache, const char **cmdline) {
+		return APT::VersionSet::FromCommandLine(Cache, cmdline, CANDINST);
+	}
+
+	static APT::VersionSet FromString(pkgCacheFile &Cache, std::string pkg,
+			APT::VersionSet::Version const &fallback, CacheSetHelper &helper,
+			bool const &onlyFromName = false);
+	static APT::VersionSet FromString(pkgCacheFile &Cache, std::string pkg,
+			APT::VersionSet::Version const &fallback) {
+		CacheSetHelper helper;
+		return APT::VersionSet::FromString(Cache, pkg, fallback, helper);
+	}
+	static APT::VersionSet FromString(pkgCacheFile &Cache, std::string pkg) {
+		return APT::VersionSet::FromString(Cache, pkg, CANDINST);
+	}
+
+	/** \brief returns all versions specified for the package
+
+	    \param Cache the package and versions are in
+	    \param P the package in question
+	    \param fallback the version(s) you want to get
+	    \param helper the helper used for display and error handling */
+	static APT::VersionSet FromPackage(pkgCacheFile &Cache, pkgCache::PkgIterator const &P,
+		VersionSet::Version const &fallback, CacheSetHelper &helper);
+	static APT::VersionSet FromPackage(pkgCacheFile &Cache, pkgCache::PkgIterator const &P,
+			APT::VersionSet::Version const &fallback) {
+		CacheSetHelper helper;
+		return APT::VersionSet::FromPackage(Cache, P, fallback, helper);
+	}
+	static APT::VersionSet FromPackage(pkgCacheFile &Cache, pkgCache::PkgIterator const &P) {
+		return APT::VersionSet::FromPackage(Cache, P, CANDINST);
+	}
+
+	struct Modifier {
+		enum Position { NONE, PREFIX, POSTFIX };
+		unsigned short ID;
+		const char * const Alias;
+		Position Pos;
+		VersionSet::Version SelectVersion;
+		Modifier (unsigned short const &id, const char * const alias, Position const &pos,
+			  VersionSet::Version const &select) : ID(id), Alias(alias), Pos(pos),
+			 SelectVersion(select) {};
+	};
+
+	static std::map<unsigned short, VersionSet> GroupedFromCommandLine(
+		pkgCacheFile &Cache, const char **cmdline,
+		std::list<VersionSet::Modifier> const &mods,
+		unsigned short const &fallback, CacheSetHelper &helper);
+	static std::map<unsigned short, VersionSet> GroupedFromCommandLine(
+		pkgCacheFile &Cache, const char **cmdline,
+		std::list<VersionSet::Modifier> const &mods,
+		unsigned short const &fallback) {
+		CacheSetHelper helper;
+		return APT::VersionSet::GroupedFromCommandLine(Cache, cmdline,
+				mods, fallback, helper);
+	}
+									/*}}}*/
+protected:								/*{{{*/
+
+	/** \brief returns the candidate version of the package
+
+	    \param Cache to be used to query for information
+	    \param Pkg we want the candidate version from this package */
+	static pkgCache::VerIterator getCandidateVer(pkgCacheFile &Cache,
+		pkgCache::PkgIterator const &Pkg, CacheSetHelper &helper);
+
+	/** \brief returns the installed version of the package
+
+	    \param Cache to be used to query for information
+	    \param Pkg we want the installed version from this package */
+	static pkgCache::VerIterator getInstalledVer(pkgCacheFile &Cache,
+		pkgCache::PkgIterator const &Pkg, CacheSetHelper &helper);
+									/*}}}*/
+};									/*}}}*/
+}
+#endif

apt-pkg/cdrom.cc

@@ -6,6 +6,8 @@
 #include<apt-pkg/cdromutl.h>
 #include<apt-pkg/strutl.h>
 #include<apt-pkg/cdrom.h>
+#include<apt-pkg/aptconfiguration.h>
+
 #include<sstream>
 #include<fstream>
 #include<config.h>
@@ -216,33 +218,23 @@ int pkgCdrom::Score(string Path)
 /* Here we drop everything that is not this machines arch */
 bool pkgCdrom::DropBinaryArch(vector<string> &List)
 {
-   char S[300];
-   snprintf(S,sizeof(S),"/binary-%s/",
-	    _config->Find("Apt::Architecture").c_str());
-   
+
    for (unsigned int I = 0; I < List.size(); I++)
    {
       const char *Str = List[I].c_str();
-      
-      const char *Res;
-      if ((Res = strstr(Str,"/binary-")) == 0)
+      const char *Start, *End;
+      if ((Start = strstr(Str,"/binary-")) == 0)
 	 continue;
 
-      // Weird, remove it.
-      if (strlen(Res) < strlen(S))
-      {
-	 List.erase(List.begin() + I);
-	 I--;
-	 continue;
-      }
-	  
-      // See if it is our arch
-      if (stringcmp(Res,Res + strlen(S),S) == 0)
-	 continue;
-      
-      // Erase it
+      // Between Start and End is the architecture
+      Start += 8;
+      if ((End = strstr(Start,"/")) != 0 && Start != End &&
+          APT::Configuration::checkArchitecture(string(Start, End)) == true)
+	 continue; // okay, architecture is accepted
+
+      // not accepted -> Erase it
       List.erase(List.begin() + I);
-      I--;
+      --I; // the next entry is at the same index after the erase
    }
    
    return true;
@@ -289,7 +281,8 @@ bool pkgCdrom::DropRepeats(vector<string> &List,const char *Name)
 	 List[J] = string();
       }
    }  
- 
+   delete[] Inodes;
+
    // Wipe erased entries
    for (unsigned int I = 0; I < List.size();)
    {
@@ -390,7 +383,7 @@ bool pkgCdrom::WriteDatabase(Configuration &Cnf)
 
    Out.close();
    
-   rename(DFile.c_str(),string(DFile + '~').c_str());
+   link(DFile.c_str(),string(DFile + '~').c_str());
    if (rename(NewFile.c_str(),DFile.c_str()) != 0)
       return _error->Errno("rename","Failed to rename %s.new to %s",
 			   DFile.c_str(),DFile.c_str());

apt-pkg/clean.cc

@@ -12,6 +12,7 @@
 #include <apt-pkg/strutl.h>
 #include <apt-pkg/error.h>
 #include <apt-pkg/configuration.h>
+#include <apt-pkg/aptconfiguration.h>
 
 #include <apti18n.h>    
 
@@ -26,7 +27,6 @@
 bool pkgArchiveCleaner::Go(string Dir,pkgCache &Cache)
 {
    bool CleanInstalled = _config->FindB("APT::Clean-Installed",true);
-   string MyArch = _config->Find("APT::Architecture");
       
    DIR *D = opendir(Dir.c_str());
    if (D == 0)
@@ -75,9 +75,9 @@ bool pkgArchiveCleaner::Go(string Dir,pkgCache &Cache)
       for (I = Start; *I != 0 && *I != '.' ;I++);
       if (*I != '.')
 	 continue;
-      string Arch = DeQuoteString(string(Start,I-Start));
+      string const Arch = DeQuoteString(string(Start,I-Start));
       
-      if (Arch != "all" && Arch != MyArch)
+      if (APT::Configuration::checkArchitecture(Arch) == false)
 	 continue;
       
       // Lookup the package

apt-pkg/contrib/cmndline.cc

@@ -135,7 +135,9 @@ bool CommandLine::Parse(int argc,const char **argv)
    for (; I != argc; I++)
       *Files++ = argv[I];
    *Files = 0;
-   
+
+   SaveInConfig(argc, argv);
+
    return true;
 }
 									/*}}}*/
@@ -351,3 +353,41 @@ bool CommandLine::DispatchArg(Dispatch *Map,bool NoMatch)
    return false;
 }
 									/*}}}*/
+// CommandLine::SaveInConfig - for output later in a logfile or so	/*{{{*/
+// ---------------------------------------------------------------------
+/* We save the commandline here to have it around later for e.g. logging.
+   It feels a bit like a hack here and isn't bulletproof, but it is better
+   than nothing after all. */
+void CommandLine::SaveInConfig(unsigned int const &argc, char const * const * const argv)
+{
+   char cmdline[100 + argc * 50];
+   unsigned int length = 0;
+   bool lastWasOption = false;
+   bool closeQuote = false;
+   for (unsigned int i = 0; i < argc && length < sizeof(cmdline); ++i, ++length)
+   {
+      for (unsigned int j = 0; argv[i][j] != '\0' && length < sizeof(cmdline)-1; ++j, ++length)
+      {
+	 cmdline[length] = argv[i][j];
+	 if (lastWasOption == true && argv[i][j] == '=')
+	 {
+	    // That is possibly an option: Quote it if it includes spaces,
+	    // the benefit is that this will eliminate also most false positives
+	    const char* c = &argv[i][j+1];
+	    for (; *c != '\0' && *c != ' '; ++c);
+	    if (*c == '\0') continue;
+	    cmdline[++length] = '"';
+	    closeQuote = true;
+	 }
+      }
+      if (closeQuote == true)
+	 cmdline[length++] = '"';
+      // Problem: detects also --hello
+      if (cmdline[length-1] == 'o')
+	 lastWasOption = true;
+      cmdline[length] = ' ';
+   }
+   cmdline[--length] = '\0';
+   _config->Set("CommandLine::AsString", cmdline);
+}
+									/*}}}*/

apt-pkg/contrib/cmndline.h

@@ -60,6 +60,7 @@ class CommandLine
    Configuration *Conf;
    bool HandleOpt(int &I,int argc,const char *argv[],
 		  const char *&Opt,Args *A,bool PreceedeMatch = false);
+   void static SaveInConfig(unsigned int const &argc, char const * const * const argv);
 
    public:
    

apt-pkg/contrib/configuration.cc

@@ -79,7 +79,7 @@ Configuration::~Configuration()
 /* This will lookup a single item by name below another item. It is a 
    helper function for the main lookup function */
 Configuration::Item *Configuration::Lookup(Item *Head,const char *S,
-					   unsigned long Len,bool Create)
+					   unsigned long const &Len,bool const &Create)
 {
    int Res = 1;
    Item *I = Head->Child;
@@ -112,7 +112,7 @@ Configuration::Item *Configuration::Lookup(Item *Head,const char *S,
 // ---------------------------------------------------------------------
 /* This performs a fully scoped lookup of a given name, possibly creating
    new items */
-Configuration::Item *Configuration::Lookup(const char *Name,bool Create)
+Configuration::Item *Configuration::Lookup(const char *Name,bool const &Create)
 {
    if (Name == 0)
       return Root->Child;
@@ -239,7 +239,7 @@ vector<string> Configuration::FindVector(const char *Name) const
 // Configuration::FindI - Find an integer value				/*{{{*/
 // ---------------------------------------------------------------------
 /* */
-int Configuration::FindI(const char *Name,int Default) const
+int Configuration::FindI(const char *Name,int const &Default) const
 {
    const Item *Itm = Lookup(Name);
    if (Itm == 0 || Itm->Value.empty() == true)
@@ -256,7 +256,7 @@ int Configuration::FindI(const char *Name,int Default) const
 // Configuration::FindB - Find a boolean type				/*{{{*/
 // ---------------------------------------------------------------------
 /* */
-bool Configuration::FindB(const char *Name,bool Default) const
+bool Configuration::FindB(const char *Name,bool const &Default) const
 {
    const Item *Itm = Lookup(Name);
    if (Itm == 0 || Itm->Value.empty() == true)
@@ -332,7 +332,7 @@ void Configuration::Set(const char *Name,const string &Value)
 // Configuration::Set - Set an integer value				/*{{{*/
 // ---------------------------------------------------------------------
 /* */
-void Configuration::Set(const char *Name,int Value)
+void Configuration::Set(const char *Name,int const &Value)
 {
    Item *Itm = Lookup(Name,true);
    if (Itm == 0)
@@ -345,7 +345,7 @@ void Configuration::Set(const char *Name,int Value)
 // Configuration::Clear - Clear an single value from a list	        /*{{{*/
 // ---------------------------------------------------------------------
 /* */
-void Configuration::Clear(const string Name, int Value)
+void Configuration::Clear(string const &Name, int const &Value)
 {
    char S[300];
    snprintf(S,sizeof(S),"%i",Value);
@@ -355,7 +355,7 @@ void Configuration::Clear(const string Name, int Value)
 // Configuration::Clear - Clear an single value from a list	        /*{{{*/
 // ---------------------------------------------------------------------
 /* */
-void Configuration::Clear(const string Name, string Value)
+void Configuration::Clear(string const &Name, string const &Value)
 {
    Item *Top = Lookup(Name.c_str(),false);
    if (Top == 0 || Top->Child == 0)
@@ -386,7 +386,7 @@ void Configuration::Clear(const string Name, string Value)
 // Configuration::Clear - Clear an entire tree				/*{{{*/
 // ---------------------------------------------------------------------
 /* */
-void Configuration::Clear(string Name)
+void Configuration::Clear(string const &Name)
 {
    Item *Top = Lookup(Name.c_str(),false);
    if (Top == 0) 
@@ -501,8 +501,8 @@ string Configuration::Item::FullTag(const Item *Stop) const
    sections like 'zone "foo.org" { .. };' This causes each section to be
    added in with a tag like "zone::foo.org" instead of being split 
    tag/value. AsSectional enables Sectional parsing.*/
-bool ReadConfigFile(Configuration &Conf,const string &FName,bool AsSectional,
-		    unsigned Depth)
+bool ReadConfigFile(Configuration &Conf,const string &FName,bool const &AsSectional,
+		    unsigned const &Depth)
 {   
    // Open the stream for reading
    ifstream F(FName.c_str(),ios::in); 
@@ -773,6 +773,8 @@ bool ReadConfigFile(Configuration &Conf,const string &FName,bool AsSectional,
 	       else
 		  return _error->Error(_("Syntax error %s:%u: Unsupported directive '%s'"),FName.c_str(),CurLine,Tag.c_str());
 	    }
+	    else if (Tag.empty() == true && NoWord == false && Word == "#clear")
+	       return _error->Error(_("Syntax error %s:%u: clear directive requires an option tree as argument"),FName.c_str(),CurLine);
 	    else
 	    {
 	       // Set the item in the configuration class
@@ -830,7 +832,7 @@ bool ReadConfigFile(Configuration &Conf,const string &FName,bool AsSectional,
 // ---------------------------------------------------------------------
 /* */
 bool ReadConfigDir(Configuration &Conf,const string &Dir,
-		   bool AsSectional, unsigned Depth)
+		   bool const &AsSectional, unsigned const &Depth)
 {
    vector<string> const List = GetListOfFilesInDir(Dir, "conf", true, true);
 
@@ -841,3 +843,54 @@ bool ReadConfigDir(Configuration &Conf,const string &Dir,
    return true;
 }
 									/*}}}*/
+// MatchAgainstConfig Constructor					/*{{{*/
+Configuration::MatchAgainstConfig::MatchAgainstConfig(char const * Config)
+{
+   std::vector<std::string> const strings = _config->FindVector(Config);
+   for (std::vector<std::string>::const_iterator s = strings.begin();
+	s != strings.end(); ++s)
+   {
+      regex_t *p = new regex_t;
+      if (regcomp(p, s->c_str(), REG_EXTENDED | REG_ICASE | REG_NOSUB) == 0)
+	 patterns.push_back(p);
+      else
+      {
+	 regfree(p);
+	 delete p;
+	 clearPatterns();
+	 _error->Warning("Regex compilation error for '%s' in configuration option '%s'",
+				s->c_str(), Config);
+	 return;
+      }
+   }
+   if (strings.size() == 0)
+      patterns.push_back(NULL);
+}
+									/*}}}*/
+// MatchAgainstConfig Destructor					/*{{{*/
+Configuration::MatchAgainstConfig::~MatchAgainstConfig()
+{
+   clearPatterns();
+}
+void Configuration::MatchAgainstConfig::clearPatterns()
+{
+   for(std::vector<regex_t *>::const_iterator p = patterns.begin();
+	p != patterns.end(); ++p)
+   {
+      if (*p == NULL) continue;
+      regfree(*p);
+      delete *p;
+   }
+}
+									/*}}}*/
+// MatchAgainstConfig::Match - returns true if a pattern matches	/*{{{*/
+bool Configuration::MatchAgainstConfig::Match(char const * str) const
+{
+   for(std::vector<regex_t *>::const_iterator p = patterns.begin();
+	p != patterns.end(); ++p)
+      if (*p != NULL && regexec(*p, str, 0, 0, 0) == 0)
+	 return true;
+
+   return false;
+}
+									/*}}}*/

apt-pkg/contrib/configuration.h

@@ -28,7 +28,7 @@
 #ifndef PKGLIB_CONFIGURATION_H
 #define PKGLIB_CONFIGURATION_H
 
-
+#include <regex.h>
 
 #include <string>
 #include <vector>
@@ -58,8 +58,8 @@ class Configuration
    Item *Root;
    bool ToFree;
    
-   Item *Lookup(Item *Head,const char *S,unsigned long Len,bool Create);
-   Item *Lookup(const char *Name,bool Create);   
+   Item *Lookup(Item *Head,const char *S,unsigned long const &Len,bool const &Create);
+   Item *Lookup(const char *Name,const bool &Create);
    inline const Item *Lookup(const char *Name) const
    {
       return ((Configuration *)this)->Lookup(Name,false);
@@ -68,32 +68,33 @@ class Configuration
    public:
 
    string Find(const char *Name,const char *Default = 0) const;
-   string Find(const string Name,const char *Default = 0) const {return Find(Name.c_str(),Default);};
+   string Find(string const &Name,const char *Default = 0) const {return Find(Name.c_str(),Default);};
+   string Find(string const &Name, string const &Default) const {return Find(Name.c_str(),Default.c_str());};
    string FindFile(const char *Name,const char *Default = 0) const;
    string FindDir(const char *Name,const char *Default = 0) const;
-   std::vector<string> FindVector(const string &Name) const;
+   std::vector<string> FindVector(string const &Name) const;
    std::vector<string> FindVector(const char *Name) const;
-   int FindI(const char *Name,int Default = 0) const;
-   int FindI(const string Name,int Default = 0) const {return FindI(Name.c_str(),Default);};
-   bool FindB(const char *Name,bool Default = false) const;
-   bool FindB(const string Name,bool Default = false) const {return FindB(Name.c_str(),Default);};
+   int FindI(const char *Name,int const &Default = 0) const;
+   int FindI(string const &Name,int const &Default = 0) const {return FindI(Name.c_str(),Default);};
+   bool FindB(const char *Name,bool const &Default = false) const;
+   bool FindB(string const &Name,bool const &Default = false) const {return FindB(Name.c_str(),Default);};
    string FindAny(const char *Name,const char *Default = 0) const;
 	      
-   inline void Set(const string Name,string Value) {Set(Name.c_str(),Value);};
+   inline void Set(const string &Name,const string &Value) {Set(Name.c_str(),Value);};
    void CndSet(const char *Name,const string &Value);
    void Set(const char *Name,const string &Value);
-   void Set(const char *Name,int Value);   
+   void Set(const char *Name,const int &Value);
    
-   inline bool Exists(const string Name) const {return Exists(Name.c_str());};
+   inline bool Exists(const string &Name) const {return Exists(Name.c_str());};
    bool Exists(const char *Name) const;
    bool ExistsAny(const char *Name) const;
 
    // clear a whole tree
-   void Clear(const string Name);
+   void Clear(const string &Name);
 
    // remove a certain value from a list (e.g. the list of "APT::Keep-Fds")
-   void Clear(const string List, string Value);
-   void Clear(const string List, int Value);
+   void Clear(string const &List, string const &Value);
+   void Clear(string const &List, int const &Value);
 
    inline const Item *Tree(const char *Name) const {return Lookup(Name);};
 
@@ -103,16 +104,34 @@ class Configuration
    Configuration(const Item *Root);
    Configuration();
    ~Configuration();
+
+   /** \brief match a string against a configurable list of patterns */
+   class MatchAgainstConfig
+   {
+     std::vector<regex_t *> patterns;
+     void clearPatterns();
+
+   public:
+     MatchAgainstConfig(char const * Config);
+     virtual ~MatchAgainstConfig();
+
+     /** \brief Returns \b true for a string matching one of the patterns */
+     bool Match(char const * str) const;
+     bool Match(std::string const &str) const { return Match(str.c_str()); };
+
+     /** \brief returns if the matcher setup was successful */
+     bool wasConstructedSuccessfully() const { return patterns.empty() == false; }
+   };
 };
 
 extern Configuration *_config;
 
 bool ReadConfigFile(Configuration &Conf,const string &FName,
-		    bool AsSectional = false,
-		    unsigned Depth = 0);
+		    bool const &AsSectional = false,
+		    unsigned const &Depth = 0);
 
 bool ReadConfigDir(Configuration &Conf,const string &Dir,
-		   bool AsSectional = false,
-		   unsigned Depth = 0);
+		   bool const &AsSectional = false,
+		   unsigned const &Depth = 0);
 
 #endif

apt-pkg/contrib/error.cc

@@ -1,16 +1,15 @@
 // -*- mode: cpp; mode: fold -*-
 // Description								/*{{{*/
-// $Id: error.cc,v 1.11 2002/03/26 07:38:58 jgg Exp $
 /* ######################################################################
-   
-   Global Erorr Class - Global error mechanism
+
+   Global Error Class - Global error mechanism
 
    We use a simple STL vector to store each error record. A PendingFlag
    is kept which indicates when the vector contains a Sever error.
-   
+
    This source is placed in the Public Domain, do with it what you will
    It was originally written by Jason Gunthorpe.
-   
+
    ##################################################################### */
 									/*}}}*/
 // Include Files							/*{{{*/
@@ -19,7 +18,6 @@
 #include <iostream>
 #include <errno.h>
 #include <stdio.h>
-#include <stdarg.h>
 #include <unistd.h>
 
 #include <string>
@@ -28,209 +26,233 @@
 #include "config.h"
    									/*}}}*/
 
-using namespace std;
-
 // Global Error Object							/*{{{*/
 /* If the implementation supports posix threads then the accessor function
    is compiled to be thread safe otherwise a non-safe version is used. A
    Per-Thread error object is maintained in much the same manner as libc
    manages errno */
 #if defined(_POSIX_THREADS) && defined(HAVE_PTHREAD)
- #include <pthread.h>
-
- static pthread_key_t ErrorKey;
- static void ErrorDestroy(void *Obj) {delete (GlobalError *)Obj;};
- static void KeyAlloc() {pthread_key_create(&ErrorKey,ErrorDestroy);};
-
- GlobalError *_GetErrorObj()
- {
-    static pthread_once_t Once = PTHREAD_ONCE_INIT;
-    pthread_once(&Once,KeyAlloc);
-    
-    void *Res = pthread_getspecific(ErrorKey);
-    if (Res == 0)
-       pthread_setspecific(ErrorKey,Res = new GlobalError);
-    return (GlobalError *)Res;
- }
+	#include <pthread.h>
+
+	static pthread_key_t ErrorKey;
+	static void ErrorDestroy(void *Obj) {delete (GlobalError *)Obj;};
+	static void KeyAlloc() {pthread_key_create(&ErrorKey,ErrorDestroy);};
+
+	GlobalError *_GetErrorObj() {
+		static pthread_once_t Once = PTHREAD_ONCE_INIT;
+		pthread_once(&Once,KeyAlloc);
+
+		void *Res = pthread_getspecific(ErrorKey);
+		if (Res == 0)
+			pthread_setspecific(ErrorKey,Res = new GlobalError);
+		return (GlobalError *)Res;
+	}
 #else
- GlobalError *_GetErrorObj()
- {
-    static GlobalError *Obj = new GlobalError;
-    return Obj;
- }
+	GlobalError *_GetErrorObj() {
+		static GlobalError *Obj = new GlobalError;
+		return Obj;
+	}
 #endif
 									/*}}}*/
-
 // GlobalError::GlobalError - Constructor				/*{{{*/
-// ---------------------------------------------------------------------
-/* */
-GlobalError::GlobalError() : List(0), PendingFlag(false)
-{
+GlobalError::GlobalError() : PendingFlag(false) {}
+									/*}}}*/
+// GlobalError::FatalE - Get part of the error string from errno	/*{{{*/
+bool GlobalError::FatalE(const char *Function,const char *Description,...) {
+	va_list args;
+	va_start(args,Description);
+	return InsertErrno(FATAL, Function, Description, args);
 }
 									/*}}}*/
 // GlobalError::Errno - Get part of the error string from errno		/*{{{*/
-// ---------------------------------------------------------------------
-/* Function indicates the stdlib function that failed and Description is
-   a user string that leads the text. Form is:
-     Description - Function (errno: strerror)
-   Carefull of the buffer overrun, sprintf.
- */
-bool GlobalError::Errno(const char *Function,const char *Description,...)
+bool GlobalError::Errno(const char *Function,const char *Description,...) {
+	va_list args;
+	va_start(args,Description);
+	return InsertErrno(ERROR, Function, Description, args);
+}
+									/*}}}*/
+// GlobalError::WarningE - Get part of the warning string from errno	/*{{{*/
+bool GlobalError::WarningE(const char *Function,const char *Description,...) {
+	va_list args;
+	va_start(args,Description);
+	return InsertErrno(WARNING, Function, Description, args);
+}
+									/*}}}*/
+// GlobalError::NoticeE - Get part of the notice string from errno	/*{{{*/
+bool GlobalError::NoticeE(const char *Function,const char *Description,...) {
+	va_list args;
+	va_start(args,Description);
+	return InsertErrno(NOTICE, Function, Description, args);
+}
+									/*}}}*/
+// GlobalError::DebugE - Get part of the debug string from errno	/*{{{*/
+bool GlobalError::DebugE(const char *Function,const char *Description,...) {
+	va_list args;
+	va_start(args,Description);
+	return InsertErrno(DEBUG, Function, Description, args);
+}
+									/*}}}*/
+// GlobalError::InsertErrno - Get part of the errortype string from errno/*{{{*/
+bool GlobalError::InsertErrno(MsgType const &type, const char *Function,
+				const char *Description,...) {
+	va_list args;
+	va_start(args,Description);
+	return InsertErrno(type, Function, Description, args);
+}
+									/*}}}*/
+// GlobalError::InsertErrno - formats an error message with the errno	/*{{{*/
+bool GlobalError::InsertErrno(MsgType type, const char* Function,
+			      const char* Description, va_list &args) {
+	char S[400];
+	snprintf(S, sizeof(S), "%s - %s (%i: %s)", Description,
+		 Function, errno, strerror(errno));
+	return Insert(type, S, args);
+}
+									/*}}}*/
+// GlobalError::Fatal - Add a fatal error to the list			/*{{{*/
+bool GlobalError::Fatal(const char *Description,...) {
+	va_list args;
+	va_start(args,Description);
+	return Insert(FATAL, Description, args);
+}
+									/*}}}*/
+// GlobalError::Error - Add an error to the list			/*{{{*/
+bool GlobalError::Error(const char *Description,...) {
+	va_list args;
+	va_start(args,Description);
+	return Insert(ERROR, Description, args);
+}
+									/*}}}*/
+// GlobalError::Warning - Add a warning to the list			/*{{{*/
+bool GlobalError::Warning(const char *Description,...) {
+	va_list args;
+	va_start(args,Description);
+	return Insert(WARNING, Description, args);
+}
+									/*}}}*/
+// GlobalError::Notice - Add a notice to the list			/*{{{*/
+bool GlobalError::Notice(const char *Description,...)
+{
+	va_list args;
+	va_start(args,Description);
+	return Insert(NOTICE, Description, args);
+}
+									/*}}}*/
+// GlobalError::Debug - Add a debug to the list				/*{{{*/
+bool GlobalError::Debug(const char *Description,...)
 {
-   va_list args;
-   va_start(args,Description);
-
-   // sprintf the description
-   char S[400];
-   vsnprintf(S,sizeof(S),Description,args);
-   snprintf(S + strlen(S),sizeof(S) - strlen(S),
-	    " - %s (%i: %s)",Function,errno,strerror(errno));
-
-   // Put it on the list
-   Item *Itm = new Item;
-   Itm->Text = S;
-   Itm->Error = true;
-   Insert(Itm);
-
-   PendingFlag = true;
-
-   return false;
-}
-									/*}}}*/
-// GlobalError::WarningE - Get part of the warn string from errno	/*{{{*/
-// ---------------------------------------------------------------------
-/* Function indicates the stdlib function that failed and Description is
-   a user string that leads the text. Form is:
-     Description - Function (errno: strerror)
-   Carefull of the buffer overrun, sprintf.
- */
-bool GlobalError::WarningE(const char *Function,const char *Description,...)
+	va_list args;
+	va_start(args,Description);
+	return Insert(DEBUG, Description, args);
+}
+									/*}}}*/
+// GlobalError::Insert - Add a errotype message to the list		/*{{{*/
+bool GlobalError::Insert(MsgType const &type, const char *Description,...)
 {
-   va_list args;
-   va_start(args,Description);
+	va_list args;
+	va_start(args,Description);
+	return Insert(type, Description, args);
+}
+									/*}}}*/
+// GlobalError::Insert - Insert a new item at the end			/*{{{*/
+bool GlobalError::Insert(MsgType type, const char* Description,
+			 va_list &args) {
+	char S[400];
+	vsnprintf(S,sizeof(S),Description,args);
+
+	Item const m(S, type);
+	Messages.push_back(m);
 
-   // sprintf the description
-   char S[400];
-   vsnprintf(S,sizeof(S),Description,args);
-   snprintf(S + strlen(S),sizeof(S) - strlen(S),
-	    " - %s (%i: %s)",Function,errno,strerror(errno));
+	if (type == ERROR || type == FATAL)
+		PendingFlag = true;
 
-   // Put it on the list
-   Item *Itm = new Item;
-   Itm->Text = S;
-   Itm->Error = false;
-   Insert(Itm);
+	if (type == FATAL || type == DEBUG)
+		std::clog << m << std::endl;
 
-   return false;
+	return false;
 }
 									/*}}}*/
-// GlobalError::Error - Add an error to the list			/*{{{*/
-// ---------------------------------------------------------------------
-/* Just vsprintfs and pushes */
-bool GlobalError::Error(const char *Description,...)
-{
-   va_list args;
-   va_start(args,Description);
+// GlobalError::PopMessage - Pulls a single message out			/*{{{*/
+bool GlobalError::PopMessage(std::string &Text) {
+	if (Messages.empty() == true)
+		return false;
+
+	Item const msg = Messages.front();
+	Messages.pop_front();
 
-   // sprintf the description
-   char S[400];
-   vsnprintf(S,sizeof(S),Description,args);
+	bool const Ret = (msg.Type == ERROR || msg.Type == FATAL);
+	Text = msg.Text;
+	if (PendingFlag == false || Ret == false)
+		return Ret;
 
-   // Put it on the list
-   Item *Itm = new Item;
-   Itm->Text = S;
-   Itm->Error = true;
-   Insert(Itm);
-   
-   PendingFlag = true;
-   
-   return false;
+	// check if another error message is pending
+	for (std::list<Item>::const_iterator m = Messages.begin();
+	     m != Messages.end(); m++)
+		if (m->Type == ERROR || m->Type == FATAL)
+			return Ret;
+
+	PendingFlag = false;
+	return Ret;
 }
 									/*}}}*/
-// GlobalError::Warning - Add a warning to the list			/*{{{*/
-// ---------------------------------------------------------------------
-/* This doesn't set the pending error flag */
-bool GlobalError::Warning(const char *Description,...)
-{
-   va_list args;
-   va_start(args,Description);
+// GlobalError::DumpErrors - Dump all of the errors/warns to cerr	/*{{{*/
+void GlobalError::DumpErrors(std::ostream &out, MsgType const &threshold,
+			     bool const &mergeStack) {
+	if (mergeStack == true)
+		for (std::list<MsgStack>::const_reverse_iterator s = Stacks.rbegin();
+		     s != Stacks.rend(); ++s)
+			Messages.insert(Messages.begin(), s->Messages.begin(), s->Messages.end());
 
-   // sprintf the description
-   char S[400];
-   vsnprintf(S,sizeof(S),Description,args);
+	for (std::list<Item>::const_iterator m = Messages.begin();
+	     m != Messages.end(); m++)
+		if (m->Type >= threshold)
+			out << (*m) << std::endl;
+	Discard();
+}
+									/*}}}*/
+// GlobalError::Discard - Discard					/*{{{*/
+void GlobalError::Discard() {
+	Messages.clear();
+	PendingFlag = false;
+};
+									/*}}}*/
+// GlobalError::empty - does our error list include anything?		/*{{{*/
+bool GlobalError::empty(MsgType const &trashhold) const {
+	if (PendingFlag == true)
+		return false;
 
-   // Put it on the list
-   Item *Itm = new Item;
-   Itm->Text = S;
-   Itm->Error = false;
-   Insert(Itm);
-   
-   return false;
+	if (Messages.empty() == true)
+		return true;
+
+	for (std::list<Item>::const_iterator m = Messages.begin();
+	     m != Messages.end(); m++)
+		if (m->Type >= trashhold)
+			return false;
+
+	return true;
 }
 									/*}}}*/
-// GlobalError::PopMessage - Pulls a single message out			/*{{{*/
-// ---------------------------------------------------------------------
-/* This should be used in a loop checking empty() each cycle. It returns
-   true if the message is an error. */
-bool GlobalError::PopMessage(string &Text)
-{
-   if (List == 0)
-      return false;
-      
-   bool Ret = List->Error;
-   Text = List->Text;
-   Item *Old = List;
-   List = List->Next;
-   delete Old;
-   
-   // This really should check the list to see if only warnings are left..
-   if (List == 0)
-      PendingFlag = false;
-   
-   return Ret;
+// GlobalError::PushToStack						/*{{{*/
+void GlobalError::PushToStack() {
+	MsgStack pack(Messages, PendingFlag);
+	Stacks.push_back(pack);
+	Discard();
 }
 									/*}}}*/
-// GlobalError::DumpErrors - Dump all of the errors/warns to cerr	/*{{{*/
-// ---------------------------------------------------------------------
-/* */
-void GlobalError::DumpErrors()
-{
-   // Print any errors or warnings found
-   string Err;
-   while (empty() == false)
-   {
-      bool Type = PopMessage(Err);
-      if (Type == true)
-	 cerr << "E: " << Err << endl;
-      else
-	 cerr << "W: " << Err << endl;
-   }
-}
-									/*}}}*/
-// GlobalError::Discard - Discard									/*{{{*/
-// ---------------------------------------------------------------------
-/* */
-void GlobalError::Discard()
-{
-   while (List != 0)
-   {
-      Item *Old = List;
-      List = List->Next;
-      delete Old;
-   }
-   
-   PendingFlag = false;
-};
+// GlobalError::RevertToStack						/*{{{*/
+void GlobalError::RevertToStack() {
+	Discard();
+	MsgStack pack = Stacks.back();
+	Messages = pack.Messages;
+	PendingFlag = pack.PendingFlag;
+	Stacks.pop_back();
+}
 									/*}}}*/
-// GlobalError::Insert - Insert a new item at the end			/*{{{*/
-// ---------------------------------------------------------------------
-/* */
-void GlobalError::Insert(Item *Itm)
-{
-   Item **End = &List;
-   for (Item *I = List; I != 0; I = I->Next)
-      End = &I->Next;
-   Itm->Next = *End;
-   *End = Itm;
+// GlobalError::MergeWithStack						/*{{{*/
+void GlobalError::MergeWithStack() {
+	MsgStack pack = Stacks.back();
+	Messages.insert(Messages.begin(), pack.Messages.begin(), pack.Messages.end());
+	PendingFlag = PendingFlag || pack.PendingFlag;
+	Stacks.pop_back();
 }
 									/*}}}*/

apt-pkg/contrib/error.h

@@ -42,43 +42,277 @@
 
 #include <apt-pkg/macros.h>
 
+#include <iostream>
+#include <list>
 #include <string>
 
-class GlobalError
+#include <stdarg.h>
+
+class GlobalError							/*{{{*/
 {
-   struct Item
-   {
-      std::string Text;
-      bool Error;
-      Item *Next;
-   };
-   
-   Item *List;
-   bool PendingFlag;
-   void Insert(Item *I);
-   
-   public:
+public:									/*{{{*/
+	/** \brief a message can have one of following severity */
+	enum MsgType {
+		/** \brief Message will be printed instantly as it is likely that
+			this error will lead to a complete crash */
+		FATAL = 40,
+		/** \brief An error does hinder the correct execution and should be corrected */
+		ERROR = 30,
+		/** \brief indicates problem that can lead to errors later on */
+		WARNING = 20,
+		/** \brief deprecation warnings, old fallback behavior, … */
+		NOTICE = 10,
+		/** \brief for developers only in areas it is hard to print something directly */
+		DEBUG = 0
+	};
 
-   // Call to generate an error from a library call.
-   bool Errno(const char *Function,const char *Description,...) __like_printf_2 __cold;
-   bool WarningE(const char *Function,const char *Description,...) __like_printf_2 __cold;
+	/** \brief add a fatal error message with errno to the list
+	 *
+	 *  \param Function name of the function generating the error
+	 *  \param Description format string for the error message
+	 *
+	 *  \return \b false
+	 */
+	bool FatalE(const char *Function,const char *Description,...) __like_printf(3) __cold;
 
-   /* A warning should be considered less severe than an error, and may be
-      ignored by the client. */
-   bool Error(const char *Description,...) __like_printf_1 __cold;
-   bool Warning(const char *Description,...) __like_printf_1 __cold;
+	/** \brief add an Error message with errno to the list
+	 *
+	 *  \param Function name of the function generating the error
+	 *  \param Description format string for the error message
+	 *
+	 *  \return \b false
+	 */
+	bool Errno(const char *Function,const char *Description,...) __like_printf(3) __cold;
 
-   // Simple accessors
-   inline bool PendingError() {return PendingFlag;};
-   inline bool empty() {return List == 0;};
-   bool PopMessage(std::string &Text);
-   void Discard();
+	/** \brief add a warning message with errno to the list
+	 *
+	 *  A warning should be considered less severe than an error and
+	 *  may be ignored by the client.
+	 *
+	 *  \param Function Name of the function generates the warning.
+	 *  \param Description Format string for the warning message.
+	 *
+	 *  \return \b false
+	 */
+	bool WarningE(const char *Function,const char *Description,...) __like_printf(3) __cold;
 
-   // Usefull routine to dump to cerr
-   void DumpErrors();
-   
-   GlobalError();
+	/** \brief add a notice message with errno to the list
+	 *
+	 *  \param Function name of the function generating the error
+	 *  \param Description format string for the error message
+	 *
+	 *  \return \b false
+	 */
+	bool NoticeE(const char *Function,const char *Description,...) __like_printf(3) __cold;
+
+	/** \brief add a debug message with errno to the list
+	 *
+	 *  \param Function name of the function generating the error
+	 *  \param Description format string for the error message
+	 *
+	 *  \return \b false
+	 */
+	bool DebugE(const char *Function,const char *Description,...) __like_printf(3) __cold;
+
+	/** \brief adds an errno message with the given type
+	 *
+	 * \param type of the error message
+	 * \param Function which failed
+	 * \param Description of the error
+	 */
+	bool InsertErrno(MsgType const &type, const char* Function,
+			 const char* Description,...) __like_printf(4) __cold;
+
+	/** \brief add an fatal error message to the list
+	 *
+	 *  Most of the stuff we consider as "error" is also "fatal" for
+	 *  the user as the application will not have the expected result,
+	 *  but a fatal message here means that it gets printed directly
+	 *  to stderr in addiction to adding it to the list as the error
+	 *  leads sometimes to crashes and a maybe duplicated message
+	 *  is better than "Segfault" as the only displayed text
+	 *
+	 *  \param Description Format string for the fatal error message.
+	 *
+	 *  \return \b false
+	 */
+	bool Fatal(const char *Description,...) __like_printf(2) __cold;
+
+	/** \brief add an Error message to the list
+	 *
+	 *  \param Description Format string for the error message.
+	 *
+	 *  \return \b false
+	 */
+	bool Error(const char *Description,...) __like_printf(2) __cold;
+
+	/** \brief add a warning message to the list
+	 *
+	 *  A warning should be considered less severe than an error and
+	 *  may be ignored by the client.
+	 *
+	 *  \param Description Format string for the message
+	 *
+	 *  \return \b false
+	 */
+	bool Warning(const char *Description,...) __like_printf(2) __cold;
+
+	/** \brief add a notice message to the list
+	 *
+	 *  A notice should be considered less severe than an error or a
+	 *  warning and can be ignored by the client without further problems
+	 *  for some times, but he should consider fixing the problem.
+	 *  This error type can be used for e.g. deprecation warnings of options.
+	 *
+	 *  \param Description Format string for the message
+	 *
+	 *  \return \b false
+	 */
+	bool Notice(const char *Description,...) __like_printf(2) __cold;
+
+	/** \brief add a debug message to the list
+	 *
+	 *  \param Description Format string for the message
+	 *
+	 *  \return \b false
+	 */
+	bool Debug(const char *Description,...) __like_printf(2) __cold;
+
+	/** \brief adds an error message with the given type
+	 *
+	 * \param type of the error message
+	 * \param Description of the error
+	 */
+	bool Insert(MsgType const &type, const char* Description,...) __like_printf(3) __cold;
+
+	/** \brief is an error in the list?
+	 *
+	 *  \return \b true if an error is included in the list, \b false otherwise
+	 */
+	inline bool PendingError() const {return PendingFlag;};
+
+	/** \brief is the list empty?
+	 *
+	 *  The default checks if the list is empty or contains only notices,
+	 *  if you want to check if also no notices happend set the parameter
+	 *  flag to \b false.
+	 *
+	 *  \param WithoutNotice does notices count, default is \b true, so no
+	 *
+	 *  \return \b true if an the list is empty, \b false otherwise
+	 */
+	bool empty(MsgType const &trashhold = WARNING) const;
+
+	/** \brief returns and removes the first (or last) message in the list
+	 *
+	 *  \param[out] Text message of the first/last item
+	 *
+	 *  \return \b true if the message was an error, \b false otherwise
+	 */
+	bool PopMessage(std::string &Text);
+
+	/** \brief clears the list of messages */
+	void Discard();
+
+	/** \brief outputs the list of messages to the given stream
+	 *
+	 *  Note that all messages are discarded, also the notices
+	 *  displayed or not.
+	 *
+	 *  \param[out] out output stream to write the messages in
+	 *  \param threshold minimim level considered
+         *  \param mergeStack 
+	 */
+	void DumpErrors(std::ostream &out, MsgType const &threshold = WARNING,
+			bool const &mergeStack = true);
+
+	/** \brief dumps the list of messages to std::cerr
+	 *
+	 *  Note that all messages are discarded, also the notices
+	 *  displayed or not.
+	 *
+	 *  \param threshold minimum level printed
+	 */
+	void inline DumpErrors(MsgType const &threshold) {
+		DumpErrors(std::cerr, threshold);
+	}
+
+        // mvo: we do this instead of using a default parameter in the
+        //      previous declaration to avoid a (subtle) API break for
+        //      e.g. sigc++ and mem_fun0
+	/** \brief dumps the messages of type WARNING or higher to std::cerr
+	 *
+	 *  Note that all messages are discarded, displayed or not.
+	 *
+	 */
+	void inline DumpErrors() {
+                DumpErrors(WARNING);
+	}
+
+	/** \brief put the current Messages into the stack
+	 *
+	 *  All "old" messages will be pushed into a stack to
+	 *  them later back, but for now the Message query will be
+	 *  empty and performs as no messages were present before.
+	 *
+	 * The stack can be as deep as you want - all stack operations
+	 * will only operate on the last element in the stack.
+	 */
+	void PushToStack();
+
+	/** \brief throw away all current messages */
+	void RevertToStack();
+
+	/** \brief merge current and stack together */
+	void MergeWithStack();
+
+	/** \brief return the deep of the stack */
+	size_t StackCount() const {
+		return Stacks.size();
+	}
+
+	GlobalError();
+									/*}}}*/
+private:								/*{{{*/
+	struct Item {
+		std::string Text;
+		MsgType Type;
+
+		Item(char const *Text, MsgType const &Type) :
+			Text(Text), Type(Type) {};
+
+		friend std::ostream& operator<< (std::ostream &out, Item i) {
+			switch(i.Type) {
+			case FATAL:
+			case ERROR: out << "E"; break;
+			case WARNING: out << "W"; break;
+			case NOTICE: out << "N"; break;
+			case DEBUG: out << "D"; break;
+			}
+			return out << ": " << i.Text;
+		}
+	};
+
+	std::list<Item> Messages;
+	bool PendingFlag;
+
+	struct MsgStack {
+		std::list<Item> const Messages;
+		bool const PendingFlag;
+
+		MsgStack(std::list<Item> const &Messages, bool const &Pending) :
+			 Messages(Messages), PendingFlag(Pending) {};
+	};
+
+	std::list<MsgStack> Stacks;
+
+	bool InsertErrno(MsgType type, const char* Function,
+			 const char* Description, va_list &args);
+	bool Insert(MsgType type, const char* Description,
+			 va_list &args);
+									/*}}}*/
 };
+									/*}}}*/
 
 // The 'extra-ansi' syntax is used to help with collisions. 
 GlobalError *_GetErrorObj();

apt-pkg/contrib/fileutl.cc

@@ -11,6 +11,7 @@
    Most of this source is placed in the Public Domain, do with it what 
    you will
    It was originally written by Jason Gunthorpe <jgg@debian.org>.
+   FileFd gzip support added by Martin Pitt <martin.pitt@canonical.com>
    
    The exception is RunScripts() it is under the GPLv2
 
@@ -18,6 +19,7 @@
 									/*}}}*/
 // Include Files							/*{{{*/
 #include <apt-pkg/fileutl.h>
+#include <apt-pkg/strutl.h>
 #include <apt-pkg/error.h>
 #include <apt-pkg/sptr.h>
 #include <apt-pkg/configuration.h>
@@ -26,6 +28,7 @@
 
 #include <cstdlib>
 #include <cstring>
+#include <cstdio>
 
 #include <iostream>
 #include <unistd.h>
@@ -197,15 +200,82 @@ bool FileExists(string File)
    return true;
 }
 									/*}}}*/
+// DirectoryExists - Check if a directory exists and is really one	/*{{{*/
+// ---------------------------------------------------------------------
+/* */
+bool DirectoryExists(string const &Path)
+{
+   struct stat Buf;
+   if (stat(Path.c_str(),&Buf) != 0)
+      return false;
+   return ((Buf.st_mode & S_IFDIR) != 0);
+}
+									/*}}}*/
+// CreateDirectory - poor man's mkdir -p guarded by a parent directory	/*{{{*/
+// ---------------------------------------------------------------------
+/* This method will create all directories needed for path in good old
+   mkdir -p style but refuses to do this if Parent is not a prefix of
+   this Path. Example: /var/cache/ and /var/cache/apt/archives are given,
+   so it will create apt/archives if /var/cache exists - on the other
+   hand if the parent is /var/lib the creation will fail as this path
+   is not a parent of the path to be generated. */
+bool CreateDirectory(string const &Parent, string const &Path)
+{
+   if (Parent.empty() == true || Path.empty() == true)
+      return false;
+
+   if (DirectoryExists(Path) == true)
+      return true;
+
+   if (DirectoryExists(Parent) == false)
+      return false;
+
+   // we are not going to create directories "into the blue"
+   if (Path.find(Parent, 0) != 0)
+      return false;
+
+   vector<string> const dirs = VectorizeString(Path.substr(Parent.size()), '/');
+   string progress = Parent;
+   for (vector<string>::const_iterator d = dirs.begin(); d != dirs.end(); ++d)
+   {
+      if (d->empty() == true)
+	 continue;
+
+      progress.append("/").append(*d);
+      if (DirectoryExists(progress) == true)
+	 continue;
+
+      if (mkdir(progress.c_str(), 0755) != 0)
+	 return false;
+   }
+   return true;
+}
+									/*}}}*/
+// CheckDirectory - ensure that the given directory exists		/*{{{*/
+// ---------------------------------------------------------------------
+/* a small wrapper around CreateDirectory to check if it exists and to
+   remove the trailing "/apt/" from the parent directory if needed */
+bool CheckDirectory(string const &Parent, string const &Path)
+{
+   if (DirectoryExists(Path) == true)
+      return true;
+
+   size_t const len = Parent.size();
+   if (len > 5 && Parent.find("/apt/", len - 6, 5) == len - 5)
+   {
+      if (CreateDirectory(Parent.substr(0,len-5), Path) == true)
+	 return true;
+   }
+   else if (CreateDirectory(Parent, Path) == true)
+      return true;
+
+   return false;
+}
+									/*}}}*/
 // GetListOfFilesInDir - returns a vector of files in the given dir	/*{{{*/
 // ---------------------------------------------------------------------
 /* If an extension is given only files with this extension are included
    in the returned vector, otherwise every "normal" file is included. */
-std::vector<string> GetListOfFilesInDir(string const &Dir, string const &Ext,
-					bool const &SortList)
-{
-   return GetListOfFilesInDir(Dir, Ext, SortList, false);
-}
 std::vector<string> GetListOfFilesInDir(string const &Dir, string const &Ext,
 					bool const &SortList, bool const &AllowNoExt)
 {
@@ -234,6 +304,7 @@ std::vector<string> GetListOfFilesInDir(string const &Dir, std::vector<string> c
    }
 
    std::vector<string> List;
+   Configuration::MatchAgainstConfig SilentIgnore("Dir::Ignore-Files-Silently");
    DIR *D = opendir(Dir.c_str());
    if (D == 0) 
    {
@@ -259,6 +330,8 @@ std::vector<string> GetListOfFilesInDir(string const &Dir, std::vector<string> c
 	    {
 	       if (Debug == true)
 		  std::clog << "Bad file: " << Ent->d_name << " → no extension" << std::endl;
+	       if (SilentIgnore.Match(Ent->d_name) == false)
+		  _error->Notice("Ignoring file '%s' in directory '%s' as it has no filename extension", Ent->d_name, Dir.c_str());
 	       continue;
 	    }
 	 }
@@ -266,6 +339,8 @@ std::vector<string> GetListOfFilesInDir(string const &Dir, std::vector<string> c
 	 {
 	    if (Debug == true)
 	       std::clog << "Bad file: " << Ent->d_name << " → bad extension »" << flExtension(Ent->d_name) << "«" << std::endl;
+	    if (SilentIgnore.Match(Ent->d_name) == false)
+	       _error->Notice("Ignoring file '%s' in directory '%s' as it has an invalid filename extension", Ent->d_name, Dir.c_str());
 	    continue;
 	 }
       }
@@ -604,10 +679,31 @@ bool FileFd::Open(string FileName,OpenMode Mode, unsigned long Perms)
       case ReadOnly:
       iFd = open(FileName.c_str(),O_RDONLY);
       break;
+
+      case ReadOnlyGzip:
+      iFd = open(FileName.c_str(),O_RDONLY);
+      if (iFd > 0) {
+	 gz = gzdopen (iFd, "r");
+	 if (gz == NULL) {
+	     close (iFd);
+	     iFd = -1;
+	 }
+      }
+      break;
       
+      case WriteAtomic:
+      {
+	 Flags |= Replace;
+	 char *name = strdup((FileName + ".XXXXXX").c_str());
+	 TemporaryFileName = string(mktemp(name));
+	 iFd = open(TemporaryFileName.c_str(),O_RDWR | O_CREAT | O_EXCL,Perms);
+	 free(name);
+	 break;
+      }
+
       case WriteEmpty:
       {
-	 struct stat Buf;
+      	 struct stat Buf;
 	 if (lstat(FileName.c_str(),&Buf) == 0 && S_ISLNK(Buf.st_mode))
 	    unlink(FileName.c_str());
 	 iFd = open(FileName.c_str(),O_RDWR | O_CREAT | O_TRUNC,Perms);
@@ -635,6 +731,24 @@ bool FileFd::Open(string FileName,OpenMode Mode, unsigned long Perms)
    SetCloseExec(iFd,true);
    return true;
 }
+
+bool FileFd::OpenDescriptor(int Fd, OpenMode Mode, bool AutoClose)
+{
+   Close();
+   Flags = (AutoClose) ? FileFd::AutoClose : 0;
+   iFd = Fd;
+   if (Mode == ReadOnlyGzip) {
+      gz = gzdopen (iFd, "r");
+      if (gz == NULL) {
+	 if (AutoClose)
+	    close (iFd);
+	 return _error->Errno("gzdopen",_("Could not open file descriptor %d"),
+			      Fd);
+      }
+   }
+   this->FileName = "";
+   return true;
+}
 									/*}}}*/
 // FileFd::~File - Closes the file					/*{{{*/
 // ---------------------------------------------------------------------
@@ -658,7 +772,10 @@ bool FileFd::Read(void *To,unsigned long Size,unsigned long *Actual)
    
    do
    {
-      Res = read(iFd,To,Size);
+      if (gz != NULL)
+         Res = gzread(gz,To,Size);
+      else
+         Res = read(iFd,To,Size);
       if (Res < 0 && errno == EINTR)
 	 continue;
       if (Res < 0)
@@ -697,7 +814,10 @@ bool FileFd::Write(const void *From,unsigned long Size)
    errno = 0;
    do
    {
-      Res = write(iFd,From,Size);
+      if (gz != NULL)
+         Res = gzwrite(gz,From,Size);
+      else
+         Res = write(iFd,From,Size);
       if (Res < 0 && errno == EINTR)
 	 continue;
       if (Res < 0)
@@ -723,7 +843,12 @@ bool FileFd::Write(const void *From,unsigned long Size)
 /* */
 bool FileFd::Seek(unsigned long To)
 {
-   if (lseek(iFd,To,SEEK_SET) != (signed)To)
+   int res;
+   if (gz)
+      res = gzseek(gz,To,SEEK_SET);
+   else
+      res = lseek(iFd,To,SEEK_SET);
+   if (res != (signed)To)
    {
       Flags |= Fail;
       return _error->Error("Unable to seek to %lu",To);
@@ -737,7 +862,12 @@ bool FileFd::Seek(unsigned long To)
 /* */
 bool FileFd::Skip(unsigned long Over)
 {
-   if (lseek(iFd,Over,SEEK_CUR) < 0)
+   int res;
+   if (gz)
+      res = gzseek(gz,Over,SEEK_CUR);
+   else
+      res = lseek(iFd,Over,SEEK_CUR);
+   if (res < 0)
    {
       Flags |= Fail;
       return _error->Error("Unable to seek ahead %lu",Over);
@@ -751,6 +881,11 @@ bool FileFd::Skip(unsigned long Over)
 /* */
 bool FileFd::Truncate(unsigned long To)
 {
+   if (gz)
+   {
+      Flags |= Fail;
+      return _error->Error("Truncating gzipped files is not implemented (%s)", FileName.c_str());
+   }
    if (ftruncate(iFd,To) != 0)
    {
       Flags |= Fail;
@@ -765,7 +900,11 @@ bool FileFd::Truncate(unsigned long To)
 /* */
 unsigned long FileFd::Tell()
 {
-   off_t Res = lseek(iFd,0,SEEK_CUR);
+   off_t Res;
+   if (gz)
+     Res = gztell(gz);
+   else
+     Res = lseek(iFd,0,SEEK_CUR);
    if (Res == (off_t)-1)
       _error->Errno("lseek","Failed to determine the current file position");
    return Res;
@@ -776,6 +915,7 @@ unsigned long FileFd::Tell()
 /* */
 unsigned long FileFd::Size()
 {
+   //TODO: For gz, do we need the actual file size here or the uncompressed length?
    struct stat Buf;
    if (fstat(iFd,&Buf) != 0)
       return _error->Errno("fstat","Unable to determine the file size");
@@ -789,14 +929,33 @@ bool FileFd::Close()
 {
    bool Res = true;
    if ((Flags & AutoClose) == AutoClose)
-      if (iFd >= 0 && close(iFd) != 0)
-	 Res &= _error->Errno("close",_("Problem closing the file"));
+   {
+      if (gz != NULL) {
+	 int const e = gzclose(gz);
+	 // gzdopen() on empty files always fails with "buffer error" here, ignore that
+	 if (e != 0 && e != Z_BUF_ERROR)
+	    Res &= _error->Errno("close",_("Problem closing the gzip file %s"), FileName.c_str());
+      } else
+	 if (iFd > 0 && close(iFd) != 0)
+	    Res &= _error->Errno("close",_("Problem closing the file %s"), FileName.c_str());
+   }
+
+   if ((Flags & Replace) == Replace && iFd >= 0) {
+      if (rename(TemporaryFileName.c_str(), FileName.c_str()) != 0)
+	 Res &= _error->Errno("rename",_("Problem renaming the file %s to %s"), TemporaryFileName.c_str(), FileName.c_str());
+
+      FileName = TemporaryFileName; // for the unlink() below.
+   }
+
    iFd = -1;
-   
+   gz = NULL;
+
    if ((Flags & Fail) == Fail && (Flags & DelOnFail) == DelOnFail &&
        FileName.empty() == false)
       if (unlink(FileName.c_str()) != 0)
-	 Res &= _error->WarningE("unlnk",_("Problem unlinking the file"));
+	 Res &= _error->WarningE("unlnk",_("Problem unlinking the file %s"), FileName.c_str());
+
+
    return Res;
 }
 									/*}}}*/

apt-pkg/contrib/fileutl.h

@@ -21,10 +21,16 @@
 #ifndef PKGLIB_FILEUTL_H
 #define PKGLIB_FILEUTL_H
 
+#include <apt-pkg/macros.h>
 
 #include <string>
 #include <vector>
 
+#include <zlib.h>
+
+/* Define this for python-apt */
+#define APT_HAS_GZIP 1
+
 using std::string;
 
 class FileFd
@@ -33,12 +39,15 @@ class FileFd
    int iFd;
  
    enum LocalFlags {AutoClose = (1<<0),Fail = (1<<1),DelOnFail = (1<<2),
-                    HitEof = (1<<3)};
+                    HitEof = (1<<3), Replace = (1<<4) };
    unsigned long Flags;
    string FileName;
-   
+   string TemporaryFileName;
+   gzFile gz;
+
    public:
-   enum OpenMode {ReadOnly,WriteEmpty,WriteExists,WriteAny,WriteTemp};
+   enum OpenMode {ReadOnly,WriteEmpty,WriteExists,WriteAny,WriteTemp,ReadOnlyGzip,
+                  WriteAtomic};
    
    inline bool Read(void *To,unsigned long Size,bool AllowEof)
    {
@@ -55,6 +64,7 @@ class FileFd
    unsigned long Tell();
    unsigned long Size();
    bool Open(string FileName,OpenMode Mode,unsigned long Perms = 0666);
+   bool OpenDescriptor(int Fd, OpenMode Mode, bool AutoClose=false);
    bool Close();
    bool Sync();
    
@@ -69,12 +79,12 @@ class FileFd
    inline string &Name() {return FileName;};
    
    FileFd(string FileName,OpenMode Mode,unsigned long Perms = 0666) : iFd(-1), 
-            Flags(0) 
+            Flags(0), gz(NULL)
    {
       Open(FileName,Mode,Perms);
    };
-   FileFd(int Fd = -1) : iFd(Fd), Flags(AutoClose) {};
-   FileFd(int Fd,bool) : iFd(Fd), Flags(0) {};
+   FileFd(int Fd = -1) : iFd(Fd), Flags(AutoClose), gz(NULL) {};
+   FileFd(int Fd,bool) : iFd(Fd), Flags(0), gz(NULL) {};
    virtual ~FileFd();
 };
 
@@ -82,11 +92,19 @@ bool RunScripts(const char *Cnf);
 bool CopyFile(FileFd &From,FileFd &To);
 int GetLock(string File,bool Errors = true);
 bool FileExists(string File);
-// FIXME: next ABI-Break: merge the two method-headers
-std::vector<string> GetListOfFilesInDir(string const &Dir, string const &Ext,
-					bool const &SortList);
+bool DirectoryExists(string const &Path) __attrib_const;
+bool CreateDirectory(string const &Parent, string const &Path);
+
+/** \brief Ensure the existence of the given Path
+ *
+ *  \param Parent directory of the Path directory - a trailing
+ *  /apt/ will be removed before CreateDirectory call.
+ *  \param Path which should exist after (successful) call
+ */
+bool CheckDirectory(string const &Parent, string const &Path);
+
 std::vector<string> GetListOfFilesInDir(string const &Dir, string const &Ext,
-					bool const &SortList, bool const &AllowNoExt);
+					bool const &SortList, bool const &AllowNoExt=false);
 std::vector<string> GetListOfFilesInDir(string const &Dir, std::vector<string> const &Ext,
 					bool const &SortList);
 string SafeGetCWD();

apt-pkg/contrib/macros.h

@@ -56,33 +56,35 @@
 
 // some nice optional GNUC features
 #if __GNUC__ >= 3
-        #define __must_check    __attribute__ ((warn_unused_result))
-        #define __deprecated    __attribute__ ((deprecated))
-        /* likely() and unlikely() can be used to mark boolean expressions
-           as (not) likely true which will help the compiler to optimise */
-        #define likely(x)       __builtin_expect (!!(x), 1)
-        #define unlikely(x)     __builtin_expect (!!(x), 0)
+	#define __must_check	__attribute__ ((warn_unused_result))
+	#define __deprecated	__attribute__ ((deprecated))
+	#define __attrib_const	__attribute__ ((__const__))
+	/* likely() and unlikely() can be used to mark boolean expressions
+	   as (not) likely true which will help the compiler to optimise */
+	#define likely(x)	__builtin_expect (!!(x), 1)
+	#define unlikely(x)	__builtin_expect (!!(x), 0)
 #else
-        #define __must_check    /* no warn_unused_result */
-        #define __deprecated    /* no deprecated */
-        #define likely(x)       (x)
-        #define unlikely(x)     (x)
+	#define __must_check	/* no warn_unused_result */
+	#define __deprecated	/* no deprecated */
+	#define __attrib_const	/* no const attribute */
+	#define likely(x)	(x)
+	#define unlikely(x)	(x)
 #endif
 
 // cold functions are unlikely() to be called
 #if (__GNUC__ == 4 && __GNUC_MINOR__ >= 3) || __GNUC__ > 4
-        #define __cold  __attribute__ ((__cold__))
+	#define __cold	__attribute__ ((__cold__))
+	#define __hot	__attribute__ ((__hot__))
 #else
-        #define __cold  /* no cold marker */
+	#define __cold	/* no cold marker */
+	#define __hot	/* no hot marker */
 #endif
 
 #ifdef __GNUG__
 // Methods have a hidden this parameter that is visible to this attribute
-	#define __like_printf_1 __attribute__ ((format (printf, 2, 3)))
-	#define __like_printf_2 __attribute__ ((format (printf, 3, 4)))
+	#define __like_printf(n)	__attribute__((format(printf, n, n + 1)))
 #else
-	#define __like_printf_1
-	#define __like_printf_2
+	#define __like_printf(n)	/* no like-printf */
 #endif
 
 #endif

apt-pkg/contrib/mmap.cc

@@ -27,6 +27,7 @@
 #include <unistd.h>
 #include <fcntl.h>
 #include <stdlib.h>
+#include <errno.h>
 
 #include <cstring>
    									/*}}}*/
@@ -35,7 +36,7 @@
 // ---------------------------------------------------------------------
 /* */
 MMap::MMap(FileFd &F,unsigned long Flags) : Flags(Flags), iSize(0),
-                     Base(0)
+                     Base(0), SyncToFd(NULL)
 {
    if ((Flags & NoImmMap) != NoImmMap)
       Map(F);
@@ -45,7 +46,7 @@ MMap::MMap(FileFd &F,unsigned long Flags) : Flags(Flags), iSize(0),
 // ---------------------------------------------------------------------
 /* */
 MMap::MMap(unsigned long Flags) : Flags(Flags), iSize(0),
-                     Base(0)
+                     Base(0), SyncToFd(NULL)
 {
 }
 									/*}}}*/
@@ -78,7 +79,24 @@ bool MMap::Map(FileFd &Fd)
    // Map it.
    Base = mmap(0,iSize,Prot,Map,Fd.Fd(),0);
    if (Base == (void *)-1)
-      return _error->Errno("mmap",_("Couldn't make mmap of %lu bytes"),iSize);
+   {
+      if (errno == ENODEV || errno == EINVAL)
+      {
+	 // The filesystem doesn't support this particular kind of mmap.
+	 // So we allocate a buffer and read the whole file into it.
+	 int const dupped_fd = dup(Fd.Fd());
+	 if (dupped_fd == -1)
+	    return _error->Errno("mmap", _("Couldn't duplicate file descriptor %i"), Fd.Fd());
+
+	 Base = new unsigned char[iSize];
+	 SyncToFd = new FileFd (dupped_fd);
+	 if (!SyncToFd->Seek(0L) || !SyncToFd->Read(Base, iSize))
+	    return false;
+      }
+      else
+	 return _error->Errno("mmap",_("Couldn't make mmap of %lu bytes"),
+	                      iSize);
+     }
 
    return true;
 }
@@ -93,10 +111,19 @@ bool MMap::Close(bool DoSync)
    
    if (DoSync == true)
       Sync();
-   
-   if (munmap((char *)Base,iSize) != 0)
-      _error->Warning("Unable to munmap");
-   
+
+   if (SyncToFd != NULL)
+   {
+      delete[] (char *)Base;
+      delete SyncToFd;
+      SyncToFd = NULL;
+   }
+   else
+   {
+      if (munmap((char *)Base, iSize) != 0)
+	 _error->WarningE("mmap", _("Unable to close mmap"));
+   }
+
    iSize = 0;
    Base = 0;
    return true;
@@ -113,8 +140,18 @@ bool MMap::Sync()
    
 #ifdef _POSIX_SYNCHRONIZED_IO   
    if ((Flags & ReadOnly) != ReadOnly)
-      if (msync((char *)Base,iSize,MS_SYNC) < 0)
-	 return _error->Errno("msync","Unable to write mmap");
+   {
+      if (SyncToFd != NULL)
+      {
+	 if (!SyncToFd->Seek(0) || !SyncToFd->Write(Base, iSize))
+	    return false;
+      }
+      else
+      {
+	 if (msync((char *)Base, iSize, MS_SYNC) < 0)
+	    return _error->Errno("msync", _("Unable to synchronize mmap"));
+      }
+   }
 #endif   
    return true;
 }
@@ -130,8 +167,19 @@ bool MMap::Sync(unsigned long Start,unsigned long Stop)
 #ifdef _POSIX_SYNCHRONIZED_IO
    unsigned long PSize = sysconf(_SC_PAGESIZE);
    if ((Flags & ReadOnly) != ReadOnly)
-      if (msync((char *)Base+(int)(Start/PSize)*PSize,Stop - Start,MS_SYNC) < 0)
-	 return _error->Errno("msync","Unable to write mmap");
+   {
+      if (SyncToFd != 0)
+      {
+	 if (!SyncToFd->Seek(0) ||
+	     !SyncToFd->Write (((char *)Base)+Start, Stop-Start))
+	    return false;
+      }
+      else
+      {
+	 if (msync((char *)Base+(int)(Start/PSize)*PSize,Stop - Start,MS_SYNC) < 0)
+	    return _error->Errno("msync", _("Unable to synchronize mmap"));
+      }
+   }
 #endif   
    return true;
 }
@@ -140,8 +188,10 @@ bool MMap::Sync(unsigned long Start,unsigned long Stop)
 // DynamicMMap::DynamicMMap - Constructor				/*{{{*/
 // ---------------------------------------------------------------------
 /* */
-DynamicMMap::DynamicMMap(FileFd &F,unsigned long Flags,unsigned long WorkSpace) :
-             MMap(F,Flags | NoImmMap), Fd(&F), WorkSpace(WorkSpace)
+DynamicMMap::DynamicMMap(FileFd &F,unsigned long Flags,unsigned long const &Workspace,
+			 unsigned long const &Grow, unsigned long const &Limit) :
+		MMap(F,Flags | NoImmMap), Fd(&F), WorkSpace(Workspace),
+		GrowFactor(Grow), Limit(Limit)
 {
    if (_error->PendingError() == true)
       return;
@@ -165,32 +215,48 @@ DynamicMMap::DynamicMMap(FileFd &F,unsigned long Flags,unsigned long WorkSpace)
 /* We try here to use mmap to reserve some space - this is much more
    cooler than the fallback solution to simply allocate a char array
    and could come in handy later than we are able to grow such an mmap */
-DynamicMMap::DynamicMMap(unsigned long Flags,unsigned long WorkSpace) :
-             MMap(Flags | NoImmMap | UnMapped), Fd(0), WorkSpace(WorkSpace)
+DynamicMMap::DynamicMMap(unsigned long Flags,unsigned long const &WorkSpace,
+			 unsigned long const &Grow, unsigned long const &Limit) :
+		MMap(Flags | NoImmMap | UnMapped), Fd(0), WorkSpace(WorkSpace),
+		GrowFactor(Grow), Limit(Limit)
 {
-   if (_error->PendingError() == true)
-      return;
+	if (_error->PendingError() == true)
+		return;
+
+	// disable Moveable if we don't grow
+	if (Grow == 0)
+		this->Flags &= ~Moveable;
+
+#ifndef __linux__
+	// kfreebsd doesn't have mremap, so we use the fallback
+	if ((this->Flags & Moveable) == Moveable)
+		this->Flags |= Fallback;
+#endif
 
 #ifdef _POSIX_MAPPED_FILES
-   // Set the permissions.
-   int Prot = PROT_READ;
-   int Map = MAP_PRIVATE | MAP_ANONYMOUS;
-   if ((Flags & ReadOnly) != ReadOnly)
-      Prot |= PROT_WRITE;
-   if ((Flags & Public) == Public)
-      Map = MAP_SHARED | MAP_ANONYMOUS;
+	if ((this->Flags & Fallback) != Fallback) {
+		// Set the permissions.
+		int Prot = PROT_READ;
+		int Map = MAP_PRIVATE | MAP_ANONYMOUS;
+		if ((this->Flags & ReadOnly) != ReadOnly)
+			Prot |= PROT_WRITE;
+		if ((this->Flags & Public) == Public)
+			Map = MAP_SHARED | MAP_ANONYMOUS;
 
-   // use anonymous mmap() to get the memory
-   Base = (unsigned char*) mmap(0, WorkSpace, Prot, Map, -1, 0);
+		// use anonymous mmap() to get the memory
+		Base = (unsigned char*) mmap(0, WorkSpace, Prot, Map, -1, 0);
 
-   if(Base == MAP_FAILED)
-      _error->Errno("DynamicMMap",_("Couldn't make mmap of %lu bytes"),WorkSpace);
-#else
-   // fallback to a static allocated space
-   Base = new unsigned char[WorkSpace];
-   memset(Base,0,WorkSpace);
+		if(Base == MAP_FAILED)
+			_error->Errno("DynamicMMap",_("Couldn't make mmap of %lu bytes"),WorkSpace);
+
+		iSize = 0;
+		return;
+	}
 #endif
-   iSize = 0;
+	// fallback to a static allocated space
+	Base = new unsigned char[WorkSpace];
+	memset(Base,0,WorkSpace);
+	iSize = 0;
 }
 									/*}}}*/
 // DynamicMMap::~DynamicMMap - Destructor				/*{{{*/
@@ -231,7 +297,7 @@ unsigned long DynamicMMap::RawAllocate(unsigned long Size,unsigned long Aln)
    {
       if(!Grow())
       {
-	 _error->Error(_("Dynamic MMap ran out of room. Please increase the size "
+	 _error->Fatal(_("Dynamic MMap ran out of room. Please increase the size "
 			 "of APT::Cache-Limit. Current value: %lu. (man 5 apt.conf)"), WorkSpace);
 	 return 0;
       }
@@ -248,7 +314,7 @@ unsigned long DynamicMMap::Allocate(unsigned long ItemSize)
    // Look for a matching pool entry
    Pool *I;
    Pool *Empty = 0;
-   for (I = Pools; I != Pools + PoolCount; I++)
+   for (I = Pools; I != Pools + PoolCount; ++I)
    {
       if (I->ItemSize == 0)
 	 Empty = I;
@@ -276,7 +342,11 @@ unsigned long DynamicMMap::Allocate(unsigned long ItemSize)
    {
       const unsigned long size = 20*1024;
       I->Count = size/ItemSize;
+      Pool* oldPools = Pools;
       Result = RawAllocate(size,ItemSize);
+      if (Pools != oldPools)
+	 I += Pools - oldPools;
+
       // Does the allocation failed ?
       if (Result == 0 && _error->PendingError())
 	 return 0;
@@ -299,7 +369,7 @@ unsigned long DynamicMMap::WriteString(const char *String,
    if (Len == (unsigned long)-1)
       Len = strlen(String);
 
-   unsigned long Result = RawAllocate(Len+1,0);
+   unsigned long const Result = RawAllocate(Len+1,0);
 
    if (Result == 0 && _error->PendingError())
       return 0;
@@ -311,30 +381,62 @@ unsigned long DynamicMMap::WriteString(const char *String,
 									/*}}}*/
 // DynamicMMap::Grow - Grow the mmap					/*{{{*/
 // ---------------------------------------------------------------------
-/* This method will try to grow the mmap we currently use. This doesn't
-   work most of the time because we can't move the mmap around in the
-   memory for now as this would require to adjust quite a lot of pointers
-   but why we should not at least try to grow it before we give up? */
-bool DynamicMMap::Grow()
-{
-#if defined(_POSIX_MAPPED_FILES) && defined(__linux__)
-   unsigned long newSize = WorkSpace + 1024*1024;
+/* This method is a wrapper around different methods to (try to) grow
+   a mmap (or our char[]-fallback). Encounterable environments:
+   1. Moveable + !Fallback + linux -> mremap with MREMAP_MAYMOVE
+   2. Moveable + !Fallback + !linux -> not possible (forbidden by constructor)
+   3. Moveable + Fallback -> realloc
+   4. !Moveable + !Fallback + linux -> mremap alone - which will fail in 99,9%
+   5. !Moveable + !Fallback + !linux -> not possible (forbidden by constructor)
+   6. !Moveable + Fallback -> not possible
+   [ While Moveable and Fallback stands for the equally named flags and
+     "linux" indicates a linux kernel instead of a freebsd kernel. ]
+   So what you can see here is, that a MMAP which want to be growable need
+   to be moveable to have a real chance but that this method will at least try
+   the nearly impossible 4 to grow it before it finally give up: Never say never. */
+bool DynamicMMap::Grow() {
+	if (Limit != 0 && WorkSpace >= Limit)
+		return _error->Error(_("Unable to increase the size of the MMap as the "
+		                       "limit of %lu bytes is already reached."), Limit);
+	if (GrowFactor <= 0)
+		return _error->Error(_("Unable to increase size of the MMap as automatic growing is disabled by user."));
 
-   if(Fd != 0)
-   {
-      Fd->Seek(newSize - 1);
-      char C = 0;
-      Fd->Write(&C,sizeof(C));
-   }
+	unsigned long const newSize = WorkSpace + GrowFactor;
 
-   Base = mremap(Base, WorkSpace, newSize, 0);
-   if(Base == MAP_FAILED)
-      return false;
+	if(Fd != 0) {
+		Fd->Seek(newSize - 1);
+		char C = 0;
+		Fd->Write(&C,sizeof(C));
+	}
 
-   WorkSpace = newSize;
-   return true;
+	unsigned long const poolOffset = Pools - ((Pool*) Base);
+
+	if ((Flags & Fallback) != Fallback) {
+#if defined(_POSIX_MAPPED_FILES) && defined(__linux__)
+   #ifdef MREMAP_MAYMOVE
+
+		if ((Flags & Moveable) == Moveable)
+			Base = mremap(Base, WorkSpace, newSize, MREMAP_MAYMOVE);
+		else
+   #endif
+			Base = mremap(Base, WorkSpace, newSize, 0);
+
+		if(Base == MAP_FAILED)
+			return false;
 #else
-   return false;
+		return false;
 #endif
+	} else {
+		if ((Flags & Moveable) != Moveable)
+			return false;
+
+		Base = realloc(Base, newSize);
+		if (Base == NULL)
+			return false;
+	}
+
+	Pools =(Pool*) Base + poolOffset;
+	WorkSpace = newSize;
+	return true;
 }
 									/*}}}*/

apt-pkg/contrib/mmap.h

@@ -44,13 +44,18 @@ class MMap
    unsigned long iSize;
    void *Base;
 
+   // In case mmap can not be used, we keep a dup of the file
+   // descriptor that should have been mmaped so that we can write to
+   // the file in Sync().
+   FileFd *SyncToFd;
+
    bool Map(FileFd &Fd);
    bool Close(bool DoSync = true);
    
    public:
 
    enum OpenFlags {NoImmMap = (1<<0),Public = (1<<1),ReadOnly = (1<<2),
-                   UnMapped = (1<<3)};
+                   UnMapped = (1<<3), Moveable = (1<<4), Fallback = (1 << 5)};
       
    // Simple accessors
    inline operator void *() {return Base;};
@@ -82,6 +87,8 @@ class DynamicMMap : public MMap
    
    FileFd *Fd;
    unsigned long WorkSpace;
+   unsigned long const GrowFactor;
+   unsigned long const Limit;
    Pool *Pools;
    unsigned int PoolCount;
 
@@ -96,8 +103,10 @@ class DynamicMMap : public MMap
    inline unsigned long WriteString(const string &S) {return WriteString(S.c_str(),S.length());};
    void UsePools(Pool &P,unsigned int Count) {Pools = &P; PoolCount = Count;};
    
-   DynamicMMap(FileFd &F,unsigned long Flags,unsigned long WorkSpace = 2*1024*1024);
-   DynamicMMap(unsigned long Flags,unsigned long WorkSpace = 2*1024*1024);
+   DynamicMMap(FileFd &F,unsigned long Flags,unsigned long const &WorkSpace = 2*1024*1024,
+	       unsigned long const &Grow = 1024*1024, unsigned long const &Limit = 0);
+   DynamicMMap(unsigned long Flags,unsigned long const &WorkSpace = 2*1024*1024,
+	       unsigned long const &Grow = 1024*1024, unsigned long const &Limit = 0);
    virtual ~DynamicMMap();
 };
 

apt-pkg/contrib/strutl.cc

@@ -198,7 +198,8 @@ bool ParseQuoteWord(const char *&String,string &Res)
    char *I;
    for (I = Buffer; I < Buffer + sizeof(Buffer) && Start != C; I++)
    {
-      if (*Start == '%' && Start + 2 < C)
+      if (*Start == '%' && Start + 2 < C &&
+	  isxdigit(Start[1]) && isxdigit(Start[2]))
       {
 	 Tmp[0] = Start[1];
 	 Tmp[1] = Start[2];
@@ -273,7 +274,8 @@ string QuoteString(const string &Str, const char *Bad)
    for (string::const_iterator I = Str.begin(); I != Str.end(); I++)
    {
       if (strchr(Bad,*I) != 0 || isprint(*I) == 0 || 
-	  *I <= 0x20 || *I >= 0x7F)
+	  *I == 0x25 || // percent '%' char
+	  *I <= 0x20 || *I >= 0x7F) // control chars
       {
 	 char Buf[10];
 	 sprintf(Buf,"%%%02x",(int)*I);
@@ -289,11 +291,17 @@ string QuoteString(const string &Str, const char *Bad)
 // ---------------------------------------------------------------------
 /* This undoes QuoteString */
 string DeQuoteString(const string &Str)
+{
+   return DeQuoteString(Str.begin(),Str.end());
+}
+string DeQuoteString(string::const_iterator const &begin,
+			string::const_iterator const &end)
 {
    string Res;
-   for (string::const_iterator I = Str.begin(); I != Str.end(); I++)
+   for (string::const_iterator I = begin; I != end; I++)
    {
-      if (*I == '%' && I + 2 < Str.end())
+      if (*I == '%' && I + 2 < end &&
+	  isxdigit(I[1]) && isxdigit(I[2]))
       {
 	 char Tmp[3];
 	 Tmp[0] = I[1];
@@ -566,7 +574,7 @@ int stringcmp(string::const_iterator A,string::const_iterator AEnd,
 int stringcasecmp(const char *A,const char *AEnd,const char *B,const char *BEnd)
 {
    for (; A != AEnd && B != BEnd; A++, B++)
-      if (toupper(*A) != toupper(*B))
+      if (tolower_ascii(*A) != tolower_ascii(*B))
 	 break;
 
    if (A == AEnd && B == BEnd)
@@ -575,7 +583,7 @@ int stringcasecmp(const char *A,const char *AEnd,const char *B,const char *BEnd)
       return 1;
    if (B == BEnd)
       return -1;
-   if (toupper(*A) < toupper(*B))
+   if (tolower_ascii(*A) < tolower_ascii(*B))
       return -1;
    return 1;
 }
@@ -584,7 +592,7 @@ int stringcasecmp(string::const_iterator A,string::const_iterator AEnd,
 		  const char *B,const char *BEnd)
 {
    for (; A != AEnd && B != BEnd; A++, B++)
-      if (toupper(*A) != toupper(*B))
+      if (tolower_ascii(*A) != tolower_ascii(*B))
 	 break;
 
    if (A == AEnd && B == BEnd)
@@ -593,7 +601,7 @@ int stringcasecmp(string::const_iterator A,string::const_iterator AEnd,
       return 1;
    if (B == BEnd)
       return -1;
-   if (toupper(*A) < toupper(*B))
+   if (tolower_ascii(*A) < tolower_ascii(*B))
       return -1;
    return 1;
 }
@@ -601,7 +609,7 @@ int stringcasecmp(string::const_iterator A,string::const_iterator AEnd,
 		  string::const_iterator B,string::const_iterator BEnd)
 {
    for (; A != AEnd && B != BEnd; A++, B++)
-      if (toupper(*A) != toupper(*B))
+      if (tolower_ascii(*A) != tolower_ascii(*B))
 	 break;
 
    if (A == AEnd && B == BEnd)
@@ -610,7 +618,7 @@ int stringcasecmp(string::const_iterator A,string::const_iterator AEnd,
       return 1;
    if (B == BEnd)
       return -1;
-   if (toupper(*A) < toupper(*B))
+   if (tolower_ascii(*A) < tolower_ascii(*B))
       return -1;
    return 1;
 }
@@ -789,28 +797,28 @@ bool ReadMessages(int Fd, vector<string> &List)
 // MonthConv - Converts a month string into a number			/*{{{*/
 // ---------------------------------------------------------------------
 /* This was lifted from the boa webserver which lifted it from 'wn-v1.07'
-   Made it a bit more robust with a few touppers though. */
+   Made it a bit more robust with a few tolower_ascii though. */
 static int MonthConv(char *Month)
 {
-   switch (toupper(*Month)) 
+   switch (tolower_ascii(*Month)) 
    {
-      case 'A':
-      return toupper(Month[1]) == 'P'?3:7;
-      case 'D':
+      case 'a':
+      return tolower_ascii(Month[1]) == 'p'?3:7;
+      case 'd':
       return 11;
-      case 'F':
+      case 'f':
       return 1;
-      case 'J':
-      if (toupper(Month[1]) == 'A')
+      case 'j':
+      if (tolower_ascii(Month[1]) == 'a')
 	 return 0;
-      return toupper(Month[2]) == 'N'?5:6;
-      case 'M':
-      return toupper(Month[2]) == 'R'?2:4;
-      case 'N':
+      return tolower_ascii(Month[2]) == 'n'?5:6;
+      case 'm':
+      return tolower_ascii(Month[2]) == 'r'?2:4;
+      case 'n':
       return 10;
-      case 'O':
+      case 'o':
       return 9;
-      case 'S':
+      case 's':
       return 8;
 
       // Pretend it is January..
@@ -819,34 +827,70 @@ static int MonthConv(char *Month)
    }   
 }
 									/*}}}*/
-// timegm - Internal timegm function if gnu is not available		/*{{{*/
+// timegm - Internal timegm if the gnu version is not available		/*{{{*/
 // ---------------------------------------------------------------------
-/* Ripped this evil little function from wget - I prefer the use of 
-   GNU timegm if possible as this technique will have interesting problems
-   with leap seconds, timezones and other.
-   
-   Converts struct tm to time_t, assuming the data in tm is UTC rather
+/* Converts struct tm to time_t, assuming the data in tm is UTC rather
    than local timezone (mktime assumes the latter).
-   
-   Contributed by Roger Beeman <beeman@cisco.com>, with the help of
-   Mark Baushke <mdb@cisco.com> and the rest of the Gurus at CISCO. */
-
-/* Turned it into an autoconf check, because GNU is not the only thing which
-   can provide timegm. -- 2002-09-22, Joel Baker */
 
-#ifndef HAVE_TIMEGM // Now with autoconf!
+   This function is a nonstandard GNU extension that is also present on
+   the BSDs and maybe other systems. For others we follow the advice of
+   the manpage of timegm and use his portable replacement. */
+#ifndef HAVE_TIMEGM
 static time_t timegm(struct tm *t)
 {
-   time_t tl, tb;
-   
-   tl = mktime (t);
-   if (tl == -1)
-      return -1;
-   tb = mktime (gmtime (&tl));
-   return (tl <= tb ? (tl + (tl - tb)) : (tl - (tb - tl)));
+   char *tz = getenv("TZ");
+   setenv("TZ", "", 1);
+   tzset();
+   time_t ret = mktime(t);
+   if (tz)
+      setenv("TZ", tz, 1);
+   else
+      unsetenv("TZ");
+   tzset();
+   return ret;
 }
 #endif
 									/*}}}*/
+// FullDateToTime - Converts a HTTP1.1 full date strings into a time_t	/*{{{*/
+// ---------------------------------------------------------------------
+/* tries to parses a full date as specified in RFC2616 Section 3.3.1
+   with one exception: All timezones (%Z) are accepted but the protocol
+   says that it MUST be GMT, but this one is equal to UTC which we will
+   encounter from time to time (e.g. in Release files) so we accept all
+   here and just assume it is GMT (or UTC) later on */
+bool RFC1123StrToTime(const char* const str,time_t &time)
+{
+   struct tm Tm;
+   setlocale (LC_ALL,"C");
+   bool const invalid =
+   // Sun, 06 Nov 1994 08:49:37 GMT  ; RFC 822, updated by RFC 1123
+      (strptime(str, "%a, %d %b %Y %H:%M:%S %Z", &Tm) == NULL &&
+   // Sunday, 06-Nov-94 08:49:37 GMT ; RFC 850, obsoleted by RFC 1036
+       strptime(str, "%A, %d-%b-%y %H:%M:%S %Z", &Tm) == NULL &&
+   // Sun Nov  6 08:49:37 1994       ; ANSI C's asctime() format
+       strptime(str, "%a %b %d %H:%M:%S %Y", &Tm) == NULL);
+   setlocale (LC_ALL,"");
+   if (invalid == true)
+      return false;
+
+   time = timegm(&Tm);
+   return true;
+}
+									/*}}}*/
+// FTPMDTMStrToTime - Converts a ftp modification date into a time_t	/*{{{*/
+// ---------------------------------------------------------------------
+/* */
+bool FTPMDTMStrToTime(const char* const str,time_t &time)
+{
+   struct tm Tm;
+   // MDTM includes no whitespaces but recommend and ignored by strptime
+   if (strptime(str, "%Y %m %d %H %M %S", &Tm) == NULL)
+      return false;
+
+   time = timegm(&Tm);
+   return true;
+}
+									/*}}}*/
 // StrToTime - Converts a string into a time_t				/*{{{*/
 // ---------------------------------------------------------------------
 /* This handles all 3 populare time formats including RFC 1123, RFC 1036
@@ -1000,6 +1044,24 @@ bool TokSplitString(char Tok,char *Input,char **List,
    return true;
 }
 									/*}}}*/
+// VectorizeString - Split a string up into a vector of strings		/*{{{*/
+// ---------------------------------------------------------------------
+/* This can be used to split a given string up into a vector, so the
+   propose is the same as in the method above and this one is a bit slower
+   also, but the advantage is that we have an iteratable vector */
+vector<string> VectorizeString(string const &haystack, char const &split)
+{
+   string::const_iterator start = haystack.begin();
+   string::const_iterator end = start;
+   vector<string> exploded;
+   do {
+      for (; end != haystack.end() && *end != split; ++end);
+      exploded.push_back(string(start, end));
+      start = end + 1;
+   } while (end != haystack.end() && (++end) != haystack.end());
+   return exploded;
+}
+									/*}}}*/
 // RegexChoice - Simple regex list/list matcher				/*{{{*/
 // ---------------------------------------------------------------------
 /* */
@@ -1099,7 +1161,7 @@ void strprintf(string &out,const char *format,...)
 char *safe_snprintf(char *Buffer,char *End,const char *Format,...)
 {
    va_list args;
-   unsigned long Did;
+   int Did;
 
    va_start(args,Format);
 
@@ -1115,10 +1177,13 @@ char *safe_snprintf(char *Buffer,char *End,const char *Format,...)
 
 // tolower_ascii - tolower() function that ignores the locale		/*{{{*/
 // ---------------------------------------------------------------------
-/* */
-int tolower_ascii(int c)
+/* This little function is the most called method we have and tries
+   therefore to do the absolut minimum - and is noteable faster than
+   standard tolower/toupper and as a bonus avoids problems with different
+   locales - we only operate on ascii chars anyway. */
+int tolower_ascii(int const c)
 {
-   if (c >= 'A' and c <= 'Z')
+   if (c >= 'A' && c <= 'Z')
       return c + 32;
    return c;
 }
@@ -1217,9 +1282,10 @@ void URI::CopyFrom(const string &U)
    else
    {
       Host.assign(At+1,SingleSlash);
-      User.assign(FirstColon,SecondColon);
+      // username and password must be encoded (RFC 3986)
+      User.assign(DeQuoteString(FirstColon,SecondColon));
       if (SecondColon < At)
-	 Password.assign(SecondColon+1,At);
+	 Password.assign(DeQuoteString(SecondColon+1,At));
    }   
    
    // Now we parse the RFC 2732 [] hostnames.

apt-pkg/contrib/strutl.h

@@ -25,19 +25,12 @@
 #include <iostream>
 #include <time.h>
 
+#include "macros.h"
+
 using std::string;
 using std::vector;
 using std::ostream;
 
-#ifdef __GNUG__
-// Methods have a hidden this parameter that is visible to this attribute
-#define APT_FORMAT2 __attribute__ ((format (printf, 2, 3)))
-#define APT_FORMAT3 __attribute__ ((format (printf, 3, 4)))
-#else
-#define APT_FORMAT2
-#define APT_FORMAT3
-#endif    
-
 bool UTF8ToCodeset(const char *codeset, const string &orig, string *dest);
 char *_strstrip(char *String);
 char *_strtabexpand(char *String,size_t Len);
@@ -45,13 +38,16 @@ bool ParseQuoteWord(const char *&String,string &Res);
 bool ParseCWord(const char *&String,string &Res);
 string QuoteString(const string &Str,const char *Bad);
 string DeQuoteString(const string &Str);
+string DeQuoteString(string::const_iterator const &begin, string::const_iterator const &end);
 string SizeToStr(double Bytes);
 string TimeToStr(unsigned long Sec);
 string Base64Encode(const string &Str);
 string OutputInDepth(const unsigned long Depth, const char* Separator="  ");
 string URItoFileName(const string &URI);
 string TimeRFC1123(time_t Date);
-bool StrToTime(const string &Val,time_t &Result);
+bool RFC1123StrToTime(const char* const str,time_t &time) __must_check;
+bool FTPMDTMStrToTime(const char* const str,time_t &time) __must_check;
+__deprecated bool StrToTime(const string &Val,time_t &Result);
 string LookupTag(const string &Message,const char *Tag,const char *Default = 0);
 int StringToBool(const string &Text,int Default = -1);
 bool ReadMessages(int Fd, vector<string> &List);
@@ -59,11 +55,12 @@ bool StrToNum(const char *Str,unsigned long &Res,unsigned Len,unsigned Base = 0)
 bool Hex2Num(const string &Str,unsigned char *Num,unsigned int Length);
 bool TokSplitString(char Tok,char *Input,char **List,
 		    unsigned long ListMax);
-void ioprintf(ostream &out,const char *format,...) APT_FORMAT2;
-void strprintf(string &out,const char *format,...) APT_FORMAT2;
-char *safe_snprintf(char *Buffer,char *End,const char *Format,...) APT_FORMAT3;
+vector<string> VectorizeString(string const &haystack, char const &split) __attrib_const;
+void ioprintf(ostream &out,const char *format,...) __like_printf(2);
+void strprintf(string &out,const char *format,...) __like_printf(2);
+char *safe_snprintf(char *Buffer,char *End,const char *Format,...) __like_printf(3);
 bool CheckDomainList(const string &Host, const string &List);
-int tolower_ascii(int c);
+int tolower_ascii(int const c) __attrib_const __hot;
 
 #define APT_MKSTRCMP(name,func) \
 inline int name(const char *A,const char *B) {return func(A,A+strlen(A),B,B+strlen(B));}; \
@@ -143,6 +140,4 @@ struct RxChoiceList
 unsigned long RegexChoice(RxChoiceList *Rxs,const char **ListBegin,
 		      const char **ListEnd);
 
-#undef APT_FORMAT2
-
 #endif

apt-pkg/contrib/weakptr.h

@@ -0,0 +1,62 @@
+/* weakptr.h - An object which supports weak pointers.
+ *
+ * Copyright (C) 2010 Julian Andres Klode <jak@debian.org>
+ * 
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
+ * MA 02110-1301, USA.
+ */
+
+#ifndef WEAK_POINTER_H
+#define WEAK_POINTER_H
+
+#include <set>
+/**
+ * Class for objects providing support for weak pointers.
+ *
+ * This class allows for the registration of certain pointers as weak,
+ * which will cause them to be set to NULL when the destructor of the
+ * object is called.
+ */
+class WeakPointable {
+private:
+    std::set<WeakPointable**> pointers;
+
+public:
+
+    /**
+     * Add a new weak pointer.
+     */
+    inline void AddWeakPointer(WeakPointable** weakptr) {
+       pointers.insert(weakptr);
+    }
+
+    /**
+     * Remove the weak pointer from the list of weak pointers.
+     */
+    inline void RemoveWeakPointer(WeakPointable **weakptr) {
+       pointers.erase(weakptr);
+    }
+
+    /**
+     * Deconstruct the object, set all weak pointers to NULL.
+     */
+    ~WeakPointable() {
+        std::set<WeakPointable**>::iterator iter = pointers.begin();
+        while (iter != pointers.end())
+            **(iter++) = NULL;
+    }
+};
+
+#endif // WEAK_POINTER_H

apt-pkg/deb/debindexfile.cc

@@ -63,9 +63,13 @@ string debSourcesIndex::SourceInfo(pkgSrcRecords::Parser const &Record,
 /* */
 pkgSrcRecords::Parser *debSourcesIndex::CreateSrcParser() const
 {
-   string SourcesURI = URItoFileName(IndexURI("Sources"));
-   return new debSrcRecordParser(_config->FindDir("Dir::State::lists") +
-				 SourcesURI,this);
+   string SourcesURI = _config->FindDir("Dir::State::lists") + 
+      URItoFileName(IndexURI("Sources"));
+   string SourcesURIgzip = SourcesURI + ".gz";
+   if (!FileExists(SourcesURI) && FileExists(SourcesURIgzip))
+      SourcesURI = SourcesURIgzip;
+
+   return new debSrcRecordParser(SourcesURI,this);
 }
 									/*}}}*/
 // SourcesIndex::Describe - Give a descriptive path to the index	/*{{{*/
@@ -106,8 +110,14 @@ string debSourcesIndex::Info(const char *Type) const
 /* */
 inline string debSourcesIndex::IndexFile(const char *Type) const
 {
-   return URItoFileName(IndexURI(Type));
+   string s = URItoFileName(IndexURI(Type));
+   string sgzip = s + ".gz";
+   if (!FileExists(s) && FileExists(sgzip))
+       return sgzip;
+   else
+       return s;
 }
+
 string debSourcesIndex::IndexURI(const char *Type) const
 {
    string Res;
@@ -149,9 +159,12 @@ unsigned long debSourcesIndex::Size() const
 // PackagesIndex::debPackagesIndex - Contructor				/*{{{*/
 // ---------------------------------------------------------------------
 /* */
-debPackagesIndex::debPackagesIndex(string URI,string Dist,string Section,bool Trusted) : 
-                  pkgIndexFile(Trusted), URI(URI), Dist(Dist), Section(Section)
+debPackagesIndex::debPackagesIndex(string const &URI, string const &Dist, string const &Section,
+					bool const &Trusted, string const &Arch) :
+                  pkgIndexFile(Trusted), URI(URI), Dist(Dist), Section(Section), Architecture(Arch)
 {
+	if (Architecture == "native")
+		Architecture = _config->Find("APT::Architecture");
 }
 									/*}}}*/
 // PackagesIndex::ArchiveInfo - Short version of the archive url	/*{{{*/
@@ -171,6 +184,8 @@ string debPackagesIndex::ArchiveInfo(pkgCache::VerIterator Ver) const
    Res += " ";
    Res += Ver.ParentPkg().Name();
    Res += " ";
+   if (Dist[Dist.size() - 1] != '/')
+      Res.append(Ver.Arch()).append(" ");
    Res += Ver.VerStr();
    return Res;
 }
@@ -204,6 +219,8 @@ string debPackagesIndex::Info(const char *Type) const
    else
       Info += Dist + '/' + Section;   
    Info += " ";
+   if (Dist[Dist.size() - 1] != '/')
+      Info += Architecture + " ";
    Info += Type;
    return Info;
 }
@@ -213,7 +230,12 @@ string debPackagesIndex::Info(const char *Type) const
 /* */
 inline string debPackagesIndex::IndexFile(const char *Type) const
 {
-   return _config->FindDir("Dir::State::lists") + URItoFileName(IndexURI(Type));
+   string s =_config->FindDir("Dir::State::lists") + URItoFileName(IndexURI(Type));
+   string sgzip = s + ".gz";
+   if (!FileExists(s) && FileExists(sgzip))
+       return sgzip;
+   else
+       return s;
 }
 string debPackagesIndex::IndexURI(const char *Type) const
 {
@@ -227,7 +249,7 @@ string debPackagesIndex::IndexURI(const char *Type) const
    }
    else
       Res = URI + "dists/" + Dist + '/' + Section +
-      "/binary-" + _config->Find("APT::Architecture") + '/';
+      "/binary-" + Architecture + '/';
    
    Res += Type;
    return Res;
@@ -255,21 +277,23 @@ unsigned long debPackagesIndex::Size() const
 // PackagesIndex::Merge - Load the index file into a cache		/*{{{*/
 // ---------------------------------------------------------------------
 /* */
-bool debPackagesIndex::Merge(pkgCacheGenerator &Gen,OpProgress &Prog) const
+bool debPackagesIndex::Merge(pkgCacheGenerator &Gen,OpProgress *Prog) const
 {
    string PackageFile = IndexFile("Packages");
-   FileFd Pkg(PackageFile,FileFd::ReadOnly);
-   debListParser Parser(&Pkg);
+   FileFd Pkg(PackageFile,FileFd::ReadOnlyGzip);
+   debListParser Parser(&Pkg, Architecture);
+
    if (_error->PendingError() == true)
       return _error->Error("Problem opening %s",PackageFile.c_str());
-   
-   Prog.SubProgress(0,Info("Packages"));
+   if (Prog != NULL)
+      Prog->SubProgress(0,Info("Packages"));
    ::URI Tmp(URI);
    if (Gen.SelectFile(PackageFile,Tmp.Host,*this) == false)
       return _error->Error("Problem with SelectFile %s",PackageFile.c_str());
 
    // Store the IMS information
    pkgCache::PkgFileIterator File = Gen.GetCurFile();
+   pkgCacheGenerator::Dynamic<pkgCache::PkgFileIterator> DynFile(File);
    struct stat St;
    if (fstat(Pkg.Fd(),&St) != 0)
       return _error->Errno("fstat","Failed to stat");
@@ -329,17 +353,23 @@ pkgCache::PkgFileIterator debPackagesIndex::FindInCache(pkgCache &Cache) const
 // TranslationsIndex::debTranslationsIndex - Contructor			/*{{{*/
 // ---------------------------------------------------------------------
 /* */
-debTranslationsIndex::debTranslationsIndex(string URI,string Dist,string Section) : 
-                  pkgIndexFile(true), URI(URI), Dist(Dist), Section(Section)
-{
-}
+debTranslationsIndex::debTranslationsIndex(string URI,string Dist,string Section,
+						char const * const Translation) :
+			pkgIndexFile(true), URI(URI), Dist(Dist), Section(Section),
+				Language(Translation)
+{}
 									/*}}}*/
 // TranslationIndex::Trans* - Return the URI to the translation files	/*{{{*/
 // ---------------------------------------------------------------------
 /* */
 inline string debTranslationsIndex::IndexFile(const char *Type) const
 {
-   return _config->FindDir("Dir::State::lists") + URItoFileName(IndexURI(Type));
+   string s =_config->FindDir("Dir::State::lists") + URItoFileName(IndexURI(Type));
+   string sgzip = s + ".gz";
+   if (!FileExists(s) && FileExists(sgzip))
+       return sgzip;
+   else
+       return s;
 }
 string debTranslationsIndex::IndexURI(const char *Type) const
 {
@@ -365,8 +395,8 @@ string debTranslationsIndex::IndexURI(const char *Type) const
 bool debTranslationsIndex::GetIndexes(pkgAcquire *Owner) const
 {
    if (TranslationsAvailable()) {
-     string TranslationFile = "Translation-" + LanguageCode();
-     new pkgAcqIndexTrans(Owner, IndexURI(LanguageCode().c_str()),
+     string const TranslationFile = string("Translation-").append(Language);
+     new pkgAcqIndexTrans(Owner, IndexURI(Language),
 			  Info(TranslationFile.c_str()),
 			  TranslationFile);
    }
@@ -385,7 +415,7 @@ string debTranslationsIndex::Describe(bool Short) const
       snprintf(S,sizeof(S),"%s",Info(TranslationFile().c_str()).c_str());
    else
       snprintf(S,sizeof(S),"%s (%s)",Info(TranslationFile().c_str()).c_str(),
-	       IndexFile(LanguageCode().c_str()).c_str());
+	       IndexFile(Language).c_str());
    return S;
 }
 									/*}}}*/
@@ -407,20 +437,20 @@ string debTranslationsIndex::Info(const char *Type) const
    return Info;
 }
 									/*}}}*/
-bool debTranslationsIndex::HasPackages() const
+bool debTranslationsIndex::HasPackages() const				/*{{{*/
 {
    if(!TranslationsAvailable())
       return false;
    
-   return FileExists(IndexFile(LanguageCode().c_str()));
+   return FileExists(IndexFile(Language));
 }
-
+									/*}}}*/
 // TranslationsIndex::Exists - Check if the index is available		/*{{{*/
 // ---------------------------------------------------------------------
 /* */
 bool debTranslationsIndex::Exists() const
 {
-   return FileExists(IndexFile(LanguageCode().c_str()));
+   return FileExists(IndexFile(Language));
 }
 									/*}}}*/
 // TranslationsIndex::Size - Return the size of the index		/*{{{*/
@@ -429,7 +459,7 @@ bool debTranslationsIndex::Exists() const
 unsigned long debTranslationsIndex::Size() const
 {
    struct stat S;
-   if (stat(IndexFile(LanguageCode().c_str()).c_str(),&S) != 0)
+   if (stat(IndexFile(Language).c_str(),&S) != 0)
       return 0;
    return S.st_size;
 }
@@ -437,18 +467,19 @@ unsigned long debTranslationsIndex::Size() const
 // TranslationsIndex::Merge - Load the index file into a cache		/*{{{*/
 // ---------------------------------------------------------------------
 /* */
-bool debTranslationsIndex::Merge(pkgCacheGenerator &Gen,OpProgress &Prog) const
+bool debTranslationsIndex::Merge(pkgCacheGenerator &Gen,OpProgress *Prog) const
 {
    // Check the translation file, if in use
-   string TranslationFile = IndexFile(LanguageCode().c_str());
+   string TranslationFile = IndexFile(Language);
    if (TranslationsAvailable() && FileExists(TranslationFile))
    {
-     FileFd Trans(TranslationFile,FileFd::ReadOnly);
+     FileFd Trans(TranslationFile,FileFd::ReadOnlyGzip);
      debListParser TransParser(&Trans);
      if (_error->PendingError() == true)
        return false;
      
-     Prog.SubProgress(0, Info(TranslationFile.c_str()));
+     if (Prog != NULL)
+	Prog->SubProgress(0, Info(TranslationFile.c_str()));
      if (Gen.SelectFile(TranslationFile,string(),*this) == false)
        return _error->Error("Problem with SelectFile %s",TranslationFile.c_str());
 
@@ -472,7 +503,7 @@ bool debTranslationsIndex::Merge(pkgCacheGenerator &Gen,OpProgress &Prog) const
 /* */
 pkgCache::PkgFileIterator debTranslationsIndex::FindInCache(pkgCache &Cache) const
 {
-   string FileName = IndexFile(LanguageCode().c_str());
+   string FileName = IndexFile(Language);
    
    pkgCache::PkgFileIterator File = Cache.FileBegin();
    for (; File.end() == false; File++)
@@ -521,16 +552,17 @@ unsigned long debStatusIndex::Size() const
 // StatusIndex::Merge - Load the index file into a cache		/*{{{*/
 // ---------------------------------------------------------------------
 /* */
-bool debStatusIndex::Merge(pkgCacheGenerator &Gen,OpProgress &Prog) const
+bool debStatusIndex::Merge(pkgCacheGenerator &Gen,OpProgress *Prog) const
 {
-   FileFd Pkg(File,FileFd::ReadOnly);
+   FileFd Pkg(File,FileFd::ReadOnlyGzip);
    if (_error->PendingError() == true)
       return false;
    debListParser Parser(&Pkg);
    if (_error->PendingError() == true)
       return false;
-   
-   Prog.SubProgress(0,File);
+
+   if (Prog != NULL)
+      Prog->SubProgress(0,File);
    if (Gen.SelectFile(File,string(),*this,pkgCache::Flag::NotSource) == false)
       return _error->Error("Problem with SelectFile %s",File.c_str());
 

apt-pkg/deb/debindexfile.h

@@ -35,7 +35,7 @@ class debStatusIndex : public pkgIndexFile
    virtual bool Exists() const;
    virtual bool HasPackages() const {return true;};
    virtual unsigned long Size() const;
-   virtual bool Merge(pkgCacheGenerator &Gen,OpProgress &Prog) const;
+   virtual bool Merge(pkgCacheGenerator &Gen,OpProgress *Prog) const;
    virtual pkgCache::PkgFileIterator FindInCache(pkgCache &Cache) const;
 
    debStatusIndex(string File);
@@ -46,6 +46,7 @@ class debPackagesIndex : public pkgIndexFile
    string URI;
    string Dist;
    string Section;
+   string Architecture;
 
    string Info(const char *Type) const;
    string IndexFile(const char *Type) const;
@@ -66,10 +67,11 @@ class debPackagesIndex : public pkgIndexFile
    virtual bool Exists() const;
    virtual bool HasPackages() const {return true;};
    virtual unsigned long Size() const;
-   virtual bool Merge(pkgCacheGenerator &Gen,OpProgress &Prog) const;
+   virtual bool Merge(pkgCacheGenerator &Gen,OpProgress *Prog) const;
    virtual pkgCache::PkgFileIterator FindInCache(pkgCache &Cache) const;
 
-   debPackagesIndex(string URI,string Dist,string Section,bool Trusted);
+   debPackagesIndex(string const &URI, string const &Dist, string const &Section,
+			bool const &Trusted, string const &Arch = "native");
 };
 
 class debTranslationsIndex : public pkgIndexFile
@@ -77,12 +79,13 @@ class debTranslationsIndex : public pkgIndexFile
    string URI;
    string Dist;
    string Section;
+   const char * const Language;
    
    string Info(const char *Type) const;
    string IndexFile(const char *Type) const;
    string IndexURI(const char *Type) const;
 
-   inline string TranslationFile() const {return "Translation-" + LanguageCode();};
+   inline string TranslationFile() const {return string("Translation-").append(Language);};
 
    public:
    
@@ -96,10 +99,10 @@ class debTranslationsIndex : public pkgIndexFile
    virtual bool Exists() const;
    virtual bool HasPackages() const;
    virtual unsigned long Size() const;
-   virtual bool Merge(pkgCacheGenerator &Gen,OpProgress &Prog) const;
+   virtual bool Merge(pkgCacheGenerator &Gen,OpProgress *Prog) const;
    virtual pkgCache::PkgFileIterator FindInCache(pkgCache &Cache) const;
 
-   debTranslationsIndex(string URI,string Dist,string Section);
+   debTranslationsIndex(string URI,string Dist,string Section, char const * const Language);
 };
 
 class debSourcesIndex : public pkgIndexFile

apt-pkg/deb/deblistparser.cc

@@ -13,11 +13,13 @@
 #include <apt-pkg/deblistparser.h>
 #include <apt-pkg/error.h>
 #include <apt-pkg/configuration.h>
+#include <apt-pkg/aptconfiguration.h>
 #include <apt-pkg/strutl.h>
 #include <apt-pkg/crc-16.h>
 #include <apt-pkg/md5.h>
 #include <apt-pkg/macros.h>
 
+#include <fnmatch.h>
 #include <ctype.h>
 									/*}}}*/
 
@@ -30,10 +32,15 @@ static debListParser::WordList PrioList[] = {{"important",pkgCache::State::Impor
 
 // ListParser::debListParser - Constructor				/*{{{*/
 // ---------------------------------------------------------------------
-/* */
-debListParser::debListParser(FileFd *File) : Tags(File)
-{
-   Arch = _config->Find("APT::architecture");
+/* Provide an architecture and only this one and "all" will be accepted
+   in Step(), if no Architecture is given we will accept every arch
+   we would accept in general with checkArchitecture() */
+debListParser::debListParser(FileFd *File, string const &Arch) : Tags(File),
+				Arch(Arch) {
+   if (Arch == "native")
+      this->Arch = _config->Find("APT::Architecture");
+   Architectures = APT::Configuration::getArchitectures();
+   MultiArchEnabled = Architectures.size() > 1;
 }
 									/*}}}*/
 // ListParser::UniqFindTagWrite - Find the tag and write a unq string	/*{{{*/
@@ -51,14 +58,41 @@ unsigned long debListParser::UniqFindTagWrite(const char *Tag)
 // ListParser::Package - Return the package name			/*{{{*/
 // ---------------------------------------------------------------------
 /* This is to return the name of the package this section describes */
-string debListParser::Package()
-{
-   string Result = Section.FindS("Package");
-   if (Result.empty() == true)
+string debListParser::Package() {
+   string const Result = Section.FindS("Package");
+   if(unlikely(Result.empty() == true))
       _error->Error("Encountered a section with no Package: header");
    return Result;
 }
 									/*}}}*/
+// ListParser::Architecture - Return the package arch			/*{{{*/
+// ---------------------------------------------------------------------
+/* This will return the Architecture of the package this section describes
+   Note that architecture "all" packages will get the architecture of the
+   Packages file parsed here. */
+string debListParser::Architecture() {
+   string const Result = Section.FindS("Architecture");
+   if (Result.empty() == true || Result == "all")
+   {
+      if (Arch.empty() == true)
+	 /* FIXME: this is a problem for installed arch all
+	    packages as we don't know from which arch this
+	    package was installed - and therefore which
+	    dependency this package resolves. */
+	 return _config->Find("APT::Architecture");
+      else
+	 return Arch;
+   }
+   return Result;
+}
+									/*}}}*/
+// ListParser::ArchitectureAll						/*{{{*/
+// ---------------------------------------------------------------------
+/* */
+bool debListParser::ArchitectureAll() {
+   return Section.FindS("Architecture") == "all";
+}
+									/*}}}*/
 // ListParser::Version - Return the version string			/*{{{*/
 // ---------------------------------------------------------------------
 /* This is to return the string describing the version in debian form,
@@ -72,17 +106,39 @@ string debListParser::Version()
 // ListParser::NewVersion - Fill in the version structure		/*{{{*/
 // ---------------------------------------------------------------------
 /* */
-bool debListParser::NewVersion(pkgCache::VerIterator Ver)
+bool debListParser::NewVersion(pkgCache::VerIterator &Ver)
 {
    // Parse the section
    Ver->Section = UniqFindTagWrite("Section");
-   Ver->Arch = UniqFindTagWrite("Architecture");
-   
+
+   // Parse multi-arch
+   if (Section.FindS("Architecture") == "all")
+      /* Arch all packages can't have a Multi-Arch field,
+         but we need a special treatment for them nonetheless */
+      Ver->MultiArch = pkgCache::Version::All;
+   else
+   {
+      string const MultiArch = Section.FindS("Multi-Arch");
+      if (MultiArch.empty() == true)
+	 Ver->MultiArch = pkgCache::Version::None;
+      else if (MultiArch == "same")
+	 Ver->MultiArch = pkgCache::Version::Same;
+      else if (MultiArch == "foreign")
+	 Ver->MultiArch = pkgCache::Version::Foreign;
+      else if (MultiArch == "allowed")
+	 Ver->MultiArch = pkgCache::Version::Allowed;
+      else
+      {
+	 _error->Warning("Unknown Multi-Arch type »%s« for package »%s«",
+			MultiArch.c_str(), Section.FindS("Package").c_str());
+	 Ver->MultiArch = pkgCache::Version::None;
+      }
+   }
+
    // Archive Size
-   Ver->Size = (unsigned)Section.FindI("Size");
-   
+   Ver->Size = Section.FindULL("Size");
    // Unpacked Size (in K)
-   Ver->InstalledSize = (unsigned)Section.FindI("Installed-Size");
+   Ver->InstalledSize = Section.FindULL("Installed-Size");
    Ver->InstalledSize *= 1024;
 
    // Priority
@@ -94,6 +150,24 @@ bool debListParser::NewVersion(pkgCache::VerIterator Ver)
 	 Ver->Priority = pkgCache::State::Extra;
    }
 
+   if (Ver->MultiArch == pkgCache::Version::All)
+   {
+      /* We maintain a "pseudo" arch=all package for architecture all versions
+	 on which these versions can depend on. This pseudo package is many used
+	 for downloading/installing: The other pseudo-packages will degenerate
+	 to a NOP in the download/install step - this package will ensure that
+	 it is downloaded only one time and installed only one time -- even if
+	 the architecture bound versions coming in and out on regular basis. */
+      if (strcmp(Ver.Arch(true),"all") == 0)
+	 return true;
+      else if (MultiArchEnabled == true)
+      {
+	 // our pseudo packages have no size to not confuse the fetcher
+	 Ver->Size = 0;
+	 Ver->InstalledSize = 0;
+      }
+   }
+
    if (ParseDepends(Ver,"Depends",pkgCache::Dep::Depends) == false)
       return false;
    if (ParseDepends(Ver,"Pre-Depends",pkgCache::Dep::PreDepends) == false)
@@ -128,10 +202,11 @@ bool debListParser::NewVersion(pkgCache::VerIterator Ver)
    only describe package properties */
 string debListParser::Description()
 {
-   if (DescriptionLanguage().empty())
+   string const lang = DescriptionLanguage();
+   if (lang.empty())
       return Section.FindS("Description");
    else
-      return Section.FindS(("Description-" + pkgIndexFile::LanguageCode()).c_str());
+      return Section.FindS(string("Description-").append(lang).c_str());
 }
                                                                         /*}}}*/
 // ListParser::DescriptionLanguage - Return the description lang string	/*{{{*/
@@ -141,7 +216,16 @@ string debListParser::Description()
    assumed to describe original description. */
 string debListParser::DescriptionLanguage()
 {
-   return Section.FindS("Description").empty() ? pkgIndexFile::LanguageCode() : "";
+   if (Section.FindS("Description").empty() == false)
+      return "";
+
+   std::vector<string> const lang = APT::Configuration::getLanguages();
+   for (std::vector<string>::const_iterator l = lang.begin();
+	l != lang.end(); l++)
+      if (Section.FindS(string("Description-").append(*l).c_str()).empty() == false)
+	 return *l;
+
+   return "";
 }
                                                                         /*}}}*/
 // ListParser::Description - Return the description_md5 MD5SumValue	/*{{{*/
@@ -167,13 +251,21 @@ MD5SumValue debListParser::Description_md5()
 // ---------------------------------------------------------------------
 /* This is called to update the package with any new information 
    that might be found in the section */
-bool debListParser::UsePackage(pkgCache::PkgIterator Pkg,
-			       pkgCache::VerIterator Ver)
+bool debListParser::UsePackage(pkgCache::PkgIterator &Pkg,
+			       pkgCache::VerIterator &Ver)
 {
    if (Pkg->Section == 0)
       Pkg->Section = UniqFindTagWrite("Section");
-   if (Section.FindFlag("Essential",Pkg->Flags,pkgCache::Flag::Essential) == false)
-      return false;
+
+   // Packages which are not from the "native" arch doesn't get the essential flag
+   // in the default "native" mode - it is also possible to mark "all" or "none".
+   // The "installed" mode is handled by ParseStatus(), See #544481 and friends.
+   string const static myArch = _config->Find("APT::Architecture");
+   string const static essential = _config->Find("pkgCacheGen::Essential", "native");
+   if ((essential == "native" && Pkg->Arch != 0 && myArch == Pkg.Arch()) ||
+       essential == "all")
+      if (Section.FindFlag("Essential",Pkg->Flags,pkgCache::Flag::Essential) == false)
+	 return false;
    if (Section.FindFlag("Important",Pkg->Flags,pkgCache::Flag::Important) == false)
       return false;
 
@@ -240,14 +332,20 @@ unsigned short debListParser::VersionHash()
    Some of the above are obsolete (I think?) flag = hold-* and 
    status = post-inst-failed, removal-failed at least.
  */
-bool debListParser::ParseStatus(pkgCache::PkgIterator Pkg,
-				pkgCache::VerIterator Ver)
+bool debListParser::ParseStatus(pkgCache::PkgIterator &Pkg,
+				pkgCache::VerIterator &Ver)
 {
    const char *Start;
    const char *Stop;
    if (Section.Find("Status",Start,Stop) == false)
       return true;
-   
+
+   // UsePackage() is responsible for setting the flag in the default case
+   bool const static essential = _config->Find("pkgCacheGen::Essential", "") == "installed";
+   if (essential == true &&
+       Section.FindFlag("Essential",Pkg->Flags,pkgCache::Flag::Essential) == false)
+      return false;
+
    // Isolate the first word
    const char *I = Start;
    for(; I < Stop && *I != ' '; I++);
@@ -376,6 +474,20 @@ const char *debListParser::ConvertRelation(const char *I,unsigned int &Op)
    return I;
 }
 
+/*
+ * CompleteArch:
+ *
+ * The complete architecture, consisting of <kernel>-<cpu>.
+ */
+static string CompleteArch(std::string& arch) {
+    if (arch == "armel")              return "linux-arm";
+    if (arch == "lpia")               return "linux-i386";
+    if (arch == "powerpcspe")         return "linux-powerpc";
+    if (arch == "uclibc-linux-armel") return "linux-arm";
+    if (arch == "uclinux-armel")      return "uclinux-arm";
+
+    return (arch.find("-") != string::npos) ? arch : "linux-" + arch;
+}
 									/*}}}*/
 // ListParser::ParseDepends - Parse a dependency element		/*{{{*/
 // ---------------------------------------------------------------------
@@ -383,7 +495,8 @@ const char *debListParser::ConvertRelation(const char *I,unsigned int &Op)
    bit by bit. */
 const char *debListParser::ParseDepends(const char *Start,const char *Stop,
 					string &Package,string &Ver,
-					unsigned int &Op, bool ParseArchFlags)
+					unsigned int &Op, bool const &ParseArchFlags,
+					bool const &StripMultiArch)
 {
    // Strip off leading space
    for (;Start != Stop && isspace(*Start) != 0; Start++);
@@ -402,7 +515,14 @@ const char *debListParser::ParseDepends(const char *Start,const char *Stop,
    
    // Stash the package name
    Package.assign(Start,I - Start);
-   
+
+   // We don't want to confuse library users which can't handle MultiArch
+   if (StripMultiArch == true) {
+      size_t const found = Package.rfind(':');
+      if (found != string::npos)
+	 Package = Package.substr(0,found);
+   }
+
    // Skip white space to the '('
    for (;I != Stop && isspace(*I) != 0 ; I++);
    
@@ -441,6 +561,7 @@ const char *debListParser::ParseDepends(const char *Start,const char *Stop,
    if (ParseArchFlags == true)
    {
       string arch = _config->Find("APT::Architecture");
+      string completeArch = CompleteArch(arch);
 
       // Parse an architecture
       if (I != Stop && *I == '[')
@@ -468,8 +589,13 @@ const char *debListParser::ParseDepends(const char *Start,const char *Stop,
 	       I++;
             }
 
-	    if (stringcmp(arch,I,End) == 0)
+	    if (stringcmp(arch,I,End) == 0) {
 	       Found = true;
+	    } else {
+	       std::string wildcard = SubstVar(string(I, End), "any", "*");
+	       if (fnmatch(wildcard.c_str(), completeArch.c_str(), 0) == 0)
+	          Found = true;
+	    }
 	    
 	    if (*End++ == ']') {
 	       I = End;
@@ -508,15 +634,16 @@ const char *debListParser::ParseDepends(const char *Start,const char *Stop,
 // ---------------------------------------------------------------------
 /* This is the higher level depends parser. It takes a tag and generates
    a complete depends tree for the given version. */
-bool debListParser::ParseDepends(pkgCache::VerIterator Ver,
+bool debListParser::ParseDepends(pkgCache::VerIterator &Ver,
 				 const char *Tag,unsigned int Type)
 {
    const char *Start;
    const char *Stop;
    if (Section.Find(Tag,Start,Stop) == false)
       return true;
-   
+
    string Package;
+   string const pkgArch = Ver.Arch(true);
    string Version;
    unsigned int Op;
 
@@ -525,8 +652,18 @@ bool debListParser::ParseDepends(pkgCache::VerIterator Ver,
       Start = ParseDepends(Start,Stop,Package,Version,Op);
       if (Start == 0)
 	 return _error->Error("Problem parsing dependency %s",Tag);
-      
-      if (NewDepends(Ver,Package,Version,Op,Type) == false)
+
+      if (MultiArchEnabled == true &&
+	  (Type == pkgCache::Dep::Conflicts ||
+	   Type == pkgCache::Dep::DpkgBreaks ||
+	   Type == pkgCache::Dep::Replaces))
+      {
+	 for (std::vector<std::string>::const_iterator a = Architectures.begin();
+	      a != Architectures.end(); ++a)
+	    if (NewDepends(Ver,Package,*a,Version,Op,Type) == false)
+	       return false;
+      }
+      else if (NewDepends(Ver,Package,pkgArch,Version,Op,Type) == false)
 	 return false;
       if (Start == Stop)
 	 break;
@@ -537,33 +674,55 @@ bool debListParser::ParseDepends(pkgCache::VerIterator Ver,
 // ListParser::ParseProvides - Parse the provides list			/*{{{*/
 // ---------------------------------------------------------------------
 /* */
-bool debListParser::ParseProvides(pkgCache::VerIterator Ver)
+bool debListParser::ParseProvides(pkgCache::VerIterator &Ver)
 {
    const char *Start;
    const char *Stop;
-   if (Section.Find("Provides",Start,Stop) == false)
-      return true;
-   
-   string Package;
-   string Version;
-   unsigned int Op;
-
-   while (1)
+   if (Section.Find("Provides",Start,Stop) == true)
    {
-      Start = ParseDepends(Start,Stop,Package,Version,Op);
-      if (Start == 0)
-	 return _error->Error("Problem parsing Provides line");
-      if (Op != pkgCache::Dep::NoOp) {
-	 _error->Warning("Ignoring Provides line with DepCompareOp for package %s", Package.c_str());
-      } else {
-	 if (NewProvides(Ver,Package,Version) == false)
-	    return false;
+      string Package;
+      string Version;
+      string const Arch = Ver.Arch(true);
+      unsigned int Op;
+
+      while (1)
+      {
+	 Start = ParseDepends(Start,Stop,Package,Version,Op);
+	 if (Start == 0)
+	    return _error->Error("Problem parsing Provides line");
+	 if (Op != pkgCache::Dep::NoOp) {
+	    _error->Warning("Ignoring Provides line with DepCompareOp for package %s", Package.c_str());
+	 } else {
+	    if (NewProvides(Ver, Package, Arch, Version) == false)
+	       return false;
+	 }
+
+	 if (Start == Stop)
+	    break;
       }
+   }
 
-      if (Start == Stop)
-	 break;
+   if (Ver->MultiArch == pkgCache::Version::Allowed)
+   {
+      string const Package = string(Ver.ParentPkg().Name()).append(":").append("any");
+      NewProvides(Ver, Package, "any", Ver.VerStr());
    }
-   
+
+   if (Ver->MultiArch != pkgCache::Version::Foreign)
+      return true;
+
+   if (MultiArchEnabled == false)
+      return true;
+
+   string const Package = Ver.ParentPkg().Name();
+   string const Version = Ver.VerStr();
+   for (std::vector<string>::const_iterator a = Architectures.begin();
+	a != Architectures.end(); ++a)
+   {
+      if (NewProvides(Ver, Package, *a, Version) == false)
+	 return false;
+   }
+
    return true;
 }
 									/*}}}*/
@@ -594,16 +753,23 @@ bool debListParser::Step()
       /* See if this is the correct Architecture, if it isn't then we
          drop the whole section. A missing arch tag only happens (in theory)
          inside the Status file, so that is a positive return */
-      const char *Start;
-      const char *Stop;
-      if (Section.Find("Architecture",Start,Stop) == false)
+      string const Architecture = Section.FindS("Architecture");
+      if (Architecture.empty() == true)
 	 return true;
 
-      if (stringcmp(Arch,Start,Stop) == 0)
-	 return true;
+      if (Arch.empty() == true || Arch == "any" || MultiArchEnabled == false)
+      {
+	 if (APT::Configuration::checkArchitecture(Architecture) == true)
+	    return true;
+      }
+      else
+      {
+	 if (Architecture == Arch)
+	    return true;
 
-      if (stringcmp(Start,Stop,"all") == 0)
-	 return true;
+	 if (Architecture == "all")
+	    return true;
+      }
 
       iOffset = Tags.Offset();
    }   
@@ -613,7 +779,7 @@ bool debListParser::Step()
 // ListParser::LoadReleaseInfo - Load the release information		/*{{{*/
 // ---------------------------------------------------------------------
 /* */
-bool debListParser::LoadReleaseInfo(pkgCache::PkgFileIterator FileI,
+bool debListParser::LoadReleaseInfo(pkgCache::PkgFileIterator &FileI,
 				    FileFd &File, string component)
 {
    pkgTagFile Tags(&File, File.Size() + 256); // XXX
@@ -621,8 +787,9 @@ bool debListParser::LoadReleaseInfo(pkgCache::PkgFileIterator FileI,
    if (Tags.Step(Section) == false)
       return false;
 
-   //mvo: I don't think we need to fill that in (it's unused since apt-0.6)
-   //FileI->Architecture = WriteUniqString(Arch);
+   // FIXME: Do we need it now for multi-arch?
+   // mvo: I don't think we need to fill that in (it's unused since apt-0.6)
+//    FileI->Architecture = WriteUniqString(Arch);
    
    // apt-secure does no longer download individual (per-section) Release
    // file. to provide Component pinning we use the section name now

apt-pkg/deb/deblistparser.h

@@ -32,12 +32,14 @@ class debListParser : public pkgCacheGenerator::ListParser
    pkgTagSection Section;
    unsigned long iOffset;
    string Arch;
-   
+   std::vector<std::string> Architectures;
+   bool MultiArchEnabled;
+
    unsigned long UniqFindTagWrite(const char *Tag);
-   bool ParseStatus(pkgCache::PkgIterator Pkg,pkgCache::VerIterator Ver);
-   bool ParseDepends(pkgCache::VerIterator Ver,const char *Tag,
+   bool ParseStatus(pkgCache::PkgIterator &Pkg,pkgCache::VerIterator &Ver);
+   bool ParseDepends(pkgCache::VerIterator &Ver,const char *Tag,
 		     unsigned int Type);
-   bool ParseProvides(pkgCache::VerIterator Ver);
+   bool ParseProvides(pkgCache::VerIterator &Ver);
    static bool GrabWord(string Word,WordList *List,unsigned char &Out);
    
    public:
@@ -46,28 +48,31 @@ class debListParser : public pkgCacheGenerator::ListParser
       
    // These all operate against the current section
    virtual string Package();
+   virtual string Architecture();
+   virtual bool ArchitectureAll();
    virtual string Version();
-   virtual bool NewVersion(pkgCache::VerIterator Ver);
+   virtual bool NewVersion(pkgCache::VerIterator &Ver);
    virtual string Description();
    virtual string DescriptionLanguage();
    virtual MD5SumValue Description_md5();
    virtual unsigned short VersionHash();
-   virtual bool UsePackage(pkgCache::PkgIterator Pkg,
-			   pkgCache::VerIterator Ver);
+   virtual bool UsePackage(pkgCache::PkgIterator &Pkg,
+			   pkgCache::VerIterator &Ver);
    virtual unsigned long Offset() {return iOffset;};
    virtual unsigned long Size() {return Section.size();};
 
    virtual bool Step();
    
-   bool LoadReleaseInfo(pkgCache::PkgFileIterator FileI,FileFd &File,
+   bool LoadReleaseInfo(pkgCache::PkgFileIterator &FileI,FileFd &File,
 			string section);
    
    static const char *ParseDepends(const char *Start,const char *Stop,
 			    string &Package,string &Ver,unsigned int &Op,
-			    bool ParseArchFlags = false);
+			    bool const &ParseArchFlags = false,
+			    bool const &StripMultiArch = false);
    static const char *ConvertRelation(const char *I,unsigned int &Op);
 
-   debListParser(FileFd *File);
+   debListParser(FileFd *File, string const &Arch = "");
 };
 
 #endif

apt-pkg/deb/debmetaindex.cc

@@ -5,11 +5,14 @@
 #include <apt-pkg/strutl.h>
 #include <apt-pkg/acquire-item.h>
 #include <apt-pkg/configuration.h>
+#include <apt-pkg/aptconfiguration.h>
 #include <apt-pkg/error.h>
 
+#include <set>
+
 using namespace std;
 
-string debReleaseIndex::Info(const char *Type, const string Section) const
+string debReleaseIndex::Info(const char *Type, string const &Section, string const &Arch) const
 {
    string Info = ::URI::SiteOnly(URI) + ' ';
    if (Dist[Dist.size() - 1] == '/')
@@ -18,7 +21,11 @@ string debReleaseIndex::Info(const char *Type, const string Section) const
          Info += Dist;
    }
    else
-      Info += Dist + '/' + Section;   
+   {
+      Info += Dist + '/' + Section;
+      if (Arch.empty() != true)
+	 Info += " " + Arch;
+   }
    Info += " ";
    Info += Type;
    return Info;
@@ -60,16 +67,21 @@ string debReleaseIndex::MetaIndexURI(const char *Type) const
    return Res;
 }
 
-string debReleaseIndex::IndexURISuffix(const char *Type, const string Section) const
+string debReleaseIndex::IndexURISuffix(const char *Type, string const &Section, string const &Arch) const
 {
    string Res ="";
    if (Dist[Dist.size() - 1] != '/')
-      Res += Section + "/binary-" + _config->Find("APT::Architecture") + '/';
+   {
+      if (Arch == "native")
+	 Res += Section + "/binary-" + _config->Find("APT::Architecture") + '/';
+      else
+	 Res += Section + "/binary-" + Arch + '/';
+   }
    return Res + Type;
 }
    
 
-string debReleaseIndex::IndexURI(const char *Type, const string Section) const
+string debReleaseIndex::IndexURI(const char *Type, string const &Section, string const &Arch) const
 {
    if (Dist[Dist.size() - 1] == '/')
    {
@@ -81,10 +93,10 @@ string debReleaseIndex::IndexURI(const char *Type, const string Section) const
       return Res + Type;
    }
    else
-      return URI + "dists/" + Dist + '/' + IndexURISuffix(Type, Section);
+      return URI + "dists/" + Dist + '/' + IndexURISuffix(Type, Section, Arch);
  }
 
-string debReleaseIndex::SourceIndexURISuffix(const char *Type, const string Section) const
+string debReleaseIndex::SourceIndexURISuffix(const char *Type, const string &Section) const
 {
    string Res ="";
    if (Dist[Dist.size() - 1] != '/')
@@ -92,7 +104,7 @@ string debReleaseIndex::SourceIndexURISuffix(const char *Type, const string Sect
    return Res + Type;
 }
 
-string debReleaseIndex::SourceIndexURI(const char *Type, const string Section) const
+string debReleaseIndex::SourceIndexURI(const char *Type, const string &Section) const
 {
    string Res;
    if (Dist[Dist.size() - 1] == '/')
@@ -107,44 +119,61 @@ string debReleaseIndex::SourceIndexURI(const char *Type, const string Section) c
       return URI + "dists/" + Dist + "/" + SourceIndexURISuffix(Type, Section);
 }
 
-debReleaseIndex::debReleaseIndex(string URI,string Dist)
-{
-   this->URI = URI;
-   this->Dist = Dist;
-   this->Indexes = NULL;
-   this->Type = "deb";
+debReleaseIndex::debReleaseIndex(string const &URI, string const &Dist) {
+	this->URI = URI;
+	this->Dist = Dist;
+	this->Indexes = NULL;
+	this->Type = "deb";
 }
 
-debReleaseIndex::~debReleaseIndex()
-{
-   for (vector<const debSectionEntry *>::const_iterator I = SectionEntries.begin();
-	I != SectionEntries.end(); I++)
-      delete *I;
+debReleaseIndex::~debReleaseIndex() {
+	for (map<string, vector<debSectionEntry const*> >::const_iterator A = ArchEntries.begin();
+	     A != ArchEntries.end(); ++A)
+		for (vector<const debSectionEntry *>::const_iterator S = A->second.begin();
+		     S != A->second.end(); ++S)
+			delete *S;
 }
 
-vector <struct IndexTarget *>* debReleaseIndex::ComputeIndexTargets() const
-{
-   vector <struct IndexTarget *>* IndexTargets = new vector <IndexTarget *>;
-   for (vector <const debSectionEntry *>::const_iterator I = SectionEntries.begin();
-	I != SectionEntries.end();
-	I++)
-   {
-      IndexTarget * Target = new IndexTarget();
-      Target->ShortDesc = (*I)->IsSrc ? "Sources" : "Packages";
-      Target->MetaKey
-	= (*I)->IsSrc ? SourceIndexURISuffix(Target->ShortDesc.c_str(), (*I)->Section)
-	              : IndexURISuffix(Target->ShortDesc.c_str(), (*I)->Section);
-      Target->URI 
-	= (*I)->IsSrc ? SourceIndexURI(Target->ShortDesc.c_str(), (*I)->Section)
-	              : IndexURI(Target->ShortDesc.c_str(), (*I)->Section);
-      
-      Target->Description = Info (Target->ShortDesc.c_str(), (*I)->Section);
-      IndexTargets->push_back (Target);
-   }
-   return IndexTargets;
+vector <struct IndexTarget *>* debReleaseIndex::ComputeIndexTargets() const {
+	vector <struct IndexTarget *>* IndexTargets = new vector <IndexTarget *>;
+
+	map<string, vector<debSectionEntry const*> >::const_iterator const src = ArchEntries.find("source");
+	if (src != ArchEntries.end()) {
+		vector<debSectionEntry const*> const SectionEntries = src->second;
+		for (vector<debSectionEntry const*>::const_iterator I = SectionEntries.begin();
+		     I != SectionEntries.end(); ++I) {
+			IndexTarget * Target = new IndexTarget();
+			Target->ShortDesc = "Sources";
+			Target->MetaKey = SourceIndexURISuffix(Target->ShortDesc.c_str(), (*I)->Section);
+			Target->URI = SourceIndexURI(Target->ShortDesc.c_str(), (*I)->Section);
+			Target->Description = Info (Target->ShortDesc.c_str(), (*I)->Section);
+			IndexTargets->push_back (Target);
+		}
+	}
+
+	// Only source release
+	if (IndexTargets->empty() == false && ArchEntries.size() == 1)
+		return IndexTargets;
+
+	for (map<string, vector<debSectionEntry const*> >::const_iterator a = ArchEntries.begin();
+	     a != ArchEntries.end(); ++a) {
+		if (a->first == "source")
+			continue;
+		for (vector <const debSectionEntry *>::const_iterator I = a->second.begin();
+		     I != a->second.end(); ++I) {
+			IndexTarget * Target = new IndexTarget();
+			Target->ShortDesc = "Packages";
+			Target->MetaKey = IndexURISuffix(Target->ShortDesc.c_str(), (*I)->Section, a->first);
+			Target->URI = IndexURI(Target->ShortDesc.c_str(), (*I)->Section, a->first);
+			Target->Description = Info (Target->ShortDesc.c_str(), (*I)->Section, a->first);
+			IndexTargets->push_back (Target);
+		}
+	}
+
+	return IndexTargets;
 }
 									/*}}}*/
-bool debReleaseIndex::GetIndexes(pkgAcquire *Owner, bool GetAll) const
+bool debReleaseIndex::GetIndexes(pkgAcquire *Owner, bool const &GetAll) const
 {
    // special case for --print-uris
    if (GetAll) {
@@ -169,17 +198,28 @@ bool debReleaseIndex::GetIndexes(pkgAcquire *Owner, bool GetAll) const
 		     ComputeIndexTargets(),
 		     new indexRecords (Dist));
 
-   // Queue the translations
-   for (vector<const debSectionEntry *>::const_iterator I = SectionEntries.begin(); 
-	I != SectionEntries.end(); I++) {
-
-      if((*I)->IsSrc)
-	 continue;
-      debTranslationsIndex i = debTranslationsIndex(URI,Dist,(*I)->Section);
-      i.GetIndexes(Owner);
-   }
-
-   return true;
+	// Queue the translations
+	std::vector<std::string> const lang = APT::Configuration::getLanguages(true);
+	map<string, set<string> > sections;
+	for (map<string, vector<debSectionEntry const*> >::const_iterator a = ArchEntries.begin();
+	     a != ArchEntries.end(); ++a) {
+		if (a->first == "source")
+			continue;
+		for (vector<debSectionEntry const*>::const_iterator I = a->second.begin();
+		     I != a->second.end(); I++)
+			sections[(*I)->Section].insert(lang.begin(), lang.end());
+	}
+
+	for (map<string, set<string> >::const_iterator s = sections.begin();
+	     s != sections.end(); ++s)
+		for (set<string>::const_iterator l = s->second.begin();
+		     l != s->second.end(); l++) {
+			if (*l == "none") continue;
+			debTranslationsIndex i = debTranslationsIndex(URI,Dist,s->first,(*l).c_str());
+			i.GetIndexes(Owner);
+		}
+
+	return true;
 }
 
 bool debReleaseIndex::IsTrusted() const
@@ -196,67 +236,123 @@ bool debReleaseIndex::IsTrusted() const
    return false;
 }
 
-vector <pkgIndexFile *> *debReleaseIndex::GetIndexFiles()
-{
-   if (Indexes != NULL)
-      return Indexes;
-
-   Indexes = new vector <pkgIndexFile*>;
-   for (vector<const debSectionEntry *>::const_iterator I = SectionEntries.begin(); 
-	I != SectionEntries.end(); I++) {
-      if ((*I)->IsSrc)
-         Indexes->push_back(new debSourcesIndex (URI, Dist, (*I)->Section, IsTrusted()));
-      else 
-      {
-         Indexes->push_back(new debPackagesIndex (URI, Dist, (*I)->Section, IsTrusted()));
-	 Indexes->push_back(new debTranslationsIndex(URI, Dist, (*I)->Section));
-      }
-   }
+vector <pkgIndexFile *> *debReleaseIndex::GetIndexFiles() {
+	if (Indexes != NULL)
+		return Indexes;
+
+	Indexes = new vector <pkgIndexFile*>;
+	map<string, vector<debSectionEntry const*> >::const_iterator const src = ArchEntries.find("source");
+	if (src != ArchEntries.end()) {
+		vector<debSectionEntry const*> const SectionEntries = src->second;
+		for (vector<debSectionEntry const*>::const_iterator I = SectionEntries.begin();
+		     I != SectionEntries.end(); I++)
+			Indexes->push_back(new debSourcesIndex (URI, Dist, (*I)->Section, IsTrusted()));
+	}
+
+	// Only source release
+	if (Indexes->empty() == false && ArchEntries.size() == 1)
+		return Indexes;
+
+	std::vector<std::string> const lang = APT::Configuration::getLanguages(true);
+	map<string, set<string> > sections;
+	for (map<string, vector<debSectionEntry const*> >::const_iterator a = ArchEntries.begin();
+	     a != ArchEntries.end(); ++a) {
+		if (a->first == "source")
+			continue;
+		for (vector<debSectionEntry const*>::const_iterator I = a->second.begin();
+		     I != a->second.end(); I++) {
+			Indexes->push_back(new debPackagesIndex (URI, Dist, (*I)->Section, IsTrusted(), a->first));
+			sections[(*I)->Section].insert(lang.begin(), lang.end());
+		}
+	}
+
+	for (map<string, set<string> >::const_iterator s = sections.begin();
+	     s != sections.end(); ++s)
+		for (set<string>::const_iterator l = s->second.begin();
+		     l != s->second.end(); l++) {
+			if (*l == "none") continue;
+			Indexes->push_back(new debTranslationsIndex(URI,Dist,s->first,(*l).c_str()));
+		}
+
+	return Indexes;
+}
 
-   return Indexes;
+void debReleaseIndex::PushSectionEntry(vector<string> const &Archs, const debSectionEntry *Entry) {
+	for (vector<string>::const_iterator a = Archs.begin();
+	     a != Archs.end(); ++a)
+		ArchEntries[*a].push_back(new debSectionEntry(Entry->Section, Entry->IsSrc));
+	delete Entry;
 }
 
-void debReleaseIndex::PushSectionEntry(const debSectionEntry *Entry)
-{
-   SectionEntries.push_back(Entry);
+void debReleaseIndex::PushSectionEntry(string const &Arch, const debSectionEntry *Entry) {
+	ArchEntries[Arch].push_back(Entry);
 }
 
-debReleaseIndex::debSectionEntry::debSectionEntry (string Section, bool IsSrc): Section(Section)
-{
-   this->IsSrc = IsSrc;
+void debReleaseIndex::PushSectionEntry(const debSectionEntry *Entry) {
+	if (Entry->IsSrc == true)
+		PushSectionEntry("source", Entry);
+	else {
+		for (map<string, vector<const debSectionEntry *> >::iterator a = ArchEntries.begin();
+		     a != ArchEntries.end(); ++a) {
+			a->second.push_back(Entry);
+		}
+	}
 }
 
+debReleaseIndex::debSectionEntry::debSectionEntry (string const &Section,
+		bool const &IsSrc): Section(Section), IsSrc(IsSrc)
+{}
+
 class debSLTypeDebian : public pkgSourceList::Type
 {
    protected:
 
-   bool CreateItemInternal(vector<metaIndex *> &List,string URI,
-			   string Dist,string Section,
-			   bool IsSrc) const
+   bool CreateItemInternal(vector<metaIndex *> &List, string const &URI,
+			   string const &Dist, string const &Section,
+			   bool const &IsSrc, map<string, string> const &Options) const
    {
-      for (vector<metaIndex *>::const_iterator I = List.begin(); 
+      map<string, string>::const_iterator const arch = Options.find("arch");
+      vector<string> const Archs =
+		(arch != Options.end()) ? VectorizeString(arch->second, ',') :
+				APT::Configuration::getArchitectures();
+
+      for (vector<metaIndex *>::const_iterator I = List.begin();
 	   I != List.end(); I++)
       {
-	 // This check insures that there will be only one Release file
-	 // queued for all the Packages files and Sources files it
-	 // corresponds to.
-		if (strcmp((*I)->GetType(), "deb") == 0)
+	 // We only worry about debian entries here
+	 if (strcmp((*I)->GetType(), "deb") != 0)
+	    continue;
+
+	 debReleaseIndex *Deb = (debReleaseIndex *) (*I);
+	 /* This check insures that there will be only one Release file
+	    queued for all the Packages files and Sources files it
+	    corresponds to. */
+	 if (Deb->GetURI() == URI && Deb->GetDist() == Dist)
 	 {
-	    debReleaseIndex *Deb = (debReleaseIndex *) (*I);
-	    // This check insures that there will be only one Release file
-	    // queued for all the Packages files and Sources files it
-	    // corresponds to.
-	    if (Deb->GetURI() == URI && Deb->GetDist() == Dist)
+	    if (IsSrc == true)
+	       Deb->PushSectionEntry("source", new debReleaseIndex::debSectionEntry(Section, IsSrc));
+	    else
 	    {
-	       Deb->PushSectionEntry(new debReleaseIndex::debSectionEntry(Section, IsSrc));
-	       return true;
+	       if (Dist[Dist.size() - 1] == '/')
+		  Deb->PushSectionEntry("any", new debReleaseIndex::debSectionEntry(Section, IsSrc));
+	       else
+		  Deb->PushSectionEntry(Archs, new debReleaseIndex::debSectionEntry(Section, IsSrc));
 	    }
+	    return true;
 	 }
       }
       // No currently created Release file indexes this entry, so we create a new one.
       // XXX determine whether this release is trusted or not
-      debReleaseIndex *Deb = new debReleaseIndex(URI,Dist);
-      Deb->PushSectionEntry (new debReleaseIndex::debSectionEntry(Section, IsSrc));
+      debReleaseIndex *Deb = new debReleaseIndex(URI, Dist);
+      if (IsSrc == true)
+	 Deb->PushSectionEntry ("source", new debReleaseIndex::debSectionEntry(Section, IsSrc));
+      else
+      {
+	 if (Dist[Dist.size() - 1] == '/')
+	    Deb->PushSectionEntry ("any", new debReleaseIndex::debSectionEntry(Section, IsSrc));
+	 else
+	    Deb->PushSectionEntry (Archs, new debReleaseIndex::debSectionEntry(Section, IsSrc));
+      }
       List.push_back(Deb);
       return true;
    }
@@ -266,10 +362,11 @@ class debSLTypeDeb : public debSLTypeDebian
 {
    public:
 
-   bool CreateItem(vector<metaIndex *> &List,string URI,
-		   string Dist,string Section) const
+   bool CreateItem(vector<metaIndex *> &List, string const &URI,
+		   string const &Dist, string const &Section,
+		   std::map<string, string> const &Options) const
    {
-      return CreateItemInternal(List, URI, Dist, Section, false);
+      return CreateItemInternal(List, URI, Dist, Section, false, Options);
    }
 
    debSLTypeDeb()
@@ -283,10 +380,11 @@ class debSLTypeDebSrc : public debSLTypeDebian
 {
    public:
 
-   bool CreateItem(vector<metaIndex *> &List,string URI,
-		   string Dist,string Section) const 
+   bool CreateItem(vector<metaIndex *> &List, string const &URI,
+		   string const &Dist, string const &Section,
+		   std::map<string, string> const &Options) const
    {
-      return CreateItemInternal(List, URI, Dist, Section, true);
+      return CreateItemInternal(List, URI, Dist, Section, true, Options);
    }
    
    debSLTypeDebSrc()

apt-pkg/deb/debmetaindex.h

@@ -5,40 +5,44 @@
 #include <apt-pkg/metaindex.h>
 #include <apt-pkg/sourcelist.h>
 
+#include <map>
+
 class debReleaseIndex : public metaIndex {
    public:
 
    class debSectionEntry
    {
       public:
-      debSectionEntry (string Section, bool IsSrc);
-      bool IsSrc;
-      string Section;
+      debSectionEntry (string const &Section, bool const &IsSrc);
+      string const Section;
+      bool const IsSrc;
    };
 
    private:
-   vector <const debSectionEntry *> SectionEntries;
+   std::map<string, vector<debSectionEntry const*> > ArchEntries;
 
    public:
 
-   debReleaseIndex(string URI, string Dist);
+   debReleaseIndex(string const &URI, string const &Dist);
    ~debReleaseIndex();
 
-   virtual string ArchiveURI(string File) const {return URI + File;};
-   virtual bool GetIndexes(pkgAcquire *Owner, bool GetAll=false) const;
+   virtual string ArchiveURI(string const &File) const {return URI + File;};
+   virtual bool GetIndexes(pkgAcquire *Owner, bool const &GetAll=false) const;
    vector <struct IndexTarget *>* ComputeIndexTargets() const;
-   string Info(const char *Type, const string Section) const;
+   string Info(const char *Type, string const &Section, string const &Arch="") const;
    string MetaIndexInfo(const char *Type) const;
    string MetaIndexFile(const char *Types) const;
    string MetaIndexURI(const char *Type) const;
-   string IndexURI(const char *Type, const string Section) const;
-   string IndexURISuffix(const char *Type, const string Section) const;
-   string SourceIndexURI(const char *Type, const string Section) const;
-   string SourceIndexURISuffix(const char *Type, const string Section) const;
+   string IndexURI(const char *Type, string const &Section, string const &Arch="native") const;
+   string IndexURISuffix(const char *Type, string const &Section, string const &Arch="native") const;
+   string SourceIndexURI(const char *Type, const string &Section) const;
+   string SourceIndexURISuffix(const char *Type, const string &Section) const;
    virtual vector <pkgIndexFile *> *GetIndexFiles();
 
    virtual bool IsTrusted() const;
 
+   void PushSectionEntry(vector<string> const &Archs, const debSectionEntry *Entry);
+   void PushSectionEntry(string const &Arch, const debSectionEntry *Entry);
    void PushSectionEntry(const debSectionEntry *Entry);
 };
 

apt-pkg/deb/debrecords.cc

@@ -11,6 +11,7 @@
 #include <apt-pkg/debrecords.h>
 #include <apt-pkg/strutl.h>
 #include <apt-pkg/error.h>
+#include <apt-pkg/aptconfiguration.h>
 #include <langinfo.h>
 									/*}}}*/
 
@@ -18,7 +19,7 @@
 // ---------------------------------------------------------------------
 /* */
 debRecordParser::debRecordParser(string FileName,pkgCache &Cache) : 
-                  File(FileName,FileFd::ReadOnly), 
+                  File(FileName,FileFd::ReadOnlyGzip), 
                   Tags(&File, std::max(Cache.Head().MaxVerFileSize, 
 				       Cache.Head().MaxDescFileSize) + 200)
 {
@@ -110,13 +111,18 @@ string debRecordParser::ShortDesc()
 string debRecordParser::LongDesc()
 {
   string orig, dest;
-  char *codeset = nl_langinfo(CODESET);
 
   if (!Section.FindS("Description").empty())
      orig = Section.FindS("Description").c_str();
-  else 
-     orig = Section.FindS(("Description-" + pkgIndexFile::LanguageCode()).c_str()).c_str();
+  else
+  {
+     vector<string> const lang = APT::Configuration::getLanguages();
+     for (vector<string>::const_iterator l = lang.begin();
+	  orig.empty() && l != lang.end(); l++)
+	orig = Section.FindS(string("Description-").append(*l).c_str());
+  }
 
+  char const * const codeset = nl_langinfo(CODESET);
   if (strcmp(codeset,"UTF-8") != 0) {
      UTF8ToCodeset(codeset, orig, &dest);
      orig = dest;

apt-pkg/deb/debsrcrecords.cc

@@ -54,7 +54,8 @@ const char **debSrcRecordParser::Binaries()
    package/version records representing the build dependency. The returned 
    array need not be freed and will be reused by the next call to this 
    function */
-bool debSrcRecordParser::BuildDepends(vector<pkgSrcRecords::Parser::BuildDepRec> &BuildDeps, bool ArchOnly)
+bool debSrcRecordParser::BuildDepends(vector<pkgSrcRecords::Parser::BuildDepRec> &BuildDeps,
+					bool const &ArchOnly, bool const &StripMultiArch)
 {
    unsigned int I;
    const char *Start, *Stop;
@@ -77,7 +78,7 @@ bool debSrcRecordParser::BuildDepends(vector<pkgSrcRecords::Parser::BuildDepRec>
       while (1)
       {
          Start = debListParser::ParseDepends(Start, Stop, 
-		     rec.Package,rec.Version,rec.Op,true);
+		     rec.Package,rec.Version,rec.Op,true, StripMultiArch);
 	 
          if (Start == 0) 
             return _error->Error("Problem parsing dependency: %s", fields[I]);

apt-pkg/deb/debsrcrecords.h

@@ -30,14 +30,14 @@ class debSrcRecordParser : public pkgSrcRecords::Parser
 
    virtual bool Restart() {return Tags.Jump(Sect,0);};
    virtual bool Step() {iOffset = Tags.Offset(); return Tags.Step(Sect);};
-   virtual bool Jump(unsigned long Off) {iOffset = Off; return Tags.Jump(Sect,Off);};
+   virtual bool Jump(unsigned long const &Off) {iOffset = Off; return Tags.Jump(Sect,Off);};
 
    virtual string Package() const {return Sect.FindS("Package");};
    virtual string Version() const {return Sect.FindS("Version");};
    virtual string Maintainer() const {return Sect.FindS("Maintainer");};
    virtual string Section() const {return Sect.FindS("Section");};
    virtual const char **Binaries();
-   virtual bool BuildDepends(vector<BuildDepRec> &BuildDeps, bool ArchOnly);
+   virtual bool BuildDepends(vector<BuildDepRec> &BuildDeps, bool const &ArchOnly, bool const &StripMultiArch = true);
    virtual unsigned long Offset() {return iOffset;};
    virtual string AsStr() 
    {
@@ -47,8 +47,8 @@ class debSrcRecordParser : public pkgSrcRecords::Parser
    };
    virtual bool Files(vector<pkgSrcRecords::File> &F);
 
-   debSrcRecordParser(string File,pkgIndexFile const *Index) 
-      : Parser(Index), Fd(File,FileFd::ReadOnly), Tags(&Fd,102400), 
+   debSrcRecordParser(string const &File,pkgIndexFile const *Index) 
+      : Parser(Index), Fd(File,FileFd::ReadOnlyGzip), Tags(&Fd,102400), 
         Buffer(0), BufSize(0) {}
    ~debSrcRecordParser();
 };

apt-pkg/deb/debsystem.cc

@@ -18,7 +18,6 @@
 #include <apt-pkg/error.h>
 #include <apt-pkg/fileutl.h>
 #include <apti18n.h>
-    
 #include <sys/types.h>
 #include <unistd.h>
 #include <dirent.h>
@@ -79,8 +78,15 @@ bool debSystem::Lock()
    {
       close(LockFD);
       LockFD = -1;
+      const char *cmd;
+      if (getenv("SUDO_USER") != NULL)
+	 cmd = "sudo dpkg --configure -a";
+      else
+	 cmd = "dpkg --configure -a";
+      // TRANSLATORS: the %s contains the recovery command, usually
+      //              dpkg --configure -a
       return _error->Error(_("dpkg was interrupted, you must manually "
-                             "run 'dpkg --configure -a' to correct the problem. "));
+                             "run '%s' to correct the problem. "), cmd);
    }
 
 	 LockCount++;
@@ -158,8 +164,8 @@ bool debSystem::Initialize(Configuration &Cnf)
    /* These really should be jammed into a generic 'Local Database' engine
       which is yet to be determined. The functions in pkgcachegen should
       be the only users of these */
-   Cnf.CndSet("Dir::State::userstatus","status.user"); // Defunct
-   Cnf.CndSet("Dir::State::status","/var/lib/dpkg/status");
+   Cnf.CndSet("Dir::State::extended_states", Cnf.FindDir("Dir::State").append("extended_states"));
+   Cnf.CndSet("Dir::State::status", Cnf.FindDir("Dir", "/").append("var/lib/dpkg/status"));
    Cnf.CndSet("Dir::Bin::dpkg","/usr/bin/dpkg");
 
    if (StatusFile) {

apt-pkg/deb/debversion.cc

@@ -190,8 +190,22 @@ int debVersioningSystem::DoCmpVersion(const char *A,const char *AEnd,
       dlhs++;
    if (drhs != rhs)
       drhs++;
-   
-   return CmpFragment(dlhs,AEnd,drhs,BEnd);
+
+   // no debian revision need to be treated like -0
+   if (*(dlhs-1) == '-' && *(drhs-1) == '-')
+      return CmpFragment(dlhs,AEnd,drhs,BEnd);
+   else if (*(dlhs-1) == '-')
+   {
+      const char* null = "0";
+      return CmpFragment(dlhs,AEnd,null, null+1);
+   }
+   else if (*(drhs-1) == '-')
+   {
+      const char* null = "0";
+      return CmpFragment(null, null+1, drhs, BEnd);
+   }
+   else
+      return 0;
 }
 									/*}}}*/
 // debVS::CheckDep - Check a single dependency				/*{{{*/

apt-pkg/deb/dpkgpm.cc

@@ -12,6 +12,7 @@
 #include <apt-pkg/error.h>
 #include <apt-pkg/configuration.h>
 #include <apt-pkg/depcache.h>
+#include <apt-pkg/pkgrecords.h>
 #include <apt-pkg/strutl.h>
 #include <apti18n.h>
 #include <apt-pkg/fileutl.h>
@@ -25,6 +26,7 @@
 #include <sys/wait.h>
 #include <signal.h>
 #include <errno.h>
+#include <string.h>
 #include <stdio.h>
 #include <string.h>
 #include <algorithm>
@@ -51,6 +53,7 @@ namespace
     std::make_pair("configure", N_("Configuring %s")),
     std::make_pair("remove",    N_("Removing %s")),
     std::make_pair("purge",    N_("Completely removing %s")),
+    std::make_pair("disappear", N_("Noting disappearance of %s")),
     std::make_pair("trigproc",  N_("Running post-installation trigger %s"))
   };
 
@@ -106,7 +109,7 @@ ionice(int PID)
 /* */
 pkgDPkgPM::pkgDPkgPM(pkgDepCache *Cache) 
    : pkgPackageManager(Cache), dpkgbuf_pos(0),
-     term_out(NULL), PackagesDone(0), PackagesTotal(0)
+     term_out(NULL), history_out(NULL), PackagesDone(0), PackagesTotal(0)
 {
 }
 									/*}}}*/
@@ -125,7 +128,19 @@ bool pkgDPkgPM::Install(PkgIterator Pkg,string File)
    if (File.empty() == true || Pkg.end() == true)
       return _error->Error("Internal Error, No file name for %s",Pkg.Name());
 
-   List.push_back(Item(Item::Install,Pkg,File));
+   // If the filename string begins with DPkg::Chroot-Directory, return the
+   // substr that is within the chroot so dpkg can access it.
+   string const chrootdir = _config->FindDir("DPkg::Chroot-Directory","/");
+   if (chrootdir != "/" && File.find(chrootdir) == 0)
+   {
+      size_t len = chrootdir.length();
+      if (chrootdir.at(len - 1) == '/')
+        len--;
+      List.push_back(Item(Item::Install,Pkg,File.substr(len)));
+   }
+   else
+      List.push_back(Item(Item::Install,Pkg,File));
+
    return true;
 }
 									/*}}}*/
@@ -342,7 +357,6 @@ bool pkgDPkgPM::RunScriptsWithPkgs(const char *Cnf)
 
    return true;
 }
-
 									/*}}}*/
 // DPkgPM::DoStdin - Read stdin and pass to slave pty			/*{{{*/
 // ---------------------------------------------------------------------
@@ -408,11 +422,12 @@ void pkgDPkgPM::ProcessDpkgStatusLine(int OutStatusFd, char *line)
       'processing: install: pkg'
       'processing: configure: pkg'
       'processing: remove: pkg'
-      'processing: purge: pkg' - but for apt is it a ignored "unknown" action
+      'processing: purge: pkg'
+      'processing: disappear: pkg'
       'processing: trigproc: trigger'
 	    
    */
-   char* list[5];
+   char* list[6];
    //        dpkg sends multiline error messages sometimes (see
    //        #374195 for a example. we should support this by
    //        either patching dpkg to not send multiline over the
@@ -455,11 +470,22 @@ void pkgDPkgPM::ProcessDpkgStatusLine(int OutStatusFd, char *line)
 	 write(OutStatusFd, status.str().c_str(), status.str().size());
       if (Debug == true)
 	 std::clog << "send: '" << status.str() << "'" << endl;
+
+      if (strncmp(action, "disappear", strlen("disappear")) == 0)
+	 handleDisappearAction(pkg_or_trigger);
       return;
    }
 
    if(strncmp(action,"error",strlen("error")) == 0)
    {
+      // urgs, sometime has ":" in its error string so that we
+      // end up with the error message split between list[3]
+      // and list[4], e.g. the message: 
+      // "failed in buffer_write(fd) (10, ret=-1): backend dpkg-deb ..."
+      // concat them again
+      if( list[4] != NULL )
+	 list[3][strlen(list[3])] = ':';
+
       status << "pmerror:" << list[1]
 	     << ":"  << (PackagesDone/float(PackagesTotal)*100.0) 
 	     << ":" << list[3]
@@ -468,6 +494,8 @@ void pkgDPkgPM::ProcessDpkgStatusLine(int OutStatusFd, char *line)
 	 write(OutStatusFd, status.str().c_str(), status.str().size());
       if (Debug == true)
 	 std::clog << "send: '" << status.str() << "'" << endl;
+      pkgFailures++;
+      WriteApportReport(list[1], list[3]);
       return;
    }
    else if(strncmp(action,"conffile",strlen("conffile")) == 0)
@@ -514,6 +542,53 @@ void pkgDPkgPM::ProcessDpkgStatusLine(int OutStatusFd, char *line)
 		<< " action: " << action << endl;
 }
 									/*}}}*/
+// DPkgPM::handleDisappearAction					/*{{{*/
+void pkgDPkgPM::handleDisappearAction(string const &pkgname)
+{
+   // record the package name for display and stuff later
+   disappearedPkgs.insert(pkgname);
+
+   pkgCache::PkgIterator Pkg = Cache.FindPkg(pkgname);
+   if (unlikely(Pkg.end() == true))
+      return;
+   // the disappeared package was auto-installed - nothing to do
+   if ((Cache[Pkg].Flags & pkgCache::Flag::Auto) == pkgCache::Flag::Auto)
+      return;
+   pkgCache::VerIterator PkgVer = Cache[Pkg].InstVerIter(Cache);
+   if (unlikely(PkgVer.end() == true))
+      return;
+   /* search in the list of dependencies for (Pre)Depends,
+      check if this dependency has a Replaces on our package
+      and if so transfer the manual installed flag to it */
+   for (pkgCache::DepIterator Dep = PkgVer.DependsList(); Dep.end() != true; ++Dep)
+   {
+      if (Dep->Type != pkgCache::Dep::Depends &&
+	  Dep->Type != pkgCache::Dep::PreDepends)
+	 continue;
+      pkgCache::PkgIterator Tar = Dep.TargetPkg();
+      if (unlikely(Tar.end() == true))
+	 continue;
+      // the package is already marked as manual
+      if ((Cache[Tar].Flags & pkgCache::Flag::Auto) != pkgCache::Flag::Auto)
+	 continue;
+      pkgCache::VerIterator TarVer =  Cache[Tar].InstVerIter(Cache);
+      if (TarVer.end() == true)
+	 continue;
+      for (pkgCache::DepIterator Rep = TarVer.DependsList(); Rep.end() != true; ++Rep)
+      {
+	 if (Rep->Type != pkgCache::Dep::Replaces)
+	    continue;
+	 if (Pkg != Rep.TargetPkg())
+	    continue;
+	 // okay, they are strongly connected - transfer manual-bit
+	 if (Debug == true)
+	    std::clog << "transfer manual-bit from disappeared »" << pkgname << "« to »" << Tar.FullName() << "«" << std::endl;
+	 Cache[Tar].Flags &= ~Flag::Auto;
+	 break;
+      }
+   }
+}
+									/*}}}*/
 // DPkgPM::DoDpkgStatusFd						/*{{{*/
 // ---------------------------------------------------------------------
 /*
@@ -552,67 +627,77 @@ void pkgDPkgPM::DoDpkgStatusFd(int statusfd, int OutStatusFd)
 }
 									/*}}}*/
 // DPkgPM::WriteHistoryTag						/*{{{*/
-void pkgDPkgPM::WriteHistoryTag(string tag, string value)
+void pkgDPkgPM::WriteHistoryTag(string const &tag, string value)
 {
-   if (value.size() > 0)
-   {
-      // poor mans rstrip(", ")
-      if (value[value.size()-2] == ',' && value[value.size()-1] == ' ')
-	 value.erase(value.size() - 2, 2);
-      fprintf(history_out, "%s: %s\n", tag.c_str(), value.c_str());
-   }
+   size_t const length = value.length();
+   if (length == 0)
+      return;
+   // poor mans rstrip(", ")
+   if (value[length-2] == ',' && value[length-1] == ' ')
+      value.erase(length - 2, 2);
+   fprintf(history_out, "%s: %s\n", tag.c_str(), value.c_str());
 }									/*}}}*/
 // DPkgPM::OpenLog							/*{{{*/
 bool pkgDPkgPM::OpenLog()
 {
-   string logdir = _config->FindDir("Dir::Log");
-   if(not FileExists(logdir))
+   string const logdir = _config->FindDir("Dir::Log");
+   if(CheckDirectory(logdir, logdir) == false)
+      // FIXME: use a better string after freeze
       return _error->Error(_("Directory '%s' missing"), logdir.c_str());
 
    // get current time
    char timestr[200];
-   time_t t = time(NULL);
-   struct tm *tmp = localtime(&t);
+   time_t const t = time(NULL);
+   struct tm const * const tmp = localtime(&t);
    strftime(timestr, sizeof(timestr), "%F  %T", tmp);
 
    // open terminal log
-   string logfile_name = flCombine(logdir,
+   string const logfile_name = flCombine(logdir,
 				   _config->Find("Dir::Log::Terminal"));
    if (!logfile_name.empty())
    {
       term_out = fopen(logfile_name.c_str(),"a");
       if (term_out == NULL)
-	 return _error->WarningE(_("Could not open file '%s'"), logfile_name.c_str());
+	 return _error->WarningE("OpenLog", _("Could not open file '%s'"), logfile_name.c_str());
       setvbuf(term_out, NULL, _IONBF, 0);
       chmod(logfile_name.c_str(), 0600);
       fprintf(term_out, "\nLog started: %s\n", timestr);
    }
 
-   // write 
-   string history_name = flCombine(logdir,
+   // write your history
+   string const history_name = flCombine(logdir,
 				   _config->Find("Dir::Log::History"));
    if (!history_name.empty())
    {
       history_out = fopen(history_name.c_str(),"a");
+      if (history_out == NULL)
+	 return _error->WarningE("OpenLog", _("Could not open file '%s'"), history_name.c_str());
       chmod(history_name.c_str(), 0644);
       fprintf(history_out, "\nStart-Date: %s\n", timestr);
       string remove, purge, install, upgrade, downgrade;
       for (pkgCache::PkgIterator I = Cache.PkgBegin(); I.end() == false; I++)
       {
 	 if (Cache[I].NewInstall())
-	    install += I.Name() + string(" (") + Cache[I].CandVersion + string("), ");
+	 {
+	    install += I.FullName(false) + string(" (") + Cache[I].CandVersion;
+	    if (Cache[I].Flags & pkgCache::Flag::Auto)
+	       install+= ", automatic";
+	    install += string("), ");
+	 }
 	 else if (Cache[I].Upgrade())
-	    upgrade += I.Name() + string(" (") + Cache[I].CurVersion + string(", ") + Cache[I].CandVersion + string("), ");
+	    upgrade += I.FullName(false) + string(" (") + Cache[I].CurVersion + string(", ") + Cache[I].CandVersion + string("), ");
 	 else if (Cache[I].Downgrade())
-	    downgrade += I.Name() + string(" (") + Cache[I].CurVersion + string(", ") + Cache[I].CandVersion + string("), ");
+	    downgrade += I.FullName(false) + string(" (") + Cache[I].CurVersion + string(", ") + Cache[I].CandVersion + string("), ");
 	 else if (Cache[I].Delete())
 	 {
 	    if ((Cache[I].iFlags & pkgDepCache::Purge) == pkgDepCache::Purge)
-	       purge += I.Name() + string(" (") + Cache[I].CurVersion + string("), ");	    
+	       purge += I.FullName(false) + string(" (") + Cache[I].CurVersion + string("), ");	    
 	    else
-	       remove += I.Name() + string(" (") + Cache[I].CurVersion + string("), ");	    
+	       remove += I.FullName(false) + string(" (") + Cache[I].CurVersion + string("), ");	    
 	 }
       }
+      if (_config->Exists("Commandline::AsString") == true)
+	 WriteHistoryTag("Commandline", _config->Find("Commandline::AsString"));
       WriteHistoryTag("Install", install);
       WriteHistoryTag("Upgrade", upgrade);
       WriteHistoryTag("Downgrade",downgrade);
@@ -643,11 +728,27 @@ bool pkgDPkgPM::CloseLog()
 
    if(history_out)
    {
-      if (dpkg_error.size() > 0)
+      if (disappearedPkgs.empty() == false)
+      {
+	 string disappear;
+	 for (std::set<std::string>::const_iterator d = disappearedPkgs.begin();
+	      d != disappearedPkgs.end(); ++d)
+	 {
+	    pkgCache::PkgIterator P = Cache.FindPkg(*d);
+	    disappear.append(*d);
+	    if (P.end() == true)
+	       disappear.append(", ");
+	    else
+	       disappear.append(" (").append(Cache[P].CurVersion).append("), ");
+	 }
+	 WriteHistoryTag("Disappeared", disappear);
+      }
+      if (dpkg_error.empty() == false)
 	 fprintf(history_out, "Error: %s\n", dpkg_error.c_str());
       fprintf(history_out, "End-Date: %s\n", timestr);
       fclose(history_out);
    }
+   history_out = NULL;
 
    return true;
 }
@@ -895,6 +996,8 @@ bool pkgDPkgPM::Go(int OutStatusFd)
 	 {
 	    if((*I).Pkg.end() == true)
 	       continue;
+	    if (I->Op == Item::Configure && disappearedPkgs.find(I->Pkg.Name()) != disappearedPkgs.end())
+	       continue;
 	    Args[n++] = I->Pkg.Name();
 	    Size += strlen(Args[n-1]);
 	 }	 
@@ -1065,7 +1168,7 @@ bool pkgDPkgPM::Go(int OutStatusFd)
 
 	 // wait for input or output here
 	 FD_ZERO(&rfds);
-	 if (!stdin_is_dev_null)
+	 if (master >= 0 && !stdin_is_dev_null)
 	    FD_SET(0, &rfds); 
 	 FD_SET(_dpkgin, &rfds);
 	 if(master >= 0)
@@ -1152,3 +1255,186 @@ void pkgDPkgPM::Reset()
    List.erase(List.begin(),List.end());
 }
 									/*}}}*/
+// pkgDpkgPM::WriteApportReport - write out error report pkg failure	/*{{{*/
+// ---------------------------------------------------------------------
+/* */
+void pkgDPkgPM::WriteApportReport(const char *pkgpath, const char *errormsg) 
+{
+   string pkgname, reportfile, srcpkgname, pkgver, arch;
+   string::size_type pos;
+   FILE *report;
+
+   if (_config->FindB("Dpkg::ApportFailureReport", false) == false)
+   {
+      std::clog << "configured to not write apport reports" << std::endl;
+      return;
+   }
+
+   // only report the first errors
+   if(pkgFailures > _config->FindI("APT::Apport::MaxReports", 3))
+   {
+      std::clog << _("No apport report written because MaxReports is reached already") << std::endl;
+      return;
+   }
+
+   // check if its not a follow up error 
+   const char *needle = dgettext("dpkg", "dependency problems - leaving unconfigured");
+   if(strstr(errormsg, needle) != NULL) {
+      std::clog << _("No apport report written because the error message indicates its a followup error from a previous failure.") << std::endl;
+      return;
+   }
+
+   // do not report disk-full failures 
+   if(strstr(errormsg, strerror(ENOSPC)) != NULL) {
+      std::clog << _("No apport report written because the error message indicates a disk full error") << std::endl;
+      return;
+   }
+
+   // do not report out-of-memory failures 
+   if(strstr(errormsg, strerror(ENOMEM)) != NULL) {
+      std::clog << _("No apport report written because the error message indicates a out of memory error") << std::endl;
+      return;
+   }
+
+   // do not report dpkg I/O errors
+   // XXX - this message is localized, but this only matches the English version.  This is better than nothing.
+   if(strstr(errormsg, "short read in buffer_copy (")) {
+      std::clog << _("No apport report written because the error message indicates a dpkg I/O error") << std::endl;
+      return;
+   }
+
+   // get the pkgname and reportfile
+   pkgname = flNotDir(pkgpath);
+   pos = pkgname.find('_');
+   if(pos != string::npos)
+      pkgname = pkgname.substr(0, pos);
+
+   // find the package versin and source package name
+   pkgCache::PkgIterator Pkg = Cache.FindPkg(pkgname);
+   if (Pkg.end() == true)
+      return;
+   pkgCache::VerIterator Ver = Cache.GetCandidateVer(Pkg);
+   if (Ver.end() == true)
+      return;
+   pkgver = Ver.VerStr() == NULL ? "unknown" : Ver.VerStr();
+   pkgRecords Recs(Cache);
+   pkgRecords::Parser &Parse = Recs.Lookup(Ver.FileList());
+   srcpkgname = Parse.SourcePkg();
+   if(srcpkgname.empty())
+      srcpkgname = pkgname;
+
+   // if the file exists already, we check:
+   // - if it was reported already (touched by apport). 
+   //   If not, we do nothing, otherwise
+   //    we overwrite it. This is the same behaviour as apport
+   // - if we have a report with the same pkgversion already
+   //   then we skip it
+   reportfile = flCombine("/var/crash",pkgname+".0.crash");
+   if(FileExists(reportfile))
+   {
+      struct stat buf;
+      char strbuf[255];
+
+      // check atime/mtime
+      stat(reportfile.c_str(), &buf);
+      if(buf.st_mtime > buf.st_atime)
+	 return;
+
+      // check if the existing report is the same version
+      report = fopen(reportfile.c_str(),"r");
+      while(fgets(strbuf, sizeof(strbuf), report) != NULL)
+      {
+	 if(strstr(strbuf,"Package:") == strbuf)
+	 {
+	    char pkgname[255], version[255];
+	    if(sscanf(strbuf, "Package: %s %s", pkgname, version) == 2)
+	       if(strcmp(pkgver.c_str(), version) == 0)
+	       {
+		  fclose(report);
+		  return;
+	       }
+	 }
+      }
+      fclose(report);
+   }
+
+   // now write the report
+   arch = _config->Find("APT::Architecture");
+   report = fopen(reportfile.c_str(),"w");
+   if(report == NULL)
+      return;
+   if(_config->FindB("DPkgPM::InitialReportOnly",false) == true)
+      chmod(reportfile.c_str(), 0);
+   else
+      chmod(reportfile.c_str(), 0600);
+   fprintf(report, "ProblemType: Package\n");
+   fprintf(report, "Architecture: %s\n", arch.c_str());
+   time_t now = time(NULL);
+   fprintf(report, "Date: %s" , ctime(&now));
+   fprintf(report, "Package: %s %s\n", pkgname.c_str(), pkgver.c_str());
+   fprintf(report, "SourcePackage: %s\n", srcpkgname.c_str());
+   fprintf(report, "ErrorMessage:\n %s\n", errormsg);
+
+   // ensure that the log is flushed
+   if(term_out)
+      fflush(term_out);
+
+   // attach terminal log it if we have it
+   string logfile_name = _config->FindFile("Dir::Log::Terminal");
+   if (!logfile_name.empty())
+   {
+      FILE *log = NULL;
+      char buf[1024];
+
+      fprintf(report, "DpkgTerminalLog:\n");
+      log = fopen(logfile_name.c_str(),"r");
+      if(log != NULL)
+      {
+	 while( fgets(buf, sizeof(buf), log) != NULL)
+	    fprintf(report, " %s", buf);
+	 fclose(log);
+      }
+   }
+
+   // log the ordering 
+   const char *ops_str[] = {"Install", "Configure","Remove","Purge"};
+   fprintf(report, "AptOrdering:\n");
+   for (vector<Item>::iterator I = List.begin(); I != List.end(); I++)
+      fprintf(report, " %s: %s\n", (*I).Pkg.Name(), ops_str[(*I).Op]);
+
+   // attach dmesg log (to learn about segfaults)
+   if (FileExists("/bin/dmesg"))
+   {
+      FILE *log = NULL;
+      char buf[1024];
+
+      fprintf(report, "Dmesg:\n");
+      log = popen("/bin/dmesg","r");
+      if(log != NULL)
+      {
+	 while( fgets(buf, sizeof(buf), log) != NULL)
+	    fprintf(report, " %s", buf);
+	 fclose(log);
+      }
+   }
+
+   // attach df -l log (to learn about filesystem status)
+   if (FileExists("/bin/df"))
+   {
+      FILE *log = NULL;
+      char buf[1024];
+
+      fprintf(report, "Df:\n");
+      log = popen("/bin/df -l","r");
+      if(log != NULL)
+      {
+	 while( fgets(buf, sizeof(buf), log) != NULL)
+	    fprintf(report, " %s", buf);
+	 fclose(log);
+      }
+   }
+
+   fclose(report);
+
+}
+									/*}}}*/

apt-pkg/deb/dpkgpm.h

@@ -32,7 +32,23 @@ class pkgDPkgPM : public pkgPackageManager
    FILE *history_out;
    string dpkg_error;
 
+   /** \brief record the disappear action and handle accordingly
+
+      dpkg let packages disappear then they have no files any longer and
+      nothing depends on them. We need to collect this as dpkg as well as
+      APT doesn't know beforehand that the package will disappear, so the
+      only possible option is to tell the user afterwards about it.
+      To enhance the experience we also try to forward the auto-install
+      flag so the disappear-causer(s) are not autoremoved next time -
+      for the transfer to happen the disappeared version needs to depend
+      on the package the flag should be forwarded to and this package
+      needs to declare a Replaces on the disappeared package.
+      \param pkgname Name of the package that disappeared
+   */
+   void handleDisappearAction(string const &pkgname);
+
    protected:
+   int pkgFailures;
 
    // progress reporting
    struct DpkgState 
@@ -49,6 +65,7 @@ class pkgDPkgPM : public pkgPackageManager
    // the int is the state that is already done (e.g. a package that is
    // going to be install is already in state "half-installed")
    map<string,unsigned int> PackageOpsDone;
+
    // progress reporting
    unsigned int PackagesDone;
    unsigned int PackagesTotal;
@@ -68,7 +85,10 @@ class pkgDPkgPM : public pkgPackageManager
    // Helpers
    bool RunScriptsWithPkgs(const char *Cnf);
    bool SendV2Pkgs(FILE *F);
-   void WriteHistoryTag(string tag, string value);
+   void WriteHistoryTag(string const &tag, string value);
+
+   // apport integration
+   void WriteApportReport(const char *pkgpath, const char *errormsg);
 
    // dpkg log
    bool OpenLog();

apt-pkg/depcache.cc

@@ -17,6 +17,7 @@
 #include <apt-pkg/fileutl.h>
 #include <apt-pkg/strutl.h>
 #include <apt-pkg/configuration.h>
+#include <apt-pkg/aptconfiguration.h>
 #include <apt-pkg/pkgsystem.h>
 #include <apt-pkg/tagfile.h>
 
@@ -164,38 +165,50 @@ bool pkgDepCache::Init(OpProgress *Prog)
 bool pkgDepCache::readStateFile(OpProgress *Prog)			/*{{{*/
 {
    FileFd state_file;
-   string state = _config->FindDir("Dir::State") + "extended_states";
+   string const state = _config->FindFile("Dir::State::extended_states");
    if(FileExists(state)) {
       state_file.Open(state, FileFd::ReadOnly);
-      int file_size = state_file.Size();
+      int const file_size = state_file.Size();
       if(Prog != NULL)
 	 Prog->OverallProgress(0, file_size, 1, 
 			       _("Reading state information"));
 
       pkgTagFile tagfile(&state_file);
       pkgTagSection section;
-      int amt=0;
-      bool debug_autoremove=_config->FindB("Debug::pkgAutoRemove",false);
+      int amt = 0;
+      bool const debug_autoremove = _config->FindB("Debug::pkgAutoRemove",false);
       while(tagfile.Step(section)) {
-	 string pkgname = section.FindS("Package");
-	 pkgCache::PkgIterator pkg=Cache->FindPkg(pkgname);
-	 // Silently ignore unknown packages and packages with no actual
-	 // version.
-	 if(!pkg.end() && !pkg.VersionList().end()) {
-	    short reason = section.FindI("Auto-Installed", 0);
-	    if(reason > 0)
-	       PkgState[pkg->ID].Flags  |= Flag::Auto;
-	    if(debug_autoremove)
-	       std::clog << "Auto-Installed : " << pkgname << std::endl;
-	    amt+=section.size();
-	    if(Prog != NULL)
-	       Prog->OverallProgress(amt, file_size, 1, 
-				     _("Reading state information"));
+	 string const pkgname = section.FindS("Package");
+	 string pkgarch = section.FindS("Architecture");
+	 if (pkgarch.empty() == true)
+	    pkgarch = "any";
+	 pkgCache::PkgIterator pkg = Cache->FindPkg(pkgname, pkgarch);
+	 // Silently ignore unknown packages and packages with no actual version.
+	 if(pkg.end() == true || pkg->VersionList == 0)
+	    continue;
+
+	 short const reason = section.FindI("Auto-Installed", 0);
+	 if(reason > 0)
+	 {
+	    PkgState[pkg->ID].Flags |= Flag::Auto;
+	    if (unlikely(debug_autoremove))
+	       std::clog << "Auto-Installed : " << pkg.FullName() << std::endl;
+	    if (pkgarch == "any")
+	    {
+	       pkgCache::GrpIterator G = pkg.Group();
+	       for (pkg = G.NextPkg(pkg); pkg.end() != true; pkg = G.NextPkg(pkg))
+		  if (pkg->VersionList != 0)
+		     PkgState[pkg->ID].Flags |= Flag::Auto;
+	    }
 	 }
+	 amt += section.size();
 	 if(Prog != NULL)
-	    Prog->OverallProgress(file_size, file_size, 1, 
+	    Prog->OverallProgress(amt, file_size, 1, 
 				  _("Reading state information"));
       }
+      if(Prog != NULL)
+	 Prog->OverallProgress(file_size, file_size, 1,
+			       _("Reading state information"));
    }
 
    return true;
@@ -203,18 +216,18 @@ bool pkgDepCache::readStateFile(OpProgress *Prog)			/*{{{*/
 									/*}}}*/
 bool pkgDepCache::writeStateFile(OpProgress *prog, bool InstalledOnly)	/*{{{*/
 {
-   bool debug_autoremove = _config->FindB("Debug::pkgAutoRemove",false);
+   bool const debug_autoremove = _config->FindB("Debug::pkgAutoRemove",false);
    
    if(debug_autoremove)
       std::clog << "pkgDepCache::writeStateFile()" << std::endl;
 
    FileFd StateFile;
-   string state = _config->FindDir("Dir::State") + "extended_states";
+   string const state = _config->FindFile("Dir::State::extended_states");
 
    // if it does not exist, create a empty one
    if(!FileExists(state)) 
    {
-      StateFile.Open(state, FileFd::WriteEmpty);
+      StateFile.Open(state, FileFd::WriteAtomic);
       StateFile.Close();
    }
 
@@ -224,7 +237,7 @@ bool pkgDepCache::writeStateFile(OpProgress *prog, bool InstalledOnly)	/*{{{*/
 			   state.c_str());
 
    FILE *OutFile;
-   string outfile = state + ".tmp";
+   string const outfile = state + ".tmp";
    if((OutFile = fopen(outfile.c_str(),"w")) == NULL)
       return _error->Error(_("Failed to write temporary StateFile %s"),
 			   outfile.c_str());
@@ -235,46 +248,72 @@ bool pkgDepCache::writeStateFile(OpProgress *prog, bool InstalledOnly)	/*{{{*/
    std::set<string> pkgs_seen;
    const char *nullreorderlist[] = {0};
    while(tagfile.Step(section)) {
-	 string pkgname = section.FindS("Package");
+	 string const pkgname = section.FindS("Package");
+	 string pkgarch = section.FindS("Architecture");
+	 if (pkgarch.empty() == true)
+	    pkgarch = "native";
 	 // Silently ignore unknown packages and packages with no actual
 	 // version.
-	 pkgCache::PkgIterator pkg=Cache->FindPkg(pkgname);
+	 pkgCache::PkgIterator pkg = Cache->FindPkg(pkgname, pkgarch);
 	 if(pkg.end() || pkg.VersionList().end()) 
 	    continue;
-	 bool newAuto = (PkgState[pkg->ID].Flags & Flag::Auto);
+	 StateCache const &P = PkgState[pkg->ID];
+	 bool newAuto = (P.Flags & Flag::Auto);
+	 // skip not installed or now-removed ones if requested
+	 if (InstalledOnly && (
+	     (pkg->CurrentVer == 0 && P.Mode != ModeInstall) ||
+	     (pkg->CurrentVer != 0 && P.Mode == ModeDelete)))
+	 {
+	    // The section is obsolete if it contains no other tag
+	    unsigned int const count = section.Count();
+	    if (count < 2 ||
+		(count == 2 && section.Exists("Auto-Installed")) ||
+		(count == 3 && section.Exists("Auto-Installed") && section.Exists("Architecture")))
+	       continue;
+	    else
+	       newAuto = false;
+	 }
 	 if(_config->FindB("Debug::pkgAutoRemove",false))
 	    std::clog << "Update existing AutoInstall info: " 
-		      << pkg.Name() << std::endl;
-	 TFRewriteData rewrite[2];
-	 rewrite[0].Tag = "Auto-Installed";
-	 rewrite[0].Rewrite = newAuto ? "1" : "0";
+		      << pkg.FullName() << std::endl;
+	 TFRewriteData rewrite[3];
+	 rewrite[0].Tag = "Architecture";
+	 rewrite[0].Rewrite = pkg.Arch();
 	 rewrite[0].NewTag = 0;
-	 rewrite[1].Tag = 0;
+	 rewrite[1].Tag = "Auto-Installed";
+	 rewrite[1].Rewrite = newAuto ? "1" : "0";
+	 rewrite[1].NewTag = 0;
+	 rewrite[2].Tag = 0;
 	 TFRewrite(OutFile, section, nullreorderlist, rewrite);
 	 fprintf(OutFile,"\n");
-	 pkgs_seen.insert(pkgname);
+	 pkgs_seen.insert(pkg.FullName());
    }
    
    // then write the ones we have not seen yet
    std::ostringstream ostr;
    for(pkgCache::PkgIterator pkg=Cache->PkgBegin(); !pkg.end(); pkg++) {
-      if(PkgState[pkg->ID].Flags & Flag::Auto) {
-	 if (pkgs_seen.find(pkg.Name()) != pkgs_seen.end()) {
+      StateCache const &P = PkgState[pkg->ID];
+      if(P.Flags & Flag::Auto) {
+	 if (pkgs_seen.find(pkg.FullName()) != pkgs_seen.end()) {
 	    if(debug_autoremove)
-	       std::clog << "Skipping already written " << pkg.Name() << std::endl;
+	       std::clog << "Skipping already written " << pkg.FullName() << std::endl;
 	    continue;
 	 }
-         // skip not installed ones if requested
-         if(InstalledOnly && pkg->CurrentVer == 0)
-            continue;
+	 // skip not installed ones if requested
+	 if (InstalledOnly && (
+	     (pkg->CurrentVer == 0 && P.Mode != ModeInstall) ||
+	     (pkg->CurrentVer != 0 && P.Mode == ModeDelete)))
+	    continue;
+	 const char* const pkgarch = pkg.Arch();
+	 if (strcmp(pkgarch, "all") == 0)
+	    continue;
 	 if(debug_autoremove)
-	    std::clog << "Writing new AutoInstall: " 
-		      << pkg.Name() << std::endl;
+	    std::clog << "Writing new AutoInstall: " << pkg.FullName() << std::endl;
 	 ostr.str(string(""));
-	 ostr << "Package: " << pkg.Name() 
+	 ostr << "Package: " << pkg.Name()
+	      << "\nArchitecture: " << pkgarch
 	      << "\nAuto-Installed: 1\n\n";
 	 fprintf(OutFile,"%s",ostr.str().c_str());
-	 fprintf(OutFile,"\n");
       }
    }
    fclose(OutFile);
@@ -299,7 +338,7 @@ bool pkgDepCache::CheckDep(DepIterator Dep,int Type,PkgIterator &Res)
    /* Check simple depends. A depends -should- never self match but 
       we allow it anyhow because dpkg does. Technically it is a packaging
       bug. Conflicts may never self match */
-   if (Dep.TargetPkg() != Dep.ParentPkg() || 
+   if (Dep.TargetPkg()->Group != Dep.ParentPkg()->Group ||
        (Dep->Type != Dep::Conflicts && Dep->Type != Dep::DpkgBreaks && Dep->Type != Dep::Obsoletes))
    {
       PkgIterator Pkg = Dep.TargetPkg();
@@ -328,9 +367,9 @@ bool pkgDepCache::CheckDep(DepIterator Dep,int Type,PkgIterator &Res)
    PkgIterator Pkg = Dep.ParentPkg();
    for (; P.end() != true; P++)
    {
-      /* Provides may never be applied against the same package if it is
-         a conflicts. See the comment above. */
-      if (P.OwnerPkg() == Pkg &&
+      /* Provides may never be applied against the same package (or group)
+         if it is a conflicts. See the comment above. */
+      if (P.OwnerPkg()->Group == Pkg->Group &&
 	  (Dep->Type == Dep::Conflicts || Dep->Type == Dep::DpkgBreaks))
 	 continue;
       
@@ -368,8 +407,60 @@ bool pkgDepCache::CheckDep(DepIterator Dep,int Type,PkgIterator &Res)
 									/*}}}*/
 // DepCache::AddSizes - Add the packages sizes to the counters		/*{{{*/
 // ---------------------------------------------------------------------
-/* Call with Mult = -1 to preform the inverse opration */
-void pkgDepCache::AddSizes(const PkgIterator &Pkg,signed long Mult)
+/* Call with Mult = -1 to preform the inverse opration
+   The Mult increases the complexity of the calulations here and is unused -
+   or do we really have a usecase for removing the size of a package two
+   times? So let us replace it with a simple bool and be done with it… */
+__deprecated void pkgDepCache::AddSizes(const PkgIterator &Pkg,signed long Mult)
+{
+   StateCache &P = PkgState[Pkg->ID];
+   
+   if (Pkg->VersionList == 0)
+      return;
+   
+   if (Pkg.State() == pkgCache::PkgIterator::NeedsConfigure && 
+       P.Keep() == true)
+      return;
+   
+   // Compute the size data
+   if (P.NewInstall() == true)
+   {
+      iUsrSize += (signed long long)(Mult*P.InstVerIter(*this)->InstalledSize);
+      iDownloadSize += (signed long long)(Mult*P.InstVerIter(*this)->Size);
+      return;
+   }
+   
+   // Upgrading
+   if (Pkg->CurrentVer != 0 && 
+       (P.InstallVer != (Version *)Pkg.CurrentVer() || 
+	(P.iFlags & ReInstall) == ReInstall) && P.InstallVer != 0)
+   {
+      iUsrSize += (signed long long)(Mult*((signed long long)P.InstVerIter(*this)->InstalledSize - 
+			(signed long long)Pkg.CurrentVer()->InstalledSize));
+      iDownloadSize += (signed long long)(Mult*P.InstVerIter(*this)->Size);
+      return;
+   }
+   
+   // Reinstall
+   if (Pkg.State() == pkgCache::PkgIterator::NeedsUnpack &&
+       P.Delete() == false)
+   {
+      iDownloadSize += (signed long long)(Mult*P.InstVerIter(*this)->Size);
+      return;
+   }
+   
+   // Removing
+   if (Pkg->CurrentVer != 0 && P.InstallVer == 0)
+   {
+      iUsrSize -= (signed long long)(Mult*Pkg.CurrentVer()->InstalledSize);
+      return;
+   }   
+}
+									/*}}}*/
+// DepCache::AddSizes - Add the packages sizes to the counters		/*{{{*/
+// ---------------------------------------------------------------------
+/* Call with Inverse = true to preform the inverse opration */
+void pkgDepCache::AddSizes(const PkgIterator &Pkg, bool const &Inverse)
 {
    StateCache &P = PkgState[Pkg->ID];
    
@@ -383,8 +474,13 @@ void pkgDepCache::AddSizes(const PkgIterator &Pkg,signed long Mult)
    // Compute the size data
    if (P.NewInstall() == true)
    {
-      iUsrSize += (signed)(Mult*P.InstVerIter(*this)->InstalledSize);
-      iDownloadSize += (signed)(Mult*P.InstVerIter(*this)->Size);
+      if (Inverse == false) {
+	 iUsrSize += P.InstVerIter(*this)->InstalledSize;
+	 iDownloadSize += P.InstVerIter(*this)->Size;
+      } else {
+	 iUsrSize -= P.InstVerIter(*this)->InstalledSize;
+	 iDownloadSize -= P.InstVerIter(*this)->Size;
+      }
       return;
    }
    
@@ -393,9 +489,15 @@ void pkgDepCache::AddSizes(const PkgIterator &Pkg,signed long Mult)
        (P.InstallVer != (Version *)Pkg.CurrentVer() || 
 	(P.iFlags & ReInstall) == ReInstall) && P.InstallVer != 0)
    {
-      iUsrSize += (signed)(Mult*((signed)P.InstVerIter(*this)->InstalledSize - 
-			(signed)Pkg.CurrentVer()->InstalledSize));
-      iDownloadSize += (signed)(Mult*P.InstVerIter(*this)->Size);
+      if (Inverse == false) {
+	 iUsrSize -= Pkg.CurrentVer()->InstalledSize;
+	 iUsrSize += P.InstVerIter(*this)->InstalledSize;
+	 iDownloadSize += P.InstVerIter(*this)->Size;
+      } else {
+	 iUsrSize -= P.InstVerIter(*this)->InstalledSize;
+	 iUsrSize += Pkg.CurrentVer()->InstalledSize;
+	 iDownloadSize -= P.InstVerIter(*this)->Size;
+      }
       return;
    }
    
@@ -403,14 +505,20 @@ void pkgDepCache::AddSizes(const PkgIterator &Pkg,signed long Mult)
    if (Pkg.State() == pkgCache::PkgIterator::NeedsUnpack &&
        P.Delete() == false)
    {
-      iDownloadSize += (signed)(Mult*P.InstVerIter(*this)->Size);
+      if (Inverse == false)
+	 iDownloadSize += P.InstVerIter(*this)->Size;
+      else
+	 iDownloadSize -= P.InstVerIter(*this)->Size;
       return;
    }
    
    // Removing
    if (Pkg->CurrentVer != 0 && P.InstallVer == 0)
    {
-      iUsrSize -= (signed)(Mult*Pkg.CurrentVer()->InstalledSize);
+      if (Inverse == false)
+	 iUsrSize -= Pkg.CurrentVer()->InstalledSize;
+      else
+	 iUsrSize += Pkg.CurrentVer()->InstalledSize;
       return;
    }   
 }
@@ -596,6 +704,107 @@ void pkgDepCache::UpdateVerState(PkgIterator Pkg)
    }
 }
 									/*}}}*/
+// DepCache::RemovePseudoInstalledPkg - MultiArch helper for Update()	/*{{{*/
+// ---------------------------------------------------------------------
+/* We "install" arch all packages for all archs if it is installed. Many
+   of these will be broken. This method will look at these broken Pkg and
+   "remove" it. */
+bool pkgDepCache::RemovePseudoInstalledPkg(PkgIterator &Pkg, std::set<unsigned long> &recheck) {
+   if (unlikely(Pkg->CurrentVer == 0))
+      return false;
+
+   VerIterator V = Pkg.CurrentVer();
+   if (V->MultiArch != Version::All)
+      return false;
+
+   // Never ever kill an "all" package - they have no dependency so they can't be broken
+   if (strcmp(Pkg.Arch(),"all") == 0)
+      return false;
+
+   unsigned char const CurDepState = VersionState(V.DependsList(),DepInstall,DepInstMin,DepInstPolicy);
+   if ((CurDepState & DepInstMin) == DepInstMin) {
+      // okay, the package isn't broken, but is the package also required?
+      // If it has no real dependencies, no installed rdepends and doesn't
+      // provide something of value, we will kill it as not required.
+      // These pseudopackages have otherwise interesting effects if they get
+      // a new dependency in a newer version…
+      for (pkgCache::DepIterator D = V.DependsList();
+	   D.end() != true; ++D)
+	 if (D.IsCritical() == true && D.ParentPkg()->Group != Pkg->Group)
+	    return false;
+      for (DepIterator D = Pkg.RevDependsList(); D.end() != true; ++D)
+      {
+	 if (D.IsCritical() == false)
+	    continue;
+	 PkgIterator const P = D.ParentPkg();
+	 if (P->Group == Pkg->Group)
+	    continue;
+	 if (P->CurrentVer != 0)
+	    return false;
+      }
+      for (PrvIterator Prv = V.ProvidesList(); Prv.end() != true; Prv++)
+	 for (DepIterator d = Prv.ParentPkg().RevDependsList();
+	      d.end() != true; ++d)
+	 {
+	    PkgIterator const P = d.ParentPkg();
+	    if (P->CurrentVer != 0 &&
+	        P->Group != Pkg->Group)
+	       return false;
+	 }
+   }
+
+   // Dependencies for this arch all package are not statisfied
+   // so we installed it only for our convenience: get right of it now.
+   RemoveSizes(Pkg);
+   RemoveStates(Pkg);
+
+   Pkg->CurrentVer = 0;
+   PkgState[Pkg->ID].InstallVer = 0;
+
+   AddStates(Pkg);
+   Update(Pkg);
+   AddSizes(Pkg);
+
+   // After the remove previously satisfied pseudo pkg could be now
+   // no longer satisfied, so we need to recheck the reverse dependencies
+   for (DepIterator d = Pkg.RevDependsList(); d.end() != true; ++d)
+   {
+      PkgIterator const P = d.ParentPkg();
+      if (P->CurrentVer != 0)
+	 recheck.insert(P.Index());
+   }
+
+   for (DepIterator d = V.DependsList(); d.end() != true; ++d)
+   {
+      PkgIterator const P = d.TargetPkg();
+      for (PrvIterator Prv = P.ProvidesList(); Prv.end() != true; ++Prv)
+      {
+	 PkgIterator const O = Prv.OwnerPkg();
+	 if (O->CurrentVer != 0)
+	    recheck.insert(O.Index());
+      }
+
+      if (P->CurrentVer != 0)
+	 recheck.insert(P.Index());
+   }
+
+   for (PrvIterator Prv = V.ProvidesList(); Prv.end() != true; Prv++)
+   {
+      for (DepIterator d = Prv.ParentPkg().RevDependsList();
+	   d.end() != true; ++d)
+      {
+	 PkgIterator const P = d.ParentPkg();
+	 if (P->CurrentVer == 0)
+	    continue;
+
+	    recheck.insert(P.Index());
+      }
+   }
+
+
+   return true;
+}
+									/*}}}*/
 // DepCache::Update - Figure out all the state information		/*{{{*/
 // ---------------------------------------------------------------------
 /* This will figure out the state of all the packages and all the 
@@ -609,9 +818,13 @@ void pkgDepCache::Update(OpProgress *Prog)
    iKeepCount = 0;
    iBrokenCount = 0;
    iBadCount = 0;
-   
+
+   std::set<unsigned long> recheck;
+
    // Perform the depends pass
    int Done = 0;
+   bool const checkMultiArch = APT::Configuration::getArchitectures().size() > 1;
+   unsigned long killed = 0;
    for (PkgIterator I = PkgBegin(); I.end() != true; I++,Done++)
    {
       if (Prog != 0 && Done%20 == 0)
@@ -619,7 +832,7 @@ void pkgDepCache::Update(OpProgress *Prog)
       for (VerIterator V = I.VersionList(); V.end() != true; V++)
       {
 	 unsigned char Group = 0;
-	 
+
 	 for (DepIterator D = V.DependsList(); D.end() != true; D++)
 	 {
 	    // Build the dependency state.
@@ -637,21 +850,158 @@ void pkgDepCache::Update(OpProgress *Prog)
 		D->Type == Dep::DpkgBreaks ||
 		D->Type == Dep::Obsoletes)
 	       State = ~State;
-	 }	 
+	 }
       }
 
-      // Compute the pacakge dependency state and size additions
+      // Compute the package dependency state and size additions
       AddSizes(I);
       UpdateVerState(I);
       AddStates(I);
+
+      if (checkMultiArch != true || I->CurrentVer == 0)
+	 continue;
+
+      VerIterator const V = I.CurrentVer();
+      if (V->MultiArch != Version::All)
+	 continue;
+
+      recheck.insert(I.Index());
+      --Done; // no progress if we need to recheck the package
    }
 
-   if (Prog != 0)      
+   if (checkMultiArch == true) {
+      /* FIXME: recheck breaks proper progress reporting as we don't know
+		how many packages we need to recheck. To lower the effect
+		a bit we increase with a kill, but we should do something more clever… */
+      while(recheck.empty() == false)
+	 for (std::set<unsigned long>::const_iterator p = recheck.begin();
+	     p != recheck.end();) {
+	    if (Prog != 0 && Done%20 == 0)
+	       Prog->Progress(Done);
+	    PkgIterator P = PkgIterator(*Cache, Cache->PkgP + *p);
+	    if (RemovePseudoInstalledPkg(P, recheck) == true) {
+	       ++killed;
+	       ++Done;
+	    }
+	    recheck.erase(p++);
+	 }
+
+      /* Okay, we have killed a great amount of pseudopackages -
+	 we have killed so many that we have now arch "all" packages
+	 without an installed pseudo package, but we NEED an installed
+	 pseudo package, so we will search now for a pseudo package
+	 we can install without breaking everything. */
+      for (GrpIterator G = Cache->GrpBegin(); G.end() != true; ++G)
+      {
+	 PkgIterator P = G.FindPkg("all");
+	 if (P.end() == true)
+	    continue;
+	 if (P->CurrentVer == 0)
+	    continue;
+	 bool installed = false;
+	 for (P = G.FindPkg("any"); P.end() != true; P = G.NextPkg(P))
+	 {
+	    if (strcmp(P.Arch(), "all") == 0)
+	       continue;
+	    if (P->CurrentVer == 0)
+	       continue;
+	    installed = true;
+	    break;
+	 }
+	 if (installed == false)
+	    recheck.insert(G.Index());
+      }
+
+      while (recheck.empty() != true)
+      {
+	 std::set<unsigned long>::const_iterator g = recheck.begin();
+	 unsigned long const G = *g;
+	 recheck.erase(g);
+	 if (unlikely(ReInstallPseudoForGroup(G, recheck) == false))
+	    _error->Warning(_("Internal error, group '%s' has no installable pseudo package"), GrpIterator(*Cache, Cache->GrpP + G).Name());
+      }
+   }
+
+   if (Prog != 0)
       Prog->Progress(Done);
 
    readStateFile(Prog);
 }
 									/*}}}*/
+// DepCache::ReInstallPseudoForGroup - MultiArch helper for Update()	/*{{{*/
+// ---------------------------------------------------------------------
+/* RemovePseudoInstalledPkg() is very successful. It even kills packages
+   to an amount that no pseudo package is left, but we need a pseudo package
+   for upgrading senarios so we need to reinstall one pseudopackage which
+   doesn't break everything. Thankfully we can't have architecture depending
+   negative dependencies so this problem is already eliminated */
+bool pkgDepCache::ReInstallPseudoForGroup(pkgCache::PkgIterator const &P, std::set<unsigned long> &recheck)
+{
+   if (P->CurrentVer != 0)
+      return true;
+   // recursive call for packages which provide this package
+   for (pkgCache::PrvIterator Prv = P.ProvidesList(); Prv.end() != true; ++Prv)
+      ReInstallPseudoForGroup(Prv.OwnerPkg(), recheck);
+   // check if we actually need to look at this group
+   unsigned long const G = P->Group;
+   std::set<unsigned long>::const_iterator Pi = recheck.find(G);
+   if (Pi == recheck.end())
+      return true;
+   recheck.erase(Pi); // remove here, so we can't fall into an endless loop
+   if (unlikely(ReInstallPseudoForGroup(G, recheck) == false))
+   {
+      recheck.insert(G);
+      return false;
+   }
+   return true;
+}
+bool pkgDepCache::ReInstallPseudoForGroup(unsigned long const &G, std::set<unsigned long> &recheck)
+{
+   std::vector<std::string> static const Archs = APT::Configuration::getArchitectures();
+   pkgCache::GrpIterator Grp(*Cache, Cache->GrpP + G);
+   if (unlikely(Grp.end() == true))
+      return false;
+   for (std::vector<std::string>::const_iterator a = Archs.begin();
+        a != Archs.end(); ++a)
+   {
+      pkgCache::PkgIterator P = Grp.FindPkg(*a);
+      if (P.end() == true)
+	 continue;
+      pkgCache::VerIterator allV = Grp.FindPkg("all").CurrentVer();
+      for (VerIterator V = P.VersionList(); V.end() != true; ++V)
+      {
+	 // search for the same version as the all package
+	 if (allV->Hash != V->Hash || strcmp(allV.VerStr(),V.VerStr()) != 0)
+	    continue;
+	 unsigned char const CurDepState = VersionState(V.DependsList(),DepInstall,DepInstMin,DepInstPolicy);
+	 // If it is broken, try to install dependencies first before retry
+	 if ((CurDepState & DepInstMin) != DepInstMin)
+	 {
+	    for (pkgCache::DepIterator D = V.DependsList(); D.end() != true; ++D)
+	    {
+	       if (D->Type != pkgCache::Dep::PreDepends && D->Type != pkgCache::Dep::Depends)
+		  continue;
+	       ReInstallPseudoForGroup(D.TargetPkg(), recheck);
+	    }
+	    unsigned char const CurDepState = VersionState(V.DependsList(),DepInstall,DepInstMin,DepInstPolicy);
+	    // if package ist still broken… try another arch
+	    if ((CurDepState & DepInstMin) != DepInstMin)
+	       break;
+	 }
+	 // dependencies satisfied: reinstall the package
+	 RemoveSizes(P);
+	 RemoveStates(P);
+	 P->CurrentVer = V.Index();
+	 PkgState[P->ID].InstallVer = V;
+	 AddStates(P);
+	 Update(P);
+	 AddSizes(P);
+	 return true;
+      }
+   }
+   return false;
+}
+									/*}}}*/
 // DepCache::Update - Update the deps list of a package	   		/*{{{*/
 // ---------------------------------------------------------------------
 /* This is a helper for update that only does the dep portion of the scan. 
@@ -799,7 +1149,7 @@ void pkgDepCache::MarkDelete(PkgIterator const &Pkg, bool rPurge,
       return;
 
    if (DebugMarker == true)
-      std::clog << OutputInDepth(Depth) << "MarkDelete " << Pkg << " FU=" << FromUser << std::endl;
+      std::clog << OutputInDepth(Depth) << (rPurge ? "MarkPurge " : "MarkDelete ") << Pkg << " FU=" << FromUser << std::endl;
 
    RemoveSizes(Pkg);
    RemoveStates(Pkg);
@@ -813,6 +1163,19 @@ void pkgDepCache::MarkDelete(PkgIterator const &Pkg, bool rPurge,
    AddStates(Pkg);   
    Update(Pkg);
    AddSizes(Pkg);
+
+   // if we remove the pseudo package, we also need to remove the "real"
+   if (Pkg->CurrentVer != 0 && Pkg.CurrentVer().Pseudo() == true)
+      MarkDelete(Pkg.Group().FindPkg("all"), rPurge, Depth+1, FromUser);
+   else if (rPurge == true && Pkg->CurrentVer == 0 &&
+	    Pkg->CurrentState != pkgCache::State::NotInstalled &&
+	    strcmp(Pkg.Arch(), "all") != 0)
+   {
+      PkgIterator const allPkg = Pkg.Group().FindPkg("all");
+      if (allPkg.end() == false && allPkg->CurrentVer == 0 &&
+	  allPkg->CurrentState != pkgCache::State::NotInstalled)
+	 MarkDelete(allPkg, rPurge, Depth+1, FromUser);
+   }
 }
 									/*}}}*/
 // DepCache::IsDeleteOk - check if it is ok to remove this package	/*{{{*/
@@ -901,6 +1264,10 @@ void pkgDepCache::MarkInstall(PkgIterator const &Pkg,bool AutoInst,
    Update(Pkg);
    AddSizes(Pkg);
 
+   // always trigger the install of the all package for a pseudo package
+   if (P.CandidateVerIter(*Cache).Pseudo() == true)
+      MarkInstall(Pkg.Group().FindPkg("all"), AutoInst, Depth, FromUser, ForceImportantDeps);
+
    if (AutoInst == false)
       return;
 
@@ -1067,10 +1434,13 @@ void pkgDepCache::MarkInstall(PkgIterator const &Pkg,bool AutoInst,
 	    VerIterator Ver(*this,*I);
 	    PkgIterator Pkg = Ver.ParentPkg();
 
-	    if (Start->Type != Dep::DpkgBreaks)
-	       MarkDelete(Pkg,false,Depth + 1, false);
-	    else if (PkgState[Pkg->ID].CandidateVer != *I)
+	    
+	       
+	    if (PkgState[Pkg->ID].CandidateVer != *I &&
+	        Start->Type == Dep::DpkgBreaks)
 	       MarkInstall(Pkg,true,Depth + 1, false, ForceImportantDeps);
+	    else
+	       MarkDelete(Pkg,false,Depth + 1, false);
 	 }
 	 continue;
       }      
@@ -1099,6 +1469,9 @@ bool pkgDepCache::IsInstallOk(PkgIterator const &Pkg,bool AutoInst,
 /* */
 void pkgDepCache::SetReInstall(PkgIterator const &Pkg,bool To)
 {
+   if (unlikely(Pkg.end() == true))
+      return;
+
    ActionGroup group(*this);
 
    RemoveSizes(Pkg);
@@ -1112,12 +1485,17 @@ void pkgDepCache::SetReInstall(PkgIterator const &Pkg,bool To)
    
    AddStates(Pkg);
    AddSizes(Pkg);
+
+   if (unlikely(Pkg.CurrentVer().end() == true) || Pkg.CurrentVer().Pseudo() == false)
+      return;
+
+   SetReInstall(Pkg.Group().FindPkg("all"), To);
 }
 									/*}}}*/
 // DepCache::SetCandidateVersion - Change the candidate version		/*{{{*/
 // ---------------------------------------------------------------------
 /* */
-void pkgDepCache::SetCandidateVersion(VerIterator TargetVer)
+void pkgDepCache::SetCandidateVersion(VerIterator TargetVer, bool const &Pseudo)
 {
    ActionGroup group(*this);
 
@@ -1135,6 +1513,28 @@ void pkgDepCache::SetCandidateVersion(VerIterator TargetVer)
    AddStates(Pkg);
    Update(Pkg);
    AddSizes(Pkg);
+
+   if (TargetVer.Pseudo() == false || Pseudo == false)
+      return;
+
+   // the version was pseudo: set all other pseudos also
+   pkgCache::GrpIterator Grp = Pkg.Group();
+   for (Pkg = Grp.FindPkg("any"); Pkg.end() == false; ++Pkg)
+   {
+      StateCache &P = PkgState[Pkg->ID];
+      if (TargetVer.SimilarVer(P.CandidateVerIter(*this)) == true ||
+	  (P.CandidateVerIter(*this).Pseudo() == false &&
+	   strcmp(Pkg.Arch(), "all") != 0))
+	 continue;
+
+      for (pkgCache::VerIterator Ver = Pkg.VersionList(); Ver.end() == false; ++Ver)
+      {
+	 if (TargetVer.SimilarVer(Ver) == false)
+	    continue;
+	 SetCandidateVersion(Ver, false);
+	 break;
+      }
+   }
 }
 
 void pkgDepCache::MarkAuto(const PkgIterator &Pkg, bool Auto)
@@ -1197,7 +1597,7 @@ const char *pkgDepCache::StateCache::StripEpoch(const char *Ver)
 // ---------------------------------------------------------------------
 /* The default just returns the highest available version that is not
    a source and automatic. */
-pkgCache::VerIterator pkgDepCache::Policy::GetCandidateVer(PkgIterator Pkg)
+pkgCache::VerIterator pkgDepCache::Policy::GetCandidateVer(PkgIterator const &Pkg)
 {
    /* Not source/not automatic versions cannot be a candidate version 
       unless they are already installed */
@@ -1232,7 +1632,7 @@ pkgCache::VerIterator pkgDepCache::Policy::GetCandidateVer(PkgIterator Pkg)
 // Policy::IsImportantDep - True if the dependency is important		/*{{{*/
 // ---------------------------------------------------------------------
 /* */
-bool pkgDepCache::Policy::IsImportantDep(DepIterator Dep)
+bool pkgDepCache::Policy::IsImportantDep(DepIterator const &Dep)
 {
    if(Dep.IsCritical())
       return true;
@@ -1251,54 +1651,6 @@ bool pkgDepCache::Policy::IsImportantDep(DepIterator Dep)
    else if(Dep->Type == pkgCache::Dep::Suggests)
      return _config->FindB("APT::Install-Suggests", false);
 
-   return false;
-}
-									/*}}}*/
-pkgDepCache::DefaultRootSetFunc::DefaultRootSetFunc()			/*{{{*/
-  : constructedSuccessfully(false)
-{
-  Configuration::Item const *Opts;
-  Opts = _config->Tree("APT::NeverAutoRemove");
-  if (Opts != 0 && Opts->Child != 0)
-    {
-      Opts = Opts->Child;
-      for (; Opts != 0; Opts = Opts->Next)
-	{
-	  if (Opts->Value.empty() == true)
-	    continue;
-
-	  regex_t *p = new regex_t;
-	  if(regcomp(p,Opts->Value.c_str(),
-		     REG_EXTENDED | REG_ICASE | REG_NOSUB) != 0)
-	    {
-	      regfree(p);
-	      delete p;
-	      _error->Error("Regex compilation error for APT::NeverAutoRemove");
-	      return;
-	    }
-
-	  rootSetRegexp.push_back(p);
-	}
-    }
-
-  constructedSuccessfully = true;
-}
-									/*}}}*/
-pkgDepCache::DefaultRootSetFunc::~DefaultRootSetFunc()			/*{{{*/
-{
-  for(unsigned int i = 0; i < rootSetRegexp.size(); i++)
-    {
-      regfree(rootSetRegexp[i]);
-      delete rootSetRegexp[i];
-    }
-}
-									/*}}}*/
-bool pkgDepCache::DefaultRootSetFunc::InRootSet(const pkgCache::PkgIterator &pkg) /*{{{*/
-{
-   for(unsigned int i = 0; i < rootSetRegexp.size(); i++)
-      if (regexec(rootSetRegexp[i], pkg.Name(), 0, 0, 0) == 0)
-	 return true;
-
    return false;
 }
 									/*}}}*/
@@ -1339,7 +1691,7 @@ bool pkgDepCache::MarkRequired(InRootSetFunc &userFunc)
 
       // debug output
       if(debug_autoremove && PkgState[p->ID].Flags & Flag::Auto)
-  	 std::clog << "AutoDep: " << p.Name() << std::endl;
+  	 std::clog << "AutoDep: " << p.FullName() << std::endl;
    }
 
    // init vars
@@ -1353,8 +1705,11 @@ bool pkgDepCache::MarkRequired(InRootSetFunc &userFunc)
    {
       if(!(PkgState[p->ID].Flags & Flag::Auto) ||
 	  (p->Flags & Flag::Essential) ||
-	  userFunc.InRootSet(p))
-          
+	  userFunc.InRootSet(p) ||
+	  // be nice even then a required package violates the policy (#583517)
+	  // and do the full mark process also for required packages
+	  (p.CurrentVer().end() != true &&
+	   p.CurrentVer()->Priority == pkgCache::State::Required))
       {
 	 // the package is installed (and set to keep)
 	 if(PkgState[p->ID].Keep() && !p.CurrentVer().end())
@@ -1373,13 +1728,18 @@ bool pkgDepCache::MarkRequired(InRootSetFunc &userFunc)
 // MarkPackage - mark a single package in Mark-and-Sweep		/*{{{*/
 void pkgDepCache::MarkPackage(const pkgCache::PkgIterator &pkg,
 			      const pkgCache::VerIterator &ver,
-			      bool follow_recommends,
-			      bool follow_suggests)
+			      bool const &follow_recommends,
+			      bool const &follow_suggests)
 {
    pkgDepCache::StateCache &state = PkgState[pkg->ID];
-   VerIterator currver            = pkg.CurrentVer();
-   VerIterator candver            = state.CandidateVerIter(*this);
-   VerIterator instver            = state.InstVerIter(*this);
+
+   // if we are marked already we are done
+   if(state.Marked)
+      return;
+
+   VerIterator const currver = pkg.CurrentVer();
+   VerIterator const candver = state.CandidateVerIter(*this);
+   VerIterator const instver = state.InstVerIter(*this);
 
 #if 0
    // If a package was garbage-collected but is now being marked, we
@@ -1405,15 +1765,11 @@ void pkgDepCache::MarkPackage(const pkgCache::PkgIterator &pkg,
       !(ver == currver && instver.end() && !ver.end()))
       return;
 
-   // if we are marked already we are done
-   if(state.Marked)
-      return;
+   bool const debug_autoremove = _config->FindB("Debug::pkgAutoRemove", false);
 
-   bool debug_autoremove = _config->FindB("Debug::pkgAutoRemove", false);
-   
    if(debug_autoremove)
      {
-       std::clog << "Marking: " << pkg.Name();
+       std::clog << "Marking: " << pkg.FullName();
        if(!ver.end())
 	 std::clog << " " << ver.VerStr();
        if(!currver.end())
@@ -1425,8 +1781,31 @@ void pkgDepCache::MarkPackage(const pkgCache::PkgIterator &pkg,
 
    state.Marked=true;
 
-   if(!ver.end())
+   if(ver.end() == true)
+      return;
+
+   // If the version belongs to a Multi-Arch all package
+   // we will mark all others in this Group with this version also
+   if (ver->MultiArch == pkgCache::Version::All &&
+	strcmp(ver.Arch(true), "all") == 0)
    {
+      GrpIterator G = pkg.Group();
+      const char* const VerStr = ver.VerStr();
+      for (PkgIterator P = G.FindPkg("any");
+	   P.end() != true; P = G.NextPkg(P))
+      {
+	 for (VerIterator V = P.VersionList();
+	      V.end() != true; ++V)
+	 {
+	    if (ver->Hash != V->Hash ||
+		strcmp(VerStr, V.VerStr()) != 0)
+	       continue;
+	    MarkPackage(P, V, follow_recommends, follow_suggests);
+	    break;
+	 }
+      }
+   }
+
      for(DepIterator d = ver.DependsList(); !d.end(); ++d)
      {
 	if(d->Type == Dep::Depends ||
@@ -1444,10 +1823,9 @@ void pkgDepCache::MarkPackage(const pkgCache::PkgIterator &pkg,
 	      {
 		if(debug_autoremove)
 		  {
-		    std::clog << "Following dep: " << d.ParentPkg().Name()
+		    std::clog << "Following dep: " << d.ParentPkg().FullName()
 			      << " " << d.ParentVer().VerStr() << " "
-			      << d.DepType() << " "
-			      << d.TargetPkg().Name();
+			      << d.DepType() << " " << d.TargetPkg().FullName();
 		    if((d->CompareOp & ~pkgCache::Dep::Or) != pkgCache::Dep::NoOp)
 		      {
 			std::clog << " (" << d.CompType() << " "
@@ -1455,7 +1833,7 @@ void pkgDepCache::MarkPackage(const pkgCache::PkgIterator &pkg,
 		      }
 		    std::clog << std::endl;
 		  }
-		 MarkPackage(V.ParentPkg(), V, 
+		 MarkPackage(V.ParentPkg(), V,
 			     follow_recommends, follow_suggests);
 	      }
 	   }
@@ -1468,17 +1846,16 @@ void pkgDepCache::MarkPackage(const pkgCache::PkgIterator &pkg,
 	      {
 		if(debug_autoremove)
 		  {
-		    std::clog << "Following dep: " << d.ParentPkg().Name()
-			      << " " << d.ParentVer().VerStr() << " "
-			      << d.DepType() << " "
-			      << d.TargetPkg().Name();
+		    std::clog << "Following dep: " << d.ParentPkg().FullName() << " "
+			      << d.ParentVer().VerStr() << " "
+			      << d.DepType() << " " << d.TargetPkg().FullName() << " ";
 		    if((d->CompareOp & ~pkgCache::Dep::Or) != pkgCache::Dep::NoOp)
 		      {
 			std::clog << " (" << d.CompType() << " "
 				  << d.TargetVer() << ")";
 		      }
 		    std::clog << ", provided by "
-			      << prv.OwnerPkg().Name() << " "
+			      << prv.OwnerPkg().FullName() << " "
 			      << prv.OwnerVer().VerStr()
 			      << std::endl;
 		  }
@@ -1489,7 +1866,6 @@ void pkgDepCache::MarkPackage(const pkgCache::PkgIterator &pkg,
 	   }
 	}
      }
-   }
 }
 									/*}}}*/
 bool pkgDepCache::Sweep()						/*{{{*/
@@ -1511,7 +1887,7 @@ bool pkgDepCache::Sweep()						/*{{{*/
      {
 	state.Garbage=true;
 	if(debug_autoremove)
-	   std::clog << "Garbage: " << p.Name() << std::endl;
+	   std::clog << "Garbage: " << p.FullName() << std::endl;
      }
   }   
 

apt-pkg/depcache.h

@@ -38,14 +38,14 @@
 #ifndef PKGLIB_DEPCACHE_H
 #define PKGLIB_DEPCACHE_H
 
-
+#include <apt-pkg/configuration.h>
 #include <apt-pkg/pkgcache.h>
 #include <apt-pkg/progress.h>
-
-#include <regex.h>
+#include <apt-pkg/error.h>
 
 #include <vector>
 #include <memory>
+#include <set>
 
 class pkgDepCache : protected pkgCache::Namespace
 {
@@ -78,8 +78,8 @@ class pkgDepCache : protected pkgCache::Namespace
     */
    void MarkPackage(const pkgCache::PkgIterator &pkg,
 		    const pkgCache::VerIterator &ver,
-		    bool follow_recommends,
-		    bool follow_suggests);
+		    bool const &follow_recommends,
+		    bool const &follow_suggests);
 
    /** \brief Update the Marked field of all packages.
     *
@@ -183,22 +183,13 @@ class pkgDepCache : protected pkgCache::Namespace
    /** \brief Returns \b true for packages matching a regular
     *  expression in APT::NeverAutoRemove.
     */
-   class DefaultRootSetFunc : public InRootSetFunc
+   class DefaultRootSetFunc : public InRootSetFunc, public Configuration::MatchAgainstConfig
    {
-     std::vector<regex_t *> rootSetRegexp;
-     bool constructedSuccessfully;
-
    public:
-     DefaultRootSetFunc();
-     ~DefaultRootSetFunc();
-
-     /** \return \b true if the class initialized successfully, \b
-      *  false otherwise.  Used to avoid throwing an exception, since
-      *  APT classes generally don't.
-      */
-     bool wasConstructedSuccessfully() const { return constructedSuccessfully; }
+     DefaultRootSetFunc() : Configuration::MatchAgainstConfig("APT::NeverAutoRemove") {};
+     virtual ~DefaultRootSetFunc() {};
 
-     bool InRootSet(const pkgCache::PkgIterator &pkg);
+     bool InRootSet(const pkgCache::PkgIterator &pkg) { return pkg.end() == true && Match(pkg.Name()); };
    };
 
    struct StateCache
@@ -265,8 +256,8 @@ class pkgDepCache : protected pkgCache::Namespace
    {
       public:
       
-      virtual VerIterator GetCandidateVer(PkgIterator Pkg);
-      virtual bool IsImportantDep(DepIterator Dep);
+      virtual VerIterator GetCandidateVer(PkgIterator const &Pkg);
+      virtual bool IsImportantDep(DepIterator const &Dep);
       
       virtual ~Policy() {};
    };
@@ -285,9 +276,11 @@ class pkgDepCache : protected pkgCache::Namespace
    pkgCache *Cache;
    StateCache *PkgState;
    unsigned char *DepState;
-   
-   double iUsrSize;
-   double iDownloadSize;
+
+   /** Stores the space changes after installation */
+   signed long long iUsrSize;
+   /** Stores how much we need to download to get the packages */
+   unsigned long long iDownloadSize;
    unsigned long iInstCount;
    unsigned long iDelCount;
    unsigned long iKeepCount;
@@ -320,8 +313,9 @@ class pkgDepCache : protected pkgCache::Namespace
    void Update(PkgIterator const &P);
    
    // Count manipulators
-   void AddSizes(const PkgIterator &Pkg,signed long Mult = 1);
-   inline void RemoveSizes(const PkgIterator &Pkg) {AddSizes(Pkg,-1);};
+   void AddSizes(const PkgIterator &Pkg, bool const &Invert = false);
+   inline void RemoveSizes(const PkgIterator &Pkg) {AddSizes(Pkg, true);};
+   void AddSizes(const PkgIterator &Pkg,signed long Mult) __deprecated;
    void AddStates(const PkgIterator &Pkg,int Add = 1);
    inline void RemoveStates(const PkgIterator &Pkg) {AddStates(Pkg,-1);};
    
@@ -330,14 +324,17 @@ class pkgDepCache : protected pkgCache::Namespace
    // Legacy.. We look like a pkgCache
    inline operator pkgCache &() {return *Cache;};
    inline Header &Head() {return *Cache->HeaderP;};
+   inline GrpIterator GrpBegin() {return Cache->GrpBegin();};
    inline PkgIterator PkgBegin() {return Cache->PkgBegin();};
+   inline GrpIterator FindGrp(string const &Name) {return Cache->FindGrp(Name);};
    inline PkgIterator FindPkg(string const &Name) {return Cache->FindPkg(Name);};
+   inline PkgIterator FindPkg(string const &Name, string const &Arch) {return Cache->FindPkg(Name, Arch);};
 
    inline pkgCache &GetCache() {return *Cache;};
    inline pkgVersioningSystem &VS() {return *Cache->VS;};
    
    // Policy implementation
-   inline VerIterator GetCandidateVer(PkgIterator Pkg) {return LocalPolicy->GetCandidateVer(Pkg);};
+   inline VerIterator GetCandidateVer(PkgIterator const &Pkg) {return LocalPolicy->GetCandidateVer(Pkg);};
    inline bool IsImportantDep(DepIterator Dep) {return LocalPolicy->IsImportantDep(Dep);};
    inline Policy &GetPolicy() {return *LocalPolicy;};
    
@@ -398,7 +395,7 @@ class pkgDepCache : protected pkgCache::Namespace
 		    bool ForceImportantDeps = false);
 
    void SetReInstall(PkgIterator const &Pkg,bool To);
-   void SetCandidateVersion(VerIterator TargetVer);
+   void SetCandidateVersion(VerIterator TargetVer, bool const &Pseudo = true);
 
    /** Set the "is automatically installed" flag of Pkg. */
    void MarkAuto(const PkgIterator &Pkg, bool Auto);
@@ -442,16 +439,13 @@ class pkgDepCache : protected pkgCache::Namespace
    virtual bool IsDeleteOk(const PkgIterator &Pkg,bool Purge = false,
 			    unsigned long Depth = 0, bool FromUser = true);
 
-   // This is for debuging
-   void Update(OpProgress *Prog = 0);
-
    // read persistent states
    bool readStateFile(OpProgress *prog);
-   bool writeStateFile(OpProgress *prog, bool InstalledOnly=false);
+   bool writeStateFile(OpProgress *prog, bool InstalledOnly=true);
    
    // Size queries
-   inline double UsrSize() {return iUsrSize;};
-   inline double DebSize() {return iDownloadSize;};
+   inline signed long long UsrSize() {return iUsrSize;};
+   inline unsigned long long DebSize() {return iDownloadSize;};
    inline unsigned long DelCount() {return iDelCount;};
    inline unsigned long KeepCount() {return iKeepCount;};
    inline unsigned long InstCount() {return iInstCount;};
@@ -460,9 +454,17 @@ class pkgDepCache : protected pkgCache::Namespace
    inline unsigned long BadCount() {return iBadCount;};
 
    bool Init(OpProgress *Prog);
-   
+   // Generate all state information
+   void Update(OpProgress *Prog = 0);
+
    pkgDepCache(pkgCache *Cache,Policy *Plcy = 0);
    virtual ~pkgDepCache();
+
+   private:
+   // Helper for Update(OpProgress) to remove pseudoinstalled arch all packages
+   bool RemovePseudoInstalledPkg(PkgIterator &Pkg, std::set<unsigned long> &recheck);
+   bool ReInstallPseudoForGroup(unsigned long const &Grp, std::set<unsigned long> &recheck);
+   bool ReInstallPseudoForGroup(pkgCache::PkgIterator const &P, std::set<unsigned long> &recheck);
 };
 
 #endif

apt-pkg/indexcopy.cc

@@ -27,6 +27,8 @@
 #include <sstream>
 #include <unistd.h>
 #include <sys/stat.h>
+#include <sys/types.h>
+#include <fcntl.h>
 #include <stdio.h>
 									/*}}}*/
 
@@ -132,7 +134,7 @@ bool IndexCopy::CopyPackages(string CDROM,string Name,vector<string> &List,
       TargetF += URItoFileName(S);
       if (_config->FindB("APT::CDROM::NoAct",false) == true)
 	 TargetF = "/dev/null";
-      FileFd Target(TargetF,FileFd::WriteEmpty);
+      FileFd Target(TargetF,FileFd::WriteAtomic);
       FILE *TargetFl = fdopen(dup(Target.Fd()),"w");
       if (_error->PendingError() == true)
 	 return false;
@@ -563,7 +565,7 @@ bool SigVerify::CopyMetaIndex(string CDROM, string CDName,		/*{{{*/
 
       FileFd Target;
       FileFd Rel;
-      Target.Open(TargetF,FileFd::WriteEmpty);
+      Target.Open(TargetF,FileFd::WriteAtomic);
       Rel.Open(prefix + file,FileFd::ReadOnly);
       if (_error->PendingError() == true)
 	 return false;
@@ -590,66 +592,34 @@ bool SigVerify::CopyAndVerify(string CDROM,string Name,vector<string> &SigList,
       indexRecords *MetaIndex = new indexRecords;
       string prefix = *I; 
 
+      string const releasegpg = *I+"Release.gpg";
+      string const release = *I+"Release";
+
       // a Release.gpg without a Release should never happen
-      if(!FileExists(*I+"Release"))
+      if(FileExists(release) == false)
       {
 	 delete MetaIndex;
 	 continue;
       }
 
-
-      // verify the gpg signature of "Release"
-      // gpg --verify "*I+Release.gpg", "*I+Release"
-      const char *Args[400];
-      unsigned int i = 0;
-
-      string gpgvpath = _config->Find("Dir::Bin::gpg", "/usr/bin/gpgv");
-      string pubringpath = _config->Find("Apt::GPGV::TrustedKeyring", "/etc/apt/trusted.gpg");
-      string releasegpg = *I+"Release.gpg";
-      string release = *I+"Release";
-
-      Args[i++] = gpgvpath.c_str();
-      Args[i++] = "--keyring";
-      Args[i++] = pubringpath.c_str();
-      Configuration::Item const *Opts;
-      Opts = _config->Tree("Acquire::gpgv::Options");
-      if (Opts != 0)
-      {
-         Opts = Opts->Child;
-	 for (; Opts != 0; Opts = Opts->Next)
-         {
-            if (Opts->Value.empty() == true)
-               continue;
-            Args[i++] = Opts->Value.c_str();
-	    if(i >= 390) { 
-	       _error->Error("Argument list from Acquire::gpgv::Options too long. Exiting.");
-	       return false;
-	    }
-         }
-      }
-      
-      Args[i++] = releasegpg.c_str();
-      Args[i++] = release.c_str();
-      Args[i++] = NULL;
-      
       pid_t pid = ExecFork();
       if(pid < 0) {
 	 _error->Error("Fork failed");
 	 return false;
       }
-      if(pid == 0) {
-	 execvp(gpgvpath.c_str(), (char**)Args);
-      }
+      if(pid == 0)
+	 RunGPGV(release, releasegpg);
+
       if(!ExecWait(pid, "gpgv")) {
 	 _error->Warning("Signature verification failed for: %s",
-			 string(*I+"Release.gpg").c_str());
+			 releasegpg.c_str());
 	 // something went wrong, don't copy the Release.gpg
 	 // FIXME: delete any existing gpg file?
 	 continue;
       }
 
       // Open the Release file and add it to the MetaIndex
-      if(!MetaIndex->Load(*I+"Release"))
+      if(!MetaIndex->Load(release))
       {
 	 _error->Error("%s",MetaIndex->ErrorText.c_str());
 	 return false;
@@ -676,6 +646,102 @@ bool SigVerify::CopyAndVerify(string CDROM,string Name,vector<string> &SigList,
       CopyMetaIndex(CDROM, Name, prefix, "Release.gpg");
    }   
 
+   return true;
+}
+									/*}}}*/
+// SigVerify::RunGPGV - returns the command needed for verify		/*{{{*/
+// ---------------------------------------------------------------------
+/* Generating the commandline for calling gpgv is somehow complicated as
+   we need to add multiple keyrings and user supplied options. Also, as
+   the cdrom code currently can not use the gpgv method we have two places
+   these need to be done - so the place for this method is wrong but better
+   than code duplication… */
+bool SigVerify::RunGPGV(std::string const &File, std::string const &FileGPG,
+			int const &statusfd, int fd[2])
+{
+   string const gpgvpath = _config->Find("Dir::Bin::gpg", "/usr/bin/gpgv");
+   // FIXME: remove support for deprecated APT::GPGV setting
+   string const trustedFile = _config->FindFile("Dir::Etc::Trusted");
+   string const trustedPath = _config->FindDir("Dir::Etc::TrustedParts");
+
+   bool const Debug = _config->FindB("Debug::Acquire::gpgv", false);
+
+   if (Debug == true)
+   {
+      std::clog << "gpgv path: " << gpgvpath << std::endl;
+      std::clog << "Keyring file: " << trustedFile << std::endl;
+      std::clog << "Keyring path: " << trustedPath << std::endl;
+   }
+
+   std::vector<string> keyrings = GetListOfFilesInDir(trustedPath, "gpg", false);
+   if (FileExists(trustedFile) == true)
+      keyrings.push_back(trustedFile);
+
+   std::vector<const char *> Args;
+   Args.reserve(30);
+
+   if (keyrings.empty() == true)
+      return false;
+
+   Args.push_back(gpgvpath.c_str());
+   Args.push_back("--ignore-time-conflict");
+
+   if (statusfd != -1)
+   {
+      Args.push_back("--status-fd");
+      char fd[10];
+      snprintf(fd, sizeof(fd), "%i", statusfd);
+      Args.push_back(fd);
+   }
+
+   for (vector<string>::const_iterator K = keyrings.begin();
+	K != keyrings.end(); ++K)
+   {
+      Args.push_back("--keyring");
+      Args.push_back(K->c_str());
+   }
+
+   Configuration::Item const *Opts;
+   Opts = _config->Tree("Acquire::gpgv::Options");
+   if (Opts != 0)
+   {
+      Opts = Opts->Child;
+      for (; Opts != 0; Opts = Opts->Next)
+      {
+	 if (Opts->Value.empty() == true)
+	    continue;
+	 Args.push_back(Opts->Value.c_str());
+      }
+   }
+
+   Args.push_back(FileGPG.c_str());
+   Args.push_back(File.c_str());
+   Args.push_back(NULL);
+
+   if (Debug == true)
+   {
+      std::clog << "Preparing to exec: " << gpgvpath;
+      for (std::vector<const char *>::const_iterator a = Args.begin(); *a != NULL; ++a)
+	 std::clog << " " << *a;
+      std::clog << std::endl;
+   }
+
+   if (statusfd != -1)
+   {
+      int const nullfd = open("/dev/null", O_RDONLY);
+      close(fd[0]);
+      // Redirect output to /dev/null; we read from the status fd
+      dup2(nullfd, STDOUT_FILENO);
+      dup2(nullfd, STDERR_FILENO);
+      // Redirect the pipe to the status fd (3)
+      dup2(fd[1], statusfd);
+
+      putenv((char *)"LANG=");
+      putenv((char *)"LC_ALL=");
+      putenv((char *)"LC_MESSAGES=");
+   }
+
+   execvp(gpgvpath.c_str(), (char **) &Args[0]);
    return true;
 }
 									/*}}}*/
@@ -773,7 +839,7 @@ bool TranslationsCopy::CopyTranslations(string CDROM,string Name,	/*{{{*/
       TargetF += URItoFileName(S);
       if (_config->FindB("APT::CDROM::NoAct",false) == true)
 	 TargetF = "/dev/null";
-      FileFd Target(TargetF,FileFd::WriteEmpty);
+      FileFd Target(TargetF,FileFd::WriteAtomic);
       FILE *TargetFl = fdopen(dup(Target.Fd()),"w");
       if (_error->PendingError() == true)
 	 return false;

apt-pkg/indexcopy.h

@@ -89,6 +89,15 @@ class SigVerify								/*{{{*/
  public:
    bool CopyAndVerify(string CDROM,string Name,vector<string> &SigList,
 		      vector<string> PkgList,vector<string> SrcList);
+
+   /** \brief generates and run the command to verify a file with gpgv */
+   static bool RunGPGV(std::string const &File, std::string const &FileOut,
+		       int const &statusfd, int fd[2]);
+   inline static bool RunGPGV(std::string const &File, std::string const &FileOut,
+			      int const &statusfd = -1) {
+      int fd[2];
+      return RunGPGV(File, FileOut, statusfd, fd);
+   };
 };
 									/*}}}*/
 

apt-pkg/indexfile.cc

@@ -8,9 +8,9 @@
    ##################################################################### */
 									/*}}}*/
 // Include Files							/*{{{*/
-#include <apt-pkg/configuration.h>
 #include <apt-pkg/indexfile.h>
 #include <apt-pkg/error.h>
+#include <apt-pkg/aptconfiguration.h>
 
 #include <clocale>
 #include <cstring>
@@ -66,34 +66,23 @@ string pkgIndexFile::SourceInfo(pkgSrcRecords::Parser const &Record,
    return string();
 }
 									/*}}}*/
-// IndexFile::TranslationsAvailable - Check if will use Translation    /*{{{*/
+// IndexFile::TranslationsAvailable - Check if will use Translation	/*{{{*/
 // ---------------------------------------------------------------------
 /* */
-bool pkgIndexFile::TranslationsAvailable()
-{
-  const string Translation = _config->Find("APT::Acquire::Translation");
-  
-  if (Translation.compare("none") != 0)
-    return CheckLanguageCode(LanguageCode().c_str());
-  else
-    return false;
+bool pkgIndexFile::TranslationsAvailable() {
+	return (APT::Configuration::getLanguages().empty() != true);
 }
 									/*}}}*/
-// IndexFile::CheckLanguageCode - Check the Language Code   	        /*{{{*/
+// IndexFile::CheckLanguageCode - Check the Language Code		/*{{{*/
 // ---------------------------------------------------------------------
-/* */
-/* common cases: de_DE, de_DE@euro, de_DE.UTF-8, de_DE.UTF-8@euro,
-                 de_DE.ISO8859-1, tig_ER
-                 more in /etc/gdm/locale.conf 
-*/
-
-bool pkgIndexFile::CheckLanguageCode(const char *Lang)
+/* No intern need for this method anymore as the check for correctness
+   is already done in getLanguages(). Note also that this check is
+   rather bad (doesn't take three character like ast into account).
+   TODO: Remove method with next API break */
+__attribute__ ((deprecated)) bool pkgIndexFile::CheckLanguageCode(const char *Lang)
 {
-  if (strlen(Lang) == 2 || 
-      strlen(Lang) == 3 ||
-      (strlen(Lang) > 3 && Lang[3] == '_') ||
-      (strlen(Lang) == 5 && Lang[2] == '_'))
-  return true;
+  if (strlen(Lang) == 2 || (strlen(Lang) == 5 && Lang[2] == '_'))
+    return true;
 
   if (strcmp(Lang,"C") != 0)
     _error->Warning("Wrong language code %s", Lang);
@@ -101,38 +90,14 @@ bool pkgIndexFile::CheckLanguageCode(const char *Lang)
   return false;
 }
 									/*}}}*/
-// IndexFile::LanguageCode - Return the Language Code            	/*{{{*/
+// IndexFile::LanguageCode - Return the Language Code			/*{{{*/
 // ---------------------------------------------------------------------
-/* return the language code */
-string pkgIndexFile::LanguageCode()
-{
-  const string Translation = _config->Find("APT::Acquire::Translation");
-
-  if (Translation.compare("environment") == 0) 
-  {
-     string lang = std::setlocale(LC_MESSAGES,NULL);
-
-     // we have a mapping of the language codes that contains all the language
-     // codes that need the country code as well 
-     // (like pt_BR, pt_PT, sv_SE, zh_*, en_*)
-     const char *need_full_langcode[] = { "cs_", 
-					  "en_", 
-					  "pt_",
-					  "sv_",
-					  "zh_", 
-					  NULL };
-     for(const char **s = need_full_langcode;*s != NULL; s++)
-	if(lang.find(*s) == 0)
-	   return lang.substr(0,5);
-     
-     if(lang.find("_") != lang.npos)
-	return lang.substr(0, lang.find("_"));
-     else if(lang.find(".") != lang.npos)
-	return lang.substr(0, lang.find("."));
-     else
-	return lang;
-  }
-  else 
-     return Translation;
+/* As we have now possibly more than one LanguageCode this method is
+   supersided by a) private classmembers or b) getLanguages().
+   TODO: Remove method with next API break */
+__attribute__ ((deprecated)) string pkgIndexFile::LanguageCode() {
+	if (TranslationsAvailable() == false)
+		return "";
+	return APT::Configuration::getLanguages()[0];
 }
 									/*}}}*/

apt-pkg/indexfile.h

@@ -74,8 +74,12 @@ class pkgIndexFile
    virtual bool Exists() const = 0;
    virtual bool HasPackages() const = 0;
    virtual unsigned long Size() const = 0;
-   virtual bool Merge(pkgCacheGenerator &/*Gen*/,OpProgress &/*Prog*/) const {return false;};
-   virtual bool MergeFileProvides(pkgCacheGenerator &/*Gen*/,OpProgress &/*Prog*/) const {return true;};
+   virtual bool Merge(pkgCacheGenerator &/*Gen*/,OpProgress* /*Prog*/) const { return false; };
+   __deprecated virtual bool Merge(pkgCacheGenerator &Gen, OpProgress &Prog) const
+      { return Merge(Gen, &Prog); };
+   virtual bool MergeFileProvides(pkgCacheGenerator &/*Gen*/,OpProgress* /*Prog*/) const {return true;};
+   __deprecated virtual bool MergeFileProvides(pkgCacheGenerator &Gen, OpProgress &Prog) const
+      {return MergeFileProvides(Gen, &Prog);};
    virtual pkgCache::PkgFileIterator FindInCache(pkgCache &Cache) const;
 
    static bool TranslationsAvailable();

apt-pkg/indexrecords.cc

@@ -7,8 +7,11 @@
 #include <apt-pkg/tagfile.h>
 #include <apt-pkg/error.h>
 #include <apt-pkg/strutl.h>
+#include <apt-pkg/configuration.h>
 #include <apti18n.h>
 #include <sys/stat.h>
+#include <clocale>
+
 									/*}}}*/
 string indexRecords::GetDist() const
 {
@@ -26,11 +29,21 @@ string indexRecords::GetExpectedDist() const
    return this->ExpectedDist;
 }
 
+time_t indexRecords::GetValidUntil() const
+{
+   return this->ValidUntil;
+}
+
 const indexRecords::checkSum *indexRecords::Lookup(const string MetaKey)
 {
    return Entries[MetaKey];
 }
 
+bool indexRecords::Exists(string const &MetaKey) const
+{
+   return Entries.count(MetaKey) == 1;
+}
+
 bool indexRecords::Load(const string Filename)				/*{{{*/
 {
    FileFd Fd(Filename, FileFd::ReadOnly);
@@ -80,9 +93,40 @@ bool indexRecords::Load(const string Filename)				/*{{{*/
    {
       strprintf(ErrorText, _("No Hash entry in Release file %s"), Filename.c_str());
       return false;
-   }  
+   }
+
+   string Label = Section.FindS("Label");
+   string StrDate = Section.FindS("Date");
+   string StrValidUntil = Section.FindS("Valid-Until");
+
+   // if we have a Valid-Until header in the Release file, use it as default
+   if (StrValidUntil.empty() == false)
+   {
+      if(RFC1123StrToTime(StrValidUntil.c_str(), ValidUntil) == false)
+      {
+	 strprintf(ErrorText, _("Invalid 'Valid-Until' entry in Release file %s"), Filename.c_str());
+	 return false;
+      }
+   }
+   // get the user settings for this archive and use what expires earlier
+   int MaxAge = _config->FindI("Acquire::Max-ValidTime", 0);
+   if (Label.empty() == true)
+      MaxAge = _config->FindI(string("Acquire::Max-ValidTime::" + Label).c_str(), MaxAge);
+
+   if(MaxAge == 0) // No user settings, use the one from the Release file
+      return true;
+
+   time_t date;
+   if (RFC1123StrToTime(StrDate.c_str(), date) == false)
+   {
+      strprintf(ErrorText, _("Invalid 'Date' entry in Release file %s"), Filename.c_str());
+      return false;
+   }
+   date += 24*60*60*MaxAge;
+
+   if (ValidUntil == 0 || ValidUntil > date)
+      ValidUntil = date;
 
-   string Strdate = Section.FindS("Date"); // FIXME: verify this somehow?
    return true;
 }
 									/*}}}*/
@@ -160,6 +204,6 @@ indexRecords::indexRecords()
 }
 
 indexRecords::indexRecords(const string ExpectedDist) :
-   ExpectedDist(ExpectedDist)
+   ExpectedDist(ExpectedDist), ValidUntil(0)
 {
 }

apt-pkg/indexrecords.h

@@ -12,6 +12,7 @@
 
 #include <map>
 #include <vector>
+#include <ctime>
 
 class indexRecords
 {
@@ -25,6 +26,8 @@ class indexRecords
    string Dist;
    string Suite;
    string ExpectedDist;
+   time_t ValidUntil;
+
    std::map<string,checkSum *> Entries;
 
    public:
@@ -34,10 +37,13 @@ class indexRecords
 
    // Lookup function
    virtual const checkSum *Lookup(const string MetaKey);
+   /** \brief tests if a checksum for this file is available */
+   bool Exists(string const &MetaKey) const;
    std::vector<std::string> MetaKeys();
 
    virtual bool Load(string Filename);
    string GetDist() const;
+   time_t GetValidUntil() const;
    virtual bool CheckDist(const string MaybeDist) const;
    string GetExpectedDist() const;
    virtual ~indexRecords(){};

apt-pkg/init.cc

@@ -51,6 +51,7 @@ bool pkgInitConfig(Configuration &Cnf)
        
    Cnf.Set("Dir::State::lists","lists/");
    Cnf.Set("Dir::State::cdroms","cdroms.list");
+   Cnf.Set("Dir::State::mirrors","mirrors/");
    
    // Cache
    Cnf.Set("Dir::Cache","var/cache/apt/");
@@ -69,6 +70,9 @@ bool pkgInitConfig(Configuration &Cnf)
    Cnf.Set("Dir::Etc::parts","apt.conf.d");
    Cnf.Set("Dir::Etc::preferences","preferences");
    Cnf.Set("Dir::Etc::preferencesparts","preferences.d");
+   string const deprecated = _config->Find("APT::GPGV::TrustedKeyring");
+   Cnf.Set("Dir::Etc::trusted", deprecated.empty() ? "trusted.gpg" : deprecated);
+   Cnf.Set("Dir::Etc::trustedparts","trusted.gpg.d");
    Cnf.Set("Dir::Bin::methods","/usr/lib/apt/methods");
    Cnf.Set("Dir::Media::MountPath","/media/apt");
 
@@ -77,6 +81,11 @@ bool pkgInitConfig(Configuration &Cnf)
    Cnf.Set("Dir::Log::Terminal","term.log");
    Cnf.Set("Dir::Log::History","history.log");
 
+   Cnf.Set("Dir::Ignore-Files-Silently::", "~$");
+   Cnf.Set("Dir::Ignore-Files-Silently::", "\\.disabled$");
+   Cnf.Set("Dir::Ignore-Files-Silently::", "\\.bak$");
+   Cnf.Set("Dir::Ignore-Files-Silently::", "\\.dpkg-[a-z]+$");
+
    // Translation
    Cnf.Set("APT::Acquire::Translation", "environment");
 
@@ -94,10 +103,10 @@ bool pkgInitConfig(Configuration &Cnf)
 
    // Read the configuration parts dir
    string Parts = Cnf.FindDir("Dir::Etc::parts");
-   if (FileExists(Parts) == true)
+   if (DirectoryExists(Parts) == true)
       Res &= ReadConfigDir(Cnf,Parts);
    else
-      _error->WarningE("FileExists",_("Unable to read %s"),Parts.c_str());
+      _error->WarningE("DirectoryExists",_("Unable to read %s"),Parts.c_str());
 
    // Read the main config file
    string FName = Cnf.FindFile("Dir::Etc::main");

apt-pkg/init.h

@@ -22,7 +22,7 @@
 // Non-ABI-Breaks should only increase RELEASE number.
 // See also buildlib/libversion.mak
 #define APT_PKG_MAJOR 4
-#define APT_PKG_MINOR 8
+#define APT_PKG_MINOR 10
 #define APT_PKG_RELEASE 0
     
 extern const char *pkgVersion;

apt-pkg/makefile

@@ -14,7 +14,7 @@ include ../buildlib/libversion.mak
 LIBRARY=apt-pkg
 MAJOR=$(LIBAPTPKG_MAJOR)
 MINOR=$(LIBAPTPKG_RELEASE)
-SLIBS=$(PTHREADLIB) $(INTLLIBS) -lutil -ldl
+SLIBS=$(PTHREADLIB) $(INTLLIBS) -lutil -ldl -lz
 APT_DOMAIN:=libapt-pkg$(LIBAPTPKG_MAJOR)
 
 # Source code for the contributed non-core things
@@ -25,7 +25,7 @@ SOURCE = contrib/mmap.cc contrib/error.cc contrib/strutl.cc \
 	 contrib/fileutl.cc 
 HEADERS = mmap.h error.h configuration.h fileutl.h  cmndline.h netrc.h\
 	  md5.h crc-16.h cdromutl.h strutl.h sptr.h sha1.h sha256.h hashes.h \
-	  macros.h
+	  macros.h weakptr.h
 
 # Source code for the core main library
 SOURCE+= pkgcache.cc version.cc depcache.cc \
@@ -35,14 +35,15 @@ SOURCE+= pkgcache.cc version.cc depcache.cc \
 	 srcrecords.cc cachefile.cc versionmatch.cc policy.cc \
 	 pkgsystem.cc indexfile.cc pkgcachegen.cc acquire-item.cc \
 	 indexrecords.cc vendor.cc vendorlist.cc cdrom.cc indexcopy.cc \
-	 aptconfiguration.cc
+	 aptconfiguration.cc cachefilter.cc cacheset.cc
 HEADERS+= algorithms.h depcache.h pkgcachegen.h cacheiterators.h \
 	  orderlist.h sourcelist.h packagemanager.h tagfile.h \
 	  init.h pkgcache.h version.h progress.h pkgrecords.h \
 	  acquire.h acquire-worker.h acquire-item.h acquire-method.h \
 	  clean.h srcrecords.h cachefile.h versionmatch.h policy.h \
 	  pkgsystem.h indexfile.h metaindex.h indexrecords.h vendor.h \
-          vendorlist.h cdrom.h indexcopy.h aptconfiguration.h
+	  vendorlist.h cdrom.h indexcopy.h aptconfiguration.h \
+	  cachefilter.h cacheset.h
 
 # Source code for the debian specific components
 # In theory the deb headers do not need to be exported..

apt-pkg/metaindex.h

@@ -33,8 +33,8 @@ class metaIndex
    virtual const char* GetType() const {return Type;}
 
    // Interface for acquire
-   virtual string ArchiveURI(string /*File*/) const = 0;
-   virtual bool GetIndexes(pkgAcquire *Owner, bool GetAll=false) const = 0;
+   virtual string ArchiveURI(string const& /*File*/) const = 0;
+   virtual bool GetIndexes(pkgAcquire *Owner, bool const &GetAll=false) const = 0;
    
    virtual vector<pkgIndexFile *> *GetIndexFiles() = 0; 
    virtual bool IsTrusted() const = 0;

apt-pkg/orderlist.cc

@@ -117,7 +117,8 @@ bool pkgOrderList::IsMissing(PkgIterator Pkg)
       return false;
 
    // Skip Packages that need configure only.
-   if (Pkg.State() == pkgCache::PkgIterator::NeedsConfigure && 
+   if ((Pkg.State() == pkgCache::PkgIterator::NeedsConfigure ||
+        Pkg.State() == pkgCache::PkgIterator::NeedsNothing) &&
        Cache[Pkg].Keep() == true)
       return false;
 
@@ -126,6 +127,11 @@ bool pkgOrderList::IsMissing(PkgIterator Pkg)
    
    if (FileList[Pkg->ID].empty() == false)
       return false;
+
+   // Missing Pseudo packages are missing if the real package is missing
+   if (pkgCache::VerIterator(Cache, Cache[Pkg].CandidateVer).Pseudo() == true)
+      return IsMissing(Pkg.Group().FindPkg("all"));
+
    return true;
 }
 									/*}}}*/
@@ -199,7 +205,7 @@ bool pkgOrderList::OrderCritical()
       {
 	 PkgIterator P(Cache,*I);
 	 if (IsNow(P) == true)
-	    clog << "  " << P.Name() << ' ' << IsMissing(P) << ',' << IsFlag(P,After) << endl;
+	    clog << "  " << P.FullName() << ' ' << IsMissing(P) << ',' << IsFlag(P,After) << endl;
       }
    }
 
@@ -272,7 +278,7 @@ bool pkgOrderList::OrderUnpack(string *FileList)
       {
 	 PkgIterator P(Cache,*I);
 	 if (IsNow(P) == true)
-	    clog << "  " << P.Name() << ' ' << IsMissing(P) << ',' << IsFlag(P,After) << endl;
+	    clog << "  " << P.FullName() << ' ' << IsMissing(P) << ',' << IsFlag(P,After) << endl;
       }
    }
 
@@ -543,7 +549,7 @@ bool pkgOrderList::VisitNode(PkgIterator Pkg)
    if (Debug == true)
    {
       for (int j = 0; j != Depth; j++) clog << ' ';
-      clog << "Visit " << Pkg.Name() << endl;
+      clog << "Visit " << Pkg.FullName() << endl;
    }
    
    Depth++;
@@ -602,7 +608,7 @@ bool pkgOrderList::VisitNode(PkgIterator Pkg)
    if (Debug == true)
    {
       for (int j = 0; j != Depth; j++) clog << ' ';
-      clog << "Leave " << Pkg.Name() << ' ' << IsFlag(Pkg,Added) << ',' << IsFlag(Pkg,AddPending) << endl;
+      clog << "Leave " << Pkg.FullName() << ' ' << IsFlag(Pkg,Added) << ',' << IsFlag(Pkg,AddPending) << endl;
    }
    
    return true;
@@ -880,13 +886,16 @@ bool pkgOrderList::DepRemove(DepIterator D)
 	    continue;
 
 	 /* We wish to see if the dep on the parent package is okay
-	    in the removed (install) state of the target pkg. */	 
+	    in the removed (install) state of the target pkg. */
+	 bool tryFixDeps = false;
 	 if (CheckDep(D) == true)
 	 {
 	    // We want to catch loops with the code below.
 	    if (IsFlag(D.ParentPkg(),AddPending) == false)
 	       continue;
 	 }
+	 else
+	    tryFixDeps = true;
 
 	 // This is the loop detection
 	 if (IsFlag(D.ParentPkg(),Added) == true || 
@@ -897,6 +906,80 @@ bool pkgOrderList::DepRemove(DepIterator D)
 	    continue;
 	 }
 
+	 if (tryFixDeps == true)
+	 {
+	    for (pkgCache::DepIterator F = D.ParentPkg().CurrentVer().DependsList();
+		 F.end() == false; ++F)
+	    {
+	       if (F->Type != pkgCache::Dep::Depends && F->Type != pkgCache::Dep::PreDepends)
+		  continue;
+	       // Check the Providers
+	       if (F.TargetPkg()->ProvidesList != 0)
+	       {
+		  pkgCache::PrvIterator Prov = F.TargetPkg().ProvidesList();
+		  for (; Prov.end() == false; ++Prov)
+		  {
+		     pkgCache::PkgIterator const P = Prov.OwnerPkg();
+		     if (IsFlag(P, InList) == true &&
+			 IsFlag(P, AddPending) == true &&
+			 IsFlag(P, Added) == false &&
+			 Cache[P].InstallVer == 0)
+			break;
+		  }
+		  if (Prov.end() == false)
+		     for (pkgCache::PrvIterator Prv = F.TargetPkg().ProvidesList();
+			  Prv.end() == false; ++Prv)
+		     {
+			pkgCache::PkgIterator const P = Prv.OwnerPkg();
+			if (IsFlag(P, InList) == true &&
+			    IsFlag(P, AddPending) == false &&
+			    Cache[P].InstallVer != 0 &&
+			    VisitNode(P) == true)
+			{
+			   Flag(P, Immediate);
+			   tryFixDeps = false;
+			   break;
+			}
+		     }
+		  if (tryFixDeps == false)
+		     break;
+	       }
+
+	       // Check for Or groups
+	       if ((F->CompareOp & pkgCache::Dep::Or) != pkgCache::Dep::Or)
+		  continue;
+	       // Lets see if the package is part of the Or group
+	       pkgCache::DepIterator S = F;
+	       for (; S.end() == false; ++S)
+	       {
+		  if (S.TargetPkg() == D.TargetPkg())
+		     break;
+		  if ((S->CompareOp & pkgCache::Dep::Or) != pkgCache::Dep::Or ||
+		      CheckDep(S)) // Or group is satisfied by another package
+		     for (;S.end() == false; ++S);
+	       }
+	       if (S.end() == true)
+		  continue;
+	       // skip to the end of the or group
+	       for (;S.end() == false && (S->CompareOp & pkgCache::Dep::Or) == pkgCache::Dep::Or; ++S);
+	       ++S;
+	       // The soon to be removed is part of the Or group
+	       // start again in the or group and find something which will serve as replacement
+	       for (; F.end() == false && F != S; ++F)
+	       {
+		  if (F.TargetPkg() == D.TargetPkg() ||
+		      IsFlag(F.TargetPkg(), InList) == false ||
+		      VisitNode(F.TargetPkg()) == false)
+		     continue;
+		  Flag(F.TargetPkg(), Immediate);
+		  tryFixDeps = false;
+		  break;
+	       }
+	       if (tryFixDeps == false)
+		  break;
+	    }
+	 }
+
 	 // Skip over missing files
 	 if (IsMissing(D.ParentPkg()) == true)
 	    continue;

apt-pkg/packagemanager.cc

@@ -80,7 +80,10 @@ bool pkgPackageManager::GetArchives(pkgAcquire *Owner,pkgSourceList *Sources,
       // Skip already processed packages
       if (List->IsNow(Pkg) == false)
 	 continue;
-	 
+
+      if (pkgCache::VerIterator(Cache, Cache[Pkg].CandidateVer).Pseudo() == true)
+	 continue;
+
       new pkgAcqArchive(Owner,Sources,Recs,Cache[Pkg].InstVerIter(Cache),
 			FileNames[Pkg->ID]);
    }
@@ -277,8 +280,10 @@ bool pkgPackageManager::ConfigureAll()
    for (pkgOrderList::iterator I = OList.begin(); I != OList.end(); I++)
    {
       PkgIterator Pkg(Cache,*I);
-      
-      if (ConfigurePkgs == true && Configure(Pkg) == false)
+
+      if (ConfigurePkgs == true &&
+	  pkgCache::VerIterator(Cache, Cache[Pkg].CandidateVer).Pseudo() == false &&
+	  Configure(Pkg) == false)
 	 return false;
       
       List->Flag(Pkg,pkgOrderList::Configured,pkgOrderList::States);
@@ -313,7 +318,9 @@ bool pkgPackageManager::SmartConfigure(PkgIterator Pkg)
    {
       PkgIterator Pkg(Cache,*I);
       
-      if (ConfigurePkgs == true && Configure(Pkg) == false)
+      if (ConfigurePkgs == true &&
+	  pkgCache::VerIterator(Cache, Cache[Pkg].CandidateVer).Pseudo() == false &&
+	  Configure(Pkg) == false)
 	 return false;
       
       List->Flag(Pkg,pkgOrderList::Configured,pkgOrderList::States);
@@ -465,7 +472,12 @@ bool pkgPackageManager::SmartRemove(PkgIterator Pkg)
       return true;
 
    List->Flag(Pkg,pkgOrderList::Configured,pkgOrderList::States);
-   return Remove(Pkg,(Cache[Pkg].iFlags & pkgDepCache::Purge) == pkgDepCache::Purge);
+
+   if (pkgCache::VerIterator(Cache, Cache[Pkg].CandidateVer).Pseudo() == false)
+      return Remove(Pkg,(Cache[Pkg].iFlags & pkgDepCache::Purge) == pkgDepCache::Purge);
+   else
+      return SmartRemove(Pkg.Group().FindPkg("all"));
+   return true;
 }
 									/*}}}*/
 // PM::SmartUnPack - Install helper					/*{{{*/
@@ -579,10 +591,23 @@ bool pkgPackageManager::SmartUnPack(PkgIterator Pkg)
    for (PrvIterator P = Cache[Pkg].InstVerIter(Cache).ProvidesList(); 
 	P.end() == false; P++)
       CheckRConflicts(Pkg,P.ParentPkg().RevDependsList(),P.ProvideVersion());
-   
-   if (Install(Pkg,FileNames[Pkg->ID]) == false)
-      return false;
-   
+
+   if (pkgCache::VerIterator(Cache, Cache[Pkg].CandidateVer).Pseudo() == false)
+   {
+      if(Install(Pkg,FileNames[Pkg->ID]) == false)
+         return false;
+   } else {
+      // Pseudo packages will not be unpacked - instead we will do this
+      // for the "real" package, but only once and if it is already
+      // configured we don't need to unpack it again…
+      PkgIterator const P = Pkg.Group().FindPkg("all");
+      if (List->IsFlag(P,pkgOrderList::UnPacked) != true &&
+	  List->IsFlag(P,pkgOrderList::Configured) != true &&
+	  P.State() != pkgCache::PkgIterator::NeedsNothing) {
+	 if (SmartUnPack(P) == false)
+	    return false;
+      }
+   }
    List->Flag(Pkg,pkgOrderList::UnPacked,pkgOrderList::States);
    
    // Perform immedate configuration of the package.

apt-pkg/packagemanager.h

@@ -28,6 +28,7 @@
 #include <iostream>
 #include <apt-pkg/pkgcache.h>
 #include <apt-pkg/depcache.h>
+#include <set>
 
 using std::string;
 
@@ -47,7 +48,15 @@ class pkgPackageManager : protected pkgCache::Namespace
    pkgDepCache &Cache;
    pkgOrderList *List;
    bool Debug;
-         
+
+   /** \brief saves packages dpkg let disappear
+
+       This way APT can retreat from trying to configure these
+       packages later on and a frontend can choose to display a
+       notice to inform the user about these disappears.
+   */
+   std::set<std::string> disappearedPkgs;
+
    bool DepAdd(pkgOrderList &Order,PkgIterator P,int Depth = 0);
    void ImmediateAdd(PkgIterator P, bool UseInstallVer, unsigned const int &Depth = 0);
    virtual OrderResult OrderInstall();
@@ -93,7 +102,10 @@ class pkgPackageManager : protected pkgCache::Namespace
    // stuff that needs to be done after the fork
    OrderResult DoInstallPostFork(int statusFd=-1);
    bool FixMissing();
-   
+
+   /** \brief returns all packages dpkg let disappear */
+   inline std::set<std::string> GetDisappearedPackages() { return disappearedPkgs; };
+
    pkgPackageManager(pkgDepCache *Cache);
    virtual ~pkgPackageManager();
 };

apt-pkg/pkgcache.cc

@@ -22,11 +22,12 @@
 // Include Files							/*{{{*/
 #include <apt-pkg/pkgcache.h>
 #include <apt-pkg/policy.h>
-#include <apt-pkg/indexfile.h>
 #include <apt-pkg/version.h>
 #include <apt-pkg/error.h>
 #include <apt-pkg/strutl.h>
 #include <apt-pkg/configuration.h>
+#include <apt-pkg/aptconfiguration.h>
+#include <apt-pkg/macros.h>
 
 #include <apti18n.h>
     
@@ -54,6 +55,7 @@ pkgCache::Header::Header()
    Dirty = false;
    
    HeaderSz = sizeof(pkgCache::Header);
+   GroupSz = sizeof(pkgCache::Group);
    PackageSz = sizeof(pkgCache::Package);
    PackageFileSz = sizeof(pkgCache::PackageFile);
    VersionSz = sizeof(pkgCache::Version);
@@ -63,6 +65,7 @@ pkgCache::Header::Header()
    VerFileSz = sizeof(pkgCache::VerFile);
    DescFileSz = sizeof(pkgCache::DescFile);
    
+   GroupCount = 0;
    PackageCount = 0;
    VersionCount = 0;
    DescriptionCount = 0;
@@ -78,7 +81,8 @@ pkgCache::Header::Header()
    StringList = 0;
    VerSysName = 0;
    Architecture = 0;
-   memset(HashTable,0,sizeof(HashTable));
+   memset(PkgHashTable,0,sizeof(PkgHashTable));
+   memset(GrpHashTable,0,sizeof(GrpHashTable));
    memset(Pools,0,sizeof(Pools));
 }
 									/*}}}*/
@@ -88,6 +92,7 @@ pkgCache::Header::Header()
 bool pkgCache::Header::CheckSizes(Header &Against) const
 {
    if (HeaderSz == Against.HeaderSz &&
+       GroupSz == Against.GroupSz &&
        PackageSz == Against.PackageSz &&
        PackageFileSz == Against.PackageFileSz &&
        VersionSz == Against.VersionSz &&
@@ -106,6 +111,10 @@ bool pkgCache::Header::CheckSizes(Header &Against) const
 /* */
 pkgCache::pkgCache(MMap *Map, bool DoMap) : Map(*Map)
 {
+   // call getArchitectures() with cached=false to ensure that the 
+   // architectures cache is re-evaulated. this is needed in cases
+   // when the APT::Architecture field changes between two cache creations
+   MultiArchEnabled = APT::Configuration::getArchitectures(false).size() > 1;
    if (DoMap == true)
       ReMap();
 }
@@ -113,10 +122,11 @@ pkgCache::pkgCache(MMap *Map, bool DoMap) : Map(*Map)
 // Cache::ReMap - Reopen the cache file					/*{{{*/
 // ---------------------------------------------------------------------
 /* If the file is already closed then this will open it open it. */
-bool pkgCache::ReMap()
+bool pkgCache::ReMap(bool const &Errorchecks)
 {
    // Apply the typecasts.
    HeaderP = (Header *)Map.Data();
+   GrpP = (Group *)Map.Data();
    PkgP = (Package *)Map.Data();
    VerFileP = (VerFile *)Map.Data();
    DescFileP = (DescFile *)Map.Data();
@@ -128,6 +138,9 @@ bool pkgCache::ReMap()
    StringItemP = (StringItem *)Map.Data();
    StrP = (char *)Map.Data();
 
+   if (Errorchecks == false)
+      return true;
+
    if (Map.Size() == 0 || HeaderP == 0)
       return _error->Error(_("Empty package cache"));
    
@@ -164,7 +177,7 @@ unsigned long pkgCache::sHash(const string &Str) const
    unsigned long Hash = 0;
    for (string::const_iterator I = Str.begin(); I != Str.end(); I++)
       Hash = 5*Hash + tolower_ascii(*I);
-   return Hash % _count(HeaderP->HashTable);
+   return Hash % _count(HeaderP->PkgHashTable);
 }
 
 unsigned long pkgCache::sHash(const char *Str) const
@@ -172,26 +185,82 @@ unsigned long pkgCache::sHash(const char *Str) const
    unsigned long Hash = 0;
    for (const char *I = Str; *I != 0; I++)
       Hash = 5*Hash + tolower_ascii(*I);
-   return Hash % _count(HeaderP->HashTable);
+   return Hash % _count(HeaderP->PkgHashTable);
 }
 
 									/*}}}*/
-// Cache::FindPkg - Locate a package by name				/*{{{*/
+// Cache::SingleArchFindPkg - Locate a package by name			/*{{{*/
 // ---------------------------------------------------------------------
-/* Returns 0 on error, pointer to the package otherwise */
-pkgCache::PkgIterator pkgCache::FindPkg(const string &Name)
+/* Returns 0 on error, pointer to the package otherwise
+   The multiArch enabled methods will fallback to this one as it is (a bit)
+   faster for single arch environments and realworld is mostly singlearch… */
+pkgCache::PkgIterator pkgCache::SingleArchFindPkg(const string &Name)
 {
    // Look at the hash bucket
-   Package *Pkg = PkgP + HeaderP->HashTable[Hash(Name)];
+   Package *Pkg = PkgP + HeaderP->PkgHashTable[Hash(Name)];
    for (; Pkg != PkgP; Pkg = PkgP + Pkg->NextPackage)
    {
       if (Pkg->Name != 0 && StrP[Pkg->Name] == Name[0] &&
-	  stringcasecmp(Name,StrP + Pkg->Name) == 0)
-	 return PkgIterator(*this,Pkg);
+          stringcasecmp(Name,StrP + Pkg->Name) == 0)
+         return PkgIterator(*this,Pkg);
    }
    return PkgIterator(*this,0);
 }
 									/*}}}*/
+// Cache::FindPkg - Locate a package by name				/*{{{*/
+// ---------------------------------------------------------------------
+/* Returns 0 on error, pointer to the package otherwise */
+pkgCache::PkgIterator pkgCache::FindPkg(const string &Name) {
+	if (MultiArchCache() == false)
+		return SingleArchFindPkg(Name);
+	size_t const found = Name.find(':');
+	if (found == string::npos)
+		return FindPkg(Name, "native");
+	string const Arch = Name.substr(found+1);
+	if (Arch == "any")
+		return FindPkg(Name, "any");
+	return FindPkg(Name.substr(0, found), Arch);
+}
+									/*}}}*/
+// Cache::FindPkg - Locate a package by name				/*{{{*/
+// ---------------------------------------------------------------------
+/* Returns 0 on error, pointer to the package otherwise */
+pkgCache::PkgIterator pkgCache::FindPkg(const string &Name, string const &Arch) {
+	if (MultiArchCache() == false) {
+		if (Arch == "native" || Arch == "all" || Arch == "any" ||
+		    Arch == _config->Find("APT::Architecture"))
+			return SingleArchFindPkg(Name);
+		else
+			return PkgIterator(*this,0);
+	}
+	/* We make a detour via the GrpIterator here as
+	   on a multi-arch environment a group is easier to
+	   find than a package (less entries in the buckets) */
+	pkgCache::GrpIterator Grp = FindGrp(Name);
+	if (Grp.end() == true)
+		return PkgIterator(*this,0);
+
+	return Grp.FindPkg(Arch);
+}
+									/*}}}*/
+// Cache::FindGrp - Locate a group by name				/*{{{*/
+// ---------------------------------------------------------------------
+/* Returns End-Pointer on error, pointer to the group otherwise */
+pkgCache::GrpIterator pkgCache::FindGrp(const string &Name) {
+	if (unlikely(Name.empty() == true))
+		return GrpIterator(*this,0);
+
+	// Look at the hash bucket for the group
+	Group *Grp = GrpP + HeaderP->GrpHashTable[sHash(Name)];
+	for (; Grp != GrpP; Grp = GrpP + Grp->Next) {
+		if (Grp->Name != 0 && StrP[Grp->Name] == Name[0] &&
+		    stringcasecmp(Name, StrP + Grp->Name) == 0)
+			return GrpIterator(*this, Grp);
+	}
+
+	return GrpIterator(*this,0);
+}
+									/*}}}*/
 // Cache::CompTypeDeb - Return a string describing the compare type	/*{{{*/
 // ---------------------------------------------------------------------
 /* This returns a string representation of the dependency compare 
@@ -241,11 +310,98 @@ const char *pkgCache::Priority(unsigned char Prio)
    return 0;
 }
 									/*}}}*/
-// Bases for iterator classes						/*{{{*/
-void pkgCache::VerIterator::_dummy() {}
-void pkgCache::DepIterator::_dummy() {}
-void pkgCache::PrvIterator::_dummy() {}
-void pkgCache::DescIterator::_dummy() {}
+// GrpIterator::FindPkg - Locate a package by arch			/*{{{*/
+// ---------------------------------------------------------------------
+/* Returns an End-Pointer on error, pointer to the package otherwise */
+pkgCache::PkgIterator pkgCache::GrpIterator::FindPkg(string Arch) const {
+	if (unlikely(IsGood() == false || S->FirstPackage == 0))
+		return PkgIterator(*Owner, 0);
+
+	static string const myArch = _config->Find("APT::Architecture");
+	/* Most of the time the package for our native architecture is
+	   the one we add at first to the cache, but this would be the
+	   last one we check, so we do it now. */
+	if (Arch == "native" || Arch == myArch) {
+		Arch = myArch;
+		pkgCache::Package *Pkg = Owner->PkgP + S->LastPackage;
+		if (stringcasecmp(Arch, Owner->StrP + Pkg->Arch) == 0)
+			return PkgIterator(*Owner, Pkg);
+	}
+
+	/* If we accept any package we simply return the "first"
+	   package in this group (the last one added). */
+	if (Arch == "any")
+		return PkgIterator(*Owner, Owner->PkgP + S->FirstPackage);
+
+	/* Iterate over the list to find the matching arch
+	   unfortunately this list includes "package noise"
+	   (= different packages with same calculated hash),
+	   so we need to check the name also */
+	for (pkgCache::Package *Pkg = PackageList(); Pkg != Owner->PkgP;
+	     Pkg = Owner->PkgP + Pkg->NextPackage) {
+		if (S->Name == Pkg->Name &&
+		    stringcasecmp(Arch, Owner->StrP + Pkg->Arch) == 0)
+			return PkgIterator(*Owner, Pkg);
+		if ((Owner->PkgP + S->LastPackage) == Pkg)
+			break;
+	}
+
+	return PkgIterator(*Owner, 0);
+}
+									/*}}}*/
+// GrpIterator::FindPreferredPkg - Locate the "best" package		/*{{{*/
+// ---------------------------------------------------------------------
+/* Returns an End-Pointer on error, pointer to the package otherwise */
+pkgCache::PkgIterator pkgCache::GrpIterator::FindPreferredPkg(bool const &PreferNonVirtual) const {
+	pkgCache::PkgIterator Pkg = FindPkg("native");
+	if (Pkg.end() == false && (PreferNonVirtual == false || Pkg->VersionList != 0))
+		return Pkg;
+
+	std::vector<std::string> const archs = APT::Configuration::getArchitectures();
+	for (std::vector<std::string>::const_iterator a = archs.begin();
+	     a != archs.end(); ++a) {
+		Pkg = FindPkg(*a);
+		if (Pkg.end() == false && (PreferNonVirtual == false || Pkg->VersionList != 0))
+			return Pkg;
+	}
+
+	if (PreferNonVirtual == true)
+		return FindPreferredPkg(false);
+	return PkgIterator(*Owner, 0);
+}
+									/*}}}*/
+// GrpIterator::NextPkg - Locate the next package in the group		/*{{{*/
+// ---------------------------------------------------------------------
+/* Returns an End-Pointer on error, pointer to the package otherwise.
+   We can't simply ++ to the next as the next package of the last will
+   be from a different group (with the same hash value) */
+pkgCache::PkgIterator pkgCache::GrpIterator::NextPkg(pkgCache::PkgIterator const &LastPkg) const {
+	if (unlikely(IsGood() == false || S->FirstPackage == 0 ||
+	    LastPkg.end() == true))
+		return PkgIterator(*Owner, 0);
+
+	if (S->LastPackage == LastPkg.Index())
+		return PkgIterator(*Owner, 0);
+
+	return PkgIterator(*Owner, Owner->PkgP + LastPkg->NextPackage);
+}
+									/*}}}*/
+// GrpIterator::operator ++ - Postfix incr				/*{{{*/
+// ---------------------------------------------------------------------
+/* This will advance to the next logical group in the hash table. */
+void pkgCache::GrpIterator::operator ++(int) 
+{
+   // Follow the current links
+   if (S != Owner->GrpP)
+      S = Owner->GrpP + S->Next;
+
+   // Follow the hash table
+   while (S == Owner->GrpP && (HashIndex+1) < (signed)_count(Owner->HeaderP->GrpHashTable))
+   {
+      HashIndex++;
+      S = Owner->GrpP + Owner->HeaderP->GrpHashTable[HashIndex];
+   }
+};
 									/*}}}*/
 // PkgIterator::operator ++ - Postfix incr				/*{{{*/
 // ---------------------------------------------------------------------
@@ -253,14 +409,14 @@ void pkgCache::DescIterator::_dummy() {}
 void pkgCache::PkgIterator::operator ++(int) 
 {
    // Follow the current links
-   if (Pkg != Owner->PkgP)
-      Pkg = Owner->PkgP + Pkg->NextPackage;
+   if (S != Owner->PkgP)
+      S = Owner->PkgP + S->NextPackage;
 
    // Follow the hash table
-   while (Pkg == Owner->PkgP && (HashIndex+1) < (signed)_count(Owner->HeaderP->HashTable))
+   while (S == Owner->PkgP && (HashIndex+1) < (signed)_count(Owner->HeaderP->PkgHashTable))
    {
       HashIndex++;
-      Pkg = Owner->PkgP + Owner->HeaderP->HashTable[HashIndex];
+      S = Owner->PkgP + Owner->HeaderP->PkgHashTable[HashIndex];
    }
 };
 									/*}}}*/
@@ -269,12 +425,12 @@ void pkgCache::PkgIterator::operator ++(int)
 /* By this we mean if it is either cleanly installed or cleanly removed. */
 pkgCache::PkgIterator::OkState pkgCache::PkgIterator::State() const
 {  
-   if (Pkg->InstState == pkgCache::State::ReInstReq ||
-       Pkg->InstState == pkgCache::State::HoldReInstReq)
+   if (S->InstState == pkgCache::State::ReInstReq ||
+       S->InstState == pkgCache::State::HoldReInstReq)
       return NeedsUnpack;
    
-   if (Pkg->CurrentState == pkgCache::State::UnPacked ||
-       Pkg->CurrentState == pkgCache::State::HalfConfigured)
+   if (S->CurrentState == pkgCache::State::UnPacked ||
+       S->CurrentState == pkgCache::State::HalfConfigured)
       // we leave triggers alone complettely. dpkg deals with
       // them in a hard-to-predict manner and if they get 
       // resolved by dpkg before apt run dpkg --configure on 
@@ -283,8 +439,8 @@ pkgCache::PkgIterator::OkState pkgCache::PkgIterator::State() const
       //Pkg->CurrentState == pkgCache::State::TriggersPending)
       return NeedsConfigure;
    
-   if (Pkg->CurrentState == pkgCache::State::HalfInstalled ||
-       Pkg->InstState != pkgCache::State::Ok)
+   if (S->CurrentState == pkgCache::State::HalfInstalled ||
+       S->InstState != pkgCache::State::Ok)
       return NeedsUnpack;
       
    return NeedsNothing;
@@ -331,7 +487,7 @@ operator<<(ostream& out, pkgCache::PkgIterator Pkg)
    string candidate = string(Pkg.CandVersion() == 0 ? "none" : Pkg.CandVersion());
    string newest = string(Pkg.VersionList().end() ? "none" : Pkg.VersionList().VerStr());
 
-   out << Pkg.Name() << " < " << current;
+   out << Pkg.Name() << " [ " << Pkg.Arch() << " ] < " << current;
    if (current != candidate)
       out << " -> " << candidate;
    if ( newest != "none" && candidate != newest)
@@ -340,17 +496,29 @@ operator<<(ostream& out, pkgCache::PkgIterator Pkg)
    return out;
 }
 									/*}}}*/
+// PkgIterator::FullName - Returns Name and (maybe) Architecture	/*{{{*/
+// ---------------------------------------------------------------------
+/* Returns a name:arch string */
+std::string pkgCache::PkgIterator::FullName(bool const &Pretty) const
+{
+   string fullname = Name();
+   if (Pretty == false ||
+       (strcmp(Arch(), "all") != 0 && _config->Find("APT::Architecture") != Arch()))
+      return fullname.append(":").append(Arch());
+   return fullname;
+}
+									/*}}}*/
 // DepIterator::IsCritical - Returns true if the dep is important	/*{{{*/
 // ---------------------------------------------------------------------
 /* Currently critical deps are defined as depends, predepends and
    conflicts (including dpkg's Breaks fields). */
-bool pkgCache::DepIterator::IsCritical()
+bool pkgCache::DepIterator::IsCritical() const
 {
-   if (Dep->Type == pkgCache::Dep::Conflicts ||
-       Dep->Type == pkgCache::Dep::DpkgBreaks ||
-       Dep->Type == pkgCache::Dep::Obsoletes ||
-       Dep->Type == pkgCache::Dep::Depends ||
-       Dep->Type == pkgCache::Dep::PreDepends)
+   if (S->Type == pkgCache::Dep::Conflicts ||
+       S->Type == pkgCache::Dep::DpkgBreaks ||
+       S->Type == pkgCache::Dep::Obsoletes ||
+       S->Type == pkgCache::Dep::Depends ||
+       S->Type == pkgCache::Dep::PreDepends)
       return true;
    return false;
 }
@@ -368,7 +536,7 @@ bool pkgCache::DepIterator::IsCritical()
    In Conjunction with the DepCache the value of Result may not be 
    super-good since the policy may have made it uninstallable. Using
    AllTargets is better in this case. */
-bool pkgCache::DepIterator::SmartTargetPkg(PkgIterator &Result)
+bool pkgCache::DepIterator::SmartTargetPkg(PkgIterator &Result) const
 {
    Result = TargetPkg();
    
@@ -417,7 +585,7 @@ bool pkgCache::DepIterator::SmartTargetPkg(PkgIterator &Result)
    provides. It includes every possible package-version that could satisfy
    the dependency. The last item in the list has a 0. The resulting pointer
    must be delete [] 'd */
-pkgCache::Version **pkgCache::DepIterator::AllTargets()
+pkgCache::Version **pkgCache::DepIterator::AllTargets() const
 {
    Version **Res = 0;
    unsigned long Size =0;
@@ -429,12 +597,12 @@ pkgCache::Version **pkgCache::DepIterator::AllTargets()
       // Walk along the actual package providing versions
       for (VerIterator I = DPkg.VersionList(); I.end() == false; I++)
       {
-	 if (Owner->VS->CheckDep(I.VerStr(),Dep->CompareOp,TargetVer()) == false)
+	 if (Owner->VS->CheckDep(I.VerStr(),S->CompareOp,TargetVer()) == false)
 	    continue;
 
-	 if ((Dep->Type == pkgCache::Dep::Conflicts ||
-	      Dep->Type == pkgCache::Dep::DpkgBreaks ||
-	      Dep->Type == pkgCache::Dep::Obsoletes) &&
+	 if ((S->Type == pkgCache::Dep::Conflicts ||
+	      S->Type == pkgCache::Dep::DpkgBreaks ||
+	      S->Type == pkgCache::Dep::Obsoletes) &&
 	     ParentPkg() == I.ParentPkg())
 	    continue;
 	 
@@ -446,12 +614,12 @@ pkgCache::Version **pkgCache::DepIterator::AllTargets()
       // Follow all provides
       for (PrvIterator I = DPkg.ProvidesList(); I.end() == false; I++)
       {
-	 if (Owner->VS->CheckDep(I.ProvideVersion(),Dep->CompareOp,TargetVer()) == false)
+	 if (Owner->VS->CheckDep(I.ProvideVersion(),S->CompareOp,TargetVer()) == false)
 	    continue;
 	 
-	 if ((Dep->Type == pkgCache::Dep::Conflicts ||
-	      Dep->Type == pkgCache::Dep::DpkgBreaks ||
-	      Dep->Type == pkgCache::Dep::Obsoletes) &&
+	 if ((S->Type == pkgCache::Dep::Conflicts ||
+	      S->Type == pkgCache::Dep::DpkgBreaks ||
+	      S->Type == pkgCache::Dep::Obsoletes) &&
 	     ParentPkg() == I.OwnerPkg())
 	    continue;
 	 
@@ -489,13 +657,37 @@ void pkgCache::DepIterator::GlobOr(DepIterator &Start,DepIterator &End)
    End = *this;
    for (bool LastOR = true; end() == false && LastOR == true;)
    {
-      LastOR = (Dep->CompareOp & pkgCache::Dep::Or) == pkgCache::Dep::Or;
+      LastOR = (S->CompareOp & pkgCache::Dep::Or) == pkgCache::Dep::Or;
       (*this)++;
       if (LastOR == true)
 	 End = (*this);
    }
 }
 									/*}}}*/
+// ostream operator to handle string representation of a dependecy	/*{{{*/
+// ---------------------------------------------------------------------
+/* */
+std::ostream& operator<<(ostream& out, pkgCache::DepIterator D)
+{
+   if (D.end() == true)
+      return out << "invalid dependency";
+
+   pkgCache::PkgIterator P = D.ParentPkg();
+   pkgCache::PkgIterator T = D.TargetPkg();
+
+   out << (P.end() ? "invalid pkg" : P.FullName(false)) << " " << D.DepType()
+	<< " on ";
+   if (T.end() == true)
+      out << "invalid pkg";
+   else
+      out << T;
+
+   if (D->Version != 0)
+      out << " (" << D.CompType() << " " << D.TargetVer() << ")";
+
+   return out;
+}
+									/*}}}*/
 // VerIterator::CompareVer - Fast version compare for same pkgs		/*{{{*/
 // ---------------------------------------------------------------------
 /* This just looks over the version list to see if B is listed before A. In
@@ -544,6 +736,23 @@ bool pkgCache::VerIterator::Automatic() const
    return false;
 }
 									/*}}}*/
+// VerIterator::Pseudo - Check if this version is a pseudo one		/*{{{*/
+// ---------------------------------------------------------------------
+/* Sometimes you have the need to express dependencies with versions
+   which doesn't really exist or exist multiply times for "different"
+   packages. We need these versions for dependency resolution but they
+   are a problem everytime we need to download/install something. */
+bool pkgCache::VerIterator::Pseudo() const
+{
+   if (S->MultiArch == pkgCache::Version::All &&
+	   strcmp(Arch(true),"all") != 0)
+   {
+	   GrpIterator const Grp = ParentPkg().Group();
+	   return (Grp->LastPackage != Grp->FirstPackage);
+   }
+   return false;
+}
+									/*}}}*/
 // VerIterator::NewestFile - Return the newest file version relation	/*{{{*/
 // ---------------------------------------------------------------------
 /* This looks at the version numbers associated with all of the sources
@@ -565,7 +774,7 @@ pkgCache::VerFileIterator pkgCache::VerIterator::NewestFile() const
 // ---------------------------------------------------------------------
 /* This describes the version from a release-centric manner. The output is a 
    list of Label:Version/Archive */
-string pkgCache::VerIterator::RelStr()
+string pkgCache::VerIterator::RelStr() const
 {
    bool First = true;
    string Res;
@@ -624,7 +833,9 @@ string pkgCache::VerIterator::RelStr()
 	 else
 	    Res += File.Site();
       }      
-   }   
+   }
+   if (S->ParentPkg != 0)
+      Res.append(" [").append(Arch()).append("]");
    return Res;
 }
 									/*}}}*/
@@ -639,7 +850,7 @@ bool pkgCache::PkgFileIterator::IsOk()
    if (stat(FileName(),&Buf) != 0)
       return false;
 
-   if (Buf.st_size != (signed)File->Size || Buf.st_mtime != File->mtime)
+   if (Buf.st_size != (signed)S->Size || Buf.st_mtime != S->mtime)
       return false;
 
    return true;
@@ -663,6 +874,8 @@ string pkgCache::PkgFileIterator::RelStr()
       Res = Res + (Res.empty() == true?"l=":",l=")  + Label();
    if (Component() != 0)
       Res = Res + (Res.empty() == true?"c=":",c=")  + Component();
+   if (Architecture() != 0)
+      Res = Res + (Res.empty() == true?"b=":",b=")  + Architecture();
    return Res;
 }
 									/*}}}*/
@@ -673,14 +886,23 @@ string pkgCache::PkgFileIterator::RelStr()
  */
 pkgCache::DescIterator pkgCache::VerIterator::TranslatedDescription() const
 {
-   pkgCache::DescIterator DescDefault = DescriptionList();
-   pkgCache::DescIterator Desc = DescDefault;
-   for (; Desc.end() == false; Desc++)
-      if (pkgIndexFile::LanguageCode() == Desc.LanguageCode())
-	 break;
-   if (Desc.end() == true) 
-      Desc = DescDefault;
-   return Desc;
+   std::vector<string> const lang = APT::Configuration::getLanguages();
+   for (std::vector<string>::const_iterator l = lang.begin();
+	l != lang.end(); l++)
+   {
+      pkgCache::DescIterator DescDefault = DescriptionList();
+      pkgCache::DescIterator Desc = DescDefault;
+
+      for (; Desc.end() == false; Desc++)
+	 if (*l == Desc.LanguageCode() ||
+	     (*l == "en" && strcmp(Desc.LanguageCode(),"") == 0))
+	    break;
+      if (Desc.end() == true) 
+	 Desc = DescDefault;
+      return Desc;
+   }
+
+   return DescriptionList();
 };
 
 									/*}}}*/

apt-pkg/pkgcache.h

@@ -1,20 +1,75 @@
 // -*- mode: cpp; mode: fold -*-
 // Description								/*{{{*/
-// $Id: pkgcache.h,v 1.25 2001/07/01 22:28:24 jgg Exp $
-/* ######################################################################
-   
-   Cache - Structure definitions for the cache file
-   
-   Please see doc/apt-pkg/cache.sgml for a more detailed description of 
-   this format. Also be sure to keep that file up-to-date!!
-   
+/**\file pkgcache.h
+   \brief pkgCache - Structure definitions for the cache file
+
+   The goal of the cache file is two fold:
+   Firstly to speed loading and processing of the package file array and
+   secondly to reduce memory consumption of the package file array.
+
+   The implementation is aimed at an environment with many primary package
+   files, for instance someone that has a Package file for their CD-ROM, a
+   Package file for the latest version of the distribution on the CD-ROM and a
+   package file for the development version. Always present is the information
+   contained in the status file which might be considered a separate package
+   file.
+
+   Please understand, this is designed as a <b>Cache file</b> it is not meant to be
+   used on any system other than the one it was created for. It is not meant to
+   be authoritative either, i.e. if a system crash or software failure occurs it
+   must be perfectly acceptable for the cache file to be in an inconsistent
+   state. Furthermore at any time the cache file may be erased without losing
+   any information.
+
+   Also the structures and storage layout is optimized for use by the APT
+   and may not be suitable for all purposes. However it should be possible
+   to extend it with associate cache files that contain other information.
+
+   To keep memory use down the cache file only contains often used fields and
+   fields that are inexpensive to store, the Package file has a full list of
+   fields. Also the client may assume that all items are perfectly valid and
+   need not perform checks against their correctness. Removal of information
+   from the cache is possible, but blanks will be left in the file, and
+   unused strings will also be present. The recommended implementation is to
+   simply rebuild the cache each time any of the data files change. It is
+   possible to add a new package file to the cache without any negative side
+   effects.
+
+   <b>Note on Pointer access</b>
    Clients should always use the CacheIterators classes for access to the
-   cache. They provide a simple STL-like method for traversing the links
-   of the datastructure.
-   
-   See pkgcachegen.h for information about generating cache structures.
-   
-   ##################################################################### */
+   cache and the data in it. They also provide a simple STL-like method for
+   traversing the links of the datastructure.
+
+   Every item in every structure is stored as the index to that structure.
+   What this means is that once the files is mmaped every data access has to
+   go through a fix up stage to get a real memory pointer. This is done
+   by taking the index, multiplying it by the type size and then adding
+   it to the start address of the memory block. This sounds complex, but
+   in C it is a single array dereference. Because all items are aligned to
+   their size and indexes are stored as multiples of the size of the structure
+   the format is immediately portable to all possible architectures - BUT the
+   generated files are -NOT-.
+
+   This scheme allows code like this to be written:
+   <example>
+     void *Map = mmap(...);
+     Package *PkgList = (Package *)Map;
+     Header *Head = (Header *)Map;
+     char *Strings = (char *)Map;
+     cout << (Strings + PkgList[Head->HashTable[0]]->Name) << endl;
+   </example>
+   Notice the lack of casting or multiplication. The net result is to return
+   the name of the first package in the first hash bucket, without error
+   checks.
+
+   The generator uses allocation pools to group similarly sized structures in
+   large blocks to eliminate any alignment overhead. The generator also
+   assures that no structures overlap and all indexes are unique. Although
+   at first glance it may seem like there is the potential for two structures
+   to exist at the same point the generator never allows this to happen.
+   (See the discussion of free space pools)
+
+   See \ref pkgcachegen.h for more information about generating cache structures. */
 									/*}}}*/
 #ifndef PKGLIB_PKGCACHE_H
 #define PKGLIB_PKGCACHE_H
@@ -32,6 +87,7 @@ class pkgCache								/*{{{*/
    public:
    // Cache element predeclarations
    struct Header;
+   struct Group;
    struct Package;
    struct PackageFile;
    struct Version;
@@ -43,6 +99,8 @@ class pkgCache								/*{{{*/
    struct DescFile;
    
    // Iterators
+   template<typename Str, typename Itr> class Iterator;
+   class GrpIterator;
    class PkgIterator;
    class VerIterator;
    class DescIterator;
@@ -51,14 +109,6 @@ class pkgCache								/*{{{*/
    class PkgFileIterator;
    class VerFileIterator;
    class DescFileIterator;
-   friend class PkgIterator;
-   friend class VerIterator;
-   friend class DescInterator;
-   friend class DepIterator;
-   friend class PrvIterator;
-   friend class PkgFileIterator;
-   friend class VerFileIterator;
-   friend class DescFileIterator;
    
    class Namespace;
    
@@ -71,12 +121,20 @@ class pkgCache								/*{{{*/
    {
       enum DepType {Depends=1,PreDepends=2,Suggests=3,Recommends=4,
 	 Conflicts=5,Replaces=6,Obsoletes=7,DpkgBreaks=8,Enhances=9};
+      /** \brief available compare operators
+
+          The lower 4 bits are used to indicate what operator is being specified and
+          the upper 4 bits are flags. OR indicates that the next package is
+          or'd with the current package. */
       enum DepCompareOp {Or=0x10,NoOp=0,LessEq=0x1,GreaterEq=0x2,Less=0x3,
 	 Greater=0x4,Equals=0x5,NotEquals=0x6};
    };
    
    struct State
    {
+      /** \brief priority of a package version
+
+          Zero is used for unparsable or absent Priority fields. */
       enum VerPriority {Important=1,Required=2,Standard=3,Optional=4,Extra=5};
       enum PkgSelectedState {Unknown=0,Install=1,Hold=2,DeInstall=3,Purge=4};
       enum PkgInstState {Ok=0,ReInstReq=1,HoldInst=2,HoldReInstReq=3};
@@ -104,6 +162,7 @@ class pkgCache								/*{{{*/
    
    // Pointers to the arrays of items
    Header *HeaderP;
+   Group *GrpP;
    Package *PkgP;
    VerFile *VerFileP;
    DescFile *DescFileP;
@@ -115,7 +174,7 @@ class pkgCache								/*{{{*/
    StringItem *StringItemP;
    char *StrP;
 
-   virtual bool ReMap();
+   virtual bool ReMap(bool const &Errorchecks = true);
    inline bool Sync() {return Map.Sync();};
    inline MMap &GetMap() {return Map;};
    inline void *DataEnd() {return ((unsigned char *)Map.Data()) + Map.Size();};
@@ -124,17 +183,24 @@ class pkgCache								/*{{{*/
    inline unsigned long Hash(const string &S) const {return sHash(S);};
    inline unsigned long Hash(const char *S) const {return sHash(S);};
 
-   // Usefull transformation things
+   // Useful transformation things
    const char *Priority(unsigned char Priority);
    
    // Accessors
+   GrpIterator FindGrp(const string &Name);
    PkgIterator FindPkg(const string &Name);
+   PkgIterator FindPkg(const string &Name, const string &Arch);
+
    Header &Head() {return *HeaderP;};
+   inline GrpIterator GrpBegin();
+   inline GrpIterator GrpEnd();
    inline PkgIterator PkgBegin();
    inline PkgIterator PkgEnd();
    inline PkgFileIterator FileBegin();
    inline PkgFileIterator FileEnd();
 
+   inline bool MultiArchCache() const { return MultiArchEnabled; };
+
    // Make me a function
    pkgVersioningSystem *VS;
    
@@ -145,19 +211,40 @@ class pkgCache								/*{{{*/
    
    pkgCache(MMap *Map,bool DoMap = true);
    virtual ~pkgCache() {};
+
+private:
+   bool MultiArchEnabled;
+   PkgIterator SingleArchFindPkg(const string &Name);
 };
 									/*}}}*/
 // Header structure							/*{{{*/
 struct pkgCache::Header
 {
-   // Signature information
+   /** \brief Signature information
+
+       This must contain the hex value 0x98FE76DC which is designed to
+       verify that the system loading the image has the same byte order
+       and byte size as the system saving the image */
    unsigned long Signature;
+   /** These contain the version of the cache file */
    short MajorVersion;
    short MinorVersion;
+   /** \brief indicates if the cache should be erased
+
+       Dirty is true if the cache file was opened for reading, the client
+       expects to have written things to it and have not fully synced it.
+       The file should be erased and rebuilt if it is true. */
    bool Dirty;
-   
-   // Size of structure values
+
+   /** \brief Size of structure values
+
+       All *Sz variables contains the sizeof() that particular structure.
+       It is used as an extra consistency check on the structure of the file.
+
+       If any of the size values do not exactly match what the client expects
+       then the client should refuse the load the file. */
    unsigned short HeaderSz;
+   unsigned short GroupSz;
    unsigned short PackageSz;
    unsigned short PackageFileSz;
    unsigned short VersionSz;
@@ -166,8 +253,13 @@ struct pkgCache::Header
    unsigned short ProvidesSz;
    unsigned short VerFileSz;
    unsigned short DescFileSz;
-   
-   // Structure counts
+
+   /** \brief Structure counts
+
+       These indicate the number of each structure contained in the cache.
+       PackageCount is especially useful for generating user state structures.
+       See Package::Id for more info. */
+   unsigned long GroupCount;
    unsigned long PackageCount;
    unsigned long VersionCount;
    unsigned long DescriptionCount;
@@ -176,154 +268,393 @@ struct pkgCache::Header
    unsigned long VerFileCount;
    unsigned long DescFileCount;
    unsigned long ProvidesCount;
-   
-   // Offsets
-   map_ptrloc FileList;              // struct PackageFile
-   map_ptrloc StringList;            // struct StringItem
-   map_ptrloc VerSysName;            // StringTable
-   map_ptrloc Architecture;          // StringTable
+
+   /** \brief index of the first PackageFile structure
+
+       The PackageFile structures are singly linked lists that represent
+       all package files that have been merged into the cache. */
+   map_ptrloc FileList;
+   /** \brief index of the first StringItem structure
+
+       The cache contains a list of all the unique strings (StringItems).
+       The parser reads this list into memory so it can match strings
+       against it.*/
+   map_ptrloc StringList;
+   /** \brief String representing the version system used */
+   map_ptrloc VerSysName;
+   /** \brief Architecture(s) the cache was built against */
+   map_ptrloc Architecture;
+   /** \brief The maximum size of a raw entry from the original Package file */
    unsigned long MaxVerFileSize;
+   /** \brief The maximum size of a raw entry from the original Translation file */
    unsigned long MaxDescFileSize;
 
-   /* Allocation pools, there should be one of these for each structure
-      excluding the header */
-   DynamicMMap::Pool Pools[8];
+   /** \brief The Pool structures manage the allocation pools that the generator uses
+
+       Start indicates the first byte of the pool, Count is the number of objects
+       remaining in the pool and ItemSize is the structure size (alignment factor)
+       of the pool. An ItemSize of 0 indicates the pool is empty. There should be
+       the same number of pools as there are structure types. The generator
+       stores this information so future additions can make use of any unused pool
+       blocks. */
+   DynamicMMap::Pool Pools[9];
    
-   // Rapid package name lookup
-   map_ptrloc HashTable[2*1048];
+   /** \brief hash tables providing rapid group/package name lookup
+
+       Each group/package name is inserted into the hash table using pkgCache::Hash(const &string)
+       By iterating over each entry in the hash table it is possible to iterate over
+       the entire list of packages. Hash Collisions are handled with a singly linked
+       list of packages based at the hash item. The linked list contains only
+       packages that match the hashing function.
+       In the PkgHashTable is it possible that multiple packages have the same name -
+       these packages are stored as a sequence in the list.
+
+       Beware: The Hashmethod assumes that the hash table sizes are equal */
+   map_ptrloc PkgHashTable[2*1048];
+   map_ptrloc GrpHashTable[2*1048];
 
    bool CheckSizes(Header &Against) const;
    Header();
 };
 									/*}}}*/
-struct pkgCache::Package						/*{{{*/
+// Group structure							/*{{{*/
+/** \brief groups architecture depending packages together
+
+    On or more packages with the same name form a group, so we have
+    a simple way to access a package built for different architectures
+    Group exists in a singly linked list of group records starting at
+    the hash index of the name in the pkgCache::Header::GrpHashTable */
+struct pkgCache::Group
+{
+   /** \brief Name of the group */
+   map_ptrloc Name;		// StringItem
+
+   // Linked List
+   /** \brief Link to the first package which belongs to the group */
+   map_ptrloc FirstPackage;	// Package
+   /** \brief Link to the last package which belongs to the group */
+   map_ptrloc LastPackage;	// Package
+   /** \brief Link to the next Group */
+   map_ptrloc Next;		// Group
+   /** \brief unique sequel ID */
+   unsigned int ID;
+
+};
+									/*}}}*/
+// Package structure							/*{{{*/
+/** \brief contains information for a single unique package
+
+    There can be any number of versions of a given package.
+    Package exists in a singly linked list of package records starting at
+    the hash index of the name in the pkgCache::Header::PkgHashTable
+
+    A package can be created for every architecture so package names are
+    not unique, but it is garanteed that packages with the same name
+    are sequencel ordered in the list. Packages with the same name can be
+    accessed with the Group.
+*/
+struct pkgCache::Package
 {
-   // Pointers
-   map_ptrloc Name;              // Stringtable
+   /** \brief Name of the package */
+   map_ptrloc Name;              // StringItem
+   /** \brief Architecture of the package */
+   map_ptrloc Arch;              // StringItem
+   /** \brief Base of a singly linked list of versions
+
+       Each structure represents a unique version of the package.
+       The version structures contain links into PackageFile and the
+       original text file as well as detailed information about the size
+       and dependencies of the specific package. In this way multiple
+       versions of a package can be cleanly handled by the system.
+       Furthermore, this linked list is guaranteed to be sorted
+       from Highest version to lowest version with no duplicate entries. */
    map_ptrloc VersionList;       // Version
+   /** \brief index to the installed version */
    map_ptrloc CurrentVer;        // Version
-   map_ptrloc Section;           // StringTable (StringItem)
-      
-   // Linked list 
+   /** \brief indicates the deduced section
+
+       Should be the index to the string "Unknown" or to the section
+       of the last parsed item. */
+   map_ptrloc Section;           // StringItem
+   /** \brief index of the group this package belongs to */
+   map_ptrloc Group;             // Group the Package belongs to
+
+   // Linked list
+   /** \brief Link to the next package in the same bucket */
    map_ptrloc NextPackage;       // Package
+   /** \brief List of all dependencies on this package */
    map_ptrloc RevDepends;        // Dependency
+   /** \brief List of all "packages" this package provide */
    map_ptrloc ProvidesList;      // Provides
 
    // Install/Remove/Purge etc
+   /** \brief state that the user wishes the package to be in */
    unsigned char SelectedState;     // What
+   /** \brief installation state of the package
+
+       This should be "ok" but in case the installation failed
+       it will be different.
+   */
    unsigned char InstState;         // Flags
+   /** \brief indicates if the package is installed */
    unsigned char CurrentState;      // State
-   
+
+   /** \brief unique sequel ID
+
+       ID is a unique value from 0 to Header->PackageCount assigned by the generator.
+       This allows clients to create an array of size PackageCount and use it to store
+       state information for the package map. For instance the status file emitter uses
+       this to track which packages have been emitted already. */
    unsigned int ID;
+   /** \brief some useful indicators of the package's state */
    unsigned long Flags;
 };
 									/*}}}*/
-struct pkgCache::PackageFile						/*{{{*/
+// Package File structure						/*{{{*/
+/** \brief stores information about the files used to generate the cache
+
+    Package files are referenced by Version structures to be able to know
+    after the generation still from which Packages file includes this Version
+    as we need this information later on e.g. for pinning. */
+struct pkgCache::PackageFile
 {
-   // Names
-   map_ptrloc FileName;        // Stringtable
-   map_ptrloc Archive;         // Stringtable
-   map_ptrloc Codename;        // Stringtable
-   map_ptrloc Component;       // Stringtable
-   map_ptrloc Version;         // Stringtable
-   map_ptrloc Origin;          // Stringtable
-   map_ptrloc Label;           // Stringtable
-   map_ptrloc Architecture;    // Stringtable
-   map_ptrloc Site;            // Stringtable
-   map_ptrloc IndexType;       // Stringtable
-   unsigned long Size;            
+   /** \brief physical disk file that this PackageFile represents */
+   map_ptrloc FileName;        // StringItem
+   /** \brief the release information
+
+       Please see the files document for a description of what the
+       release information means. */
+   map_ptrloc Archive;         // StringItem
+   map_ptrloc Codename;        // StringItem
+   map_ptrloc Component;       // StringItem
+   map_ptrloc Version;         // StringItem
+   map_ptrloc Origin;          // StringItem
+   map_ptrloc Label;           // StringItem
+   map_ptrloc Architecture;    // StringItem
+   /** \brief The site the index file was fetched from */
+   map_ptrloc Site;            // StringItem
+   /** \brief indicates what sort of index file this is
+
+       @TODO enumerate at least the possible indexes */
+   map_ptrloc IndexType;       // StringItem
+   /** \brief Size of the file
+
+       Used together with the modification time as a
+       simple check to ensure that the Packages
+       file has not been altered since Cache generation. */
+   unsigned long Size;
+   /** \brief Modification time for the file */
+   time_t mtime;
+
+   /* @TODO document PackageFile::Flags */
    unsigned long Flags;
-   
+
    // Linked list
+   /** \brief Link to the next PackageFile in the Cache */
    map_ptrloc NextFile;        // PackageFile
+   /** \brief unique sequel ID */
    unsigned int ID;
-   time_t mtime;                  // Modification time for the file
 };
 									/*}}}*/
-struct pkgCache::VerFile						/*{{{*/
+// VerFile structure							/*{{{*/
+/** \brief associates a version with a PackageFile
+
+    This allows a full description of all Versions in all files
+    (and hence all sources) under consideration. */
+struct pkgCache::VerFile
 {
+   /** \brief index of the package file that this version was found in */
    map_ptrloc File;           // PackageFile
+   /** \brief next step in the linked list */
    map_ptrloc NextFile;       // PkgVerFile
+   /** \brief position in the package file */
    map_ptrloc Offset;         // File offset
-   unsigned short Size;
+   /* @TODO document pkgCache::VerFile::Size */
+   unsigned long Size;
 };
 									/*}}}*/
-struct pkgCache::DescFile						/*{{{*/
+// DescFile structure							/*{{{*/
+/** \brief associates a description with a Translation file */
+struct pkgCache::DescFile
 {
+   /** \brief index of the file that this description was found in */
    map_ptrloc File;           // PackageFile
+   /** \brief next step in the linked list */
    map_ptrloc NextFile;       // PkgVerFile
+   /** \brief position in the file */
    map_ptrloc Offset;         // File offset
-   unsigned short Size;
+   /* @TODO document pkgCache::DescFile::Size */
+   unsigned long Size;
 };
 									/*}}}*/
-struct pkgCache::Version						/*{{{*/
+// Version structure							/*{{{*/
+/** \brief information for a single version of a package
+
+    The version list is always sorted from highest version to lowest
+    version by the generator. Equal version numbers are either merged
+    or handled as separate versions based on the Hash value. */
+struct pkgCache::Version
 {
-   map_ptrloc VerStr;            // Stringtable
-   map_ptrloc Section;           // StringTable (StringItem)
-   map_ptrloc Arch;              // StringTable
-      
-   // Lists
+   /** \brief complete version string */
+   map_ptrloc VerStr;            // StringItem
+   /** \brief section this version is filled in */
+   map_ptrloc Section;           // StringItem
+   /** \brief stores the MultiArch capabilities of this version
+
+       None is the default and doesn't trigger special behaviour,
+       Foreign means that this version can fulfill dependencies even
+       if it is built for another architecture as the requester.
+       Same indicates that builds for different architectures can
+       be co-installed on the system and All is the marker for a
+       version with the Architecture: all. */
+   enum {None, All, Foreign, Same, Allowed} MultiArch;
+
+   /** \brief references all the PackageFile's that this version came from
+
+       FileList can be used to determine what distribution(s) the Version
+       applies to. If FileList is 0 then this is a blank version.
+       The structure should also have a 0 in all other fields excluding
+       pkgCache::Version::VerStr and Possibly pkgCache::Version::NextVer. */
    map_ptrloc FileList;          // VerFile
+   /** \brief next (lower or equal) version in the linked list */
    map_ptrloc NextVer;           // Version
+   /** \brief next description in the linked list */
    map_ptrloc DescriptionList;   // Description
+   /** \brief base of the dependency list */
    map_ptrloc DependsList;       // Dependency
+   /** \brief links to the owning package
+
+       This allows reverse dependencies to determine the package */
    map_ptrloc ParentPkg;         // Package
+   /** \brief list of pkgCache::Provides */
    map_ptrloc ProvidesList;      // Provides
-   
-   map_ptrloc Size;              // These are the .deb size
-   map_ptrloc InstalledSize;
+
+   /** \brief archive size for this version
+
+       For Debian this is the size of the .deb file. */
+   unsigned long long Size;      // These are the .deb size
+   /** \brief uncompressed size for this version */
+   unsigned long long InstalledSize;
+   /** \brief characteristic value representing this version
+
+       No two packages in existence should have the same VerStr
+       and Hash with different contents. */
    unsigned short Hash;
+   /** \brief unique sequel ID */
    unsigned int ID;
+   /** \brief parsed priority value */
    unsigned char Priority;
 };
 									/*}}}*/
-struct pkgCache::Description						/*{{{*/
+// Description structure						/*{{{*/
+/** \brief datamember of a linked list of available description for a version */
+struct pkgCache::Description
 {
-   // Language Code store the description translation language code. If
-   // the value has a 0 lenght then this is readed using the Package
-   // file else the Translation-CODE are used.
-   map_ptrloc language_code;     // StringTable
-   map_ptrloc md5sum;            // StringTable
+   /** \brief Language code of this description (translation)
+
+       If the value has a 0 length then this is read using the Package
+       file else the Translation-CODE file is used. */
+   map_ptrloc language_code;     // StringItem
+   /** \brief MD5sum of the original description
+
+       Used to map Translations of a description to a version
+       and to check that the Translation is up-to-date. */
+   map_ptrloc md5sum;            // StringItem
 
-   // Linked list 
+   /* @TODO document pkgCache::Description::FileList */
    map_ptrloc FileList;          // DescFile
+   /** \brief next translation for this description */
    map_ptrloc NextDesc;          // Description
+   /** \brief the text is a description of this package */
    map_ptrloc ParentPkg;         // Package
 
+   /** \brief unique sequel ID */
    unsigned int ID;
 };
 									/*}}}*/
-struct pkgCache::Dependency						/*{{{*/
+// Dependency structure							/*{{{*/
+/** \brief information for a single dependency record
+
+    The records are split up like this to ease processing by the client.
+    The base of the linked list is pkgCache::Version::DependsList.
+    All forms of dependencies are recorded here including Depends,
+    Recommends, Suggests, Enhances, Conflicts, Replaces and Breaks. */
+struct pkgCache::Dependency
 {
-   map_ptrloc Version;         // Stringtable
+   /** \brief string of the version the dependency is applied against */
+   map_ptrloc Version;         // StringItem
+   /** \brief index of the package this depends applies to
+
+       The generator will - if the package does not already exist -
+       create a blank (no version records) package. */
    map_ptrloc Package;         // Package
+   /** \brief next dependency of this version */
    map_ptrloc NextDepends;     // Dependency
+   /** \brief next reverse dependency of this package */
    map_ptrloc NextRevDepends;  // Dependency
+   /** \brief version of the package which has the reverse depends */
    map_ptrloc ParentVer;       // Version
-   
-   // Specific types of depends
-   map_ptrloc ID;   
+
+   /** \brief unique sequel ID */
+   map_ptrloc ID;
+   /** \brief Dependency type - Depends, Recommends, Conflicts, etc */
    unsigned char Type;
+   /** \brief comparison operator specified on the depends line
+
+       If the high bit is set then it is a logical OR with the previous record. */
    unsigned char CompareOp;
 };
 									/*}}}*/
-struct pkgCache::Provides						/*{{{*/
+// Provides structure							/*{{{*/
+/** \brief handles virtual packages
+
+    When a Provides: line is encountered a new provides record is added
+    associating the package with a virtual package name.
+    The provides structures are linked off the package structures.
+    This simplifies the analysis of dependencies and other aspects A provides
+    refers to a specific version of a specific package, not all versions need to
+    provide that provides.*/
+struct pkgCache::Provides
 {
-   map_ptrloc ParentPkg;        // Pacakge
+   /** \brief index of the package providing this */
+   map_ptrloc ParentPkg;        // Package
+   /** \brief index of the version this provide line applies to */
    map_ptrloc Version;          // Version
-   map_ptrloc ProvideVersion;   // Stringtable
+   /** \brief version in the provides line (if any)
+
+       This version allows dependencies to depend on specific versions of a
+       Provides, as well as allowing Provides to override existing packages.
+       This is experimental. Note that Debian doesn't allow versioned provides */
+   map_ptrloc ProvideVersion;   // StringItem
+   /** \brief next provides (based of package) */
    map_ptrloc NextProvides;     // Provides
+   /** \brief next provides (based of version) */
    map_ptrloc NextPkgProv;      // Provides
 };
 									/*}}}*/
-struct pkgCache::StringItem						/*{{{*/
+// StringItem structure							/*{{{*/
+/** \brief used for generating single instances of strings
+
+    Some things like Section Name are are useful to have as unique tags.
+    It is part of a linked list based at pkgCache::Header::StringList
+
+    All strings are simply inlined any place in the file that is natural
+    for the writer. The client should make no assumptions about the positioning
+    of strings. All StringItems should be null-terminated. */
+struct pkgCache::StringItem
 {
-   map_ptrloc String;        // Stringtable
+   /** \brief string this refers to */
+   map_ptrloc String;        // StringItem
+   /** \brief Next link in the chain */
    map_ptrloc NextItem;      // StringItem
 };
 									/*}}}*/
 #include <apt-pkg/cacheiterators.h>
 
+inline pkgCache::GrpIterator pkgCache::GrpBegin() 
+       {return GrpIterator(*this);};
+inline pkgCache::GrpIterator pkgCache::GrpEnd() 
+       {return GrpIterator(*this,GrpP);};
 inline pkgCache::PkgIterator pkgCache::PkgBegin() 
        {return PkgIterator(*this);};
 inline pkgCache::PkgIterator pkgCache::PkgEnd() 
@@ -337,7 +668,7 @@ inline pkgCache::PkgFileIterator pkgCache::FileEnd()
 class pkgCache::Namespace						/*{{{*/
 {   
    public:
-
+   typedef pkgCache::GrpIterator GrpIterator;
    typedef pkgCache::PkgIterator PkgIterator;
    typedef pkgCache::VerIterator VerIterator;
    typedef pkgCache::DescIterator DescIterator;

apt-pkg/pkgcachegen.h

@@ -23,6 +23,8 @@
 #include <apt-pkg/pkgcache.h>
 #include <apt-pkg/md5.h>
 
+#include <vector>
+
 class pkgSourceList;
 class OpProgress;
 class MMap;
@@ -31,16 +33,32 @@ class pkgIndexFile;
 class pkgCacheGenerator							/*{{{*/
 {
    private:
-   
+
    pkgCache::StringItem *UniqHash[26];
-   
+   map_ptrloc WriteStringInMap(std::string const &String) { return WriteStringInMap(String.c_str()); };
+   map_ptrloc WriteStringInMap(const char *String);
+   map_ptrloc WriteStringInMap(const char *String, const unsigned long &Len);
+   map_ptrloc AllocateInMap(const unsigned long &size);
+
    public:
    
    class ListParser;
    friend class ListParser;
-   
+
+   template<typename Iter> class Dynamic {
+      public:
+      static std::vector<Iter*> toReMap;
+      Dynamic(Iter &I) {
+	 toReMap.push_back(&I);
+      }
+
+      ~Dynamic() {
+	 toReMap.pop_back();
+      }
+   };
+
    protected:
-   
+
    DynamicMMap &Map;
    pkgCache Cache;
    OpProgress *Progress;
@@ -51,9 +69,13 @@ class pkgCacheGenerator							/*{{{*/
    // Flag file dependencies
    bool FoundFileDeps;
    
-   bool NewPackage(pkgCache::PkgIterator &Pkg,const string &PkgName);
+   bool NewGroup(pkgCache::GrpIterator &Grp,const string &Name);
+   bool NewPackage(pkgCache::PkgIterator &Pkg,const string &Name, const string &Arch);
    bool NewFileVer(pkgCache::VerIterator &Ver,ListParser &List);
    bool NewFileDesc(pkgCache::DescIterator &Desc,ListParser &List);
+   bool NewDepends(pkgCache::PkgIterator &Pkg, pkgCache::VerIterator &Ver,
+		   string const &Version, unsigned int const &Op,
+		   unsigned int const &Type, map_ptrloc *OldDepLast);
    unsigned long NewVersion(pkgCache::VerIterator &Ver,const string &VerStr,unsigned long Next);
    map_ptrloc NewDescription(pkgCache::DescIterator &Desc,const string &Lang,const MD5SumValue &md5sum,map_ptrloc Next);
 
@@ -72,7 +94,15 @@ class pkgCacheGenerator							/*{{{*/
 
    bool HasFileDeps() {return FoundFileDeps;};
    bool MergeFileProvides(ListParser &List);
-      
+   bool FinishCache(OpProgress *Progress);
+
+   static bool MakeStatusCache(pkgSourceList &List,OpProgress *Progress,
+			MMap **OutMap = 0,bool AllowMem = false);
+   static bool MakeOnlyStatusCache(OpProgress *Progress,DynamicMMap **OutMap);
+   static DynamicMMap* CreateDynamicMMap(FileFd *CacheF, unsigned long Flags = 0);
+
+   void ReMap(void const * const oldMap, void const * const newMap);
+
    pkgCacheGenerator(DynamicMMap *Map,OpProgress *Progress);
    ~pkgCacheGenerator();
 };
@@ -94,26 +124,28 @@ class pkgCacheGenerator::ListParser
 
    inline unsigned long WriteUniqString(string S) {return Owner->WriteUniqString(S);};
    inline unsigned long WriteUniqString(const char *S,unsigned int Size) {return Owner->WriteUniqString(S,Size);};
-   inline unsigned long WriteString(const string &S) {return Owner->Map.WriteString(S);};
-   inline unsigned long WriteString(const char *S,unsigned int Size) {return Owner->Map.WriteString(S,Size);};
-   bool NewDepends(pkgCache::VerIterator Ver,const string &Package,
+   inline unsigned long WriteString(const string &S) {return Owner->WriteStringInMap(S);};
+   inline unsigned long WriteString(const char *S,unsigned int Size) {return Owner->WriteStringInMap(S,Size);};
+   bool NewDepends(pkgCache::VerIterator &Ver,const string &Package, const string &Arch,
 		   const string &Version,unsigned int Op,
 		   unsigned int Type);
-   bool NewProvides(pkgCache::VerIterator Ver,const string &Package,
-		    const string &Version);
+   bool NewProvides(pkgCache::VerIterator &Ver,const string &PkgName,
+		    const string &PkgArch, const string &Version);
    
    public:
    
    // These all operate against the current section
    virtual string Package() = 0;
+   virtual string Architecture() = 0;
+   virtual bool ArchitectureAll() = 0;
    virtual string Version() = 0;
-   virtual bool NewVersion(pkgCache::VerIterator Ver) = 0;
+   virtual bool NewVersion(pkgCache::VerIterator &Ver) = 0;
    virtual string Description() = 0;
    virtual string DescriptionLanguage() = 0;
    virtual MD5SumValue Description_md5() = 0;
    virtual unsigned short VersionHash() = 0;
-   virtual bool UsePackage(pkgCache::PkgIterator Pkg,
-			   pkgCache::VerIterator Ver) = 0;
+   virtual bool UsePackage(pkgCache::PkgIterator &Pkg,
+			   pkgCache::VerIterator &Ver) = 0;
    virtual unsigned long Offset() = 0;
    virtual unsigned long Size() = 0;
    
@@ -121,16 +153,18 @@ class pkgCacheGenerator::ListParser
    
    inline bool HasFileDeps() {return FoundFileDeps;};
    virtual bool CollectFileProvides(pkgCache &Cache,
-				    pkgCache::VerIterator Ver) {return true;};
+				    pkgCache::VerIterator &Ver) {return true;};
 
    ListParser() : FoundFileDeps(false) {};
    virtual ~ListParser() {};
 };
 									/*}}}*/
+
 bool pkgMakeStatusCache(pkgSourceList &List,OpProgress &Progress,
 			MMap **OutMap = 0,bool AllowMem = false);
 bool pkgMakeOnlyStatusCache(OpProgress &Progress,DynamicMMap **OutMap);
 
+
 #ifdef APT_COMPATIBILITY
 #if APT_COMPATIBILITY != 986
 #warning "Using APT_COMPATIBILITY"
@@ -138,7 +172,7 @@ bool pkgMakeOnlyStatusCache(OpProgress &Progress,DynamicMMap **OutMap);
 MMap *pkgMakeStatusCacheMem(pkgSourceList &List,OpProgress &Progress)
 {
    MMap *Map = 0;
-   if (pkgMakeStatusCache(List,Progress,&Map,true) == false)
+   if (pkgCacheGenerator::MakeStatusCache(List,&Progress,&Map,true) == false)
       return 0;
    return Map;
 }

apt-pkg/policy.cc

@@ -115,12 +115,16 @@ bool pkgPolicy::InitDefaults()
 // ---------------------------------------------------------------------
 /* Evaluate the package pins and the default list to deteremine what the
    best package is. */
-pkgCache::VerIterator pkgPolicy::GetCandidateVer(pkgCache::PkgIterator Pkg)
+pkgCache::VerIterator pkgPolicy::GetCandidateVer(pkgCache::PkgIterator const &Pkg)
 {
    // Look for a package pin and evaluate it.
    signed Max = GetPriority(Pkg);
    pkgCache::VerIterator Pref = GetMatch(Pkg);
 
+   // Alternatives in case we can not find our package pin (Bug#512318).
+   signed MaxAlt = 0;
+   pkgCache::VerIterator PrefAlt;
+
    // no package = no candidate version
    if (Pkg.end() == true)
       return Pref;
@@ -143,6 +147,16 @@ pkgCache::VerIterator pkgPolicy::GetCandidateVer(pkgCache::PkgIterator Pkg)
     */
    for (pkgCache::VerIterator Ver = Pkg.VersionList(); Ver.end() == false; Ver++)
    {
+      /* Lets see if this version is the installed version */
+      bool instVer = (Pkg.CurrentVer() == Ver);
+      if (Ver.Pseudo() == true && instVer == false)
+      {
+	 pkgCache::PkgIterator const allPkg = Ver.ParentPkg().Group().FindPkg("all");
+	 if (allPkg->CurrentVer != 0 && allPkg.CurrentVer()->Hash == Ver->Hash &&
+	     strcmp(allPkg.CurVersion(), Ver.VerStr()) == 0)
+	    instVer = true;
+      }
+
       for (pkgCache::VerFileIterator VF = Ver.FileList(); VF.end() == false; VF++)
       {
 	 /* If this is the status file, and the current version is not the
@@ -151,18 +165,23 @@ pkgCache::VerIterator pkgPolicy::GetCandidateVer(pkgCache::PkgIterator Pkg)
 	    out bogus entries that may be due to config-file states, or
 	    other. */
 	 if ((VF.File()->Flags & pkgCache::Flag::NotSource) == pkgCache::Flag::NotSource &&
-	     Pkg.CurrentVer() != Ver)
+	     instVer == false)
 	    continue;
-	 	 	 
+
 	 signed Prio = PFPriority[VF.File()->ID];
 	 if (Prio > Max)
 	 {
 	    Pref = Ver;
 	    Max = Prio;
+	 }
+	 if (Prio > MaxAlt)
+	 {
+	    PrefAlt = Ver;
+	    MaxAlt = Prio;
 	 }	 
       }      
       
-      if (Pkg.CurrentVer() == Ver && Max < 1000)
+      if (instVer == true && Max < 1000)
       {
 	 /* Elevate our current selection (or the status file itself)
 	    to the Pseudo-status priority. */
@@ -175,6 +194,12 @@ pkgCache::VerIterator pkgPolicy::GetCandidateVer(pkgCache::PkgIterator Pkg)
 	    break;
       }            
    }
+   // If we do not find our candidate, use the one with the highest pin.
+   // This means that if there is a version available with pin > 0; there
+   // will always be a candidate (Closes: #512318)
+   if (!Pref.IsGood() && MaxAlt > 0)
+       Pref = PrefAlt;
+
    return Pref;
 }
 									/*}}}*/
@@ -187,49 +212,51 @@ pkgCache::VerIterator pkgPolicy::GetCandidateVer(pkgCache::PkgIterator Pkg)
 void pkgPolicy::CreatePin(pkgVersionMatch::MatchType Type,string Name,
 			  string Data,signed short Priority)
 {
-   Pin *P = 0;
-   
    if (Name.empty() == true)
-      P = &*Defaults.insert(Defaults.end(),PkgPin());
-   else
    {
-      // Get a spot to put the pin
-      pkgCache::PkgIterator Pkg = Cache->FindPkg(Name);
-      if (Pkg.end() == true)
+      Pin *P = &*Defaults.insert(Defaults.end(),PkgPin());
+      P->Type = Type;
+      P->Priority = Priority;
+      P->Data = Data;
+      return;
+   }
+
+   // Get a spot to put the pin
+   pkgCache::GrpIterator Grp = Cache->FindGrp(Name);
+   for (pkgCache::PkgIterator Pkg = Grp.FindPkg("any");
+	Pkg.end() != true; Pkg = Grp.NextPkg(Pkg))
+   {
+      Pin *P = 0;
+      if (Pkg.end() == false)
+	 P = Pins + Pkg->ID;
+      else
       {
 	 // Check the unmatched table
-	 for (vector<PkgPin>::iterator I = Unmatched.begin(); 
+	 for (vector<PkgPin>::iterator I = Unmatched.begin();
 	      I != Unmatched.end() && P == 0; I++)
 	    if (I->Pkg == Name)
 	       P = &*I;
-	 
+
 	 if (P == 0)
-	    P = &*Unmatched.insert(Unmatched.end(),PkgPin());      
+	    P = &*Unmatched.insert(Unmatched.end(),PkgPin());
       }
-      else
-      {
-	 P = Pins + Pkg->ID;
-      }      
+      P->Type = Type;
+      P->Priority = Priority;
+      P->Data = Data;
    }
-   
-   // Set..
-   P->Type = Type;
-   P->Priority = Priority;
-   P->Data = Data;
 }
 									/*}}}*/
 // Policy::GetMatch - Get the matching version for a package pin	/*{{{*/
 // ---------------------------------------------------------------------
 /* */
-pkgCache::VerIterator pkgPolicy::GetMatch(pkgCache::PkgIterator Pkg)
+pkgCache::VerIterator pkgPolicy::GetMatch(pkgCache::PkgIterator const &Pkg)
 {
    const Pin &PPkg = Pins[Pkg->ID];
-   if (PPkg.Type != pkgVersionMatch::None)
-   {
-      pkgVersionMatch Match(PPkg.Data,PPkg.Type);
-      return Match.Find(Pkg);
-   }
-   return pkgCache::VerIterator(*Pkg.Cache());
+   if (PPkg.Type == pkgVersionMatch::None)
+      return pkgCache::VerIterator(*Pkg.Cache());
+
+   pkgVersionMatch Match(PPkg.Data,PPkg.Type);
+   return Match.Find(Pkg);
 }
 									/*}}}*/
 // Policy::GetPriority - Get the priority of the package pin		/*{{{*/
@@ -274,9 +301,9 @@ bool ReadPinDir(pkgPolicy &Plcy,string Dir)
    if (Dir.empty() == true)
       Dir = _config->FindDir("Dir::Etc::PreferencesParts");
 
-   if (FileExists(Dir) == false)
+   if (DirectoryExists(Dir) == false)
    {
-      _error->WarningE("FileExists",_("Unable to read %s"),Dir.c_str());
+      _error->WarningE("DirectoryExists",_("Unable to read %s"),Dir.c_str());
       return true;
    }
 

apt-pkg/policy.h

@@ -72,11 +72,11 @@ class pkgPolicy : public pkgDepCache::Policy
    inline signed short GetPriority(pkgCache::PkgFileIterator const &File) 
        {return PFPriority[File->ID];};
    signed short GetPriority(pkgCache::PkgIterator const &Pkg);
-   pkgCache::VerIterator GetMatch(pkgCache::PkgIterator Pkg);
+   pkgCache::VerIterator GetMatch(pkgCache::PkgIterator const &Pkg);
 
    // Things for the cache interface.
-   virtual pkgCache::VerIterator GetCandidateVer(pkgCache::PkgIterator Pkg);
-   virtual bool IsImportantDep(pkgCache::DepIterator Dep) {return pkgDepCache::Policy::IsImportantDep(Dep);};
+   virtual pkgCache::VerIterator GetCandidateVer(pkgCache::PkgIterator const &Pkg);
+   virtual bool IsImportantDep(pkgCache::DepIterator const &Dep) {return pkgDepCache::Policy::IsImportantDep(Dep);};
    bool InitDefaults();
    
    pkgPolicy(pkgCache *Owner);

apt-pkg/sourcelist.cc

@@ -72,13 +72,58 @@ bool pkgSourceList::Type::FixupURI(string &URI) const
    Weird types may override this. */
 bool pkgSourceList::Type::ParseLine(vector<metaIndex *> &List,
 				    const char *Buffer,
-				    unsigned long CurLine,
-				    string File) const
+				    unsigned long const &CurLine,
+				    string const &File) const
 {
+   for (;Buffer != 0 && isspace(*Buffer); ++Buffer); // Skip whitespaces
+
+   // Parse option field if it exists
+   // e.g.: [ option1=value1 option2=value2 ]
+   map<string, string> Options;
+   if (Buffer != 0 && Buffer[0] == '[')
+   {
+      ++Buffer; // ignore the [
+      for (;Buffer != 0 && isspace(*Buffer); ++Buffer); // Skip whitespaces
+      while (*Buffer != ']')
+      {
+	 // get one option, e.g. option1=value1
+	 string option;
+	 if (ParseQuoteWord(Buffer,option) == false)
+	    return _error->Error(_("Malformed line %lu in source list %s ([option] unparseable)"),CurLine,File.c_str());
+
+	 if (option.length() < 3)
+	    return _error->Error(_("Malformed line %lu in source list %s ([option] too short)"),CurLine,File.c_str());
+
+	 // accept options even if the last has no space before the ]-end marker
+	 if (option.at(option.length()-1) == ']')
+	 {
+	    for (; *Buffer != ']'; --Buffer);
+	    option.resize(option.length()-1);
+	 }
+
+	 size_t const needle = option.find('=');
+	 if (needle == string::npos)
+	    return _error->Error(_("Malformed line %lu in source list %s ([%s] is not an assignment)"),CurLine,File.c_str(), option.c_str());
+
+	 string const key = string(option, 0, needle);
+	 string const value = string(option, needle + 1, option.length());
+
+	 if (key.empty() == true)
+	    return _error->Error(_("Malformed line %lu in source list %s ([%s] has no key)"),CurLine,File.c_str(), option.c_str());
+
+	 if (value.empty() == true)
+	    return _error->Error(_("Malformed line %lu in source list %s ([%s] key %s has no value)"),CurLine,File.c_str(),option.c_str(),key.c_str());
+
+	 Options[key] = value;
+      }
+      ++Buffer; // ignore the ]
+      for (;Buffer != 0 && isspace(*Buffer); ++Buffer); // Skip whitespaces
+   }
+
    string URI;
    string Dist;
-   string Section;   
-   
+   string Section;
+
    if (ParseQuoteWord(Buffer,URI) == false)
       return _error->Error(_("Malformed line %lu in source list %s (URI)"),CurLine,File.c_str());
    if (ParseQuoteWord(Buffer,Dist) == false)
@@ -93,7 +138,7 @@ bool pkgSourceList::Type::ParseLine(vector<metaIndex *> &List,
       if (ParseQuoteWord(Buffer,Section) == true)
 	 return _error->Error(_("Malformed line %lu in source list %s (absolute dist)"),CurLine,File.c_str());
       Dist = SubstVar(Dist,"$(ARCH)",_config->Find("APT::Architecture"));
-      return CreateItem(List,URI,Dist,Section);
+      return CreateItem(List, URI, Dist, Section, Options);
    }
    
    // Grab the rest of the dists
@@ -102,7 +147,7 @@ bool pkgSourceList::Type::ParseLine(vector<metaIndex *> &List,
    
    do
    {
-      if (CreateItem(List,URI,Dist,Section) == false)
+      if (CreateItem(List, URI, Dist, Section, Options) == false)
 	 return false;
    }
    while (ParseQuoteWord(Buffer,Section) == true);
@@ -154,15 +199,15 @@ bool pkgSourceList::ReadMainList()
    
    if (FileExists(Main) == true)
       Res &= ReadAppend(Main);
-   else if (FileExists(Parts) == false)
+   else if (DirectoryExists(Parts) == false)
       // Only warn if there are no sources.list.d.
-      _error->WarningE("FileExists",_("Unable to read %s"),Main.c_str());
+      _error->WarningE("DirectoryExists", _("Unable to read %s"), Parts.c_str());
 
-   if (FileExists(Parts) == true)
+   if (DirectoryExists(Parts) == true)
       Res &= ReadSourceDir(Parts);
    else if (FileExists(Main) == false)
       // Only warn if there is no sources.list file.
-      _error->WarningE("FileExists",_("Unable to read %s"),Parts.c_str());
+      _error->WarningE("FileExists", _("Unable to read %s"), Main.c_str());
 
    return Res;
 }
@@ -239,36 +284,7 @@ bool pkgSourceList::ReadAppend(string File)
       if (Parse == 0)
 	 return _error->Error(_("Type '%s' is not known on line %u in source list %s"),LineType.c_str(),CurLine,File.c_str());
       
-      // Vendor name specified
-      if (C[0] == '[')
-      {
-	 string VendorID;
-	 
-	 if (ParseQuoteWord(C,VendorID) == false)
-	     return _error->Error(_("Malformed line %u in source list %s (vendor id)"),CurLine,File.c_str());
-
-	 if (VendorID.length() < 2 || VendorID.end()[-1] != ']')
-	     return _error->Error(_("Malformed line %u in source list %s (vendor id)"),CurLine,File.c_str());
-	 VendorID = string(VendorID,1,VendorID.size()-2);
-	 
-// 	 for (vector<const Vendor *>::const_iterator iter = VendorList.begin();
-// 	      iter != VendorList.end(); iter++) 
-// 	 {
-// 	    if ((*iter)->GetVendorID() == VendorID)
-// 	    {
-// 	      if (_config->FindB("Debug::sourceList", false)) 
-// 		std::cerr << "Comparing VendorID \"" << VendorID << "\" with \"" << (*iter)->GetVendorID() << '"' << std::endl;
-// 	       Verifier = *iter;
-// 	       break;
-// 	    }
-// 	 }
-
-// 	 if (Verifier == 0)
-// 	    return _error->Error(_("Unknown vendor ID '%s' in line %u of source list %s"),
-// 				 VendorID.c_str(),CurLine,File.c_str());
-      }
-
-      if (Parse->ParseLine(SrcList,C,CurLine,File) == false)
+      if (Parse->ParseLine(SrcList, C, CurLine, File) == false)
 	 return false;
    }
    return true;

apt-pkg/sourcelist.h

@@ -29,6 +29,7 @@
 
 #include <string>
 #include <vector>
+#include <map>
 #include <apt-pkg/pkgcache.h>
 #include <apt-pkg/metaindex.h>
 
@@ -57,9 +58,10 @@ class pkgSourceList
       bool FixupURI(string &URI) const;
       virtual bool ParseLine(vector<metaIndex *> &List,
 			     const char *Buffer,
-			     unsigned long CurLine,string File) const;
-      virtual bool CreateItem(vector<metaIndex *> &List,string URI,
-			      string Dist,string Section) const = 0;
+			     unsigned long const &CurLine,string const &File) const;
+      virtual bool CreateItem(vector<metaIndex *> &List,string const &URI,
+			      string const &Dist,string const &Section,
+			      std::map<string, string> const &Options) const = 0;
       Type();
       virtual ~Type() {};
    };

apt-pkg/srcrecords.cc

@@ -77,7 +77,7 @@ bool pkgSrcRecords::Restart()
 /* This searches on both source package names and output binary names and
    returns the first found. A 'cursor' like system is used to allow this
    function to be called multiple times to get successive entries */
-pkgSrcRecords::Parser *pkgSrcRecords::Find(const char *Package,bool SrcOnly)
+pkgSrcRecords::Parser *pkgSrcRecords::Find(const char *Package,bool const &SrcOnly)
 {
    if (Current == Files.end())
       return 0;
@@ -116,7 +116,7 @@ pkgSrcRecords::Parser *pkgSrcRecords::Find(const char *Package,bool SrcOnly)
 // Parser::BuildDepType - Convert a build dep to a string		/*{{{*/
 // ---------------------------------------------------------------------
 /* */
-const char *pkgSrcRecords::Parser::BuildDepType(unsigned char Type)
+const char *pkgSrcRecords::Parser::BuildDepType(unsigned char const &Type)
 {
    const char *fields[] = {"Build-Depends", 
                            "Build-Depends-Indep",

apt-pkg/srcrecords.h

@@ -59,7 +59,7 @@ class pkgSrcRecords
       
       virtual bool Restart() = 0;
       virtual bool Step() = 0;
-      virtual bool Jump(unsigned long Off) = 0;
+      virtual bool Jump(unsigned long const &Off) = 0;
       virtual unsigned long Offset() = 0;
       virtual string AsStr() = 0;
       
@@ -69,8 +69,8 @@ class pkgSrcRecords
       virtual string Section() const = 0;
       virtual const char **Binaries() = 0;   // Ownership does not transfer
 
-      virtual bool BuildDepends(vector<BuildDepRec> &BuildDeps, bool ArchOnly) = 0;
-      static const char *BuildDepType(unsigned char Type);
+      virtual bool BuildDepends(vector<BuildDepRec> &BuildDeps, bool const &ArchOnly, bool const &StripMultiArch = true) = 0;
+      static const char *BuildDepType(unsigned char const &Type);
 
       virtual bool Files(vector<pkgSrcRecords::File> &F) = 0;
       
@@ -90,7 +90,7 @@ class pkgSrcRecords
    bool Restart();
 
    // Locate a package by name
-   Parser *Find(const char *Package,bool SrcOnly = false);
+   Parser *Find(const char *Package,bool const &SrcOnly = false);
    
    pkgSrcRecords(pkgSourceList &List);
    ~pkgSrcRecords();

apt-pkg/tagfile.cc

@@ -193,17 +193,8 @@ bool pkgTagFile::Jump(pkgTagSection &Tag,unsigned long Offset)
 									/*}}}*/
 // TagSection::Scan - Scan for the end of the header information	/*{{{*/
 // ---------------------------------------------------------------------
-/* This looks for the first double new line in the data stream. It also
-   indexes the tags in the section. This very simple hash function for the
-   last 8 letters gives very good performance on the debian package files */
-inline static unsigned long AlphaHash(const char *Text, const char *End = 0)
-{
-   unsigned long Res = 0;
-   for (; Text != End && *Text != ':' && *Text != 0; Text++)
-      Res = ((unsigned long)(*Text) & 0xDF) ^ (Res << 1);
-   return Res & 0xFF;
-}
-
+/* This looks for the first double new line in the data stream.
+   It also indexes the tags in the section. */
 bool pkgTagSection::Scan(const char *Start,unsigned long MaxLength)
 {
    const char *End = Start + MaxLength;
@@ -374,6 +365,30 @@ signed int pkgTagSection::FindI(const char *Tag,signed long Default) const
    return Result;
 }
 									/*}}}*/
+// TagSection::FindULL - Find an unsigned long long integer		/*{{{*/
+// ---------------------------------------------------------------------
+/* */
+unsigned long long pkgTagSection::FindULL(const char *Tag, unsigned long long const &Default) const
+{
+   const char *Start;
+   const char *Stop;
+   if (Find(Tag,Start,Stop) == false)
+      return Default;
+
+   // Copy it into a temp buffer so we can use strtoull
+   char S[100];
+   if ((unsigned)(Stop - Start) >= sizeof(S))
+      return Default;
+   strncpy(S,Start,Stop-Start);
+   S[Stop - Start] = 0;
+   
+   char *End;
+   unsigned long long Result = strtoull(S,&End,10);
+   if (S == End)
+      return Default;
+   return Result;
+}
+									/*}}}*/
 // TagSection::FindFlag - Locate a yes/no type flag			/*{{{*/
 // ---------------------------------------------------------------------
 /* The bits marked in Flag are masked on/off in Flags */
@@ -418,6 +433,7 @@ static const char *iTFRewritePackageOrder[] = {
                           "Section",
                           "Installed-Size",
                           "Maintainer",
+                          "Original-Maintainer",
                           "Architecture",
                           "Source",
                           "Version",
@@ -447,6 +463,7 @@ static const char *iTFRewriteSourceOrder[] = {"Package",
                                       "Priority",
                                       "Section",
                                       "Maintainer",
+				      "Original-Maintainer",
                                       "Build-Depends",
                                       "Build-Depends-Indep",
                                       "Build-Conflicts",

apt-pkg/tagfile.h

@@ -33,7 +33,18 @@ class pkgTagSection
    unsigned int AlphaIndexes[0x100];
    
    unsigned int TagCount;
-     
+
+   /* This very simple hash function for the last 8 letters gives
+      very good performance on the debian package files */
+   inline static unsigned long AlphaHash(const char *Text, const char *End = 0)
+   {
+      unsigned long Res = 0;
+      for (; Text != End && *Text != ':' && *Text != 0; Text++)
+	 Res = ((unsigned long)(*Text) & 0xDF) ^ (Res << 1);
+      return Res & 0xFF;
+   }
+
+
    protected:
    const char *Stop;
 
@@ -46,6 +57,7 @@ class pkgTagSection
    bool Find(const char *Tag,unsigned &Pos) const;
    string FindS(const char *Tag) const;
    signed int FindI(const char *Tag,signed long Default = 0) const ;
+   unsigned long long FindULL(const char *Tag, unsigned long long const &Default = 0) const;
    bool FindFlag(const char *Tag,unsigned long &Flags,
 		 unsigned long Flag) const;
    bool Scan(const char *Start,unsigned long MaxLength);
@@ -54,6 +66,8 @@ class pkgTagSection
    virtual void TrimRecord(bool BeforeRecord, const char* &End);
    
    inline unsigned int Count() const {return TagCount;};
+   inline bool Exists(const char* const Tag) {return AlphaIndexes[AlphaHash(Tag)] != 0;}
+ 
    inline void Get(const char *&Start,const char *&Stop,unsigned int I) const
                    {Start = Section + Indexes[I]; Stop = Section + Indexes[I+1];}
 	    

apt-pkg/versionmatch.cc

@@ -18,6 +18,10 @@
 
 #include <stdio.h>
 #include <ctype.h>
+#include <fnmatch.h>
+#include <sys/types.h>
+#include <regex.h>
+
 									/*}}}*/
 
 // VersionMatch::pkgVersionMatch - Constructor				/*{{{*/
@@ -100,6 +104,8 @@ pkgVersionMatch::pkgVersionMatch(string Data,MatchType Type) : Type(Type)
 	    RelLabel = Fragments[J]+2;
 	 else if (stringcasecmp(Fragments[J],Fragments[J]+2,"c=") == 0)
 	    RelComponent = Fragments[J]+2;
+	 else if (stringcasecmp(Fragments[J],Fragments[J]+2,"b=") == 0)
+	    RelArchitecture = Fragments[J]+2;
       }
 
       if (RelVerStr.end()[-1] == '*')
@@ -112,7 +118,10 @@ pkgVersionMatch::pkgVersionMatch(string Data,MatchType Type) : Type(Type)
    
    if (Type == Origin)
    {
-      OrSite = Data;
+      if (Data[0] == '"' && Data.length() >= 2 && Data.end()[-1] == '"')
+	 OrSite = Data.substr(1, Data.length() - 2);
+      else
+	 OrSite = Data;
       return;
    }   
 }
@@ -149,6 +158,8 @@ pkgCache::VerIterator pkgVersionMatch::Find(pkgCache::PkgIterator Pkg)
       {
 	 if (MatchVer(Ver.VerStr(),VerStr,VerPrefixMatch) == true)
 	    return Ver;
+	 if (ExpressionMatches(VerStr, Ver.VerStr()) == true)
+	    return Ver;
 	 continue;
       }
       
@@ -160,6 +171,36 @@ pkgCache::VerIterator pkgVersionMatch::Find(pkgCache::PkgIterator Pkg)
    // This will be Ended by now.
    return Ver;
 }
+
+#ifndef FNM_CASEFOLD
+#define FNM_CASEFOLD 0
+#endif
+
+bool pkgVersionMatch::ExpressionMatches(const char *pattern, const char *string)
+{
+   if (pattern[0] == '/') {
+      bool res = false;
+      size_t length = strlen(pattern);
+      if (pattern[length - 1] == '/') {
+	 regex_t preg;
+	 char *regex = strdup(pattern + 1);
+	 regex[length - 2] = '\0';
+	 if (regcomp(&preg, regex, REG_EXTENDED | REG_ICASE) != 0) {
+	    _error->Warning("Invalid regular expression: %s", regex);
+	 } else if (regexec(&preg, string, 0, NULL, 0) == 0) {
+	    res = true;
+	 }
+	 free(regex);
+	 regfree(&preg);
+	 return res;
+      }
+   }
+   return fnmatch(pattern, string, FNM_CASEFOLD) == 0;
+}
+bool pkgVersionMatch::ExpressionMatches(const std::string& pattern, const char *string)
+{
+    return ExpressionMatches(pattern.c_str(), string);
+}
 									/*}}}*/
 // VersionMatch::FileMatch - Match against an index file		/*{{{*/
 // ---------------------------------------------------------------------
@@ -178,38 +219,42 @@ bool pkgVersionMatch::FileMatch(pkgCache::PkgFileIterator File)
       if (RelVerStr.empty() == true && RelOrigin.empty() == true &&
 	  RelArchive.empty() == true && RelLabel.empty() == true &&
 	  RelRelease.empty() == true && RelCodename.empty() == true &&
-	  RelComponent.empty() == true)
+	  RelComponent.empty() == true && RelArchitecture.empty() == true)
 	 return false;
 
       if (RelVerStr.empty() == false)
 	 if (File->Version == 0 ||
-	     MatchVer(File.Version(),RelVerStr,RelVerPrefixMatch) == false)
+	     (MatchVer(File.Version(),RelVerStr,RelVerPrefixMatch) == false &&
+	      ExpressionMatches(RelVerStr, File.Version()) == false))
 	    return false;
       if (RelOrigin.empty() == false)
-	 if (File->Origin == 0 ||
-	     stringcasecmp(RelOrigin,File.Origin()) != 0)
+	 if (File->Origin == 0 || !ExpressionMatches(RelOrigin,File.Origin()))
 	    return false;
       if (RelArchive.empty() == false)
 	 if (File->Archive == 0 ||
-	     stringcasecmp(RelArchive,File.Archive()) != 0)
+	     !ExpressionMatches(RelArchive,File.Archive()))
             return false;
       if (RelCodename.empty() == false)
 	 if (File->Codename == 0 ||
-	     stringcasecmp(RelCodename,File.Codename()) != 0)
+	     !ExpressionMatches(RelCodename,File.Codename()))
             return false;
       if (RelRelease.empty() == false)
 	 if ((File->Archive == 0 ||
-	     stringcasecmp(RelRelease,File.Archive()) != 0) &&
+	     !ExpressionMatches(RelRelease,File.Archive())) &&
              (File->Codename == 0 ||
-	      stringcasecmp(RelRelease,File.Codename()) != 0))
+	      !ExpressionMatches(RelRelease,File.Codename())))
 	       return false;
       if (RelLabel.empty() == false)
 	 if (File->Label == 0 ||
-	     stringcasecmp(RelLabel,File.Label()) != 0)
+	     !ExpressionMatches(RelLabel,File.Label()))
 	    return false;
       if (RelComponent.empty() == false)
 	 if (File->Component == 0 ||
-	     stringcasecmp(RelComponent,File.Component()) != 0)
+	     !ExpressionMatches(RelComponent,File.Component()))
+	    return false;
+      if (RelArchitecture.empty() == false)
+	 if (File->Architecture == 0 ||
+	     !ExpressionMatches(RelArchitecture,File.Architecture()))
 	    return false;
       return true;
    }
@@ -217,12 +262,12 @@ bool pkgVersionMatch::FileMatch(pkgCache::PkgFileIterator File)
    if (Type == Origin)
    {
       if (OrSite.empty() == false) {
-	 if (File->Site == 0 || OrSite != File.Site())
+	 if (File->Site == 0)
 	    return false;
       } else // so we are talking about file:// or status file
-	 if (strcmp(File.Site(),"") == 0 && File->Archive != 0) // skip the status file
+	 if (strcmp(File.Site(),"") == 0 && File->Archive != 0 && strcmp(File.Archive(),"now") == 0) // skip the status file
 	    return false;
-      return (OrSite == File.Site());		/* both strings match */
+      return (ExpressionMatches(OrSite, File.Site())); /* both strings match */
    }
 
    return false;

apt-pkg/versionmatch.h

@@ -23,6 +23,7 @@
       Codename (n=) e.g. etch, lenny, squeeze, sid
       Label (l=)
       Component (c=)
+      Binary Architecture (b=)
    If there are no equals signs in the string then it is scanned in short
    form - if it starts with a number it is Version otherwise it is an
    Archive or a Codename.
@@ -55,6 +56,7 @@ class pkgVersionMatch
    string RelArchive;
    string RelLabel;
    string RelComponent;
+   string RelArchitecture;
    bool MatchAll;
    
    // Origin Matching
@@ -65,6 +67,8 @@ class pkgVersionMatch
    enum MatchType {None = 0,Version,Release,Origin} Type;
    
    bool MatchVer(const char *A,string B,bool Prefix);
+   bool ExpressionMatches(const char *pattern, const char *string);
+   bool ExpressionMatches(const std::string& pattern, const char *string);
    bool FileMatch(pkgCache::PkgFileIterator File);
    pkgCache::VerIterator Find(pkgCache::PkgIterator Pkg);
 			       

buildlib/apti18n.h.in

@@ -11,8 +11,10 @@
 # include <libintl.h>
 # ifdef APT_DOMAIN
 #   define _(x) dgettext(APT_DOMAIN,x)
+#   define P_(msg,plural,n) dngettext(APT_DOMAIN,msg,plural,n)
 # else
 #   define _(x) gettext(x)
+#   define P_(msg,plural,n) ngettext(msg,plural,n)
 # endif
 # define N_(x) x
 #else
@@ -21,5 +23,6 @@
 # define textdomain(a)
 # define bindtextdomain(a, b)
 # define _(x) x
+# define P_(msg,plural,n) (n == 1 ? msg : plural)
 # define N_(x) x
 #endif

buildlib/library.mak

@@ -16,11 +16,11 @@
 # See defaults.mak for information about LOCAL
 
 # Some local definitions
-LOCAL := lib$(LIBRARY)$(LIBEXT).so.$(MAJOR).$(MINOR)
+LOCAL := lib$(LIBRARY).so.$(MAJOR).$(MINOR)
 $(LOCAL)-OBJS := $(addprefix $(OBJ)/,$(addsuffix .opic,$(notdir $(basename $(SOURCE)))))
 $(LOCAL)-DEP := $(addprefix $(DEP)/,$(addsuffix .opic.d,$(notdir $(basename $(SOURCE)))))
 $(LOCAL)-HEADERS := $(addprefix $(INCLUDE)/,$(HEADERS))
-$(LOCAL)-SONAME := lib$(LIBRARY)$(LIBEXT).so.$(MAJOR)
+$(LOCAL)-SONAME := lib$(LIBRARY).so.$(MAJOR)
 $(LOCAL)-SLIBS := $(SLIBS)
 $(LOCAL)-LIBRARY := $(LIBRARY)
 
@@ -29,7 +29,7 @@ include $(PODOMAIN_H)
 
 # Install the command hooks
 headers: $($(LOCAL)-HEADERS)
-library: $(LIB)/lib$(LIBRARY).so $(LIB)/lib$(LIBRARY)$(LIBEXT).so.$(MAJOR)
+library: $(LIB)/lib$(LIBRARY).so $(LIB)/lib$(LIBRARY).so.$(MAJOR)
 clean: clean/$(LOCAL)
 veryclean: veryclean/$(LOCAL)
 
@@ -44,14 +44,14 @@ veryclean/$(LOCAL): clean/$(LOCAL)
 	-rm -f $($(@F)-HEADERS) $(LIB)/lib$($(@F)-LIBRARY)*.so*
 
 # Build rules for the two symlinks
-.PHONY: $(LIB)/lib$(LIBRARY)$(LIBEXT).so.$(MAJOR) $(LIB)/lib$(LIBRARY).so
-$(LIB)/lib$(LIBRARY)$(LIBEXT).so.$(MAJOR): $(LIB)/lib$(LIBRARY)$(LIBEXT).so.$(MAJOR).$(MINOR)
+.PHONY: $(LIB)/lib$(LIBRARY).so.$(MAJOR) $(LIB)/lib$(LIBRARY).so
+$(LIB)/lib$(LIBRARY).so.$(MAJOR): $(LIB)/lib$(LIBRARY).so.$(MAJOR).$(MINOR)
 	ln -sf $(<F) $@
-$(LIB)/lib$(LIBRARY).so: $(LIB)/lib$(LIBRARY)$(LIBEXT).so.$(MAJOR).$(MINOR)
+$(LIB)/lib$(LIBRARY).so: $(LIB)/lib$(LIBRARY).so.$(MAJOR).$(MINOR)
 	ln -sf $(<F) $@
 
 # The binary build rule
-$(LIB)/lib$(LIBRARY)$(LIBEXT).so.$(MAJOR).$(MINOR): $($(LOCAL)-HEADERS) $($(LOCAL)-OBJS)
+$(LIB)/lib$(LIBRARY).so.$(MAJOR).$(MINOR): $($(LOCAL)-HEADERS) $($(LOCAL)-OBJS)
 	-rm -f $(LIB)/lib$($(@F)-LIBRARY)*.so* 2> /dev/null
 	echo Building shared library $@
 	$(CXX) $(CXXFLAGS) $(LDFLAGS) $(PICFLAGS) $(LFLAGS) $(LFLAGS_SO)\

buildlib/libversion.mak

@@ -12,10 +12,3 @@ LIBAPTPKG_RELEASE=$(shell grep -E '^\#define APT_PKG_RELEASE' $(BASE)/apt-pkg/in
 # The versionnumber is extracted from apt-inst/makefile - see also there.
 LIBAPTINST_MAJOR=$(shell egrep '^MAJOR=' $(BASE)/apt-inst/makefile |cut -d '=' -f 2)
 LIBAPTINST_MINOR=$(shell egrep '^MINOR=' $(BASE)/apt-inst/makefile |cut -d '=' -f 2)
-
-# FIXME: In previous releases this lovely variable includes
-# the detected libc and libdc++ version. As this is bogus we
-# want to drop this, but this a ABI break.
-# And we don't want to do this now. So we hardcode a value here,
-# and drop it later on (hopefully as fast as possible).
-LIBEXT=-libc6.9-6

cmdline/apt-cdrom.cc

@@ -147,7 +147,7 @@ bool DoAdd(CommandLine &)
    pkgCdrom cdrom;
    bool res = true;
 
-   bool AutoDetect = _config->FindB("Acquire::cdrom::AutoDetect");
+   bool AutoDetect = _config->FindB("Acquire::cdrom::AutoDetect", true);
    unsigned int count = 0;
    
    if (AutoDetect && UdevCdroms.Dlopen())
@@ -266,20 +266,18 @@ int main(int argc,const char *argv[])					/*{{{*/
       return ShowHelp();
 
    // Deal with stdout not being a tty
-   if (isatty(STDOUT_FILENO) && _config->FindI("quiet",0) < 1)
+   if (isatty(STDOUT_FILENO) && _config->FindI("quiet", -1) == -1)
       _config->Set("quiet","1");
    
    // Match the operation
    CmdL.DispatchArg(Cmds);
 
    // Print any errors or warnings found during parsing
-   if (_error->empty() == false)
-   {
-      bool Errors = _error->PendingError();
+   bool const Errors = _error->PendingError();
+   if (_config->FindI("quiet",0) > 0)
       _error->DumpErrors();
-      return Errors == true?100:0;
-   }
-   
-   return 0;
+   else
+      _error->DumpErrors(GlobalError::DEBUG);
+   return Errors == true ? 100 : 0;
 }
 									/*}}}*/

cmdline/apt-extracttemplates.cc

@@ -296,8 +296,7 @@ bool Go(CommandLine &CmdL)
 	MMap *Map = 0;
 	pkgSourceList List;
 	List.ReadMainList();
-	OpProgress Prog;
-	pkgMakeStatusCache(List,Prog,&Map,true);
+	pkgCacheGenerator::MakeStatusCache(List,NULL,&Map,true);
 	if (Map == 0)
 	   return false;
 	DebFile::Cache = new pkgCache(Map);

cmdline/apt-key

@@ -146,6 +146,7 @@ if [ "$1" = "--keyring" ]; then
 else
 	#echo "generate list"
 	TRUSTEDFILE="/etc/apt/trusted.gpg"
+	eval $(apt-config shell TRUSTEDFILE Apt::GPGV::TrustedKeyring)
 	if [ -r "$TRUSTEDFILE" ]; then
 		GPG="$GPG --keyring $TRUSTEDFILE"
 	fi

cmdline/apt-mark

@@ -8,7 +8,7 @@ import os.path
 try:
     import apt_pkg
 except ImportError:
-    print "Error importing apt_pkg, is python-apt installed?"
+    print >> sys.stderr, "Error importing apt_pkg, is python-apt installed?"
     sys.exit(1)
     
 actions = { "markauto" : 1,
@@ -19,10 +19,10 @@ def show_automatic(filename):
     if not os.path.exists(STATE_FILE):
         return
     auto = set()
-    tagfile = apt_pkg.ParseTagFile(open(STATE_FILE))
-    while tagfile.Step():
-        pkgname = tagfile.Section.get("Package")
-        autoInst = tagfile.Section.get("Auto-Installed")
+    tagfile = apt_pkg.TagFile(open(STATE_FILE))
+    for section in tagfile:
+        pkgname = section.get("Package")
+        autoInst = section.get("Auto-Installed")
         if int(autoInst):
             auto.add(pkgname)
     print "\n".join(sorted(auto))
@@ -33,24 +33,24 @@ def mark_unmark_automatic(filename, action, pkgs):
     # open the statefile
     if os.path.exists(STATE_FILE):
         try:
-            tagfile = apt_pkg.ParseTagFile(open(STATE_FILE))
+            tagfile = apt_pkg.TagFile(open(STATE_FILE))
             outfile = open(STATE_FILE+".tmp","w")
         except IOError, msg:
             print "%s, are you root?" % (msg)
             sys.exit(1)
-        while tagfile.Step():
-            pkgname = tagfile.Section.get("Package")
-            autoInst = tagfile.Section.get("Auto-Installed")
+        for section in tagfile:
+            pkgname = section.get("Package")
+            autoInst = section.get("Auto-Installed")
             if pkgname in pkgs:
                 if options.verbose:
                     print "changing %s to %s" % (pkgname,action)
-                newsec = apt_pkg.RewriteSection(tagfile.Section,
+                newsec = apt_pkg.rewrite_section(section,
                                        [],
                                        [ ("Auto-Installed",str(action)) ])
                 pkgs.remove(pkgname)
                 outfile.write(newsec+"\n")
             else:
-                outfile.write(str(tagfile.Section)+"\n")
+                outfile.write(str(section)+"\n")
         if action == 1:
             for pkgname in pkgs:
                 if options.verbose:
@@ -68,6 +68,7 @@ if __name__ == "__main__":
     # option parsing
     parser = OptionParser()
     parser.usage = "%prog [options] {markauto|unmarkauto} packages..."
+    parser.epilog = "apt-mark is deprecated, use apt-get markauto/unmarkauto."
     parser.add_option("-f", "--file", action="store", type="string",
                       dest="filename",
                       help="read/write a different file")
@@ -76,9 +77,13 @@ if __name__ == "__main__":
                       help="print verbose status messages to stdout")
     (options, args) = parser.parse_args()
 
+    if not args:
+        parser.print_help()
+        sys.exit(1)
+
     # get the state-file
     if not options.filename:
-        STATE_FILE = apt_pkg.Config.FindDir("Dir::State") + "extended_states"
+        STATE_FILE = apt_pkg.config.find_dir("Dir::State") + "extended_states"
     else:
         STATE_FILE=options.filename
 

cmdline/apt-report-mirror-failure

@@ -0,0 +1,29 @@
+#!/usr/bin/python
+#
+# This is a stub that is meant to support failure reporting of 
+# mirrors to a central database
+#
+# its currently not used
+
+import sys
+import urllib
+import apt_pkg
+
+apt_pkg.init()
+url = apt_pkg.Config.find("Acquire::Mirror::ReportFailures", "")
+                          #"http://people.ubuntu.com:9000/mirror-failure")
+                          #"http://localhost:9000/mirror-failure")
+if not url:
+    sys.exit(0)
+
+print "Reporting mirror failure to '%s'" % url
+
+data = {}
+data['mirror'] = sys.argv[1]
+data['failurl'] = sys.argv[2]
+data['error'] = sys.argv[3]
+f = urllib.urlopen(url, urllib.urlencode(data))
+f.read()
+f.close()
+
+

cmdline/makefile

@@ -58,3 +58,9 @@ SOURCE=apt-mark
 TO=$(BIN)
 TARGET=program
 include $(COPY_H)
+
+# The apt-report-mirror-failure program
+#SOURCE=apt-report-mirror-failure
+#TO=$(BIN)
+#TARGET=program
+#include $(COPY_H)

configure.in

@@ -18,7 +18,7 @@ AC_CONFIG_AUX_DIR(buildlib)
 AC_CONFIG_HEADER(include/config.h:buildlib/config.h.in include/apti18n.h:buildlib/apti18n.h.in)
 
 dnl -- SET THIS TO THE RELEASE VERSION --
-AC_DEFINE_UNQUOTED(VERSION,"0.7.25.4")
+AC_DEFINE_UNQUOTED(VERSION,"0.7.26~exp6")
 PACKAGE="apt"
 AC_DEFINE_UNQUOTED(PACKAGE,"$PACKAGE")
 AC_SUBST(PACKAGE)
@@ -83,6 +83,10 @@ AC_CHECK_LIB(curl, curl_easy_init,
 
 AC_SUBST(BDBLIB)
 
+AC_CHECK_LIB(z, gzopen,
+	[AC_CHECK_HEADER(zlib.h, [], AC_MSG_ERROR([failed: zlib.h not found]))],
+	AC_MSG_ERROR([failed: Need libz]))
+
 dnl Converts the ARCH to be something singular for this general CPU family
 dnl This is often the dpkg architecture string.
 dnl First check against the full canonical canoncial-system-type in $target
@@ -202,7 +206,7 @@ ah_GCC3DEP
 dnl It used to be that the user could select translations and that could get
 dnl passed to the makefiles, but now that can only work if you use special
 dnl gettext approved makefiles, so this feature is unsupported by this.
-ALL_LINGUAS="bg bs ca cs cy da de dz el en_GB es eu fi fr gl hu it ja ko ku nb nl nn pl pt_BR pt ro ru sk sl sv th tl uk vi zn_CN zh_TW"
+ALL_LINGUAS="bg bs ca cs cy da de dz el es eu fi fr gl hu it ja ko ku nb nl nn pl pt_BR pt ro ru sk sl sv th tl uk vi zn_CN zh_TW"
 AM_GNU_GETTEXT(external)
 if test x"$USE_NLS" = "xyes"; then
    AC_DEFINE(USE_NLS)

debian/NEWS

@@ -1,8 +1,15 @@
+apt (0.7.26~exp3) experimental; urgency=low
+
+  * apt-ftparchive now reads the standard configuration files in
+    /etc/apt/apt.conf and /etc/apt/apt.conf.d.
+
+ -- Julian Andres Klode <jak@debian.org>  Fri, 26 Mar 2010 15:34:16 +0100
+
 apt (0.7.24) unstable; urgency=low
 
   * Already included in the last version but now with better documentation
     is the possibility to add/prefer different compression types while
-    downloading archive informations, which can decrease the time needed for
+    downloading archive information, which can decrease the time needed for
     update on slow machines. See apt.conf (5) manpage for details.
   * APT manages his manpage translations now with po4a, thanks to Nicolas
     François and Kurasawa Nozomu, who also provide the ja translation.

debian/apt-doc.docs

@@ -1 +1,2 @@
-README.progress-reporting
+README.progress-reporting
+README.MultiArch

debian/apt.conf.autoremove

@@ -1,9 +1,24 @@
 APT
 {
-  NeverAutoRemove  
-  { 
-	"^linux-image.*";  
+  NeverAutoRemove
+  {
+	"^firmware-linux.*";
+	"^linux-firmware$";
+	"^linux-image.*";
+	"^kfreebsd-image.*";
 	"^linux-restricted-modules.*";
-	"^kfreebsd-image.*";  
+	"^linux-ubuntu-modules-.*";
+  };
+
+  Never-MarkAuto-Sections
+  {
+	"metapackages";
+	"restricted/metapackages";
+	"universe/metapackages";
+	"multiverse/metapackages";
+	"oldlibs";
+	"restricted/oldlibs";
+	"universe/oldlibs";
+	"multiverse/oldlibs";
   };
 };

debian/apt.cron.daily

@@ -320,6 +320,15 @@ debug_echo()
 
 # ------------------------ main ----------------------------
 
+# Backup the 7 last versions of APT's extended_states file
+# shameless copy from dpkg cron
+if cd /var/backups ; then
+    if ! cmp -s apt.extended_states.0 /var/lib/apt/extended_states; then
+	cp -p /var/lib/apt/extended_states apt.extended_states
+	savelog -c 7 apt.extended_states >/dev/null
+    fi
+fi
+
 # check apt-config exstance
 if ! which apt-config >/dev/null ; then
 	exit 0

debian/apt.dirs

@@ -9,6 +9,7 @@ etc/apt/trusted.gpg.d
 etc/logrotate.d
 var/cache/apt/archives/partial
 var/lib/apt/lists/partial
+var/lib/apt/mirrors/partial
 var/lib/apt/periodic
 var/log/apt
 usr/share/bug/apt